Re: [Libvir] [RFC]OpenVZ XML def
Shuveb Hussain wrote:
Daemon though? Can't the libvirt code just invoke the OpenVZ commands directly?
Yeah, that is very much possible. But if it were a daemon, it could be a Python daemon, since text processing is much simpler and I don't have to use the str*( ) funcs in C. What is your comment?
I'm a big advocate of using a sane language instead of C, but libvirt is written in C for better or worse. The problem with a separate daemon is management of that daemon. Something needs to start it and stop it, it needs to have sockets with the right permissions and so on. What happens if the process runs and can't find the daemon? What is the path to the socket? etc. etc. At the moment we have two daemons - the qemu daemon (which has two separate functions intertwined), and a remote daemon (necessary, because there's no other way to communicate to a remote machine). One idea would be to have the Python code fork off from the main program and communicate over a pipe. With this, there is no daemon management problem. +----------------+ | Program | | - - - - - - - -| | libvirt | +----------------+ | | fork(2) and pipe(2) | +-------------+ | Python code | +-------------+ But I'm not the person who ultimately decides what would be acceptable in libvirt. You'd need to get general agreement on the best way to do this. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Hello,
Daemon though? Can't the libvirt code just invoke the OpenVZ commands directly?
Yeah, that is very much possible. But if it were a daemon, it could be a Python daemon, since text processing is much simpler and I don't have to use the str*( ) funcs in C. What is your comment?
I'm a big advocate of using a sane language instead of C, but libvirt is written in C for better or worse.
The problem with a separate daemon is management of that daemon. Something needs to start it and stop it, it needs to have sockets with the right permissions and so on. What happens if the process runs and can't find the daemon? What is the path to the socket? etc. etc. At the moment we have two daemons - the qemu daemon (which has two separate functions intertwined), and a remote daemon (necessary, because there's no other way to communicate to a remote machine).
One idea would be to have the Python code fork off from the main program and communicate over a pipe. With this, there is no daemon management problem.
+----------------+ | Program | | - - - - - - - -| | libvirt | +----------------+ | | fork(2) and pipe(2) | +-------------+ | Python code | +-------------+
But I'm not the person who ultimately decides what would be acceptable in libvirt. You'd need to get general agreement on the best way to do this.
The OpenVZ utils code, as I previously mentioned is GPL and a bigger issue is that the code is simple and straight forward. So, even if I wanted to add OpenVZ support into Libvirt, it ends up looking very close to the original GPL code. If the other Libvirt maintainers don't have an issue with what Richard is suggesting, then I can go ahead and do it. I have the freedom to work on this full time now. I can either write a Python helper(since I have to deal with a lot of text), or write C code that will be well integrated into Libvirt. Either way, it is going to be popen()ing the OpenVZ utilities. Which of this is better? Thanks, -- Shuveb Hussain. When you lose, be patient. When you achieve, be even more patient. EasyVZ: http://easyvz.sourceforge.net Blog: http://binarykarma.org
On Thu, Jul 05, 2007 at 12:28:40PM +0530, Shuveb Hussain wrote:
Daemon though? Can't the libvirt code just invoke the OpenVZ commands directly?
Yeah, that is very much possible. But if it were a daemon, it could be a Python daemon, since text processing is much simpler and I don't have to use the str*( ) funcs in C. What is your comment?
I'm a big advocate of using a sane language instead of C, but libvirt is written in C for better or worse.
The problem with a separate daemon is management of that daemon. Something needs to start it and stop it, it needs to have sockets with the right permissions and so on. What happens if the process runs and can't find the daemon? What is the path to the socket? etc. etc. At the moment we have two daemons - the qemu daemon (which has two separate functions intertwined), and a remote daemon (necessary, because there's no other way to communicate to a remote machine).
One idea would be to have the Python code fork off from the main program and communicate over a pipe. With this, there is no daemon management problem.
+----------------+ | Program | | - - - - - - - -| | libvirt | +----------------+ | | fork(2) and pipe(2) | +-------------+ | Python code | +-------------+
But I'm not the person who ultimately decides what would be acceptable in libvirt. You'd need to get general agreement on the best way to do this.
The OpenVZ utils code, as I previously mentioned is GPL and a bigger issue is that the code is simple and straight forward. So, even if I wanted to add OpenVZ support into Libvirt, it ends up looking very close to the original GPL code. If the other Libvirt maintainers don't have an issue with what Richard is suggesting, then I can go ahead and do it. I have the freedom to work on this full time now. I can either
Excellent! No issue, go ahead :-)
write a Python helper(since I have to deal with a lot of text), or write C code that will be well integrated into Libvirt. Either way, it is going to be popen()ing the OpenVZ utilities. Which of this is better?
Oh at the C level, definitely ! The python layer is really only a language binding wrapper and should remain that way ! Even if we fork as a new process the python side of libvirt is packaged separately, and really the core should be written in C, this would also allow to merge back your code in the existing daemon code if needed. I think that forking and using a pipe is not ideal, but that's a good first step, and eventually we may find a way to improve this if more people start to use it. Look at how many driver we wrote for Xen and how all this has evolved over time ! For existing interprocess fork/pipe communications see the libvirt xen proxy code, src/proxy_internal.c and src/proxy_internal.h is the part on the libvirt side and proxy/libvirt_proxy.c for the forked part. Your design is likely to be very very similar, and you can probably reuse all that code nearly directly for the communication with your daemon, the code is done and debugged and that should be a good first step to bootstrap your development (something like openvz_internal.[ch] in src and an openvzd/ subdirectory would fit well with existing infrastructure). I don't know how OpenVZ works, and if a single shared daemon would be okay or if it is better to create one daemon per OpenVZ instance, probably the last is the best but it really depends on how the tools work. I think the key is to try to share your design and development as you progress (release early, release often motto :-) to gain traction and help. Don't hesistate to ask, I'm also fine with commiting code which may not work yet and is not complete, best is to make it optional in configure.in and off by default. In a nutshell, go ahead, and please let us help you as you progress :-) thanks ! Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Hi Daniel,
Oh at the C level, definitely ! The python layer is really only a language binding wrapper and should remain that way ! Even if we fork as a new process the python side of libvirt is packaged separately, and really the core should be written in C, this would also allow to merge back your code in the existing daemon code if needed.
I understand the Python bindings part. I was wondering if the daemon itself could be written in Python, since I have to deal with a lot of text parsing from the OpenVZ utils. The LibVirt part that will talk to the daemon over the proxy/pipe code will anyways have to be C. Regards, -- Shuveb Hussain. When you lose, be patient. When you achieve, be even more patient. EasyVZ: http://easyvz.sourceforge.net Blog: http://binarykarma.org
On Thu, Jul 05, 2007 at 02:06:08PM +0530, Shuveb Hussain wrote:
Hi Daniel,
Oh at the C level, definitely ! The python layer is really only a language binding wrapper and should remain that way ! Even if we fork as a new process the python side of libvirt is packaged separately, and really the core should be written in C, this would also allow to merge back your code in the existing daemon code if needed.
I understand the Python bindings part. I was wondering if the daemon itself could be written in Python, since I have to deal with a lot of text parsing from the OpenVZ utils. The LibVirt part that will talk to the daemon over the proxy/pipe code will anyways have to be C.
I would really prefer C so we can merge daemon code if needed. I know a bit about textual parsing, I'm not afraid by this, in my experience doing proper pipe protocol is way harder and we have that working which you can reuse nearly as is. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
I would really prefer C so we can merge daemon code if needed. I know a bit about textual parsing, I'm not afraid by this, in my experience doing proper pipe protocol is way harder and we have that working which you can reuse nearly as is.
OK Daniel. I'm taking in your suggestions and starting work. Thanks, -- Shuveb Hussain. When you lose, be patient. When you achieve, be even more patient. EasyVZ: http://easyvz.sourceforge.net Blog: http://binarykarma.org
On Thu, Jul 05, 2007 at 02:32:27PM +0530, Shuveb Hussain wrote:
I would really prefer C so we can merge daemon code if needed. I know a bit about textual parsing, I'm not afraid by this, in my experience doing proper pipe protocol is way harder and we have that working which you can reuse nearly as is.
OK Daniel. I'm taking in your suggestions and starting work.
I can give an hand for parsing, I assume it's the textual output from the openvz control tools, key part is usually to try to extract some common structure in the text and do subroutines for each kind of objects emited. If needed send me samples of them and I can try to do part of this independantly. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Shuveb Hussain wrote:
Hi Daniel,
Oh at the C level, definitely ! The python layer is really only a language binding wrapper and should remain that way ! Even if we fork as a new process the python side of libvirt is packaged separately, and really the core should be written in C, this would also allow to merge back your code in the existing daemon code if needed.
I understand the Python bindings part. I was wondering if the daemon itself could be written in Python, since I have to deal with a lot of text parsing from the OpenVZ utils. The LibVirt part that will talk to the daemon over the proxy/pipe code will anyways have to be C.
As I said:
I'm a big advocate of using a sane language instead of C, but libvirt is written in C for better or worse.
Can you give an example of the kind of text processing which you need to do? I assume that this is just parsing the output of the OpenVZ command line utilities, or are there other things that need to be parsed? We can definitely help in this area. Writing a separate Python daemon has several problems: (1) It introduces a dependency on Python for base libvirt.[*] (2) You have to have a protocol to talk over the pipe, and it's not clear what that would be. (3) You have to create, manage and tear down the pipe and subprocess. Error handling in particular is complicated. What happens if the subprocess goes away suddenly? Or if you can't fork? (4) Pipes / subprocesses / Python don't work well on Windows, not that I personally care much about that, but we shouldn't exclude the possibility that people will wish to run libvirt on Windows. Rich. [*] Although currently the Python bindings are distributed with libvirt, I'd like to see them broken out as a separate package. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
participants (3)
-
Daniel Veillard -
Richard W.M. Jones -
Shuveb Hussain