4:51 a.m.
Shuveb Hussain wrote:
>
> Daemon though? Can't the libvirt code just invoke the OpenVZ commands
> directly?
Yeah, that is very much possible. But if it were a daemon, it could be
a Python daemon, since text processing is much simpler and I don't
have to use the str*( ) funcs in C. What is your comment?
I'm a big advocate of using a sane language instead of C, but libvirt is
written in C for better or worse.
The problem with a separate daemon is management of that daemon.
Something needs to start it and stop it, it needs to have sockets with
the right permissions and so on. What happens if the process runs and
can't find the daemon? What is the path to the socket? etc. etc. At
the moment we have two daemons - the qemu daemon (which has two separate
functions intertwined), and a remote daemon (necessary, because there's
no other way to communicate to a remote machine).
One idea would be to have the Python code fork off from the main program
and communicate over a pipe. With this, there is no daemon management
problem.
+----------------+
| Program |
| - - - - - - - -|
| libvirt |
+----------------+
|
| fork(2) and pipe(2)
|
+-------------+
| Python code |
+-------------+
But I'm not the person who ultimately decides what would be acceptable
in libvirt. You'd need to get general agreement on the best way to do this.
Rich.
--
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in
England and Wales under Company Registration No. 03798903
1:58 a.m.
New subject: [Libvir] [RFC]OpenVZ XML def
Hello,
>> Daemon though? Can't the libvirt code just invoke the
OpenVZ commands
>> directly?
>
> Yeah, that is very much possible. But if it were a daemon, it could be
> a Python daemon, since text processing is much simpler and I don't
> have to use the str*( ) funcs in C. What is your comment?
I'm a big advocate of using a sane language instead of C, but libvirt is
written in C for better or worse.
The problem with a separate daemon is management of that daemon.
Something needs to start it and stop it, it needs to have sockets with
the right permissions and so on. What happens if the process runs and
can't find the daemon? What is the path to the socket? etc. etc. At
the moment we have two daemons - the qemu daemon (which has two separate
functions intertwined), and a remote daemon (necessary, because there's
no other way to communicate to a remote machine).
One idea would be to have the Python code fork off from the main program
and communicate over a pipe. With this, there is no daemon management
problem.
+----------------+
| Program |
| - - - - - - - -|
| libvirt |
+----------------+
|
| fork(2) and pipe(2)
|
+-------------+
| Python code |
+-------------+
But I'm not the person who ultimately decides what would be acceptable
in libvirt. You'd need to get general agreement on the best way to do this.
The OpenVZ utils code, as I previously mentioned is GPL and a bigger
issue is that the code is simple and straight forward. So, even if I
wanted to add OpenVZ support into Libvirt, it ends up looking very
close to the original GPL code. If the other Libvirt maintainers don't
have an issue with what Richard is suggesting, then I can go ahead and
do it. I have the freedom to work on this full time now. I can either
write a Python helper(since I have to deal with a lot of text), or
write C code that will be well integrated into Libvirt. Either way, it
is going to be popen()ing the OpenVZ utilities. Which of this is
better?
Thanks,
--
Shuveb Hussain.
When you lose, be patient. When you achieve, be even more patient.
EasyVZ: http://easyvz.sourceforge.net
Blog: http://binarykarma.org
2:58 a.m.
New subject: [Libvir] [RFC]OpenVZ XML def
On Thu, Jul 05, 2007 at 12:28:40PM +0530, Shuveb Hussain wrote:
>>> Daemon though? Can't the libvirt code just invoke
the OpenVZ commands
>>> directly?
>>
>> Yeah, that is very much possible. But if it were a daemon, it could be
>> a Python daemon, since text processing is much simpler and I don't
>> have to use the str*( ) funcs in C. What is your comment?
>
>I'm a big advocate of using a sane language instead of C, but libvirt is
>written in C for better or worse.
>
>The problem with a separate daemon is management of that daemon.
>Something needs to start it and stop it, it needs to have sockets with
>the right permissions and so on. What happens if the process runs and
>can't find the daemon? What is the path to the socket? etc. etc. At
>the moment we have two daemons - the qemu daemon (which has two separate
>functions intertwined), and a remote daemon (necessary, because there's
>no other way to communicate to a remote machine).
>
>One idea would be to have the Python code fork off from the main program
>and communicate over a pipe. With this, there is no daemon management
>problem.
>
> +----------------+
> | Program |
> | - - - - - - - -|
> | libvirt |
> +----------------+
> |
> | fork(2) and pipe(2)
> |
> +-------------+
> | Python code |
> +-------------+
>
>But I'm not the person who ultimately decides what would be acceptable
>in libvirt. You'd need to get general agreement on the best way to do
>this.
>
The OpenVZ utils code, as I previously mentioned is GPL and a bigger
issue is that the code is simple and straight forward. So, even if I
wanted to add OpenVZ support into Libvirt, it ends up looking very
close to the original GPL code. If the other Libvirt maintainers don't
have an issue with what Richard is suggesting, then I can go ahead and
do it. I have the freedom to work on this full time now. I can either
Excellent! No issue, go ahead :-)
write a Python helper(since I have to deal with a lot of text), or
write C code that will be well integrated into Libvirt. Either way, it
is going to be popen()ing the OpenVZ utilities. Which of this is
better?
Oh at the C level, definitely ! The python layer is really only a language
binding wrapper and should remain that way ! Even if we fork as a new process
the python side of libvirt is packaged separately, and really the core should
be written in C, this would also allow to merge back your code in the existing
daemon code if needed.
I think that forking and using a pipe is not ideal, but that's a good first
step, and eventually we may find a way to improve this if more people start
to use it. Look at how many driver we wrote for Xen and how all this has
evolved over time ! For existing interprocess fork/pipe communications
see the libvirt xen proxy code, src/proxy_internal.c and src/proxy_internal.h
is the part on the libvirt side and proxy/libvirt_proxy.c for the forked
part. Your design is likely to be very very similar, and you can probably
reuse all that code nearly directly for the communication with your daemon,
the code is done and debugged and that should be a good first step to
bootstrap your development (something like openvz_internal.[ch] in src
and an openvzd/ subdirectory would fit well with existing infrastructure).
I don't know how OpenVZ works, and if a single shared daemon would be
okay or if it is better to create one daemon per OpenVZ instance, probably
the last is the best but it really depends on how the tools work.
I think the key is to try to share your design and development as you progress
(release early, release often motto :-) to gain traction and help. Don't
hesistate to ask, I'm also fine with commiting code which may not work yet
and is not complete, best is to make it optional in configure.in and off
by default.
In a nutshell, go ahead, and please let us help you as you progress :-)
thanks !
Daniel
--
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard | virtualization library http://libvirt.org/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
3:36 a.m.
New subject: [Libvir] [RFC]OpenVZ XML def
Hi Daniel,
Oh at the C level, definitely ! The python layer is really only a
language
binding wrapper and should remain that way ! Even if we fork as a new process
the python side of libvirt is packaged separately, and really the core should
be written in C, this would also allow to merge back your code in the existing
daemon code if needed.
I understand the Python bindings part. I was wondering if the daemon
itself could be written in Python, since I have to deal with a lot of
text parsing from the OpenVZ utils. The LibVirt part that will talk to
the daemon over the proxy/pipe code will anyways have to be C.
Regards,
--
Shuveb Hussain.
When you lose, be patient. When you achieve, be even more patient.
EasyVZ: http://easyvz.sourceforge.net
Blog: http://binarykarma.org
3:58 a.m.
New subject: [Libvir] [RFC]OpenVZ XML def
On Thu, Jul 05, 2007 at 02:06:08PM +0530, Shuveb Hussain wrote:
Hi Daniel,
> Oh at the C level, definitely ! The python layer is really only a
> language
>binding wrapper and should remain that way ! Even if we fork as a new
>process
>the python side of libvirt is packaged separately, and really the core
>should
>be written in C, this would also allow to merge back your code in the
>existing
>daemon code if needed.
I understand the Python bindings part. I was wondering if the daemon
itself could be written in Python, since I have to deal with a lot of
text parsing from the OpenVZ utils. The LibVirt part that will talk to
the daemon over the proxy/pipe code will anyways have to be C.
I would really prefer C so we can merge daemon code if needed.
I know a bit about textual parsing, I'm not afraid by this, in my
experience doing proper pipe protocol is way harder and we have that
working which you can reuse nearly as is.
Daniel
--
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard | virtualization library http://libvirt.org/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
4:02 a.m.
New subject: [Libvir] [RFC]OpenVZ XML def
I would really prefer C so we can merge daemon code if needed.
I know a bit about textual parsing, I'm not afraid by this, in my
experience doing proper pipe protocol is way harder and we have that
working which you can reuse nearly as is.
OK Daniel. I'm taking in your suggestions and starting work.
Thanks,
--
Shuveb Hussain.
When you lose, be patient. When you achieve, be even more patient.
EasyVZ: http://easyvz.sourceforge.net
Blog: http://binarykarma.org
4:06 a.m.
New subject: [Libvir] [RFC]OpenVZ XML def
On Thu, Jul 05, 2007 at 02:32:27PM +0530, Shuveb Hussain wrote:
> I would really prefer C so we can merge daemon code if needed.
>I know a bit about textual parsing, I'm not afraid by this, in my
>experience doing proper pipe protocol is way harder and we have that
>working which you can reuse nearly as is.
OK Daniel. I'm taking in your suggestions and starting work.
I can give an hand for parsing, I assume it's the textual output
from the openvz control tools, key part is usually to try to extract
some common structure in the text and do subroutines for each kind
of objects emited. If needed send me samples of them and I can try
to do part of this independantly.
Daniel
--
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard | virtualization library http://libvirt.org/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
5:15 a.m.
New subject: [Libvir] [RFC]OpenVZ XML def
Shuveb Hussain wrote:
Hi Daniel,
> Oh at the C level, definitely ! The python layer is really only a
> language
> binding wrapper and should remain that way ! Even if we fork as a new
> process
> the python side of libvirt is packaged separately, and really the core
> should
> be written in C, this would also allow to merge back your code in the
> existing
> daemon code if needed.
I understand the Python bindings part. I was wondering if the daemon
itself could be written in Python, since I have to deal with a lot of
text parsing from the OpenVZ utils. The LibVirt part that will talk to
the daemon over the proxy/pipe code will anyways have to be C.
As I said:
I'm a big advocate of using a sane language instead of C, but
libvirt is written in C for better or worse.
Can you give an example of the kind of text processing which you need to do?
I assume that this is just parsing the output of the OpenVZ command line
utilities, or are there other things that need to be parsed? We can
definitely help in this area.
Writing a separate Python daemon has several problems:
(1) It introduces a dependency on Python for base libvirt.[*]
(2) You have to have a protocol to talk over the pipe, and it's not
clear what that would be.
(3) You have to create, manage and tear down the pipe and subprocess.
Error handling in particular is complicated. What happens if the
subprocess goes away suddenly? Or if you can't fork?
(4) Pipes / subprocesses / Python don't work well on Windows, not that I
personally care much about that, but we shouldn't exclude the
possibility that people will wish to run libvirt on Windows.
Rich.
[*] Although currently the Python bindings are distributed with libvirt,
I'd like to see them broken out as a separate package.
--
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in
England and Wales under Company Registration No. 03798903
6163
days inactive
6207
days old
7 comments
3 participants
participants (3)
-
Daniel Veillard -
Richard W.M. Jones -
Shuveb Hussain