[Libvir] support for hvm guests
Daniel, FYI, I'm working on a patch to support hvm guests. Attached is the work thus far. Calling "virsh dumpxml some_hvm_guest" will (almost) correctly dump xml config info on hvm guests. Needs to handle "type=ioemu" component of VIF specification. Currently in the XML there is a type attribute for the interface element that specifies bridge, route, etc. How would you like to structure the interface element and its children to account for hvm guests? Using "virsh create file" where file contains XML config for hvm guests may be close to working but I haven't tested it yet. There are several options that need to be added to the XML to support hvm guests. If we can agree on the structure I can add the functionality. Regards, Jim
On Fri, Apr 07, 2006 at 04:10:36PM -0600, Jim Fehlig wrote:
Daniel,
FYI, I'm working on a patch to support hvm guests.
Okay, cool. Note that I have no such hardware handy (but could get remote access to those) so that's a kind a scenario I will have some troubles running regression tests against in the near future.
Attached is the work thus far.
I see, also includes the end of life events patch now in CVS.
Calling "virsh dumpxml some_hvm_guest" will (almost) correctly dump xml config info on hvm guests. Needs to handle "type=ioemu" component of VIF specification. Currently in the XML there is a type attribute for the interface element that specifies bridge, route, etc. How would you like to structure the interface element and its children to account for hvm guests?
Well <interface type='iommu'> and then as sub elements the extra informations needed, sounds the closer to the existing format. But is that making sense from a logical point of view ?
Using "virsh create file" where file contains XML config for hvm guests may be close to working but I haven't tested it yet. There are several options that need to be added to the XML to support hvm guests. If we can agree on the structure I can add the functionality.
My two main goals when extending the XML format are: - try to keep it logical and as generic as possible - keep it fully documented second point should be easier if first point is followed :-) So what else is needed than the IOMMU network interfaces for hvm guests ? Also since I can't really test it right now, please feel in charge of driving this :-) Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel Veillard wrote:
On Fri, Apr 07, 2006 at 04:10:36PM -0600, Jim Fehlig wrote:
Daniel,
FYI, I'm working on a patch to support hvm guests.
[snip]
Calling "virsh dumpxml some_hvm_guest" will (almost) correctly dump xml config info on hvm guests. Needs to handle "type=ioemu" component of VIF specification. Currently in the XML there is a type attribute for the interface element that specifies bridge, route, etc. How would you like to structure the interface element and its children to account for hvm guests?
Well <interface type='iommu'> and then as sub elements the extra informations needed, sounds the closer to the existing format. But is that making sense from a logical point of view ?
Type 'bridge' can be valid for hvm guests. Perhaps it is better to not expose ioemu in the XML for now as there will probably be changes in this area anyway. We know the domain is type hvm, so just add 'ioemu:' where necessary when talking to xend/xenstore. I'm currently thinking about the following XML for OS element when type is hvm: <os> <type>hvm</type> <kernel>/usr/lib/xen/boot/hvmloader</kernel> <device_model>/usr/lib/xen/bin/qemu-dm</device_model> <boot dev='c'> <graphics type='vnc'> <cdrom dev='/dev/hdd'> </os> boot, graphics, and cdrom elements are optional. - boot: The device from which the guest should boot. If not specified the guest will be booted from hard disk. 'a' = floppy, 'c' = hard disk, 'd' = cdrom - graphics: Graphics model to use for the guest. Valid type attribute values are 'vnc' and 'sdl'. If not specified the guest will be launched with sdl. - cdrom: The cdrom device, e.g. '/dev/hdd', '/path/to/iso/plan9.iso' The patch I'm currently working on assumes this XML format. What do you think? Regards, Jim
On Fri, Apr 28, 2006 at 06:07:43PM -0600, Jim Fehlig wrote:
boot, graphics, and cdrom elements are optional. - boot: The device from which the guest should boot. If not specified the guest will be booted from hard disk. 'a' = floppy, 'c' = hard disk, 'd' = cdrom
Please, use full names (floppy | hdd | cdrom) instead strange M$ abbreviation. Karel -- Karel Zak <kzak@redhat.com>
First sorry for being a bit late answering, On Fri, Apr 28, 2006 at 06:07:43PM -0600, Jim Fehlig wrote:
Type 'bridge' can be valid for hvm guests. Perhaps it is better to not expose ioemu in the XML for now as there will probably be changes in this area anyway. We know the domain is type hvm, so just add 'ioemu:' where necessary when talking to xend/xenstore. I'm currently thinking about the following XML for OS element when type is hvm:
<os> <type>hvm</type> <kernel>/usr/lib/xen/boot/hvmloader</kernel> <device_model>/usr/lib/xen/bin/qemu-dm</device_model> <boot dev='c'> <graphics type='vnc'> <cdrom dev='/dev/hdd'> </os>
Hum, I agree with Karel on the use of <boot dev='c' />, that's not really a logical naming, can we avoid that and use more strcutured names like '/dev/hda' . Also I don't understand why graphics and cdrom are not defined in the <devices> section for example <devices> <disk type='block'> <source dev='hdd'/> <target dev='hdd'/> <cdrom/> </disk> <graphic type='vnc'> </graphic> </devices>
boot, graphics, and cdrom elements are optional. - boot: The device from which the guest should boot. If not specified the guest will be booted from hard disk. 'a' = floppy, 'c' = hard disk, 'd' = cdrom
not fond of that
- graphics: Graphics model to use for the guest. Valid type attribute values are 'vnc' and 'sdl'. If not specified the guest will be launched with sdl.
use a graphic element in the devices section instead, that way it stay structured. Also be make code which may allow to pass extra informations in the future as subelements, for example the vnc device may define authentication mechanism, and default resolution/depth may also be passed down in the future
- cdrom: The cdrom device, e.g. '/dev/hdd', '/path/to/iso/plan9.iso'
Well should be a disk device too, with an extra <cdrom/> subelement, and a source and a target as usual.
The patch I'm currently working on assumes this XML format. What do you think?
that I would prefer thing to stay more in line with existing format details from http://libvirt.org/format.html, it's just semantic sugar but design error there are usually a burden in the long term, thanks ! Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel Veillard wrote:
First sorry for being a bit late answering,
On Fri, Apr 28, 2006 at 06:07:43PM -0600, Jim Fehlig wrote:
Type 'bridge' can be valid for hvm guests. Perhaps it is better to not expose ioemu in the XML for now as there will probably be changes in this area anyway. We know the domain is type hvm, so just add 'ioemu:' where necessary when talking to xend/xenstore. I'm currently thinking about the following XML for OS element when type is hvm:
<os> <type>hvm</type> <kernel>/usr/lib/xen/boot/hvmloader</kernel> <device_model>/usr/lib/xen/bin/qemu-dm</device_model> <boot dev='c'> <graphics type='vnc'> <cdrom dev='/dev/hdd'> </os>
Hum, I agree with Karel on the use of <boot dev='c' />, that's not really a logical naming, can we avoid that and use more strcutured names like '/dev/hda' . Also I don't understand why graphics and cdrom are not defined in the <devices> section for example
<devices> <disk type='block'> <source dev='hdd'/> <target dev='hdd'/> <cdrom/> </disk> <graphic type='vnc'> </graphic>
</devices>
boot, graphics, and cdrom elements are optional. - boot: The device from which the guest should boot. If not specified the guest will be booted from hard disk. 'a' = floppy, 'c' = hard disk, 'd' = cdrom
not fond of that
- graphics: Graphics model to use for the guest. Valid type attribute values are 'vnc' and 'sdl'. If not specified the guest will be launched with sdl.
use a graphic element in the devices section instead, that way it stay structured. Also be make code which may allow to pass extra informations in the future as subelements, for example the vnc device may define authentication mechanism, and default resolution/depth may also be passed down in the future
- cdrom: The cdrom device, e.g. '/dev/hdd', '/path/to/iso/plan9.iso'
Well should be a disk device too, with an extra <cdrom/> subelement, and a source and a target as usual.
The patch I'm currently working on assumes this XML format. What do you think?
that I would prefer thing to stay more in line with existing format details from http://libvirt.org/format.html, it's just semantic sugar but design error there are usually a burden in the long term,
OK, these are great comments that I will incorporate into the patch. Currently I can get info on hvm domains but unable to create them. The create request is sent to xend but then blocks reading from xend in xend_req(). No problem creating the domain using xm. I have looked at the config coming into xend and it is identical between xm and virsh/libvirt. Need to poke around xend and see why there is no response to the virsh/libvirt create request for hvm domains. Thanks for the comments. Jim
On Wed, May 03, 2006 at 09:43:36AM -0600, Jim Fehlig wrote:
OK, these are great comments that I will incorporate into the patch.
Okay thanks :-)
Currently I can get info on hvm domains but unable to create them. The create request is sent to xend but then blocks reading from xend in xend_req(). No problem creating the domain using xm. I have looked at the config coming into xend and it is identical between xm and virsh/libvirt. Need to poke around xend and see why there is no response to the virsh/libvirt create request for hvm domains.
Hum, basically to debug this kind of things I spend my time in /var/log/xend.log especially since it prints what it received at creation time, you can at least check the informations coming from libvirt and from xm look alike. Unfortunately I won't be able to help you, first because I'm about to take vacations for 3 weeks :-), second because I don't have one of those new Intel CPUs, but I am sure others will give an hand if needed. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel Veillard wrote:
On Wed, May 03, 2006 at 09:43:36AM -0600, Jim Fehlig wrote:
Currently I can get info on hvm domains but unable to create them. The create request is sent to xend but then blocks reading from xend in xend_req(). No problem creating the domain using xm. I have looked at the config coming into xend and it is identical between xm and virsh/libvirt. Need to poke around xend and see why there is no response to the virsh/libvirt create request for hvm domains.
Hum, basically to debug this kind of things I spend my time in /var/log/xend.log especially since it prints what it received at creation time, you can at least check the informations coming from libvirt and from xm look alike. Unfortunately I won't be able to help you, first because I'm about to take vacations for 3 weeks :-), second because I don't have one of those new Intel CPUs, but I am sure others will give an hand if needed.
Well, I have found the problem but not a fix :-). When posting the create op to xend, the libvirt code currently waits for a response from xend (xend_req() is called after posting in xend_post()). For hvm guests, xend does not respond and libvirt blocks indefinitely on read. If I skip over the call to xend_req() for the create operation, wait_for_devices() and subsequently unpause() are called and the hvm guest is launched. This "fix" is just an experimental hack and an appropriate fix would be to have xend respond to the create op for hvm guests. Being unfamiliar with the xend code, perhaps someone (Anthony :-)) could direct me to the appropriate module in xend? Recall that all of this works for paravirtualized guests so it may not be hard to find even for someone that has just dabbled in the code. Regards, Jim
Jim Fehlig wrote:
Daniel Veillard wrote:
On Wed, May 03, 2006 at 09:43:36AM -0600, Jim Fehlig wrote:
Currently I can get info on hvm domains but unable to create them. The create request is sent to xend but then blocks reading from xend in xend_req(). No problem creating the domain using xm. I have looked at the config coming into xend and it is identical between xm and virsh/libvirt. Need to poke around xend and see why there is no response to the virsh/libvirt create request for hvm domains.
Hum, basically to debug this kind of things I spend my time in /var/log/xend.log especially since it prints what it received at creation time, you can at least check the informations coming from libvirt and from xm look alike. Unfortunately I won't be able to help you, first because I'm about to take vacations for 3 weeks :-), second because I don't have one of those new Intel CPUs, but I am sure others will give an hand if needed.
Well, I have found the problem but not a fix :-). When posting the create op to xend, the libvirt code currently waits for a response from xend (xend_req() is called after posting in xend_post()). For hvm guests, xend does not respond and libvirt blocks indefinitely on read. If I skip over the call to xend_req() for the create operation, wait_for_devices() and subsequently unpause() are called and the hvm guest is launched.
What do you mean by Xend does not respond? The path is a normal domain_create(). The path within Xend is server/XMLRPCServer.py but that's a very thin wrapper around a direct call to XendDomain.py:domain_create(). domain_create() should always return. If it's not, there's a bug in Xend I think. Can you give me a very concrete test case for this? The exact parameters being passed to domain_create() (it should just be an S-Expression config). Regards, Anthony Liguori
This "fix" is just an experimental hack and an appropriate fix would be to have xend respond to the create op for hvm guests. Being unfamiliar with the xend code, perhaps someone (Anthony :-)) could direct me to the appropriate module in xend? Recall that all of this works for paravirtualized guests so it may not be hard to find even for someone that has just dabbled in the code.
Regards, Jim
Anthony Liguori wrote:
Jim Fehlig wrote:
Daniel Veillard wrote:
On Wed, May 03, 2006 at 09:43:36AM -0600, Jim Fehlig wrote:
Currently I can get info on hvm domains but unable to create them. The create request is sent to xend but then blocks reading from xend in xend_req(). No problem creating the domain using xm. I have looked at the config coming into xend and it is identical between xm and virsh/libvirt. Need to poke around xend and see why there is no response to the virsh/libvirt create request for hvm domains.
Hum, basically to debug this kind of things I spend my time in /var/log/xend.log especially since it prints what it received at creation time, you can at least check the informations coming from libvirt and from xm look alike. Unfortunately I won't be able to help you, first because I'm about to take vacations for 3 weeks :-), second because I don't have one of those new Intel CPUs, but I am sure others will give an hand if needed.
Well, I have found the problem but not a fix :-). When posting the create op to xend, the libvirt code currently waits for a response from xend (xend_req() is called after posting in xend_post()). For hvm guests, xend does not respond and libvirt blocks indefinitely on read. If I skip over the call to xend_req() for the create operation, wait_for_devices() and subsequently unpause() are called and the hvm guest is launched.
What do you mean by Xend does not respond?
OK, let me try this again after some experimentation today. First, I'm using CVS libvirt and it does not use xmlrpc but the "old" S-expression over http - so I take a different code path to domain_create(). I set a breakpoint (pdb.set_trace()) in domain_create() and attempted to launch the hvm guest with xm and virsh/libvirt. The stack trace from xm is (Pdb) bt /usr/lib/python2.4/threading.py(442)__bootstrap() -> self.run() /usr/lib/python2.4/threading.py(422)run() -> self.__target(*self.__args, **self.__kwargs) /usr/lib/python2.4/SocketServer.py(463)process_request_thread() -> self.finish_request(request, client_address) /usr/lib/python2.4/SocketServer.py(254)finish_request() -> self.RequestHandlerClass(request, client_address, self) /usr/lib/python2.4/SocketServer.py(521)__init__() -> self.handle() /usr/lib/python2.4/BaseHTTPServer.py(316)handle() -> self.handle_one_request() /usr/lib/python2.4/BaseHTTPServer.py(310)handle_one_request() -> method() /usr/lib/python2.4/SimpleXMLRPCServer.py(432)do_POST() -> response = self.server._marshaled_dispatch( /usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py(103)_marshaled_dispatch() -> response = self._dispatch(method, params) /usr/lib/python2.4/SimpleXMLRPCServer.py(406)_dispatch() -> return func(*params) /usr/lib/python2.4/site-packages/xen/xend/server/XMLRPCServer.py(65)domain_create() -> info = XendDomain.instance().domain_create(config)
/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py(227)domain_create() -> self.domains_lock.acquire() (Pdb) and the stack trace using virsh/libvirt: (Pdb) bt /usr/lib/python2.4/threading.py(442)__bootstrap() -> self.run() /usr/lib/python2.4/threading.py(422)run() -> self.__target(*self.__args, **self.__kwargs) /usr/lib/python2.4/site-packages/xen/web/httpserver.py(288)run() -> self.processRequest(sock, addr) /usr/lib/python2.4/site-packages/xen/web/httpserver.py(316)processRequest() -> rp.process() /usr/lib/python2.4/site-packages/xen/web/httpserver.py(110)process() -> res = req.process() /usr/lib/python2.4/site-packages/xen/web/httpserver.py(140)process() -> return self.render(resource) /usr/lib/python2.4/site-packages/xen/web/httpserver.py(180)render() -> val = resource.render(self) /usr/lib/python2.4/site-packages/xen/web/resource.py(59)render() -> return meth(req) /usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py(135)render_POST() -> return self.perform(req) /usr/lib/python2.4/site-packages/xen/web/SrvBase.py(85)perform() -> return op_method(op, req) /usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py(77)op_create() -> dominfo = self.xd.domain_create(config)
/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py(227)domain_create() -> self.domains_lock.acquire() (Pdb) Continuing in the debugger I find that all routines complete up to process() in httpserver.py line 110 (note that my line numbers may be skewed a little due to insertion of debug code). After executing the statement "res = req.process()", I take the else clause of "if isinstance(res, ThreadRequest):" - which calls req.finish() which in turn closes the connection. So no response is sent to libvirt, which is blocked waiting to read something from xend. As mentioned before, if I ignore reading a response from xend on the create op and go directly to wait_for_devices and unpause then the guest is launched as expected. BTW, I do get responses from xend on wait_for_devices and unpause ops.
The path is a normal domain_create(). The path within Xend is server/XMLRPCServer.py but that's a very thin wrapper around a direct call to XendDomain.py:domain_create(). domain_create() should always return. If it's not, there's a bug in Xend I think.
Not in the case of libvirt - see above.
Can you give me a very concrete test case for this? The exact parameters being passed to domain_create() (it should just be an S-Expression config).
Sent in a previous post. I will look into this more tomorrow - have other fires that need attention now. Thanks for the help :-). Jim
Jim Fehlig wrote:
Anthony Liguori wrote:
[snip]
Continuing in the debugger I find that all routines complete up to process() in httpserver.py line 110 (note that my line numbers may be skewed a little due to insertion of debug code). After executing the statement "res = req.process()", I take the else clause of "if isinstance(res, ThreadRequest):" - which calls req.finish() which in turn closes the connection. So no response is sent to libvirt, which is blocked waiting to read something from xend.
As mentioned before, if I ignore reading a response from xend on the create op and go directly to wait_for_devices and unpause then the guest is launched as expected. BTW, I do get responses from xend on wait_for_devices and unpause ops.
The path is a normal domain_create(). The path within Xend is server/XMLRPCServer.py but that's a very thin wrapper around a direct call to XendDomain.py:domain_create(). domain_create() should always return. If it's not, there's a bug in Xend I think.
Not in the case of libvirt - see above.
Can you give me a very concrete test case for this? The exact parameters being passed to domain_create() (it should just be an S-Expression config).
Sent in a previous post.
I will look into this more tomorrow - have other fires that need attention now.
Tomorrow ended up being several days later :-). I looked at this again today and must say that this is a frustrating bug - feeling like some CS101 student. So I was wrong in stating that xend does not respond to the create op issued by libvirt. I get the expected response from xend but the connection is not closed upon completing the request, thus I set in some read loop in libvirt waiting for eof. I stepped through the request processing code in xend for create op for both paravirtualized and hvm guests and all seems fine, i.e. request is processed, finished, socket closed, etc. in httpserver.py. Looking at a packet trace shows connection is terminated for paravirt case (which breaks out of the read loop in libvirt) but not so for hvm case. Don't think I want to waste anymore time with this bug since the http / sexp interface is going away - which brings me to a question. Why is libvirt still using this interface and not xmlrpc? Perhaps my time would be better spent moving libvirt to this interface. I would like to submit the hvm patch for libvirt but can't reliably test it at this point. Regards, Jim
On Tue, May 09, 2006 at 05:57:52PM -0600, Jim Fehlig wrote:
closed, etc. in httpserver.py. Looking at a packet trace shows connection is terminated for paravirt case (which breaks out of the read loop in libvirt) but not so for hvm case.
very strange indeed ...
Don't think I want to waste anymore time with this bug since the http / sexp interface is going away - which brings me to a question. Why is libvirt still using this interface and not xmlrpc? Perhaps my time
Basically it took longuer than expect to get 3.0.2 in the fedora environment and I didn't started to work on it before leaving. Sorry :-\
would be better spent moving libvirt to this interface. I would like to submit the hvm patch for libvirt but can't reliably test it at this point.
Hum, there are other areas we didn't completely test in the code, that should not prevent it from posting it. Maybe it's not worth commiting to CVS right now, but having the code on the archived list meansthis can be looked at by more people :-) Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Jim Fehlig wrote:
Tomorrow ended up being several days later :-). I looked at this again today and must say that this is a frustrating bug - feeling like some CS101 student.
So I was wrong in stating that xend does not respond to the create op issued by libvirt. I get the expected response from xend but the connection is not closed upon completing the request, thus I set in some read loop in libvirt waiting for eof. I stepped through the request processing code in xend for create op for both paravirtualized and hvm guests and all seems fine, i.e. request is processed, finished, socket closed, etc. in httpserver.py. Looking at a packet trace shows connection is terminated for paravirt case (which breaks out of the read loop in libvirt) but not so for hvm case.
The proper way to do the S-Expression RPC (or XML-RPC for that matter) is to look at the Content-Length header in the response and read back that much data (close()'ing the connection once you've gotten that amount). We shouldn't wait for EOF to occur to stop reading. There was a patch recently that changed the read() behavior to read in larger amounts. It appears that when this was added, the behavior was changed to expect EOF. Sorry, I should have looked at that patch more carefully. If you revert xend_post to the older code, things should go back to working.
Don't think I want to waste anymore time with this bug since the http / sexp interface is going away - which brings me to a question. Why is libvirt still using this interface and not xmlrpc? Perhaps my time would be better spent moving libvirt to this interface. I would like to submit the hvm patch for libvirt but can't reliably test it at this point.
wrt XML-RPC, I think I'm to blame for that. I've not gotten around to modifying the internals of libvirt to switch between XML-RPC/S-Expression. I'd suggest just reverting the xend_post to the older code so you can keep making progress. Regards, Anthony Liguori
Regards, Jim
On Wed, May 10, 2006 at 11:53:02AM -0500, Anthony Liguori wrote:
Don't think I want to waste anymore time with this bug since the http / sexp interface is going away - which brings me to a question. Why is libvirt still using this interface and not xmlrpc? Perhaps my time would be better spent moving libvirt to this interface. I would like to submit the hvm patch for libvirt but can't reliably test it at this point.
wrt XML-RPC, I think I'm to blame for that. I've not gotten around to modifying the internals of libvirt to switch between XML-RPC/S-Expression. I'd suggest just reverting the xend_post to the older code so you can keep making progress.
Note, I've commited some XMP-RPC tests and minor xml/xmlrpc code changes to libvirt. It seems that Anthony's xmlrpc code works fine against Python SimpleXMLRPCServer. For more details see CVS. Karel -- Karel Zak <kzak@redhat.com>
Karel Zak wrote:
On Wed, May 10, 2006 at 11:53:02AM -0500, Anthony Liguori wrote:
Don't think I want to waste anymore time with this bug since the http / sexp interface is going away - which brings me to a question. Why is libvirt still using this interface and not xmlrpc? Perhaps my time would be better spent moving libvirt to this interface. I would like to submit the hvm patch for libvirt but can't reliably test it at this point.
wrt XML-RPC, I think I'm to blame for that. I've not gotten around to modifying the internals of libvirt to switch between XML-RPC/S-Expression. I'd suggest just reverting the xend_post to the older code so you can keep making progress.
Note, I've commited some XMP-RPC tests and minor xml/xmlrpc code changes to libvirt. It seems that Anthony's xmlrpc code works fine against Python SimpleXMLRPCServer. For more details see CVS.
As if there ever was any doubt ;-) I ran most of that code through valgrind and tested it against the Xend XML-RPC support. The remaining work should be just plumbing the libvir functions to use the XML-RPC functions. The hardest part is probably dealing with the S-Expressions since in XML-RPC they're going to be returned as an Array instead of a string which means someone needs to implement an XmlRpcArray -> struct sexpr conversion function. The other outstanding issue is that I used libxml2's nanoHttp interface which won't work for a domain socket. I think we may need to implement our own HTTP code (or just borrow the code in the S-Expression interface code). Regards, Anthony Liguori
Karel
On Wed, May 10, 2006 at 03:20:15PM -0500, Anthony Liguori wrote:
Karel Zak wrote:
On Wed, May 10, 2006 at 11:53:02AM -0500, Anthony Liguori wrote:
Don't think I want to waste anymore time with this bug since the http / sexp interface is going away - which brings me to a question. Why is libvirt still using this interface and not xmlrpc? Perhaps my time would be better spent moving libvirt to this interface. I would like to submit the hvm patch for libvirt but can't reliably test it at this point.
wrt XML-RPC, I think I'm to blame for that. I've not gotten around to modifying the internals of libvirt to switch between XML-RPC/S-Expression. I'd suggest just reverting the xend_post to the older code so you can keep making progress.
Note, I've commited some XMP-RPC tests and minor xml/xmlrpc code changes to libvirt. It seems that Anthony's xmlrpc code works fine against Python SimpleXMLRPCServer. For more details see CVS.
As if there ever was any doubt ;-)
No doubt about your code, but about my dirty hands that touch the code :-) I'd like to add maximum unit tests. It's good way how we can detect possible regressions in future changes.
I ran most of that code through valgrind and tested it against the Xend XML-RPC support. The remaining work should be just plumbing the libvir functions to use the XML-RPC functions. The hardest part is probably dealing with the S-Expressions since in XML-RPC they're going to be returned as an Array instead of a string which means someone needs to implement an XmlRpcArray -> struct sexpr conversion function.
The other outstanding issue is that I used libxml2's nanoHttp interface which won't work for a domain socket. I think we may need to implement our own HTTP code (or just borrow the code in the S-Expression interface code).
I think only one HTTP code in the library will be the best solution (= reuse the code in the S-Expression interface). Karel -- Karel Zak <kzak@redhat.com>
Anthony Liguori wrote:
Jim Fehlig wrote:
Tomorrow ended up being several days later :-). I looked at this again today and must say that this is a frustrating bug - feeling like some CS101 student.
So I was wrong in stating that xend does not respond to the create op issued by libvirt. I get the expected response from xend but the connection is not closed upon completing the request, thus I set in some read loop in libvirt waiting for eof. I stepped through the request processing code in xend for create op for both paravirtualized and hvm guests and all seems fine, i.e. request is processed, finished, socket closed, etc. in httpserver.py. Looking at a packet trace shows connection is terminated for paravirt case (which breaks out of the read loop in libvirt) but not so for hvm case.
The proper way to do the S-Expression RPC (or XML-RPC for that matter) is to look at the Content-Length header in the response and read back that much data (close()'ing the connection once you've gotten that amount). We shouldn't wait for EOF to occur to stop reading.
There was a patch recently that changed the read() behavior to read in larger amounts. It appears that when this was added, the behavior was changed to expect EOF. Sorry, I should have looked at that patch more carefully. If you revert xend_post to the older code, things should go back to working.
grr... Yes, this worked. So I'm able to launch hvm guests now but in the process of incorporating comments about the XML format for such guests I have come across another issue that could be handled in libvirt but perhaps a better place would be xend. The issue is structure of s-exp sent to xend. Currently xend appears to only accept things such as (vnc 1), (cdrom /dev/hdd) within the (image (hvm ...)) node. Well xend does accept something like "... (image (hvm ...(boot d))) (cdrom /dev/hdd) (vnc 1)...", but the resulting domain is not configured properly. With this config I get the following in xend.log: [2006-05-10 17:11:48 xend] DEBUG (image:267) args: cdrom, val: None This list has agreed that cdrom and graphics should be in the devices section of libvirt's XML format, e.g. <os> <type>hvm</type> ... <boot dev='hdd'> <os> <devices> <disk type='block'> <source dev='hdd'/> <target dev='hdd'/> <cdrom/> </disk> <graphics type='vnc'> </graphics> ... </devices> I can certainly add code in libvirt to handle converting this XML to the s-exp currently expected by xend but it makes for a larger patch. It almost seems better to have xend accept definition of cdrom and graphics outside of image node, e.g. (image (hvm ... (boot d) ) ) (cdrom /dev/hdd) (vnc 1) (other graphics-related options) Any thoughts before I go off and rework the XML -> s-exp code in libvirt? Regards, Jim
Jim Fehlig wrote:
Anthony Liguori wrote:
Jim Fehlig wrote:
Tomorrow ended up being several days later :-). I looked at this again today and must say that this is a frustrating bug - feeling like some CS101 student.
So I was wrong in stating that xend does not respond to the create op issued by libvirt. I get the expected response from xend but the connection is not closed upon completing the request, thus I set in some read loop in libvirt waiting for eof. I stepped through the request processing code in xend for create op for both paravirtualized and hvm guests and all seems fine, i.e. request is processed, finished, socket closed, etc. in httpserver.py. Looking at a packet trace shows connection is terminated for paravirt case (which breaks out of the read loop in libvirt) but not so for hvm case.
The proper way to do the S-Expression RPC (or XML-RPC for that matter) is to look at the Content-Length header in the response and read back that much data (close()'ing the connection once you've gotten that amount). We shouldn't wait for EOF to occur to stop reading.
There was a patch recently that changed the read() behavior to read in larger amounts. It appears that when this was added, the behavior was changed to expect EOF. Sorry, I should have looked at that patch more carefully. If you revert xend_post to the older code, things should go back to working.
grr... Yes, this worked. So I'm able to launch hvm guests now but in the process of incorporating comments about the XML format for such guests I have come across another issue that could be handled in libvirt but perhaps a better place would be xend. The issue is structure of s-exp sent to xend. Currently xend appears to only accept things such as (vnc 1), (cdrom /dev/hdd) within the (image (hvm ...)) node. Well xend does accept something like "... (image (hvm ...(boot d))) (cdrom /dev/hdd) (vnc 1)...", but the resulting domain is not configured properly. With this config I get the following in xend.log:
wrt the S-Expression format, that stuff is all changing in the very near future. I've been sitting on patches to remove the S-Expression stuff completely for a few weeks now. The thing that's kept me from submitting it is that I haven't gotten around to enabling the XML-RPC support in libvirt. As soon as we get XML-RPC support working in libvirt, we can start making some aggressive changes to the Xend side of the API. Regards, Anthony Liguori
[2006-05-10 17:11:48 xend] DEBUG (image:267) args: cdrom, val: None
This list has agreed that cdrom and graphics should be in the devices section of libvirt's XML format, e.g.
<os> <type>hvm</type> ... <boot dev='hdd'> <os> <devices> <disk type='block'> <source dev='hdd'/> <target dev='hdd'/> <cdrom/> </disk> <graphics type='vnc'> </graphics> ... </devices>
I can certainly add code in libvirt to handle converting this XML to the s-exp currently expected by xend but it makes for a larger patch. It almost seems better to have xend accept definition of cdrom and graphics outside of image node, e.g.
(image (hvm ... (boot d) ) ) (cdrom /dev/hdd) (vnc 1) (other graphics-related options)
Any thoughts before I go off and rework the XML -> s-exp code in libvirt?
Regards, Jim
Hi Jim, On Wed, May 10, 2006 at 06:25:07PM -0600, Jim Fehlig wrote:
This list has agreed that cdrom and graphics should be in the devices section of libvirt's XML format, e.g.
<os> <type>hvm</type> ... <boot dev='hdd'> <os> <devices> <disk type='block'> <source dev='hdd'/> <target dev='hdd'/> <cdrom/> </disk> <graphics type='vnc'> </graphics> ... </devices>
I can certainly add code in libvirt to handle converting this XML to the s-exp currently expected by xend but it makes for a larger patch. It almost seems better to have xend accept definition of cdrom and graphics outside of image node, e.g.
(image (hvm ... (boot d) ) ) (cdrom /dev/hdd) (vnc 1) (other graphics-related options)
Any thoughts before I go off and rework the XML -> s-exp code in libvirt?
Did you get any further dealing with this issue ? It'd be nice to get the support for HVM guests into libvirt in the near future, since I'm working on some higher level UI tools which could be improved with more detail about HVM guests. I could probably find time to do some coding on the patches if there are areas you've not currently got time to address yourself Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
Hi Jim,
[snip]
Did you get any further dealing with this issue ? It'd be nice to get the support for HVM guests into libvirt in the near future, since I'm working on some higher level UI tools which could be improved with more detail about HVM guests. I could probably find time to do some coding on the patches if there are areas you've not currently got time to address yourself
I put this patch on the back burner given Anthony's response in https://www.redhat.com/archives/libvir-list/2006-May/msg00017.html I recently considered continuing this work until we started the discussions on Xen Management API :-). To be honest, now I'm not sure what to do. I will be on vacation next week but probably have time to finish this patch before leaving. However this code will certainly be thrown out at some point. Guess the question is how far out is "some point"? Is it worth doing the xml -> sexp conversion and working around some of the inconveniences in xend discussed earlier in this thread? If we think the current state of affairs will exist for several more months then perhaps so. Thoughts? Jim
On Wed, Jun 28, 2006 at 10:59:20AM -0600, Jim Fehlig wrote:
Daniel P. Berrange wrote:
Hi Jim,
[snip]
Did you get any further dealing with this issue ? It'd be nice to get the support for HVM guests into libvirt in the near future, since I'm working on some higher level UI tools which could be improved with more detail about HVM guests. I could probably find time to do some coding on the patches if there are areas you've not currently got time to address yourself
I put this patch on the back burner given Anthony's response in https://www.redhat.com/archives/libvir-list/2006-May/msg00017.html
I recently considered continuing this work until we started the discussions on Xen Management API :-). To be honest, now I'm not sure what to do. I will be on vacation next week but probably have time to finish this patch before leaving. However this code will certainly be thrown out at some point. Guess the question is how far out is "some point"? Is it worth doing the xml -> sexp conversion and working around some of the inconveniences in xend discussed earlier in this thread? If we think the current state of affairs will exist for several more months then perhaps so.
Unfortunately I think we'll need to get an implementation done that works with SEXPR. Even if everything progresses smoothly with the new XML-RPC protocol for XenD, realistically it is going to be 2-3 months into the future before its anywhere near ready for use by libvirt, and even then there'll be a non-trivial existing userbase still on the SEXPR based XenD for quite a while after that. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
Unfortunately I think we'll need to get an implementation done that works with SEXPR. Even if everything progresses smoothly with the new XML-RPC protocol for XenD, realistically it is going to be 2-3 months into the future before its anywhere near ready for use by libvirt, and even then there'll be a non-trivial existing userbase still on the SEXPR based XenD for quite a while after that.
Here is a first cut of the patch for review. I have minimally tested the "get" side of the patch, e.g. virsh dumpxml <some_hvm_domain>. Have not tried the "put" side, e.g. virsh create. I hope to test/fix the create side tomorrow as well as test the get side with more interesting configurations. I haven't quite decided on how to handle the cdrom yet. I'm hardly and expert on libxml so perhaps someone can suggest how to elegantly extract the cdrom info from the incoming xml :-). See the TODO related to cdrom in xml.c. Regards, Jim
Hello, I am getting this error on trying to create a xen domain Unable to create domain . This is the code snippet. I am unable to figure out the root cause . Any help will be appreciated .. Thanks, Dakshina /* XML descriptor to create a new Domain */ const char *xmlDesc="<domain type='xen' id='18'
<name>newxen</name> <os><type>linux</type> <kernel> /boot/vmlinuz-2.6.12-xenU </kernel> <root>/dev/hda5</root><cmdline>3</cmdline></os><memory>65536</memory><vcpu>1</vcpu><devices><disk type='file'><source file='/copyfed.img'/> </disk></devices></domain>";
static void createNewDomain(){ virConnectPtr conn = NULL; /* the hypervisor connection */ virDomainPtr dom = NULL; /* the domain being checked */ virDomainInfo info; /* the informations being fetched */ virErrorPtr err; /* Ptr to the error structure*/ int errorCode; char xmlDesc[250]; unsigned int flags=0; /* NULL means connect to local Xen hypervisor */ conn = virConnectOpenReadOnly(NULL); if (conn == NULL) { fprintf(stderr, "Failed to connect to hypervisor\n"); goto error; } printf("Connected to the hypervisor successfully"); dom=virDomainCreateLinux(conn,xmlDesc,0); if (dom == NULL) { fprintf(stderr,"Unable to create a domain\n"); goto error; } dom = virDomainLookupByID(conn, 0); printf(" %s\n" ,virDomainGetXMLDesc(dom,0)); error: #if 0 err=virGetLastError(); printf("Error is %s\n" ,err->message); #endif errorCode=virConnCopyLastError(conn,err); printf("Error is %d\n" ,errorCode); if (dom != NULL) virDomainFree(dom); if (conn != NULL) virConnectClose(conn); } __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On Mon, Jul 03, 2006 at 02:06:30AM -0700, dasari dakshina wrote:
Hello, I am getting this error on trying to create a xen domain Unable to create domain . This is the code snippet. I am unable to figure out the root cause . Any help will be appreciated .. [...] conn = virConnectOpenReadOnly(NULL);
First you should not open a Read Only connection to do a domain creation this is obviously an operation with side effect. Second the best way to debug what's happening is check /var/log/xend.log . Xend gives back nearly no debug information when an operation fails, and libvirt really can't guess whatb went wrong, check what xend reported. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Hello Daniel, Thanks. I used the virConnectOpen() function. I did check for the log file entries .I deleted the log file ,stopped xend and created a new log file and ran xend .I dont find any help there. Is there any other mistake that I am making while using tht API / writing the XML description? Thanks, Dakshina --- Daniel Veillard <veillard@redhat.com> wrote:
On Mon, Jul 03, 2006 at 02:06:30AM -0700, dasari dakshina wrote:
Hello, I am getting this error on trying to create a xen domain Unable to create domain . This is the code snippet. I am unable to figure out the root cause . Any help will be appreciated .. [...] conn = virConnectOpenReadOnly(NULL);
First you should not open a Read Only connection to do a domain creation this is obviously an operation with side effect. Second the best way to debug what's happening is check /var/log/xend.log . Xend gives back nearly no debug information when an operation fails, and libvirt really can't guess whatb went wrong, check what xend reported.
Daniel
-- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On Mon, Jul 03, 2006 at 03:55:15AM -0700, dasari dakshina wrote:
Hello Daniel, Thanks. I used the virConnectOpen() function. I did check for the log file entries .I deleted the log file ,stopped xend and created a new log file and ran xend .
I dont find any help there.
What does this mean ? No log or you don't understand the logs or there is no error in the logs ?
Is there any other mistake that I am making while using tht API / writing the XML description?
I don't know. You should use virsh and see how it work. I would be very surprized if you only got that "domain create failed" error without any additional informations either from libvirt or from xend logs. This may happen. But there is way too much information I don't have, I can't reproduce your problem as is, so you must find more on your side. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
participants (6)
-
Anthony Liguori -
Daniel P. Berrange -
Daniel Veillard -
dasari dakshina -
Jim Fehlig -
Karel Zak