[libvirt-users] with current libvirt git virsh expects vbox, refuses kvm define
Okay, just installed libvirt from today's git on two Ubuntu systems. The first one went fine (except for continuation of the problem that virsh and virt-manager cannot connect to 0.8.3 systems that's been there in the 0.9.x libvirts) ... but on the second, after install, "virsh list --all" presents nothing. And: # virsh define tu1004.xml error: Failed to define domain from tu1004.xml error: internal error unexpected domain type kvm, expecting vbox These are _exactly_ the same xml files as are still working on the first system. And in both cases the new libvirt was built with defaults. Long ago the second system did have VirtualBox on it, so that may have been found while it was setting up config, but that's no good reason for KVM to fail here, right? Must be a thing with the git. Ran the 0.9.3 release built on this system previously without this problem. Whit
于 2011年07月27日 08:04, Whit Blauvelt 写道:
Okay, just installed libvirt from today's git on two Ubuntu systems. The first one went fine (except for continuation of the problem that virsh and virt-manager cannot connect to 0.8.3 systems that's been there in the 0.9.x libvirts) ... but on the second, after install, "virsh list --all" presents nothing. And:
# virsh define tu1004.xml error: Failed to define domain from tu1004.xml error: internal error unexpected domain type kvm, expecting vbox
These are _exactly_ the same xml files as are still working on the first system. And in both cases the new libvirt was built with defaults. Long ago the second system did have VirtualBox on it, so that may have been found while it was setting up config, but that's no good reason for KVM to fail here, right?
Must be a thing with the git. Ran the 0.9.3 release built on this system previously without this problem.
Whit
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
Hi, Whit What's the output of "# virsh -V" on your second ubuntu box? I guess your libvirt on that box might not be compiled with qemu driver. Another possiablity is the libvirt is compiled with both qemu driver and vbox driver, but when you try to creat a new connection, vbox was the first succesfully connected one. In this case, you can try like below: # virsh -c qemu:///sytem list --all Or you can set the ENV variable VIRSH_DEFAULT_CONNECT_URI or LIBVIRT_DEFAULT_URI to the URI of the hypervisor you want to connect to. Osier
What's the output of "# virsh -V" on your second ubuntu box? I guess your libvirt on that box might not be compiled with qemu driver.
# virsh -V Virsh command line tool of libvirt 0.9.3 ... Compiled with support for: Hypervisors: QEmu/KVM UML OpenVZ VirtualBox LXC ESX Test Networking: Remote Daemon Network Bridging Nwfilter VirtualPort Storage: Dir Filesystem SCSI Multipath LVM Miscellaneous: SELinux Secrets Debug Readline
Another possiablity is the libvirt is compiled with both qemu driver and vbox driver, but when you try to creat a new connection, vbox was the first succesfully connected one. In this case, you can try like below:
Why? Ah, I do have a couple stray vbox processes somehow: root 25265 0.0 0.1 86076 4304 ? S 19:34 0:00 /usr/lib/virtualbox/VBoxXPCOMIPCD root 25274 0.0 0.1 209964 6672 ? Sl 19:34 0:02 /usr/lib/virtualbox/VBoxSVC --pipe 4 --auto-shutdown But that should cause it to deny it knows how to handle Qemu/KVM?
# virsh -c qemu:///sytem list --all
Okay, that one works.
Or you can set the ENV variable VIRSH_DEFAULT_CONNECT_URI or LIBVIRT_DEFAULT_URI to the URI of the hypervisor you want to connect to.
The first (at least) also works. Thanks. Whit
2011/7/27 Whit Blauvelt <whit.virt@transpect.com>:
What's the output of "# virsh -V" on your second ubuntu box? I guess your libvirt on that box might not be compiled with qemu driver.
# virsh -V Virsh command line tool of libvirt 0.9.3 ...
Compiled with support for: Hypervisors: QEmu/KVM UML OpenVZ VirtualBox LXC ESX Test Networking: Remote Daemon Network Bridging Nwfilter VirtualPort Storage: Dir Filesystem SCSI Multipath LVM Miscellaneous: SELinux Secrets Debug Readline
Another possiablity is the libvirt is compiled with both qemu driver and vbox driver, but when you try to creat a new connection, vbox was the first succesfully connected one. In this case, you can try like below:
Why? Ah, I do have a couple stray vbox processes somehow:
root 25265 0.0 0.1 86076 4304 ? S 19:34 0:00 /usr/lib/virtualbox/VBoxXPCOMIPCD root 25274 0.0 0.1 209964 6672 ? Sl 19:34 0:02 /usr/lib/virtualbox/VBoxSVC --pipe 4 --auto-shutdown
But that should cause it to deny it knows how to handle Qemu/KVM?
The point is that libvirt autodetects the available hypervisors at runtime when you don't specify a connection URI. For example, just running virsh results in autodetecting VirtualBox because you have it installed in a way that it's still working and due to the way libvirt works internally VirtualBox comes before QEMU in the autodetection list.
# virsh -c qemu:///sytem list --all
Okay, that one works.
Here you're telling libvirt to connect to QEMU explicitly, that's why it doesn't do autodetection here. The initial "error: internal error unexpected domain type kvm, expecting vbox" you saw was added recently to prevent incompatible driver/config combinations. In you're case it highlighted that autodetection didn't work for you as expected. -- Matthias Bolte http://photron.blogspot.com
On Wed, Jul 27, 2011 at 09:21:08AM +0200, Matthias Bolte wrote:
The point is that libvirt autodetects the available hypervisors at runtime when you don't specify a connection URI. For example, just running virsh results in autodetecting VirtualBox because you have it installed in a way that it's still working and due to the way libvirt works internally VirtualBox comes before QEMU in the autodetection list.
But shouldn't it autodetect _all_ available hypervisors and then, when an xml file specifies kvm - which is available - use it rather than complain it has a problem with it? You're describing the current behavior as if it's to be accepted. Clearly having libvirt recognize that the kvm hypervisor is available, and automatically using it when the xml file specifies it, does not take some extraordinary level of clever programming. "The xml specifies kvm. Do I have kvm? Why, yes I do!"
The initial "error: internal error unexpected domain type kvm, expecting vbox" you saw was added recently to prevent incompatible driver/config combinations. In you're case it highlighted that autodetection didn't work for you as expected.
Fine, so when it recognizes it has the wrong driver for the config, it should grab the right driver. No reason for it to complain to me about it. It should simply do its job. Right? This is a bug that it doesn't? Whit
2011/7/27 Whit Blauvelt <whit.virt@transpect.com>:
On Wed, Jul 27, 2011 at 09:21:08AM +0200, Matthias Bolte wrote:
The point is that libvirt autodetects the available hypervisors at runtime when you don't specify a connection URI. For example, just running virsh results in autodetecting VirtualBox because you have it installed in a way that it's still working and due to the way libvirt works internally VirtualBox comes before QEMU in the autodetection list.
But shouldn't it autodetect _all_ available hypervisors and then, when an xml file specifies kvm - which is available - use it rather than complain it has a problem with it? You're describing the current behavior as if it's to be accepted. Clearly having libvirt recognize that the kvm hypervisor is available, and automatically using it when the xml file specifies it, does not take some extraordinary level of clever programming. "The xml specifies kvm. Do I have kvm? Why, yes I do!"
The initial "error: internal error unexpected domain type kvm, expecting vbox" you saw was added recently to prevent incompatible driver/config combinations. In you're case it highlighted that autodetection didn't work for you as expected.
Fine, so when it recognizes it has the wrong driver for the config, it should grab the right driver. No reason for it to complain to me about it. It should simply do its job. Right? This is a bug that it doesn't?
Whit
That's not how libvirt works. The architecture is different than you seem to expect it. When you open a connection libvirt picks a driver for it. If you're using virsh you may not be aware of the connection as this is happening mostly implicit. If you tell libvirt which driver/hypervisor you want then it'll try to use that one. For example when you do virsh -c qemu:///system then libvirt will ask it's connection/hypervisor drivers until it find one that accepts this URI. In case of qemu:///system the QEMU driver will accept it in the end (after going through the remote driver, but that's transparent for the user). If you just run virsh then libvirt will again ask it's drivers one after another if they can accept an empty connection URI. In you case the VirtualBox driver accepted it because it found a local VirtualBox installation that it can use. Once the connection is opened and has a driver assigned then calls to the libvirt API are dispatched to the selected driver. So when you do a virsh define then this will call the implementation of the virDomainDefine function in the VirtualBox driver (because that one was autoselected). There is no driver selection in the API after the connection is opened. For libvirt to work as you describe it virsh define (actually virDomainDefine) needs to parse the XML to the the type and pick a driver based on that. That might work to some extend in the local case where qemu:///system could be deduced from type=kvm, but won't work in the general or remote case. For example a domain config with type vmware can be used with the VMware ESX and the VMware Workstation driver, so how to pick the driver here? Even when you solve this by using vmware-esx and vmware-workstation instead then you need to tell libvirt in the virDomainDefine call where the ESX server is and with what options and credentials you want open the connection. In the current libvirt architecture you do that when you open the connection via the connection URI instead. Your suggestion also implies that virsh list lists all domains on all available hypervisors. This requires name/ID/UUID uniqueness across multiple hypervisors, that gets hairy fast. Also you have the problem of specifying the connection details to virsh list, maybe even for multiple remote connections. So there is no bug in general here, as this is how libvirt is designed to work. One could argue that libvirt should be able to pick the right driver in virDomainDefine, but that is not as simple as it sounds. Also it doesn't fit to the way things currently work, so I don't see that happening anytime soon. -- Matthias Bolte http://photron.blogspot.com
That's not how libvirt works. The architecture is different than you seem to expect it.
Matthias, Even with all you say - and I thank you for your patient explanation - there remains a question: Why should 0.9.3+ git suddenly decide that the default driver should be vbox, when 0.8.3 and 0.9.2 and 0.9.3, all on the same system, and the second two similarly compiled with their defaults from the tar, went to kvm instead? There was no other change on the system. And the vbox processes I found appear to have been started by libvirt itself, since vbox hasn't been run on this system in over a year, during which time it's been rebooted without vbox being invoked at all. If kvm is on the system as an option - which it has been all along - it should be the libvirt default. KVM is the preferred hypervisor for all the major distros. Having the libvirt default be a random, round robin selection rather by ordered priority, in which kvm if available comes out on top, is not good design. Regards, Whit
2011/7/27 Whit Blauvelt <whit.virt@transpect.com>:
That's not how libvirt works. The architecture is different than you seem to expect it.
Matthias,
Even with all you say - and I thank you for your patient explanation - there remains a question: Why should 0.9.3+ git suddenly decide that the default driver should be vbox, when 0.8.3 and 0.9.2 and 0.9.3, all on the same system, and the second two similarly compiled with their defaults from the tar, went to kvm instead?
First, there is no default driver and nothing should have changed in the way drivers a selected. The only thing that has changed in this regard is that the type checking for the domain XML config was made stricter after 0.9.3. In 0.9.3 and before the VirtualBox driver happily accepted a domain config with type kvm. Basically all drivers didn't check whether or not a domain config was meant for them. Now the VirtualVtox driver only accepts type vbox and the QEMU driver accepts type qemu, kvm, kqemu and xen (xen for xenner).
There was no other change on the system. And the vbox processes I found appear to have been started by libvirt itself, since vbox hasn't been run on this system in over a year, during which time it's been rebooted without vbox being invoked at all.
I currently have no explanation for why libvirt 0.9.3+ picks VirtualBox for you now when it autoselected QEMU before. Are you sure that libvirt 0.9.3 and before had VirtualBox support compiled in? If not then VirtualBox can not supersede QEMU in the autodetection. That might explain it, but I don't see how that can happen apart from you explicitly disabling VirtualBox support at configure time.
If kvm is on the system as an option - which it has been all along - it should be the libvirt default. KVM is the preferred hypervisor for all the major distros. Having the libvirt default be a random, round robin selection rather by ordered priority, in which kvm if available comes out on top, is not good design.
You're right the autodetection process is suboptimal and should be improved. But the selection is not random, the problem is that VirtualBox comes before QEMU in the probing order. This is due to how the remote driver interferes with the detection process. -- Matthias Bolte http://photron.blogspot.com
Are you sure that libvirt 0.9.3 and before had VirtualBox support compiled in? If not then VirtualBox can not supersede QEMU in the autodetection. That might explain it, but I don't see how that can happen apart from you explicitly disabling VirtualBox support at configure time.
libvirt 0.9.3: configure:57866: Drivers configure:57868: configure:57870: Xen: no configure:57872: QEMU: yes configure:57874: UML: yes configure:57876: OpenVZ: yes configure:57878: VMware: yes configure:57880: VBox: yes configure:57882: XenAPI: no configure:57884: xenlight: no configure:57886: LXC: yes configure:57888: PHYP: no configure:57890: ESX: no configure:57892: Test: yes configure:57894: Remote: yes configure:57896: Network: yes configure:57898: Libvirtd: yes configure:57900: netcf: no configure:57902: macvtap: yes configure:57904: virtport: yes libvirt 0.9.3 + git (yesterday's) configure:58371: Drivers configure:58373: configure:58375: Xen: no configure:58377: QEMU: yes configure:58379: UML: yes configure:58381: OpenVZ: yes configure:58383: VMware: yes configure:58385: VBox: yes configure:58387: XenAPI: no configure:58389: xenlight: no configure:58391: LXC: yes configure:58393: PHYP: no configure:58395: ESX: yes configure:58397: Test: yes configure:58399: Remote: yes configure:58401: Network: yes configure:58403: Libvirtd: yes configure:58405: netcf: no configure:58407: macvtap: yes configure:58409: virtport: yes (Hmm. Picked up ESX meanwhile.) Best, Whit
On Wed, Jul 27, 2011 at 08:42:59AM -0400, Whit Blauvelt wrote: [..]
be accepted. Clearly having libvirt recognize that the kvm hypervisor is available, and automatically using it when the xml file specifies it, does not take some extraordinary level of clever programming. "The xml specifies kvm. Do I have kvm? Why, yes I do!"
Virsh first opens a connection to an hypervisor, and then tries to run the commands (can be multiple one for example in shall mode). What you're suggestion is to do some introspection on the XML, and then change the way virsh works just for that command, that's not that trivial nor very clean.
The initial "error: internal error unexpected domain type kvm, expecting vbox" you saw was added recently to prevent incompatible driver/config combinations. In you're case it highlighted that autodetection didn't work for you as expected.
Fine, so when it recognizes it has the wrong driver for the config, it should grab the right driver. No reason for it to complain to me about it. It should simply do its job. Right? This is a bug that it doesn't?
On the other hand what was improved recently is that if a command like create uses the "wrong" hypervisor type in the XML we are now reporting the expected hypervisor type instead of just failing the operation as you got. If you try with 0.9.4-rc1 it should have that improvement, Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On 07/27/2011 03:21 AM, Matthias Bolte wrote:
2011/7/27 Whit Blauvelt <whit.virt@transpect.com>:
What's the output of "# virsh -V" on your second ubuntu box? I guess your libvirt on that box might not be compiled with qemu driver.
# virsh -V Virsh command line tool of libvirt 0.9.3 ...
Compiled with support for:  Hypervisors: QEmu/KVM UML OpenVZ VirtualBox LXC ESX Test  Networking: Remote Daemon Network Bridging Nwfilter VirtualPort  Storage: Dir Filesystem SCSI Multipath LVM  Miscellaneous: SELinux Secrets Debug Readline
Another possiablity is the libvirt is compiled with both qemu driver and vbox driver, but when you try to creat a new connection, vbox was the first succesfully connected one. In this case, you can try like below:
Why? Ah, I do have a couple stray vbox processes somehow:
root   25265  0.0  0.1  86076  4304 ?     S   19:34  0:00 /usr/lib/virtualbox/VBoxXPCOMIPCD root   25274  0.0  0.1 209964  6672 ?     Sl  19:34  0:02 /usr/lib/virtualbox/VBoxSVC --pipe 4 --auto-shutdown
But that should cause it to deny it knows how to handle Qemu/KVM?
The point is that libvirt autodetects the available hypervisors at runtime when you don't specify a connection URI. For example, just running virsh results in autodetecting VirtualBox because you have it installed in a way that it's still working and due to the way libvirt works internally VirtualBox comes before QEMU in the autodetection list.
I think this should be changed actually. I think it's clear that there are far more libvirt+kvm users than libvirt+vbox users, we should adjust the driver probing to match. Unfortunately it doesn't look like a simple change. - Cole
2011/7/27 Cole Robinson <crobinso@redhat.com>:
On 07/27/2011 03:21 AM, Matthias Bolte wrote:
2011/7/27 Whit Blauvelt <whit.virt@transpect.com>:
What's the output of "# virsh -V" on your second ubuntu box? I guess your libvirt on that box might not be compiled with qemu driver.
# virsh -V Virsh command line tool of libvirt 0.9.3 ...
Compiled with support for:  Hypervisors: QEmu/KVM UML OpenVZ VirtualBox LXC ESX Test  Networking: Remote Daemon Network Bridging Nwfilter VirtualPort  Storage: Dir Filesystem SCSI Multipath LVM  Miscellaneous: SELinux Secrets Debug Readline
Another possiablity is the libvirt is compiled with both qemu driver and vbox driver, but when you try to creat a new connection, vbox was the first succesfully connected one. In this case, you can try like below:
Why? Ah, I do have a couple stray vbox processes somehow:
root   25265  0.0  0.1  86076  4304 ?     S   19:34  0:00 /usr/lib/virtualbox/VBoxXPCOMIPCD root   25274  0.0  0.1 209964  6672 ?     Sl  19:34  0:02 /usr/lib/virtualbox/VBoxSVC --pipe 4 --auto-shutdown
But that should cause it to deny it knows how to handle Qemu/KVM?
The point is that libvirt autodetects the available hypervisors at runtime when you don't specify a connection URI. For example, just running virsh results in autodetecting VirtualBox because you have it installed in a way that it's still working and due to the way libvirt works internally VirtualBox comes before QEMU in the autodetection list.
I think this should be changed actually. I think it's clear that there are far more libvirt+kvm users than libvirt+vbox users, we should adjust the driver probing to match. Unfortunately it doesn't look like a simple change.
Sure, the problem is that QEMU goes through the remote driver (because it's a daemon/stateful driver) and that one is last in the local probing list. I don't currently see how we can give QEMU a higher position in the list, except by special casing it in the probing process. -- Matthias Bolte http://photron.blogspot.com
participants (5)
-
Cole Robinson -
Daniel Veillard -
Matthias Bolte -
Osier Yang -
Whit Blauvelt