[libvirt-users] libvirt doesn't boot kVM, hangs, qemu on 100% CPU
Hi folks, it's been adventurous. Yesterday night I've started debugging this particular issue of why my KVMs don't boot on Ubuntu 11.10. A first hint was apparmor, which seemed to deny access to the LVM partitions I had assigned as disks. After correctly configuring apparmor, or even disabling it the issue was still the same. After a wild goose chase, I decided to update everything, I now have * libvirt 0.9.8 (built from Ubuntu 12.04 sources) * qemu 1.0 (built from Ubuntu 12.04 sources) * SeaBIOS (built from git master: pre-1.6.4-20120221_203841) I even installed sgabios to see what's going on without having to connect with virt-manager every time. So what's going on? What am I trying to do? I'm trying to boot an Ubuntu amd64 VM, which looks pretty much the same as the Ubuntu host, except that I uninstalled libvirt and qemu-kvm, which felt out of place. This I'm trying to accomplish by booting it with the host's kernel and initrd (os/kernel and os/initrd). So much for the starting conditions. You can find my configuration here: http://sprunge.us/gCVj (I'm attaching it just in case the pastebin goes stale). The first thing that is slightly irritating is that it doesn't validate: """ Relax-NG validity error : Extra element kernel in interleave web.xml:9: element kernel: Relax-NG validity error : Element os failed to validate content web.xml:8: element type: Relax-NG validity error : Error validating value web.xml:8: element type: Relax-NG validity error : Element type failed to validate content web.xml:1: element domain: Relax-NG validity error : Invalid sequence in interleave web.xml:1: element domain: Relax-NG validity error : Element domain failed to validate content web.xml fails to validate """ Interestingly, even if I remove some of the elements that `virt-xml-validate` complains about via `virsh edit`, next time I `edit`, or `dumpxml` it'll be back in place. With sgabios in place, I can fortunately just c/p to show you what a boot looks like: """ SeaBIOS (version pre-1.6.4-20120221_203841-bacon) iPXE (http://ipxe.org) 00:03.0 C100 PCI2.10 PnP PMM+001C9B80+00189B80 C100 iPXE (http://ipxe.org) 00:04.0 C200 PCI2.10 PnP PMM 001C9B80 00189B80 C200 Booting from ROM... """ And that's it. And that's wrong. But I can't seem to figure out how and why. I'll be hugely in debt for your help and welcome any suggestions, ready to provide as much information as you need to solve this issue. So long, i -- Igor Galić Tel: +43 (0) 664 886 22 883 Mail: i.galic@brainsware.org URL: http://brainsware.org/ GPG: 6880 4155 74BD FD7C B515 2EA5 4B1D 9E08 A097 C9AE
On 02/21/2012 03:46 PM, Igor Galić wrote:
So much for the starting conditions. You can find my configuration here: http://sprunge.us/gCVj (I'm attaching it just in case the pastebin goes stale).
The first thing that is slightly irritating is that it doesn't validate:
""" Relax-NG validity error : Extra element kernel in interleave web.xml:9: element kernel: Relax-NG validity error : Element os failed to validate content web.xml:8: element type: Relax-NG validity error : Error validating value web.xml:8: element type: Relax-NG validity error : Element type failed to validate content web.xml:1: element domain: Relax-NG validity error : Invalid sequence in interleave web.xml:1: element domain: Relax-NG validity error : Element domain failed to validate content web.xml fails to validate
Right now, the RNG says you can have either <kernel> or <boot> under <os>, but not both. But the C code doesn't reject attempts to have both, and I have to wonder if we are running into problems by allowing both. And unfortunately, none of the tests/qemuxml2argvdata test files include any use of <kernel>, so we aren't exercising this part of XML parsing. Definitely some bugs to be fixed, but my problem is that I don't know what behavior should be legal. Is there ever a use case to combine both <kernel> and <boot> in the same image? Or are they really distinct (<kernel> says to boot using an image in the host, <boot> says to boot by using a kernel found in the guest device, whether that be a guest disk device or a guest network interface for PXE boot)? I'm hoping Dan has a bit more experience on this, given his work on libvirt-sandbox.
"""
Interestingly, even if I remove some of the elements that `virt-xml-validate` complains about via `virsh edit`, next time I `edit`, or `dumpxml` it'll be back in place.
Hmm - that makes it sound like once we have both methods in memory, we don't really have a way to remove one again. Also quite fishy. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 02/21/2012 03:46 PM, Igor Galić wrote:
Hi folks,
it's been adventurous. Yesterday night I've started debugging this particular issue of why my KVMs don't boot on Ubuntu 11.10.
On IRC, we identified the culprit:
<uuid>8cfcb7b0-10ee-7d08-9b64-9f39c154292a</uuid> <memory>2048</memory> <currentMemory>2048</currentMemory>
There ain't no way on earth you're going to boot a kernel in 2 megabytes of memory! I propose enhancing the XML; on output, libvirt should produce: <memory units='k'>2048</memory> => 2048 * kibibyte the output unit must remain the same as it has always been, but the new attribute will make it easier for humans reading the XML to spot blunders like what spawned this thread. On input, the optional attribute is more useful - we can use it to provide a multiplier (of course, the result will be rounded up to k, and again rounded up to any higher granularity per the hypervisor): b => bytes k, KiB => kibibyte (1024) KB => kilobyte (1000) M, MiB => mebibyte (1024*1024) MB => megabyte (1000000) and so on for at least G and T (do we need P, E, Z, or Y? and I'm jealous if you have a machine with 1Y memory). Thoughts before I propose such a patch? -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, Feb 22, 2012 at 04:42:59PM -0700, Eric Blake wrote:
On 02/21/2012 03:46 PM, Igor Galić wrote:
Hi folks,
it's been adventurous. Yesterday night I've started debugging this particular issue of why my KVMs don't boot on Ubuntu 11.10.
On IRC, we identified the culprit:
<uuid>8cfcb7b0-10ee-7d08-9b64-9f39c154292a</uuid> <memory>2048</memory> <currentMemory>2048</currentMemory>
There ain't no way on earth you're going to boot a kernel in 2 megabytes of memory!
I propose enhancing the XML; on output, libvirt should produce:
<memory units='k'>2048</memory> => 2048 * kibibyte
the output unit must remain the same as it has always been, but the new attribute will make it easier for humans reading the XML to spot blunders like what spawned this thread.
On input, the optional attribute is more useful - we can use it to provide a multiplier (of course, the result will be rounded up to k, and again rounded up to any higher granularity per the hypervisor): b => bytes k, KiB => kibibyte (1024) KB => kilobyte (1000) M, MiB => mebibyte (1024*1024) MB => megabyte (1000000) and so on for at least G and T (do we need P, E, Z, or Y? and I'm jealous if you have a machine with 1Y memory).
Thoughts before I propose such a patch?
I like the idea, maybe use units=KiB just to be completely clear? Dave
Hey, On Wed, Feb 22, 2012 at 04:42:59PM -0700, Eric Blake wrote:
There ain't no way on earth you're going to boot a kernel in 2 megabytes of memory!
I propose enhancing the XML; on output, libvirt should produce:
<memory units='k'>2048</memory> => 2048 * kibibyte
the output unit must remain the same as it has always been, but the new attribute will make it easier for humans reading the XML to spot blunders like what spawned this thread.
[...]
Thoughts before I propose such a patch?
For what it's worth, there are other places with unitless sizes, there's at least <video vram=...>, everything under <memtune> and <bandwidth> (not something measured in bytes though), and these 3 <pool> children: <allocation>, <capacity> and <available>. (NB: looked at your patches now and saw you handle memtune) Also, <volume><allocation> and <volume><capacity> already support a "unit" attribute which accept the K, M, G, T, P, E values (xxxbyte), for consistency it would be better for the new <memory> attribute to use "unit" and not "units" as suggested by Mathias, and for both "unit" attributes to accept the same values. Christophe
Hey folks, After two painful days of debugging, I've finally found the answer to the pains. It lies in my misinterpretation of the unit of <memory> The unit is k, while I assumed m. The first thing someone helpfully suggested on IRC, when seeing <memory>20480</memory> was that 20G might be too much for my poor OS - so I shaved off a 0, going from 20M, which was already too little for booting a Linux kernel down to 2. Whelp. I've already sent an RFE to seabios@ asking if they would perhaps very much mind adding a printout of the amount of RAM available, like any other BIOS does. Let's see where that goes. Meanwhile I talked to Eric and convinced him that adding <memory units= would be a good thing. See his next mail in a different thread. Well so much for that. Thanks to everybody who helped me step through the stack. You've been a great crowd! So long, i -- Igor Galić Tel: +43 (0) 664 886 22 883 Mail: i.galic@brainsware.org URL: http://brainsware.org/ GPG: 6880 4155 74BD FD7C B515 2EA5 4B1D 9E08 A097 C9AE
participants (4)
-
Christophe Fergeau -
Dave Allan -
Eric Blake -
Igor Galić