On Thu, 2016-07-21 at 16:31 +0100, Daniel P. Berrange wrote:
It seems that we could solve this by just changing the logic in qemuDomainAssignDevicePCISlots() so that when it is auto-assigning addresses, it always uses 00:00.0 for the USB controller when guest arch is ppc64. Existing guests deployed from a libvirt version using -device won't be affected, because we'll have recorded a PCI address for that in the XML and continue to use that. ie the auto-allocation logic won't run for them. Existing guests deployed from a libvirt version using -usb should then get the fixed PCI address of 00:00.0 when they upgrade to the fix libvirt (assuming they didn't get run with a broken libvirt in between). Mh, that could actually work! Thanks :) I'll try to cook up a patch tomorrow. Unfortunately Martin pointed out a problem with this plan: even though they're not actually using it, old guests have been assigned a PCI address for the USB controller. Which means we can't tell whether <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller is an old guest that should be assigned the 00:00.0 address or a new guest that should keep using the 00:03.0 address :( If we revert 8156493d8db95de91dd9ace743df0fd4dff98281 we'll go back to using -usb and the PCI addr reported in the XML is just completely wrong vs what's actually being used. If the user requests an explicit addr for the USB controller, we'll ignore it and that could cause boot failure if the user has requested something else in addr 00:00.0 I think we have no choice but to stick with using -device, because that is clearly the preferred syntax long term, and it allows us to actually honour the addresses requested in the XML, which the original code was not doing. IOW the old code < 1.3.1 using -usb was clearly broken, so I don't think reverting is an option we can take. Furthermore reverting it will instead break anyone with libvirt 1.3.1 -> 2.0.0
It was actually more broken than I remembered - it used -usb on ppc64 because the condition for not using it was to have PIIX3, which is IIUC x86-specific hardware.
So no matter what we do some portion of users are screwed. On balance I think users of libvirt < 1.3.1 will just have to take the pain. We can document a procedure to minimize that pain. Specifically if you have libvirt < 1.3.1 and want to upgrade them 1. Use virsh save-image-edit to change the XML for all existing saved images, and fix the PCI address slot to be 0 instead of 3. This should allow new libvirt to load the image, since itsXML will now reflect the reality of what was used.
I tried this with libvirt-1.2.17-13.el7.ppc64le. The save image contains just <controller type='usb' index='0'/> If you add a PCI address that's not 00:00.0, it will be ignored; 00:00.0 will cause save-image-edit to fail with XML error: Invalid PCI address 0000:00:00, at least one of domain, bus, or slot must be > 0
2. For any running guests edit /var/run/libvirt/qemu/*.xml to again fix the PCI address slot from 3 to 0. Then restart libvirtd so it reloads its config. This should then allow those running guests to be live migrated
This fails with the same error as above; moreover, the guest will disappear. The QEMU process will still be running, of course. I'm afraid the only way for people running libvirt < 1.3.1 to make their guests live migratable is to tweak the XML so that it looks like <controller type='usb' model='pci-ohci' ... and shutdown the guests. When they are started again, they will get -device pci-ohci,... on the command line and live migration (back and forth) will work as expected. I'd be extremely happy if I were proven wrong, though :) -- Andrea Bolognani / Red Hat / Virtualization