[libvirt] cpuset / numa and qemu in TCG mode
Hello all, This is with reference to OpenStack (nova) bug 1439280 [1]. The symptom is when nova(-compute) tries to launch an instance/VM it errors out with: "libvirtError: Requested operation is not valid: cpu affinity is not supported" This only happens with using qemu in TCG mode. After looking at the domain XML and the docs at [2] It seems to me the problem is either: 1) the libvirt driver in nova is generating valid XML that is an invalid domain description ; or 2) NUMA support in qemu (TCG) mode is broken. Exploring these options a little further: Option 1: the libvirt driver in nova is generating valid XML that is an invalid domain description ==== This is the relevant section of the domain XML --- <vcpu placement='static' cpuset='0-1'>1</vcpu> <cputune> <shares>1024</shares> </cputune> --- To my reading of [2] the domain XML should either avoid specifying the 'cpuset' in the vcpu node ; or include 'emulatorpin' in the cputune node. Have I understood the documentation correctly? If so it would seem that the correct fix is in nova to teach the libvirt driver to generate the correct XML for this virtulisation type. Option 2: NUMA support in qemu (TCG) mode is broken. ==== Is cpuset and NUMA architecture expected to work in qemu when running in TCG mode? If it is then clearly we need to dig into this and fix qemu. Yours Tony. [1] https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1439280 [2] https://libvirt.org/formatdomain.html#elementsCPUAllocation
On Mon, May 11, 2015 at 10:52:46AM +1000, Tony Breeds wrote:
Hello all, This is with reference to OpenStack (nova) bug 1439280 [1].
The symptom is when nova(-compute) tries to launch an instance/VM it errors out with: "libvirtError: Requested operation is not valid: cpu affinity is not supported" This only happens with using qemu in TCG mode.
I can see where the error might have come from. QEMU doesn't have vcpu threads if ran in TCG mode, so if libvirt tries to pin some of them, this will happen. *But* upstream handles this perfectly. I just tried and it works. I'll explain why a few lines down the road.
After looking at the domain XML and the docs at [2] It seems to me the problem is either: 1) the libvirt driver in nova is generating valid XML that is an invalid domain description ; or 2) NUMA support in qemu (TCG) mode is broken.
I don't get what this has to do with NUMA, but anyway I just think this is a bug in older libvirt actually.
Exploring these options a little further:
Option 1: the libvirt driver in nova is generating valid XML that is an invalid domain description ====
This is the relevant section of the domain XML --- <vcpu placement='static' cpuset='0-1'>1</vcpu> <cputune> <shares>1024</shares> </cputune> ---
To my reading of [2] the domain XML should either avoid specifying the 'cpuset' in the vcpu node ; or include 'emulatorpin' in the cputune node.
Yes, that's because we can't differentiate which threads do what with TCG accel. Because of the you must only specify one pinning for all threads (I/O threads, CPU threads and the emulator thread), which is done exactly using the 'cpuset' attribute as that is valid for the whole machine. Thatnsk to that we don't have to differentiate anything and just use the cpuset for the whole machine -> no error should occur there.
Have I understood the documentation correctly? If so it would seem that the correct fix is in nova to teach the libvirt driver to generate the correct XML for this virtulisation type.
You understood the documentation correctly, it was just libvirt acting up a bit.
Option 2: NUMA support in qemu (TCG) mode is broken. ====
Is cpuset and NUMA architecture expected to work in qemu when running in TCG mode? If it is then clearly we need to dig into this and fix qemu.
I'm still not sure why are you bringing NUMA into the mix as there is nothing NUMA related in the bug or this mail. But to complete my answer, even though there is no practical benefit to using NUMA on TCG accelerated QEMU machine, it should still work. Here's the XML I just fired up using libvirt 1.2.15 just to try it out. And it simply works. <domain type='qemu'> <name>dummy-tcg</name> <uuid>946f637c-dcaa-4392-8926-b54476eddd43</uuid> <memory unit='KiB'>131072</memory> <currentMemory unit='KiB'>65536</currentMemory> <vcpu placement='static' cpuset='3-5,7'>4</vcpu> <os> <type arch='x86_64' machine='pc-q35-2.3'>hvm</type> <boot dev='hd'/> </os> <cpu> <numa> <cell id='0' cpus='0,2' memory='65536' unit='KiB'/> <cell id='1' cpus='1,3' memory='65536' unit='KiB'/> </numa> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <memballoon model='none'/> </devices> </domain>
Yours Tony.
[1] https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1439280 [2] https://libvirt.org/formatdomain.html#elementsCPUAllocation
On Mon, May 11, 2015 at 10:52:17AM +0200, Martin Kletzander wrote:
On Mon, May 11, 2015 at 10:52:46AM +1000, Tony Breeds wrote:
Hello all, This is with reference to OpenStack (nova) bug 1439280 [1].
The symptom is when nova(-compute) tries to launch an instance/VM it errors out with: "libvirtError: Requested operation is not valid: cpu affinity is not supported" This only happens with using qemu in TCG mode.
I can see where the error might have come from. QEMU doesn't have vcpu threads if ran in TCG mode, so if libvirt tries to pin some of them, this will happen. *But* upstream handles this perfectly. I just tried and it works. I'll explain why a few lines down the road.
Okay, that's good to know.
After looking at the domain XML and the docs at [2] It seems to me the problem is either: 1) the libvirt driver in nova is generating valid XML that is an invalid domain description ; or 2) NUMA support in qemu (TCG) mode is broken.
I don't get what this has to do with NUMA, but anyway I just think this is a bug in older libvirt actually.
The NUMA references are baddness on my part. They come from the fact that the code in nova that does the CPUpinning is tied up with NUMA support. So with that on mind I'd reword my summary as: "CPU Pinning in qemu (TCG) mode is broken."
Yes, that's because we can't differentiate which threads do what with TCG accel. Because of the you must only specify one pinning for all threads (I/O threads, CPU threads and the emulator thread), which is done exactly using the 'cpuset' attribute as that is valid for the whole machine. Thatnsk to that we don't have to differentiate anything and just use the cpuset for the whole machine -> no error should occur there.
Ok I think I follow that.
Have I understood the documentation correctly? If so it would seem that the correct fix is in nova to teach the libvirt driver to generate the correct XML for this virtulisation type.
You understood the documentation correctly, it was just libvirt acting up a bit.
Okay that good to know. So we can work around this in nova by either: a) Check for libvirt 1.2.15 in the approriate TCG code path and elide the cpuset ; or b) including 'emulatorpin' in the cputune node 'a' is easy to do and is basically what I have proposwed in the bug. Having said that which is the better way to work around libvirt acting up a bit in older versions? Ther nova side will be pretty easy regardless. I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
I'm still not sure why are you bringing NUMA into the mix as there is nothing NUMA related in the bug or this mail. But to complete my answer, even though there is no practical benefit to using NUMA on TCG accelerated QEMU machine, it should still work.
Okay so as I said above NUMA as such is a mistake. Thank tyou for confirming that when the correct XML is generated the instance/guest/VM works just fine. Yours Tony.
On Mon, May 11, 2015 at 07:24:47PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 10:52:17AM +0200, Martin Kletzander wrote:
On Mon, May 11, 2015 at 10:52:46AM +1000, Tony Breeds wrote:
Hello all, This is with reference to OpenStack (nova) bug 1439280 [1].
The symptom is when nova(-compute) tries to launch an instance/VM it errors out with: "libvirtError: Requested operation is not valid: cpu affinity is not supported" This only happens with using qemu in TCG mode.
I can see where the error might have come from. QEMU doesn't have vcpu threads if ran in TCG mode, so if libvirt tries to pin some of them, this will happen. *But* upstream handles this perfectly. I just tried and it works. I'll explain why a few lines down the road.
Okay, that's good to know.
After looking at the domain XML and the docs at [2] It seems to me the problem is either: 1) the libvirt driver in nova is generating valid XML that is an invalid domain description ; or 2) NUMA support in qemu (TCG) mode is broken.
I don't get what this has to do with NUMA, but anyway I just think this is a bug in older libvirt actually.
The NUMA references are baddness on my part. They come from the fact that the code in nova that does the CPUpinning is tied up with NUMA support. So with that on mind I'd reword my summary as:
"CPU Pinning in qemu (TCG) mode is broken."
Oh, OK, I thought there is some connection I must've missed :)
Yes, that's because we can't differentiate which threads do what with TCG accel. Because of the you must only specify one pinning for all threads (I/O threads, CPU threads and the emulator thread), which is done exactly using the 'cpuset' attribute as that is valid for the whole machine. Thatnsk to that we don't have to differentiate anything and just use the cpuset for the whole machine -> no error should occur there.
Ok I think I follow that.
Have I understood the documentation correctly? If so it would seem that the correct fix is in nova to teach the libvirt driver to generate the correct XML for this virtulisation type.
You understood the documentation correctly, it was just libvirt acting up a bit.
Okay that good to know.
So we can work around this in nova by either: a) Check for libvirt 1.2.15 in the approriate TCG code path and elide the cpuset ; or b) including 'emulatorpin' in the cputune node
'a' is easy to do and is basically what I have proposwed in the bug. Having said that which is the better way to work around libvirt acting up a bit in older versions?
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it). Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
I'm still not sure why are you bringing NUMA into the mix as there is nothing NUMA related in the bug or this mail. But to complete my answer, even though there is no practical benefit to using NUMA on TCG accelerated QEMU machine, it should still work.
Okay so as I said above NUMA as such is a mistake. Thank tyou for confirming that when the correct XML is generated the instance/guest/VM works just fine.
Yours Tony.
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000 qemu: fix setting of VM CPU affinity with TCG --- A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong. If you don't beat me to it I'll request that backport to 1.2.13 *and* ask the Ubuntu guys to take it as well. I have to admit I'm still in 2 minds on the nova side. Adding a wart to the libvirt driver for this specific bug for distros/vendors that are using 1.2.12 seems a bit gross but .... Daniel what do you think? Yours Tony.
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG ---
A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong.
If you don't beat me to it I'll request that backport to 1.2.13 *and* ask the Ubuntu guys to take it as well.
I have to admit I'm still in 2 minds on the nova side. Adding a wart to the libvirt driver for this specific bug for distros/vendors that are using 1.2.12 seems a bit gross but ....
I think this is something distros should really fix in their libvirt packages - backporting this kind of fix is exactly the kind job of maintainers should be doing. Particularly since it only affects TCG mode, I don't think Nova really should try to workaround it Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, May 12, 2015 at 09:51:24AM +0100, Daniel P. Berrange wrote:
I think this is something distros should really fix in their libvirt packages - backporting this kind of fix is exactly the kind job of maintainers should be doing.
Particularly since it only affects TCG mode, I don't think Nova really should try to workaround it
Thanks Daniel. Yours Tony.
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG ---
A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong.
1.2.13 has the commit already in the release and 1.2.12-maint has it as a first back-port right after release. The problem is that there was no maintenance release of 1.2.12 yet. Maybe they would use 1.2.12.1 if it existed. I Cc'd Guido as an upstream debian maintainer, maybe he'll have some insights. @Guido: would it help if we created a maintenance release from the v1.2.12-maint branch? Or is the only thing missing the fact that the launchpad bug is not moved to libvirt?
If you don't beat me to it I'll request that backport to 1.2.13 *and* ask the Ubuntu guys to take it as well.
I have to admit I'm still in 2 minds on the nova side. Adding a wart to the libvirt driver for this specific bug for distros/vendors that are using 1.2.12 seems a bit gross but ....
Daniel what do you think?
Yours Tony.
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
1.2.13 has the commit already in the release and 1.2.12-maint has it as a first back-port right after release. The problem is that there was no maintenance release of 1.2.12 yet. Maybe they would use 1.2.12.1 if it existed.
Perhaps. At this point we're well and truly in realms I can't influence. I thank you for you help keeping me on track here. I think by and large the issue is preety much solved (and was before I started this thread a couple of days ago.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some insights. @Guido: would it help if we created a maintenance release from the v1.2.12-maint branch? Or is the only thing missing the fact that the launchpad bug is not moved to libvirt?
I just added: --- I was pointed at the v1.2.12-maint head in the libvirt git which contains this fix already. http://libvirt.org/git/?p=libvirt.git;a=shortlog;h=refs/heads/v1.2.12-maint I suggest we close the nova issue with won't fix and get the correctly backported patch into the libvirt package. --- https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1439280/comments/41 It's up to the Ubuntu team now. Yours Tony.
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG ---
A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong.
1.2.13 has the commit already in the release and 1.2.12-maint has it as a first back-port right after release. The problem is that there was no maintenance release of 1.2.12 yet. Maybe they would use 1.2.12.1 if it existed.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some insights. @Guido: would it help if we created a maintenance release from the v1.2.12-maint branch? Or is the only thing missing the fact that the launchpad bug is not moved to libvirt?
Basically Ubuntu takes the version that is in Debian testing, unstable or experimental at time of their release and adds pathes. There's little to no sync unfortunately (except for the awesome efforts to sync the apparmor stuff) Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 + maintenance patches (which I intend to switch over to the maintenance releases soonish and support hopefully Cole with these), oldstable has 0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the releases). I've added Serge since he might to jump onto 1.2.12.1 maintenance relasese. I'm happy about any additional notifications for things that should go into distributions. Cheers, -- Guido
Quoting Guido Günther (agx@sigxcpu.org):
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG ---
A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong.
1.2.13 has the commit already in the release and 1.2.12-maint has it as a first back-port right after release. The problem is that there was no maintenance release of 1.2.12 yet. Maybe they would use 1.2.12.1 if it existed.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some insights. @Guido: would it help if we created a maintenance release from the v1.2.12-maint branch? Or is the only thing missing the fact that the launchpad bug is not moved to libvirt?
Thanks guys, I'm going to cherrypick that patch into vivid right now. (It should be in wily as that is 1.2.15).
Basically Ubuntu takes the version that is in Debian testing, unstable or experimental at time of their release and adds pathes. There's little to no sync unfortunately (except for the awesome efforts to sync the apparmor stuff)
Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 + maintenance patches (which I intend to switch over to the maintenance releases soonish and support hopefully Cole with these), oldstable has 0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the releases).
I've added Serge since he might to jump onto 1.2.12.1 maintenance relasese.
If you mean contributing by adding patches that we add to vivid, I'm definately willing to do that. Note that vivid has a pretty short lifecycle so it wouldn't go on for very long. If people are willing to tag patches with '[1.2.12.1-maint]' on this list I'm willing to support this longer.
I'm happy about any additional notifications for things that should go into distributions.
Cheers, -- Guido
Thanks! -serge
On Wed, May 13, 2015 at 03:31:09PM +0000, Serge Hallyn wrote:
Quoting Guido Günther (agx@sigxcpu.org):
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG ---
A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong.
1.2.13 has the commit already in the release and 1.2.12-maint has it as a first back-port right after release. The problem is that there was no maintenance release of 1.2.12 yet. Maybe they would use 1.2.12.1 if it existed.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some insights. @Guido: would it help if we created a maintenance release from the v1.2.12-maint branch? Or is the only thing missing the fact that the launchpad bug is not moved to libvirt?
Thanks guys, I'm going to cherrypick that patch into vivid right now. (It should be in wily as that is 1.2.15).
Basically Ubuntu takes the version that is in Debian testing, unstable or experimental at time of their release and adds pathes. There's little to no sync unfortunately (except for the awesome efforts to sync the apparmor stuff)
Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 + maintenance patches (which I intend to switch over to the maintenance releases soonish and support hopefully Cole with these), oldstable has 0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the releases).
I've added Serge since he might to jump onto 1.2.12.1 maintenance relasese.
If you mean contributing by adding patches that we add to vivid, I'm definately willing to do that. Note that vivid has a pretty short lifecycle so it wouldn't go on for very long. If people are willing to tag patches with '[1.2.12.1-maint]' on this list I'm willing to support this longer.
Rather than tagging patches with -maint, we just back-port them straight into the maintenance branch. If there are more distributions using the maintenance branch or would be if there was more review, etc., I'm sure we could come up with some solution to suit everyone (or majority at least). After patches are back-ported, Cole does a maintenance release which should make it even more easier (e.g. you don't want to follow up the list or the git repo) since the release announcement goes to libvirt-announce as with other releases. However, I'm not sure when Cole does those releases (if it's time-, commit- or random-based). Anyway, as I said earlier, we should definitely come up with how to do this so it helps other maintainers out there. I'm offering my help.
I'm happy about any additional notifications for things that should go into distributions.
Cheers, -- Guido
Thanks!
-serge
[Adding Cole to Cc as I forgot to do that before sending the mail] On Thu, May 14, 2015 at 08:58:03AM +0200, Martin Kletzander wrote:
On Wed, May 13, 2015 at 03:31:09PM +0000, Serge Hallyn wrote:
Quoting Guido Günther (agx@sigxcpu.org):
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
>Ther nova side will be pretty easy regardless. > >I'd say the best solution would be to back port the 'fix' but that seems like a >lot of effort given the number of distros and libvirt versions potentiall >involved. >
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG ---
A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong.
1.2.13 has the commit already in the release and 1.2.12-maint has it as a first back-port right after release. The problem is that there was no maintenance release of 1.2.12 yet. Maybe they would use 1.2.12.1 if it existed.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some insights. @Guido: would it help if we created a maintenance release from the v1.2.12-maint branch? Or is the only thing missing the fact that the launchpad bug is not moved to libvirt?
Thanks guys, I'm going to cherrypick that patch into vivid right now. (It should be in wily as that is 1.2.15).
Basically Ubuntu takes the version that is in Debian testing, unstable or experimental at time of their release and adds pathes. There's little to no sync unfortunately (except for the awesome efforts to sync the apparmor stuff)
Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 + maintenance patches (which I intend to switch over to the maintenance releases soonish and support hopefully Cole with these), oldstable has 0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the releases).
I've added Serge since he might to jump onto 1.2.12.1 maintenance relasese.
If you mean contributing by adding patches that we add to vivid, I'm definately willing to do that. Note that vivid has a pretty short lifecycle so it wouldn't go on for very long. If people are willing to tag patches with '[1.2.12.1-maint]' on this list I'm willing to support this longer.
Rather than tagging patches with -maint, we just back-port them straight into the maintenance branch. If there are more distributions using the maintenance branch or would be if there was more review, etc., I'm sure we could come up with some solution to suit everyone (or majority at least).
After patches are back-ported, Cole does a maintenance release which should make it even more easier (e.g. you don't want to follow up the list or the git repo) since the release announcement goes to libvirt-announce as with other releases.
However, I'm not sure when Cole does those releases (if it's time-, commit- or random-based). Anyway, as I said earlier, we should definitely come up with how to do this so it helps other maintainers out there. I'm offering my help.
I'm happy about any additional notifications for things that should go into distributions.
Cheers, -- Guido
Thanks!
-serge
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Hi, On Wed, May 13, 2015 at 03:31:09PM +0000, Serge Hallyn wrote:
Quoting Guido Günther (agx@sigxcpu.org):
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more importantly working around bug in underlying software is something that shouldn't be done at all IMHO. Let the maintainers backport whatever needs to be done.
I agree with you in an ideal world but there are times when we do need to add work arounds in $project_x to work around issues in $project_y.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like a lot of effort given the number of distros and libvirt versions potentiall involved.
If you want the fix to be distro-agnostic, there's nothing easier than back-porting the fix into our upstream maintenance branches. Those should make the life of distro maintainers easy (although it looks like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us with doing an additional backport to 1.2.12 and getting the cloud-archive team to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need to be back-ported. Having known your libvirt version, it shouldn't be too hard looking for the differences and finding the right commit. When back-porting request is made on the list, it is usually acted upon. If you can't find the exact commit, let me know and I'll do my best to help.
So a git bisect points at: --- commit a103bb105c0c189c3973311ff1826972b5bc6ad6 Author: Daniel P. Berrange <berrange@redhat.com> Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG ---
A small amount of reading implies to me that we'd be looking at backporting a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I could be wrong.
1.2.13 has the commit already in the release and 1.2.12-maint has it as a first back-port right after release. The problem is that there was no maintenance release of 1.2.12 yet. Maybe they would use 1.2.12.1 if it existed.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some insights. @Guido: would it help if we created a maintenance release from the v1.2.12-maint branch? Or is the only thing missing the fact that the launchpad bug is not moved to libvirt?
Thanks guys, I'm going to cherrypick that patch into vivid right now. (It should be in wily as that is 1.2.15).
Basically Ubuntu takes the version that is in Debian testing, unstable or experimental at time of their release and adds pathes. There's little to no sync unfortunately (except for the awesome efforts to sync the apparmor stuff)
Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 + maintenance patches (which I intend to switch over to the maintenance releases soonish and support hopefully Cole with these), oldstable has 0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the releases).
I've added Serge since he might to jump onto 1.2.12.1 maintenance relasese.
If you mean contributing by adding patches that we add to vivid, I'm definately willing to do that. Note that vivid has a pretty short lifecycle so it wouldn't go on for very long. If people are willing to tag patches with '[1.2.12.1-maint]' on this list I'm willing to support this longer.
I rather meant that you might be interested to follow a v1.2.12-maint branch. Maintaining it is even better ;) Cheers, -- Guido
participants (5)
-
Daniel P. Berrange -
Guido Günther -
Martin Kletzander -
Serge Hallyn -
Tony Breeds