7:52 p.m.
Hello all,
This is with reference to OpenStack (nova) bug 1439280 [1].
The symptom is when nova(-compute) tries to launch an instance/VM it errors out with:
"libvirtError: Requested operation is not valid: cpu affinity is not
supported"
This only happens with using qemu in TCG mode.
After looking at the domain XML and the docs at [2] It seems to me the problem is
either:
1) the libvirt driver in nova is generating valid XML that is an invalid
domain description ; or
2) NUMA support in qemu (TCG) mode is broken.
Exploring these options a little further:
Option 1: the libvirt driver in nova is generating valid XML that is an
invalid domain description
====
This is the relevant section of the domain XML
---
<vcpu placement='static' cpuset='0-1'>1</vcpu>
<cputune>
<shares>1024</shares>
</cputune>
---
To my reading of [2] the domain XML should either avoid specifying the 'cpuset'
in the vcpu node ; or include 'emulatorpin' in the cputune node.
Have I understood the documentation correctly? If so it would seem that the
correct fix is in nova to teach the libvirt driver to generate the correct XML
for this virtulisation type.
Option 2: NUMA support in qemu (TCG) mode is broken.
====
Is cpuset and NUMA architecture expected to work in qemu when running in TCG
mode? If it is then clearly we need to dig into this and fix qemu.
Yours Tony.
[1] https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1439280
[2] https://libvirt.org/formatdomain.html#elementsCPUAllocation
3:52 a.m.
On Mon, May 11, 2015 at 10:52:46AM +1000, Tony Breeds wrote:
Hello all,
This is with reference to OpenStack (nova) bug 1439280 [1].
The symptom is when nova(-compute) tries to launch an instance/VM it errors out with:
"libvirtError: Requested operation is not valid: cpu affinity is not
supported"
This only happens with using qemu in TCG mode.
I can see where the error might have come from. QEMU doesn't have
vcpu threads if ran in TCG mode, so if libvirt tries to pin some of
them, this will happen. *But* upstream handles this perfectly. I
just tried and it works. I'll explain why a few lines down the road.
After looking at the domain XML and the docs at [2] It seems to me
the problem is either:
1) the libvirt driver in nova is generating valid XML that is an invalid
domain description ; or
2) NUMA support in qemu (TCG) mode is broken.
I don't get what this has to do with NUMA, but anyway I just think
this is a bug in older libvirt actually.
Exploring these options a little further:
Option 1: the libvirt driver in nova is generating valid XML that is an
invalid domain description
====
This is the relevant section of the domain XML
---
<vcpu placement='static' cpuset='0-1'>1</vcpu>
<cputune>
<shares>1024</shares>
</cputune>
---
To my reading of [2] the domain XML should either avoid specifying the 'cpuset'
in the vcpu node ; or include 'emulatorpin' in the cputune node.
Yes, that's because we can't differentiate which threads do what with
TCG accel. Because of the you must only specify one pinning for all
threads (I/O threads, CPU threads and the emulator thread), which is
done exactly using the 'cpuset' attribute as that is valid for the
whole machine. Thatnsk to that we don't have to differentiate
anything and just use the cpuset for the whole machine -> no error
should occur there.
Have I understood the documentation correctly? If so it would seem
that the
correct fix is in nova to teach the libvirt driver to generate the correct XML
for this virtulisation type.
You understood the documentation correctly, it was just libvirt acting
up a bit.
Option 2: NUMA support in qemu (TCG) mode is broken.
====
Is cpuset and NUMA architecture expected to work in qemu when running in TCG
mode? If it is then clearly we need to dig into this and fix qemu.
I'm still not sure why are you bringing NUMA into the mix as there is
nothing NUMA related in the bug or this mail. But to complete my
answer, even though there is no practical benefit to using NUMA on TCG
accelerated QEMU machine, it should still work.
Here's the XML I just fired up using libvirt 1.2.15 just to try it
out. And it simply works.
<domain type='qemu'>
<name>dummy-tcg</name>
<uuid>946f637c-dcaa-4392-8926-b54476eddd43</uuid>
<memory unit='KiB'>131072</memory>
<currentMemory unit='KiB'>65536</currentMemory>
<vcpu placement='static' cpuset='3-5,7'>4</vcpu>
<os>
<type arch='x86_64' machine='pc-q35-2.3'>hvm</type>
<boot dev='hd'/>
</os>
<cpu>
<numa>
<cell id='0' cpus='0,2' memory='65536'
unit='KiB'/>
<cell id='1' cpus='1,3' memory='65536'
unit='KiB'/>
</numa>
</cpu>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-kvm</emulator>
<controller type='sata' index='0'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x1f' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1'
model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01'
slot='0x01' function='0x0'/>
</controller>
<memballoon model='none'/>
</devices>
</domain>
4:24 a.m.
On Mon, May 11, 2015 at 10:52:17AM +0200, Martin Kletzander wrote:
On Mon, May 11, 2015 at 10:52:46AM +1000, Tony Breeds wrote:
>
>Hello all,
> This is with reference to OpenStack (nova) bug 1439280 [1].
>
>The symptom is when nova(-compute) tries to launch an instance/VM it errors out
with:
> "libvirtError: Requested operation is not valid: cpu affinity is not
supported"
>This only happens with using qemu in TCG mode.
>
I can see where the error might have come from. QEMU doesn't have
vcpu threads if ran in TCG mode, so if libvirt tries to pin some of
them, this will happen. *But* upstream handles this perfectly. I
just tried and it works. I'll explain why a few lines down the road.
Okay, that's good to know.
>After looking at the domain XML and the docs at [2] It seems to
me the problem is either:
>1) the libvirt driver in nova is generating valid XML that is an invalid
> domain description ; or
>2) NUMA support in qemu (TCG) mode is broken.
>
I don't get what this has to do with NUMA, but anyway I just think
this is a bug in older libvirt actually.
The NUMA references are baddness on my part. They come from the fact that the
code in nova that does the CPUpinning is tied up with NUMA support. So with
that on mind I'd reword my summary as:
"CPU Pinning in qemu (TCG) mode is broken."
Yes, that's because we can't differentiate which threads do
what with
TCG accel. Because of the you must only specify one pinning for all
threads (I/O threads, CPU threads and the emulator thread), which is
done exactly using the 'cpuset' attribute as that is valid for the
whole machine. Thatnsk to that we don't have to differentiate
anything and just use the cpuset for the whole machine -> no error
should occur there.
Ok I think I follow that.
>Have I understood the documentation correctly? If so it would
seem that the
>correct fix is in nova to teach the libvirt driver to generate the correct XML
>for this virtulisation type.
>
You understood the documentation correctly, it was just libvirt acting
up a bit.
Okay that good to know.
So we can work around this in nova by either:
a) Check for libvirt 1.2.15 in the approriate TCG code path and elide the
cpuset ; or
b) including 'emulatorpin' in the cputune node
'a' is easy to do and is basically what I have proposwed in the bug. Having said
that
which is the better way to work around libvirt acting up a bit in older versions?
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like
a
lot of effort given the number of distros and libvirt versions potentiall
involved.
I'm still not sure why are you bringing NUMA into the mix as
there is
nothing NUMA related in the bug or this mail. But to complete my
answer, even though there is no practical benefit to using NUMA on TCG
accelerated QEMU machine, it should still work.
Okay so as I said above NUMA as such is a mistake. Thank tyou for confirming that
when the correct XML is generated the instance/guest/VM works just fine.
Yours Tony.
6:14 a.m.
On Mon, May 11, 2015 at 07:24:47PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 10:52:17AM +0200, Martin Kletzander wrote:
> On Mon, May 11, 2015 at 10:52:46AM +1000, Tony Breeds wrote:
> >
> >Hello all,
> > This is with reference to OpenStack (nova) bug 1439280 [1].
> >
> >The symptom is when nova(-compute) tries to launch an instance/VM it errors out
with:
> > "libvirtError: Requested operation is not valid: cpu affinity is not
supported"
> >This only happens with using qemu in TCG mode.
> >
>
> I can see where the error might have come from. QEMU doesn't have
> vcpu threads if ran in TCG mode, so if libvirt tries to pin some of
> them, this will happen. *But* upstream handles this perfectly. I
> just tried and it works. I'll explain why a few lines down the road.
Okay, that's good to know.
> >After looking at the domain XML and the docs at [2] It seems to me the problem
is either:
> >1) the libvirt driver in nova is generating valid XML that is an invalid
> > domain description ; or
> >2) NUMA support in qemu (TCG) mode is broken.
> >
>
> I don't get what this has to do with NUMA, but anyway I just think
> this is a bug in older libvirt actually.
The NUMA references are baddness on my part. They come from the fact that the
code in nova that does the CPUpinning is tied up with NUMA support. So with
that on mind I'd reword my summary as:
"CPU Pinning in qemu (TCG) mode is broken."
Oh, OK, I thought there is some connection I must've missed :)
> Yes, that's because we can't differentiate which threads
do what with
> TCG accel. Because of the you must only specify one pinning for all
> threads (I/O threads, CPU threads and the emulator thread), which is
> done exactly using the 'cpuset' attribute as that is valid for the
> whole machine. Thatnsk to that we don't have to differentiate
> anything and just use the cpuset for the whole machine -> no error
> should occur there.
Ok I think I follow that.
> >Have I understood the documentation correctly? If so it would seem that the
> >correct fix is in nova to teach the libvirt driver to generate the correct XML
> >for this virtulisation type.
> >
>
> You understood the documentation correctly, it was just libvirt acting
> up a bit.
Okay that good to know.
So we can work around this in nova by either:
a) Check for libvirt 1.2.15 in the approriate TCG code path and elide the
cpuset ; or
b) including 'emulatorpin' in the cputune node
'a' is easy to do and is basically what I have proposwed in the bug. Having said
that
which is the better way to work around libvirt acting up a bit in older versions?
Determining this by version might not be reliable, but more
importantly working around bug in underlying software is something
that shouldn't be done at all IMHO. Let the maintainers backport
whatever needs to be done.
Ther nova side will be pretty easy regardless.
I'd say the best solution would be to back port the 'fix' but that seems like
a
lot of effort given the number of distros and libvirt versions potentiall
involved.
If you want the fix to be distro-agnostic, there's nothing easier than
back-porting the fix into our upstream maintenance branches. Those
should make the life of distro maintainers easy (although it looks
like not many distros use it).
Having said that I'm not sure which commit(s) are those that need to
be back-ported. Having known your libvirt version, it shouldn't be
too hard looking for the differences and finding the right commit.
When back-porting request is made on the list, it is usually acted
upon. If you can't find the exact commit, let me know and I'll do my
best to help.
> I'm still not sure why are you bringing NUMA into the mix as
there is
> nothing NUMA related in the bug or this mail. But to complete my
> answer, even though there is no practical benefit to using NUMA on TCG
> accelerated QEMU machine, it should still work.
Okay so as I said above NUMA as such is a mistake. Thank tyou for confirming that
when the correct XML is generated the instance/guest/VM works just fine.
Yours Tony.
2:27 a.m.
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
Determining this by version might not be reliable, but more
importantly working around bug in underlying software is something
that shouldn't be done at all IMHO. Let the maintainers backport
whatever needs to be done.
I agree with you in an ideal world but there are times when we do need
to add work arounds in $project_x to work around issues in $project_y.
>Ther nova side will be pretty easy regardless.
>
>I'd say the best solution would be to back port the 'fix' but that seems
like a
>lot of effort given the number of distros and libvirt versions potentiall
>involved.
>
If you want the fix to be distro-agnostic, there's nothing easier than
back-porting the fix into our upstream maintenance branches. Those
should make the life of distro maintainers easy (although it looks
like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive
is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us
with doing an additional backport to 1.2.12 and getting the cloud-archive team
to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to
say for sure that some vendor isn't running libvirt 1.2.12 also.
Having said that I'm not sure which commit(s) are those that need
to
be back-ported. Having known your libvirt version, it shouldn't be
too hard looking for the differences and finding the right commit.
When back-porting request is made on the list, it is usually acted
upon. If you can't find the exact commit, let me know and I'll do my
best to help.
So a git bisect points at:
---
commit a103bb105c0c189c3973311ff1826972b5bc6ad6
Author: Daniel P. Berrange <berrange(a)redhat.com>
Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG
---
A small amount of reading implies to me that we'd be looking at backporting
a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains
b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I
could be wrong.
If you don't beat me to it I'll request that backport to 1.2.13 *and* ask the
Ubuntu guys to take it as well.
I have to admit I'm still in 2 minds on the nova side. Adding a wart to the
libvirt driver for this specific bug for distros/vendors that are using 1.2.12
seems a bit gross but ....
Daniel what do you think?
Yours Tony.
3:51 a.m.
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
> Determining this by version might not be reliable, but more
> importantly working around bug in underlying software is something
> that shouldn't be done at all IMHO. Let the maintainers backport
> whatever needs to be done.
I agree with you in an ideal world but there are times when we do need
to add work arounds in $project_x to work around issues in $project_y.
> >Ther nova side will be pretty easy regardless.
> >
> >I'd say the best solution would be to back port the 'fix' but that
seems like a
> >lot of effort given the number of distros and libvirt versions potentiall
> >involved.
> >
>
> If you want the fix to be distro-agnostic, there's nothing easier than
> back-porting the fix into our upstream maintenance branches. Those
> should make the life of distro maintainers easy (although it looks
> like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive
is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us
with doing an additional backport to 1.2.12 and getting the cloud-archive team
to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to
say for sure that some vendor isn't running libvirt 1.2.12 also.
> Having said that I'm not sure which commit(s) are those that need to
> be back-ported. Having known your libvirt version, it shouldn't be
> too hard looking for the differences and finding the right commit.
> When back-porting request is made on the list, it is usually acted
> upon. If you can't find the exact commit, let me know and I'll do my
> best to help.
So a git bisect points at:
---
commit a103bb105c0c189c3973311ff1826972b5bc6ad6
Author: Daniel P. Berrange <berrange(a)redhat.com>
Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG
---
A small amount of reading implies to me that we'd be looking at backporting
a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains
b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I
could be wrong.
If you don't beat me to it I'll request that backport to 1.2.13 *and* ask the
Ubuntu guys to take it as well.
I have to admit I'm still in 2 minds on the nova side. Adding a wart to the
libvirt driver for this specific bug for distros/vendors that are using 1.2.12
seems a bit gross but ....
I think this is something distros should really fix in their libvirt
packages - backporting this kind of fix is exactly the kind job of
maintainers should be doing.
Particularly since it only affects TCG mode, I don't think Nova really
should try to workaround it
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
4:28 a.m.
On Tue, May 12, 2015 at 09:51:24AM +0100, Daniel P. Berrange wrote:
I think this is something distros should really fix in their libvirt
packages - backporting this kind of fix is exactly the kind job of
maintainers should be doing.
Particularly since it only affects TCG mode, I don't think Nova really
should try to workaround it
Thanks Daniel.
Yours Tony.
4:14 a.m.
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
> Determining this by version might not be reliable, but more
> importantly working around bug in underlying software is something
> that shouldn't be done at all IMHO. Let the maintainers backport
> whatever needs to be done.
I agree with you in an ideal world but there are times when we do need
to add work arounds in $project_x to work around issues in $project_y.
> >Ther nova side will be pretty easy regardless.
> >
> >I'd say the best solution would be to back port the 'fix' but that
seems like a
> >lot of effort given the number of distros and libvirt versions potentiall
> >involved.
> >
>
> If you want the fix to be distro-agnostic, there's nothing easier than
> back-porting the fix into our upstream maintenance branches. Those
> should make the life of distro maintainers easy (although it looks
> like not many distros use it).
And this is part of the problem. If I understand correctly Ubuntu cloud-archive
is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us
with doing an additional backport to 1.2.12 and getting the cloud-archive team
to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard to
say for sure that some vendor isn't running libvirt 1.2.12 also.
> Having said that I'm not sure which commit(s) are those that need to
> be back-ported. Having known your libvirt version, it shouldn't be
> too hard looking for the differences and finding the right commit.
> When back-porting request is made on the list, it is usually acted
> upon. If you can't find the exact commit, let me know and I'll do my
> best to help.
So a git bisect points at:
---
commit a103bb105c0c189c3973311ff1826972b5bc6ad6
Author: Daniel P. Berrange <berrange(a)redhat.com>
Date: Tue Feb 10 15:59:57 2015 +0000
qemu: fix setting of VM CPU affinity with TCG
---
A small amount of reading implies to me that we'd be looking at backporting
a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains
b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I
could be wrong.
1.2.13 has the commit already in the release and 1.2.12-maint has it
as a first back-port right after release. The problem is that there
was no maintenance release of 1.2.12 yet. Maybe they would use
1.2.12.1 if it existed.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some
insights. @Guido: would it help if we created a maintenance release
from the v1.2.12-maint branch? Or is the only thing missing the fact
that the launchpad bug is not moved to libvirt?
If you don't beat me to it I'll request that backport to
1.2.13 *and* ask the
Ubuntu guys to take it as well.
I have to admit I'm still in 2 minds on the nova side. Adding a wart to the
libvirt driver for this specific bug for distros/vendors that are using 1.2.12
seems a bit gross but ....
Daniel what do you think?
Yours Tony.
4:27 a.m.
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
1.2.13 has the commit already in the release and 1.2.12-maint has it
as a first back-port right after release. The problem is that there
was no maintenance release of 1.2.12 yet. Maybe they would use
1.2.12.1 if it existed.
Perhaps. At this point we're well and truly in realms I can't influence.
I thank you for you help keeping me on track here. I think by and large
the issue is preety much solved (and was before I started this thread
a couple of days ago.
I Cc'd Guido as an upstream debian maintainer, maybe he'll
have some
insights. @Guido: would it help if we created a maintenance release
from the v1.2.12-maint branch? Or is the only thing missing the fact
that the launchpad bug is not moved to libvirt?
I just added:
---
I was pointed at the v1.2.12-maint head in the libvirt git which contains this fix
already.
http://libvirt.org/git/?p=libvirt.git;a=shortlog;h=refs/heads/v1.2.12-maint
I suggest we close the nova issue with won't fix and get the correctly backported
patch into the libvirt package.
---
https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1439280/comments/41
It's up to the Ubuntu team now.
Yours Tony.
11:46 a.m.
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
>On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
>
>>Determining this by version might not be reliable, but more
>>importantly working around bug in underlying software is something
>>that shouldn't be done at all IMHO. Let the maintainers backport
>>whatever needs to be done.
>
>I agree with you in an ideal world but there are times when we do need
>to add work arounds in $project_x to work around issues in $project_y.
>
>>>Ther nova side will be pretty easy regardless.
>>>
>>>I'd say the best solution would be to back port the 'fix' but
that seems like a
>>>lot of effort given the number of distros and libvirt versions potentiall
>>>involved.
>>>
>>
>>If you want the fix to be distro-agnostic, there's nothing easier than
>>back-porting the fix into our upstream maintenance branches. Those
>>should make the life of distro maintainers easy (although it looks
>>like not many distros use it).
>
>And this is part of the problem. If I understand correctly Ubuntu cloud-archive
>is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us
>with doing an additional backport to 1.2.12 and getting the cloud-archive team
>to take it[1] or Adding a hack to nova. And that's just Ubuntu It's hard
to
>say for sure that some vendor isn't running libvirt 1.2.12 also.
>
>>Having said that I'm not sure which commit(s) are those that need to
>>be back-ported. Having known your libvirt version, it shouldn't be
>>too hard looking for the differences and finding the right commit.
>>When back-porting request is made on the list, it is usually acted
>>upon. If you can't find the exact commit, let me know and I'll do my
>>best to help.
>
>So a git bisect points at:
>---
>commit a103bb105c0c189c3973311ff1826972b5bc6ad6
>Author: Daniel P. Berrange <berrange(a)redhat.com>
>Date: Tue Feb 10 15:59:57 2015 +0000
>
> qemu: fix setting of VM CPU affinity with TCG
>---
>
>A small amount of reading implies to me that we'd be looking at backporting
>a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that contains
>b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I
>could be wrong.
>
1.2.13 has the commit already in the release and 1.2.12-maint has it
as a first back-port right after release. The problem is that there
was no maintenance release of 1.2.12 yet. Maybe they would use
1.2.12.1 if it existed.
I Cc'd Guido as an upstream debian maintainer, maybe he'll have some
insights. @Guido: would it help if we created a maintenance release
from the v1.2.12-maint branch? Or is the only thing missing the fact
that the launchpad bug is not moved to libvirt?
Basically Ubuntu takes the version that is in Debian testing, unstable
or experimental at time of their release and adds pathes. There's little
to no sync unfortunately (except for the awesome efforts to sync the
apparmor stuff)
Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 +
maintenance patches (which I intend to switch over to the maintenance
releases soonish and support hopefully Cole with these), oldstable has
0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the
releases).
I've added Serge since he might to jump onto 1.2.12.1 maintenance
relasese.
I'm happy about any additional notifications for things that should go
into distributions.
Cheers,
-- Guido
10:31 a.m.
Quoting Guido Günther (agx(a)sigxcpu.org):
On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
> On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
> >On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
> >
> >>Determining this by version might not be reliable, but more
> >>importantly working around bug in underlying software is something
> >>that shouldn't be done at all IMHO. Let the maintainers backport
> >>whatever needs to be done.
> >
> >I agree with you in an ideal world but there are times when we do need
> >to add work arounds in $project_x to work around issues in $project_y.
> >
> >>>Ther nova side will be pretty easy regardless.
> >>>
> >>>I'd say the best solution would be to back port the 'fix'
but that seems like a
> >>>lot of effort given the number of distros and libvirt versions
potentiall
> >>>involved.
> >>>
> >>
> >>If you want the fix to be distro-agnostic, there's nothing easier than
> >>back-porting the fix into our upstream maintenance branches. Those
> >>should make the life of distro maintainers easy (although it looks
> >>like not many distros use it).
> >
> >And this is part of the problem. If I understand correctly Ubuntu
cloud-archive
> >is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves us
> >with doing an additional backport to 1.2.12 and getting the cloud-archive team
> >to take it[1] or Adding a hack to nova. And that's just Ubuntu It's
hard to
> >say for sure that some vendor isn't running libvirt 1.2.12 also.
> >
> >>Having said that I'm not sure which commit(s) are those that need to
> >>be back-ported. Having known your libvirt version, it shouldn't be
> >>too hard looking for the differences and finding the right commit.
> >>When back-porting request is made on the list, it is usually acted
> >>upon. If you can't find the exact commit, let me know and I'll do
my
> >>best to help.
> >
> >So a git bisect points at:
> >---
> >commit a103bb105c0c189c3973311ff1826972b5bc6ad6
> >Author: Daniel P. Berrange <berrange(a)redhat.com>
> >Date: Tue Feb 10 15:59:57 2015 +0000
> >
> > qemu: fix setting of VM CPU affinity with TCG
> >---
> >
> >A small amount of reading implies to me that we'd be looking at backporting
> >a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that
contains
> >b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but I
> >could be wrong.
> >
>
> 1.2.13 has the commit already in the release and 1.2.12-maint has it
> as a first back-port right after release. The problem is that there
> was no maintenance release of 1.2.12 yet. Maybe they would use
> 1.2.12.1 if it existed.
>
> I Cc'd Guido as an upstream debian maintainer, maybe he'll have some
> insights. @Guido: would it help if we created a maintenance release
> from the v1.2.12-maint branch? Or is the only thing missing the fact
> that the launchpad bug is not moved to libvirt?
Thanks guys, I'm going to cherrypick that patch into vivid right now. (It
should be in wily as that is 1.2.15).
Basically Ubuntu takes the version that is in Debian testing,
unstable
or experimental at time of their release and adds pathes. There's little
to no sync unfortunately (except for the awesome efforts to sync the
apparmor stuff)
Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 +
maintenance patches (which I intend to switch over to the maintenance
releases soonish and support hopefully Cole with these), oldstable has
0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the
releases).
I've added Serge since he might to jump onto 1.2.12.1 maintenance
relasese.
If you mean contributing by adding patches that we add to vivid, I'm
definately willing to do that. Note that vivid has a pretty short
lifecycle so it wouldn't go on for very long. If people are willing
to tag patches with '[1.2.12.1-maint]' on this list I'm willing to
support this longer.
I'm happy about any additional notifications for things that
should go
into distributions.
Cheers,
-- Guido
Thanks!
-serge
1:58 a.m.
On Wed, May 13, 2015 at 03:31:09PM +0000, Serge Hallyn wrote:
Quoting Guido Günther (agx(a)sigxcpu.org):
> On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
> > On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
> > >On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
> > >
> > >>Determining this by version might not be reliable, but more
> > >>importantly working around bug in underlying software is something
> > >>that shouldn't be done at all IMHO. Let the maintainers backport
> > >>whatever needs to be done.
> > >
> > >I agree with you in an ideal world but there are times when we do need
> > >to add work arounds in $project_x to work around issues in $project_y.
> > >
> > >>>Ther nova side will be pretty easy regardless.
> > >>>
> > >>>I'd say the best solution would be to back port the
'fix' but that seems like a
> > >>>lot of effort given the number of distros and libvirt versions
potentiall
> > >>>involved.
> > >>>
> > >>
> > >>If you want the fix to be distro-agnostic, there's nothing easier
than
> > >>back-porting the fix into our upstream maintenance branches. Those
> > >>should make the life of distro maintainers easy (although it looks
> > >>like not many distros use it).
> > >
> > >And this is part of the problem. If I understand correctly Ubuntu
cloud-archive
> > >is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves
us
> > >with doing an additional backport to 1.2.12 and getting the cloud-archive
team
> > >to take it[1] or Adding a hack to nova. And that's just Ubuntu
It's hard to
> > >say for sure that some vendor isn't running libvirt 1.2.12 also.
> > >
> > >>Having said that I'm not sure which commit(s) are those that need
to
> > >>be back-ported. Having known your libvirt version, it shouldn't be
> > >>too hard looking for the differences and finding the right commit.
> > >>When back-porting request is made on the list, it is usually acted
> > >>upon. If you can't find the exact commit, let me know and I'll
do my
> > >>best to help.
> > >
> > >So a git bisect points at:
> > >---
> > >commit a103bb105c0c189c3973311ff1826972b5bc6ad6
> > >Author: Daniel P. Berrange <berrange(a)redhat.com>
> > >Date: Tue Feb 10 15:59:57 2015 +0000
> > >
> > > qemu: fix setting of VM CPU affinity with TCG
> > >---
> > >
> > >A small amount of reading implies to me that we'd be looking at
backporting
> > >a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that
contains
> > >b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only, but
I
> > >could be wrong.
> > >
> >
> > 1.2.13 has the commit already in the release and 1.2.12-maint has it
> > as a first back-port right after release. The problem is that there
> > was no maintenance release of 1.2.12 yet. Maybe they would use
> > 1.2.12.1 if it existed.
> >
> > I Cc'd Guido as an upstream debian maintainer, maybe he'll have some
> > insights. @Guido: would it help if we created a maintenance release
> > from the v1.2.12-maint branch? Or is the only thing missing the fact
> > that the launchpad bug is not moved to libvirt?
Thanks guys, I'm going to cherrypick that patch into vivid right now. (It
should be in wily as that is 1.2.15).
> Basically Ubuntu takes the version that is in Debian testing, unstable
> or experimental at time of their release and adds pathes. There's little
> to no sync unfortunately (except for the awesome efforts to sync the
> apparmor stuff)
>
> Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 +
> maintenance patches (which I intend to switch over to the maintenance
> releases soonish and support hopefully Cole with these), oldstable has
> 0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the
> releases).
>
> I've added Serge since he might to jump onto 1.2.12.1 maintenance
> relasese.
If you mean contributing by adding patches that we add to vivid, I'm
definately willing to do that. Note that vivid has a pretty short
lifecycle so it wouldn't go on for very long. If people are willing
to tag patches with '[1.2.12.1-maint]' on this list I'm willing to
support this longer.
Rather than tagging patches with -maint, we just back-port them
straight into the maintenance branch. If there are more distributions
using the maintenance branch or would be if there was more review,
etc., I'm sure we could come up with some solution to suit everyone
(or majority at least).
After patches are back-ported, Cole does a maintenance release which
should make it even more easier (e.g. you don't want to follow up the
list or the git repo) since the release announcement goes to
libvirt-announce as with other releases.
However, I'm not sure when Cole does those releases (if it's time-,
commit- or random-based). Anyway, as I said earlier, we should
definitely come up with how to do this so it helps other maintainers
out there. I'm offering my help.
> I'm happy about any additional notifications for things that
should go
> into distributions.
>
> Cheers,
> -- Guido
>
Thanks!
-serge
2:01 a.m.
[Adding Cole to Cc as I forgot to do that before sending the mail]
On Thu, May 14, 2015 at 08:58:03AM +0200, Martin Kletzander wrote:
On Wed, May 13, 2015 at 03:31:09PM +0000, Serge Hallyn wrote:
>Quoting Guido Günther (agx(a)sigxcpu.org):
>>On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
>>> On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
>>> >On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
>>> >
>>> >>Determining this by version might not be reliable, but more
>>> >>importantly working around bug in underlying software is something
>>> >>that shouldn't be done at all IMHO. Let the maintainers
backport
>>> >>whatever needs to be done.
>>> >
>>> >I agree with you in an ideal world but there are times when we do need
>>> >to add work arounds in $project_x to work around issues in $project_y.
>>> >
>>> >>>Ther nova side will be pretty easy regardless.
>>> >>>
>>> >>>I'd say the best solution would be to back port the
'fix' but that seems like a
>>> >>>lot of effort given the number of distros and libvirt versions
potentiall
>>> >>>involved.
>>> >>>
>>> >>
>>> >>If you want the fix to be distro-agnostic, there's nothing easier
than
>>> >>back-porting the fix into our upstream maintenance branches. Those
>>> >>should make the life of distro maintainers easy (although it looks
>>> >>like not many distros use it).
>>> >
>>> >And this is part of the problem. If I understand correctly Ubuntu
cloud-archive
>>> >is using libvirt 1.2.12 which is *NOT* a maintenance release so that
leaves us
>>> >with doing an additional backport to 1.2.12 and getting the cloud-archive
team
>>> >to take it[1] or Adding a hack to nova. And that's just Ubuntu
It's hard to
>>> >say for sure that some vendor isn't running libvirt 1.2.12 also.
>>> >
>>> >>Having said that I'm not sure which commit(s) are those that need
to
>>> >>be back-ported. Having known your libvirt version, it shouldn't
be
>>> >>too hard looking for the differences and finding the right commit.
>>> >>When back-porting request is made on the list, it is usually acted
>>> >>upon. If you can't find the exact commit, let me know and
I'll do my
>>> >>best to help.
>>> >
>>> >So a git bisect points at:
>>> >---
>>> >commit a103bb105c0c189c3973311ff1826972b5bc6ad6
>>> >Author: Daniel P. Berrange <berrange(a)redhat.com>
>>> >Date: Tue Feb 10 15:59:57 2015 +0000
>>> >
>>> > qemu: fix setting of VM CPU affinity with TCG
>>> >---
>>> >
>>> >A small amount of reading implies to me that we'd be looking at
backporting
>>> >a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that
contains
>>> >b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only,
but I
>>> >could be wrong.
>>> >
>>>
>>> 1.2.13 has the commit already in the release and 1.2.12-maint has it
>>> as a first back-port right after release. The problem is that there
>>> was no maintenance release of 1.2.12 yet. Maybe they would use
>>> 1.2.12.1 if it existed.
>>>
>>> I Cc'd Guido as an upstream debian maintainer, maybe he'll have some
>>> insights. @Guido: would it help if we created a maintenance release
>>> from the v1.2.12-maint branch? Or is the only thing missing the fact
>>> that the launchpad bug is not moved to libvirt?
>
>Thanks guys, I'm going to cherrypick that patch into vivid right now. (It
>should be in wily as that is 1.2.15).
>
>>Basically Ubuntu takes the version that is in Debian testing, unstable
>>or experimental at time of their release and adds pathes. There's little
>>to no sync unfortunately (except for the awesome efforts to sync the
>>apparmor stuff)
>>
>>Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 +
>>maintenance patches (which I intend to switch over to the maintenance
>>releases soonish and support hopefully Cole with these), oldstable has
>>0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the
>>releases).
>>
>>I've added Serge since he might to jump onto 1.2.12.1 maintenance
>>relasese.
>
>If you mean contributing by adding patches that we add to vivid, I'm
>definately willing to do that. Note that vivid has a pretty short
>lifecycle so it wouldn't go on for very long. If people are willing
>to tag patches with '[1.2.12.1-maint]' on this list I'm willing to
>support this longer.
>
Rather than tagging patches with -maint, we just back-port them
straight into the maintenance branch. If there are more distributions
using the maintenance branch or would be if there was more review,
etc., I'm sure we could come up with some solution to suit everyone
(or majority at least).
After patches are back-ported, Cole does a maintenance release which
should make it even more easier (e.g. you don't want to follow up the
list or the git repo) since the release announcement goes to
libvirt-announce as with other releases.
However, I'm not sure when Cole does those releases (if it's time-,
commit- or random-based). Anyway, as I said earlier, we should
definitely come up with how to do this so it helps other maintainers
out there. I'm offering my help.
>>I'm happy about any additional notifications for things that should go
>>into distributions.
>>
>>Cheers,
>> -- Guido
>>
>
>Thanks!
>
>-serge
--
libvir-list mailing list
libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
2:21 p.m.
Hi,
On Wed, May 13, 2015 at 03:31:09PM +0000, Serge Hallyn wrote:
Quoting Guido Günther (agx(a)sigxcpu.org):
> On Tue, May 12, 2015 at 11:14:09AM +0200, Martin Kletzander wrote:
> > On Tue, May 12, 2015 at 05:27:34PM +1000, Tony Breeds wrote:
> > >On Mon, May 11, 2015 at 01:14:58PM +0200, Martin Kletzander wrote:
> > >
> > >>Determining this by version might not be reliable, but more
> > >>importantly working around bug in underlying software is something
> > >>that shouldn't be done at all IMHO. Let the maintainers backport
> > >>whatever needs to be done.
> > >
> > >I agree with you in an ideal world but there are times when we do need
> > >to add work arounds in $project_x to work around issues in $project_y.
> > >
> > >>>Ther nova side will be pretty easy regardless.
> > >>>
> > >>>I'd say the best solution would be to back port the
'fix' but that seems like a
> > >>>lot of effort given the number of distros and libvirt versions
potentiall
> > >>>involved.
> > >>>
> > >>
> > >>If you want the fix to be distro-agnostic, there's nothing easier
than
> > >>back-porting the fix into our upstream maintenance branches. Those
> > >>should make the life of distro maintainers easy (although it looks
> > >>like not many distros use it).
> > >
> > >And this is part of the problem. If I understand correctly Ubuntu
cloud-archive
> > >is using libvirt 1.2.12 which is *NOT* a maintenance release so that leaves
us
> > >with doing an additional backport to 1.2.12 and getting the cloud-archive
team
> > >to take it[1] or Adding a hack to nova. And that's just Ubuntu
It's hard to
> > >say for sure that some vendor isn't running libvirt 1.2.12 also.
> > >
> > >>Having said that I'm not sure which commit(s) are those that need
to
> > >>be back-ported. Having known your libvirt version, it shouldn't
be
> > >>too hard looking for the differences and finding the right commit.
> > >>When back-porting request is made on the list, it is usually acted
> > >>upon. If you can't find the exact commit, let me know and I'll
do my
> > >>best to help.
> > >
> > >So a git bisect points at:
> > >---
> > >commit a103bb105c0c189c3973311ff1826972b5bc6ad6
> > >Author: Daniel P. Berrange <berrange(a)redhat.com>
> > >Date: Tue Feb 10 15:59:57 2015 +0000
> > >
> > > qemu: fix setting of VM CPU affinity with TCG
> > >---
> > >
> > >A small amount of reading implies to me that we'd be looking at
backporting
> > >a103bb105c0c189c3973311ff1826972b5bc6ad6 to any maintenance branch that
contains
> > >b07f3d821dfb11a118ee75ea275fd6ab737d9500. Which I think is 1.2.13 only,
but I
> > >could be wrong.
> > >
> >
> > 1.2.13 has the commit already in the release and 1.2.12-maint has it
> > as a first back-port right after release. The problem is that there
> > was no maintenance release of 1.2.12 yet. Maybe they would use
> > 1.2.12.1 if it existed.
> >
> > I Cc'd Guido as an upstream debian maintainer, maybe he'll have some
> > insights. @Guido: would it help if we created a maintenance release
> > from the v1.2.12-maint branch? Or is the only thing missing the fact
> > that the launchpad bug is not moved to libvirt?
Thanks guys, I'm going to cherrypick that patch into vivid right now. (It
should be in wily as that is 1.2.15).
> Basically Ubuntu takes the version that is in Debian testing, unstable
> or experimental at time of their release and adds pathes. There's little
> to no sync unfortunately (except for the awesome efforts to sync the
> apparmor stuff)
>
> Debian itself is not using 1.2.12 anywhere. Current stable has 1.2.9 +
> maintenance patches (which I intend to switch over to the maintenance
> releases soonish and support hopefully Cole with these), oldstable has
> 0.9.12.3 and unstable/sid has 1.2.15 (and will keep following the
> releases).
>
> I've added Serge since he might to jump onto 1.2.12.1 maintenance
> relasese.
If you mean contributing by adding patches that we add to vivid, I'm
definately willing to do that. Note that vivid has a pretty short
lifecycle so it wouldn't go on for very long. If people are willing
to tag patches with '[1.2.12.1-maint]' on this list I'm willing to
support this longer.
I rather meant that you might be interested to follow a v1.2.12-maint
branch. Maintaining it is even better ;)
Cheers,
-- Guido
3476
days inactive
3483
days old
13 comments
5 participants
participants (5)
-
Daniel P. Berrange -
Guido Günther -
Martin Kletzander -
Serge Hallyn -
Tony Breeds