- Devel - libvirt List Archives

[libvirt] [PATCH v2 0/9] Few VFIO related fixes
by Shivaprasad G Bhat 02 Nov '15

02 Nov '15

The following series fixes few VFIO related host crashes during hotplug/unplug or bind/unbind. --- Shivaprasad G Bhat (9): Implement virPCIIsAKnownStub function Initialize the stubDriver of pci devices if bound to a valid one Add iommu group number info to virPCIDevice Refuse to reattach from vfio if the iommu group is in use by any domain Wait for vfio-pci device cleanups before reassinging the device to host driver Split reprobe action from the virPCIUnbindFromStub into a new function Pass activeDevs and inactiveDevs to virPCIDeviceUnbindFromStub and virPCIDeviceBindToStub Postpone reprobing till all the devices in iommu group are unbound from vfio Wait for iommmu device to go away before reprobing the host driver src/libvirt_private.syms | 1 src/util/virhostdev.c | 18 +++ src/util/virpci.c | 289 ++++++++++++++++++++++++++++++++++++++-------- src/util/virpci.h | 2 tests/virpcimock.c | 187 +++++++++++++++++++++++++++--- tests/virpcitest.c | 152 ++++++++++++++++++++---- 6 files changed, 555 insertions(+), 94 deletions(-) -- Signature

3 14

[libvirt] Happy Birthday Libvirt
by Daniel Veillard 02 Nov '15

02 Nov '15

Hello everybody, as the first commits were done on Nov 2 2005, libvirt is more or less ten years old. This has been quite a bit of a journey, but a very successful one in my opinion, the initial goal was to provide a long term stable API for virtualization in the Red Hat ecosystem, and I'm very happy to see the goal has been reached, and we went way beyond that. In the end I think libvirt helped avoid the "balkanization" which is classic in highly competitive domains where you end up with one vertical set of tools per vendor. I also think that libvirt played a key role to keep free and open source software in the main virtualization offerings and even more so for cloud management systems. So it's time to celebrate, there is obviously lot to be done still on libvirt (as the steady stream of patches show !), but I would like to thank everybody who has contributed to the project so far, with a special mention to Dan Berrange who has been leading the effort technically for most of that time. I would also say that I have been very pelased with how the project has been working in that decade, the community is technical but friendly, and it's my hope libvirt, the project as well as the code, will remain as successful for the next decade ! So Thank You everybody, for making this project and community successful ! Enjoy that very special day :-) Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0

[libvirt] [PATCH 0/8] Few vfio related fixes
by Shivaprasad G Bhat 02 Nov '15

02 Nov '15

The series fixes 3 issues which lead to host crash. The actual fixes are in p3, p4, p7 and p8. I'll fill in more details / descriptions in v2. The make check fails on virpcitest now, which I am still debugging. I anticipate some rework from review on p7 which I continue to refine. Sending now, to get any feedback that might change things drastically. --- Shivaprasad G Bhat (8): Initialize the stubDriver of pci devices if bound to a valid one Add iommu group number info to virPCIDevice Refuse to reattach from vfio if the iommu group is in use by any domain Wait for vfio-pci device cleanups before reassinging the device to host driver Split reprobe action from the virPCIUnbindFromStub into a new function Pass activeDevs and inactiveDevs to virPCIDeviceUnbindFromStub and virPCIDeviceBindToStub Postpone reprobing till all the devices in iommu group are unbound from vfio Wait for iommmu device to go away before reprobing the host driver src/util/virhostdev.c | 18 ++++ src/util/virpci.c | 206 ++++++++++++++++++++++++++++++++++++++----------- 2 files changed, 177 insertions(+), 47 deletions(-) -- Signature

4 19

[libvirt] Entering freeze for libvirt-1.2.21
by Daniel Veillard 02 Nov '15

02 Nov '15

As pointed our on Tuesday it's time for a new release. I have tagged the release candidate 1 in git and pushed signed tarball and rpms to the usual place at: ftp://libvirt.org/libvirt/ Based on my limited testing this works just fine, but that's very limited and doesn't test portability at all, so please give it a try ! I will likely push rc2 on the week-end and the final version on Tuesday or Wednesday, please raise issues if you fine any, thanks, Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 5

[libvirt] Outreachy 2015 DecemberMarch- Enhancing libvirt-designer
by Riyafa Abdul Hameed 01 Nov '15

01 Nov '15

Dear sir, I am Computer Science and Engineering undergraduate at University of Moratuwa, Sri Lanka. I am a keen learner and am much interested in open source. I have worked with WSO2 <http://wso2.com/> on open source projects and am quite familiar with FOSS development. I have a good knowledge in C and in using XML. I would like to participate in "Outreachy 2015 DecemberMarch" with QEMU, but it was already quite late when I came to know of this program as its application deadline closes by tomorrow. I would like to apply for the project "Enhancing libvirt-designer" this time for Outreachy. I sent a mail to the mentor, Michal Privoznik and I couldn't get in touch with him. Because it is already too late can anyone please suggest what I can do before I apply for the program by tomorrow? Thank you. Yours sincerely, Riyafa -- Riyafa Abdul Hameed Undergraduate, University of Moratuwa Email: riyafa3(a)gmail.com Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa> <http://twitter.com/Riyafa1>

1 0

[libvirt] [PATCH 1/3] cpu_map.xml: add cmt feature to x86
by Qiaowei Ren 31 Oct '15

31 Oct '15

Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family) introduced CMT (Cache Monitoring Technology) to measure the usage of cache by applications running on the platform. This patch add it into x86 part of cpu_map.xml. Signed-off-by: Qiaowei Ren <qiaowei.ren(a)intel.com> --- .gnulib | 2 +- src/cpu/cpu_map.xml | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/.gnulib b/.gnulib index f39477d..106a386 160000 --- a/.gnulib +++ b/.gnulib @@ -1 +1 @@ -Subproject commit f39477dba778e99392948dd3dd19ec0d46aee932 +Subproject commit 106a3866d01f9dd57ab4f10dbeb0d5a8db73a9f7 diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml index b9e95cf..14ccbd8 100644 --- a/src/cpu/cpu_map.xml +++ b/src/cpu/cpu_map.xml @@ -317,6 +317,9 @@ <feature name='rtm'> <cpuid function='0x00000007' ebx='0x00000800'/> </feature> + <feature name='cmt'> + <cpuid function='0x00000007' ebx='0x00001000'/> + </feature> <feature name='rdseed'> <cpuid function='0x00000007' ebx='0x00040000'/> </feature> -- 1.9.1

5 10

[libvirt] [python PATCH] fix crash introduced by commit 1d39dbaf
by Pavel Hrdina 31 Oct '15

31 Oct '15

Some of the libvirt_*Wrap functions steals the reference and we need to set the item in array to NULL no not free it on success. Those three places was accidentally removed by commit 1d39dbaf. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1270977 Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- Pushed as fix for critical crasher. libvirt-override.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/libvirt-override.c b/libvirt-override.c index c6c1da8..5f1c98f 100644 --- a/libvirt-override.c +++ b/libvirt-override.c @@ -2448,6 +2448,7 @@ libvirt_virDomainListAllSnapshots(PyObject *self ATTRIBUTE_UNUSED, for (i = 0; i < c_retval; i++) { VIR_PY_LIST_SET_GOTO(py_retval, i, libvirt_virDomainSnapshotPtrWrap(snaps[i]), error); + snaps[i] = NULL; } cleanup: @@ -2544,9 +2545,11 @@ libvirt_virDomainSnapshotListAllChildren(PyObject *self ATTRIBUTE_UNUSED, if (!(py_retval = PyList_New(c_retval))) goto cleanup; - for (i = 0; i < c_retval; i++) + for (i = 0; i < c_retval; i++) { VIR_PY_LIST_SET_GOTO(py_retval, i, libvirt_virDomainSnapshotPtrWrap(snaps[i]), error); + snaps[i] = NULL; + } cleanup: for (i = 0; i < c_retval; i++) @@ -3117,9 +3120,11 @@ libvirt_virConnectListAllNetworks(PyObject *self ATTRIBUTE_UNUSED, if (!(py_retval = PyList_New(c_retval))) goto cleanup; - for (i = 0; i < c_retval; i++) + for (i = 0; i < c_retval; i++) { VIR_PY_LIST_SET_GOTO(py_retval, i, libvirt_virNetworkPtrWrap(nets[i]), error); + nets[i] = NULL; + } cleanup: for (i = 0; i < c_retval; i++) -- 2.6.2

1 0

[libvirt] RFC: Improve performance of macvtap device creation
by Tony Krowiak 30 Oct '15

30 Oct '15

For a guest domain defined with a large number of macvtap devices, it takes an exceedingly long time to boot the guest. In a test of a guest domain configured with 82 macvtap devices, it took over two minutes for the guest to boot. An strace of the ioctl calls during guest start up showed the SIOCGIFFLAGS ioctl literally being invoked 3,403 times. I was able to isolate the source of the ioctl calls to the*virNetDevMacVLanCreateWithVPortProfile* function in*virnetdevmacvlan.c*. The macvtap interface name is created by looping over a counter variable, starting with zero, and appending the counter value to 'macvtap'. With each iteration, a call is made to*virNetDevExists* (SIOCGIFFLAGS ioctl) to determine if a device with that name already exists, until a unique name is created. In the test case cited above, to create an interface name for the 82nd macvtap device, the*virNetDevExists* function will be called for interface names 'macvtap0' to 'macvtap80' before it is determined that 'mavtap81' can be used. So if N is the number of macvtap interfaces defined for a guest, the SIOCGIFFLAGS ioctl will be invoked (N x N + N)/2 times to find an unused macvtap device names. That's assuming only one guest is being started, who knows how many times the ioctl may have to be called in an installation running a large number of guests defined with macvtap devices. I was able to reduce the amount of time for starting a guest domain defined with 82 macvtap devices from over 2 minutes to about 14 seconds by keeping track of the interface name suffixes previously used. I defined two static bit maps (virBitmap), one each for macvtap and macvlan device name suffixes. When a macvtap/macvlan device is created, the index of the next clear bit (virBitmapNextClearBit) is retrieved to create the name. If an interface with that name does not exist, the device is created and the bit at the index used to create the interface name is set (virBitmapSetBit). When a macvtap/macvlan device is deleted, if the interface name has the pattern 'macvtap%d' or 'macvlan%d', the suffix is parsed into a bit index and used to clear the (virBitMapClearBit) bit in the respective bitmap. I am not sure that is the best design because there is no way to track interface names used to create macvtap devices outside of libvirt, for example using the ip command. There may also be other issues I've not contemplated. I included a couple of additional ideas below and am looking for comments or other suggestions that I have not considered. * Define a global counter variable initialized to 0, that gets incremented each time an interface name is created, to keep track of the last used interface name suffix. At some maximum value, the counter will be set back to 0. * Append a random number to 'macvlan' or 'macvtap' when creating the interface name. Of course, the number of digits would have to be limited so the interface name would not exceed the maximum allowed. * Create the interface name in code that has more knowledge of the environment and pass the name into the *virNetDevMacVLanCreateWithVPortProfile* function via the *tgifname* parameter. For example, the *qemuBuildCommandLine* function in *qemu_command.c* contains the loop that iterates over the network devices defined for the guest domain that ultimately get created via the *virNetDevMacVLanCreateWithVPortProfile* function. That function has access to the network device configuration and at the very least could ensure none of the names previously defined for the guest aren't used. I believe it would be matter of creating a macvtap interface name - e.g., maybe a call to some function in *virnetdevmacvlan.c* - and setting the name in the virDomainNetDef structure prior to invoking *qemuBuildInterfaceCommandLine*? There are shortcomings in all of these ideas, so if you have a better one, feel free to present it.

4 7

[libvirt] RAM backend and guest ABI (was Re: [Qemu-devel] [PATCH v2] pc: memhp: enforce minimal 128Mb) alignment for pc-dimm
by Eduardo Habkost 30 Oct '15

30 Oct '15

(CCing Michal and libvir-list, so libvirt team is aware of this restriction) On Thu, Oct 29, 2015 at 02:36:37PM +0100, Igor Mammedov wrote: > On Tue, 27 Oct 2015 14:36:35 -0200 > Eduardo Habkost <ehabkost(a)redhat.com> wrote: > > > On Tue, Oct 27, 2015 at 10:14:56AM +0100, Igor Mammedov wrote: > > > On Tue, 27 Oct 2015 10:53:08 +0200 > > > "Michael S. Tsirkin" <mst(a)redhat.com> wrote: > > > > > > > On Tue, Oct 27, 2015 at 09:48:37AM +0100, Igor Mammedov wrote: > > > > > On Tue, 27 Oct 2015 10:31:21 +0200 > > > > > "Michael S. Tsirkin" <mst(a)redhat.com> wrote: > > > > > > > > > > > On Mon, Oct 26, 2015 at 02:24:32PM +0100, Igor Mammedov wrote: > > > > > > > Yep it's workaround but it works around QEMU's broken virtio > > > > > > > implementation in a simple way without need for guest side changes. > > > > > > > > > > > > > > Without foreseeable virtio fix it makes memory hotplug unusable and even > > > > > > > more so if there were a virtio fix it won't fix old guests since you've > > > > > > > said that virtio fix would require changes of both QEMU and guest sides. > > > > > > > > > > > > What makes it not foreseeable? > > > > > > Apparently only the fact that we have a work-around in place so no one > > > > > > works on it. I can code it up pretty quickly, but I'm flat out of time > > > > > > for testing as I'm going on vacation soon, and hard freeze is pretty > > > > > > close. > > > > > I can lend a hand for testing part. > > > > > > > > > > > > > > > > > GPA space is kind of cheap, but wasting it in chunks of 512M > > > > > > seems way too aggressive. > > > > > hotplug region is sized with 1Gb alignment reserve per DIMM so we aren't > > > > > actually wasting anything here. > > > > > > > > > > > > > If I allocate two 1G DIMMs, what will be the gap size? 512M? 1G? > > > > It's too much either way. > > > minimum would be 512, and if backend is 1Gb-hugepage gap will be > > > backend's natural alignment (i.e. 1Gb). > > > > Is backend configuration even allowed to affect the machine ABI? We need > > to be able to change backend configuration when migrating the VM to > > another host. > for now, one has to use the same type of backend on both sides > i.e. if source uses 1Gb huge pages backend then target also > need to use it. > The page size of the backend don't even depend on QEMU arguments, but on the kernel command-line or hugetlbfs mount options. So it's possible to have exactly the same QEMU command-line on source and destination (with an explicit versioned machine-type), and get a VM that can't be migrated? That means we are breaking our guarantees about migration and guest ABI. > We could change this for the next machine type to always force > max alignment (1Gb), then it would be possible to change > between backends with different alignments. I'm not sure what's the best solution here. If always using 1GB is too aggressive, we could require management to ask for an explicit alignment as a -machine option if they know they will need a specific backend page size. BTW, are you talking about the behavior introduced by aa8580cddf011e8cedcf87f7a0fdea7549fc4704 ("pc: memhp: force gaps between DIMM's GPA") only, or the backend page size was already affecting GPA allocation before that commit? -- Eduardo

2 1

[libvirt] [PATCH v2 0/3] improve virsh attach-interface
by Pavel Hrdina 30 Oct '15

30 Oct '15

new in v2: - removed netdev name as an option for --source parameter - removed --driver parameter, the default is vfio for new qemu and that's good enough Pavel Hrdina (3): virsh-nodedev: makes struct and functions for NodeDevice list available virsh-domain: update attach-interface to support type=hostdev virsh.pod: update and improve a attach-interface section tools/virsh-domain.c | 34 +++++++++++++++++++-- tools/virsh-nodedev.c | 16 ++++------ tools/virsh-nodedev.h | 11 +++++++ tools/virsh.pod | 82 +++++++++++++++++++++++++++++++-------------------- 4 files changed, 98 insertions(+), 45 deletions(-) -- 2.6.2

2 13