- Devel - libvirt List Archives

[libvirt] Entering freeze for libvirt-1.2.21
by Daniel Veillard 02 Nov '15

02 Nov '15

As pointed our on Tuesday it's time for a new release. I have tagged the release candidate 1 in git and pushed signed tarball and rpms to the usual place at: ftp://libvirt.org/libvirt/ Based on my limited testing this works just fine, but that's very limited and doesn't test portability at all, so please give it a try ! I will likely push rc2 on the week-end and the final version on Tuesday or Wednesday, please raise issues if you fine any, thanks, Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 5

[libvirt] Outreachy 2015 DecemberMarch- Enhancing libvirt-designer
by Riyafa Abdul Hameed 01 Nov '15

01 Nov '15

Dear sir, I am Computer Science and Engineering undergraduate at University of Moratuwa, Sri Lanka. I am a keen learner and am much interested in open source. I have worked with WSO2 <http://wso2.com/> on open source projects and am quite familiar with FOSS development. I have a good knowledge in C and in using XML. I would like to participate in "Outreachy 2015 DecemberMarch" with QEMU, but it was already quite late when I came to know of this program as its application deadline closes by tomorrow. I would like to apply for the project "Enhancing libvirt-designer" this time for Outreachy. I sent a mail to the mentor, Michal Privoznik and I couldn't get in touch with him. Because it is already too late can anyone please suggest what I can do before I apply for the program by tomorrow? Thank you. Yours sincerely, Riyafa -- Riyafa Abdul Hameed Undergraduate, University of Moratuwa Email: riyafa3(a)gmail.com Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa> <http://twitter.com/Riyafa1>

1 0

[libvirt] [PATCH 1/3] cpu_map.xml: add cmt feature to x86
by Qiaowei Ren 31 Oct '15

31 Oct '15

Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family) introduced CMT (Cache Monitoring Technology) to measure the usage of cache by applications running on the platform. This patch add it into x86 part of cpu_map.xml. Signed-off-by: Qiaowei Ren <qiaowei.ren(a)intel.com> --- .gnulib | 2 +- src/cpu/cpu_map.xml | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/.gnulib b/.gnulib index f39477d..106a386 160000 --- a/.gnulib +++ b/.gnulib @@ -1 +1 @@ -Subproject commit f39477dba778e99392948dd3dd19ec0d46aee932 +Subproject commit 106a3866d01f9dd57ab4f10dbeb0d5a8db73a9f7 diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml index b9e95cf..14ccbd8 100644 --- a/src/cpu/cpu_map.xml +++ b/src/cpu/cpu_map.xml @@ -317,6 +317,9 @@ <feature name='rtm'> <cpuid function='0x00000007' ebx='0x00000800'/> </feature> + <feature name='cmt'> + <cpuid function='0x00000007' ebx='0x00001000'/> + </feature> <feature name='rdseed'> <cpuid function='0x00000007' ebx='0x00040000'/> </feature> -- 1.9.1

5 10

[libvirt] [python PATCH] fix crash introduced by commit 1d39dbaf
by Pavel Hrdina 31 Oct '15

31 Oct '15

Some of the libvirt_*Wrap functions steals the reference and we need to set the item in array to NULL no not free it on success. Those three places was accidentally removed by commit 1d39dbaf. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1270977 Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- Pushed as fix for critical crasher. libvirt-override.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/libvirt-override.c b/libvirt-override.c index c6c1da8..5f1c98f 100644 --- a/libvirt-override.c +++ b/libvirt-override.c @@ -2448,6 +2448,7 @@ libvirt_virDomainListAllSnapshots(PyObject *self ATTRIBUTE_UNUSED, for (i = 0; i < c_retval; i++) { VIR_PY_LIST_SET_GOTO(py_retval, i, libvirt_virDomainSnapshotPtrWrap(snaps[i]), error); + snaps[i] = NULL; } cleanup: @@ -2544,9 +2545,11 @@ libvirt_virDomainSnapshotListAllChildren(PyObject *self ATTRIBUTE_UNUSED, if (!(py_retval = PyList_New(c_retval))) goto cleanup; - for (i = 0; i < c_retval; i++) + for (i = 0; i < c_retval; i++) { VIR_PY_LIST_SET_GOTO(py_retval, i, libvirt_virDomainSnapshotPtrWrap(snaps[i]), error); + snaps[i] = NULL; + } cleanup: for (i = 0; i < c_retval; i++) @@ -3117,9 +3120,11 @@ libvirt_virConnectListAllNetworks(PyObject *self ATTRIBUTE_UNUSED, if (!(py_retval = PyList_New(c_retval))) goto cleanup; - for (i = 0; i < c_retval; i++) + for (i = 0; i < c_retval; i++) { VIR_PY_LIST_SET_GOTO(py_retval, i, libvirt_virNetworkPtrWrap(nets[i]), error); + nets[i] = NULL; + } cleanup: for (i = 0; i < c_retval; i++) -- 2.6.2

1 0

[libvirt] RFC: Improve performance of macvtap device creation
by Tony Krowiak 30 Oct '15

30 Oct '15

For a guest domain defined with a large number of macvtap devices, it takes an exceedingly long time to boot the guest. In a test of a guest domain configured with 82 macvtap devices, it took over two minutes for the guest to boot. An strace of the ioctl calls during guest start up showed the SIOCGIFFLAGS ioctl literally being invoked 3,403 times. I was able to isolate the source of the ioctl calls to the*virNetDevMacVLanCreateWithVPortProfile* function in*virnetdevmacvlan.c*. The macvtap interface name is created by looping over a counter variable, starting with zero, and appending the counter value to 'macvtap'. With each iteration, a call is made to*virNetDevExists* (SIOCGIFFLAGS ioctl) to determine if a device with that name already exists, until a unique name is created. In the test case cited above, to create an interface name for the 82nd macvtap device, the*virNetDevExists* function will be called for interface names 'macvtap0' to 'macvtap80' before it is determined that 'mavtap81' can be used. So if N is the number of macvtap interfaces defined for a guest, the SIOCGIFFLAGS ioctl will be invoked (N x N + N)/2 times to find an unused macvtap device names. That's assuming only one guest is being started, who knows how many times the ioctl may have to be called in an installation running a large number of guests defined with macvtap devices. I was able to reduce the amount of time for starting a guest domain defined with 82 macvtap devices from over 2 minutes to about 14 seconds by keeping track of the interface name suffixes previously used. I defined two static bit maps (virBitmap), one each for macvtap and macvlan device name suffixes. When a macvtap/macvlan device is created, the index of the next clear bit (virBitmapNextClearBit) is retrieved to create the name. If an interface with that name does not exist, the device is created and the bit at the index used to create the interface name is set (virBitmapSetBit). When a macvtap/macvlan device is deleted, if the interface name has the pattern 'macvtap%d' or 'macvlan%d', the suffix is parsed into a bit index and used to clear the (virBitMapClearBit) bit in the respective bitmap. I am not sure that is the best design because there is no way to track interface names used to create macvtap devices outside of libvirt, for example using the ip command. There may also be other issues I've not contemplated. I included a couple of additional ideas below and am looking for comments or other suggestions that I have not considered. * Define a global counter variable initialized to 0, that gets incremented each time an interface name is created, to keep track of the last used interface name suffix. At some maximum value, the counter will be set back to 0. * Append a random number to 'macvlan' or 'macvtap' when creating the interface name. Of course, the number of digits would have to be limited so the interface name would not exceed the maximum allowed. * Create the interface name in code that has more knowledge of the environment and pass the name into the *virNetDevMacVLanCreateWithVPortProfile* function via the *tgifname* parameter. For example, the *qemuBuildCommandLine* function in *qemu_command.c* contains the loop that iterates over the network devices defined for the guest domain that ultimately get created via the *virNetDevMacVLanCreateWithVPortProfile* function. That function has access to the network device configuration and at the very least could ensure none of the names previously defined for the guest aren't used. I believe it would be matter of creating a macvtap interface name - e.g., maybe a call to some function in *virnetdevmacvlan.c* - and setting the name in the virDomainNetDef structure prior to invoking *qemuBuildInterfaceCommandLine*? There are shortcomings in all of these ideas, so if you have a better one, feel free to present it.

4 7

[libvirt] RAM backend and guest ABI (was Re: [Qemu-devel] [PATCH v2] pc: memhp: enforce minimal 128Mb) alignment for pc-dimm
by Eduardo Habkost 30 Oct '15

30 Oct '15

(CCing Michal and libvir-list, so libvirt team is aware of this restriction) On Thu, Oct 29, 2015 at 02:36:37PM +0100, Igor Mammedov wrote: > On Tue, 27 Oct 2015 14:36:35 -0200 > Eduardo Habkost <ehabkost(a)redhat.com> wrote: > > > On Tue, Oct 27, 2015 at 10:14:56AM +0100, Igor Mammedov wrote: > > > On Tue, 27 Oct 2015 10:53:08 +0200 > > > "Michael S. Tsirkin" <mst(a)redhat.com> wrote: > > > > > > > On Tue, Oct 27, 2015 at 09:48:37AM +0100, Igor Mammedov wrote: > > > > > On Tue, 27 Oct 2015 10:31:21 +0200 > > > > > "Michael S. Tsirkin" <mst(a)redhat.com> wrote: > > > > > > > > > > > On Mon, Oct 26, 2015 at 02:24:32PM +0100, Igor Mammedov wrote: > > > > > > > Yep it's workaround but it works around QEMU's broken virtio > > > > > > > implementation in a simple way without need for guest side changes. > > > > > > > > > > > > > > Without foreseeable virtio fix it makes memory hotplug unusable and even > > > > > > > more so if there were a virtio fix it won't fix old guests since you've > > > > > > > said that virtio fix would require changes of both QEMU and guest sides. > > > > > > > > > > > > What makes it not foreseeable? > > > > > > Apparently only the fact that we have a work-around in place so no one > > > > > > works on it. I can code it up pretty quickly, but I'm flat out of time > > > > > > for testing as I'm going on vacation soon, and hard freeze is pretty > > > > > > close. > > > > > I can lend a hand for testing part. > > > > > > > > > > > > > > > > > GPA space is kind of cheap, but wasting it in chunks of 512M > > > > > > seems way too aggressive. > > > > > hotplug region is sized with 1Gb alignment reserve per DIMM so we aren't > > > > > actually wasting anything here. > > > > > > > > > > > > > If I allocate two 1G DIMMs, what will be the gap size? 512M? 1G? > > > > It's too much either way. > > > minimum would be 512, and if backend is 1Gb-hugepage gap will be > > > backend's natural alignment (i.e. 1Gb). > > > > Is backend configuration even allowed to affect the machine ABI? We need > > to be able to change backend configuration when migrating the VM to > > another host. > for now, one has to use the same type of backend on both sides > i.e. if source uses 1Gb huge pages backend then target also > need to use it. > The page size of the backend don't even depend on QEMU arguments, but on the kernel command-line or hugetlbfs mount options. So it's possible to have exactly the same QEMU command-line on source and destination (with an explicit versioned machine-type), and get a VM that can't be migrated? That means we are breaking our guarantees about migration and guest ABI. > We could change this for the next machine type to always force > max alignment (1Gb), then it would be possible to change > between backends with different alignments. I'm not sure what's the best solution here. If always using 1GB is too aggressive, we could require management to ask for an explicit alignment as a -machine option if they know they will need a specific backend page size. BTW, are you talking about the behavior introduced by aa8580cddf011e8cedcf87f7a0fdea7549fc4704 ("pc: memhp: force gaps between DIMM's GPA") only, or the backend page size was already affecting GPA allocation before that commit? -- Eduardo

2 1

[libvirt] [PATCH v2 0/3] improve virsh attach-interface
by Pavel Hrdina 30 Oct '15

30 Oct '15

new in v2: - removed netdev name as an option for --source parameter - removed --driver parameter, the default is vfio for new qemu and that's good enough Pavel Hrdina (3): virsh-nodedev: makes struct and functions for NodeDevice list available virsh-domain: update attach-interface to support type=hostdev virsh.pod: update and improve a attach-interface section tools/virsh-domain.c | 34 +++++++++++++++++++-- tools/virsh-nodedev.c | 16 ++++------ tools/virsh-nodedev.h | 11 +++++++ tools/virsh.pod | 82 +++++++++++++++++++++++++++++++-------------------- 4 files changed, 98 insertions(+), 45 deletions(-) -- 2.6.2

2 13

[libvirt] qemu-agent-command via isa-serial for freebsd
by Vasiliy Tolstov 30 Oct '15

30 Oct '15

Hi! I'm need to control freebsd guest via qemu-ga, as i see qemu-ga already supports isa-serial connection. How i need to configure domain via libvirt to able virsh qemu-agent-command to this guest vm? -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

3 10

[libvirt] [PATCH] util: set error if DAD is not finished
by Luyao Huang 30 Oct '15

30 Oct '15

If DAD not finished in 5 seconds, user will get an unknown error like this: # virsh net-start ipv6 error: Failed to start network ipv6 error: An error occurred, but the cause is unknown Call virReportError to set an error. Signed-off-by: Luyao Huang <lhuang(a)redhat.com> --- I found the DAD will take 7 seconds on my machine, and i cannot create a network which use ipv6 now :( . Can we offer a way allow user to change this timeout ? maybe add a configuration file option in libvirtd.conf. src/util/virnetdev.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/util/virnetdev.c b/src/util/virnetdev.c index 9789e93..c8861e9 100644 --- a/src/util/virnetdev.c +++ b/src/util/virnetdev.c @@ -1398,7 +1398,13 @@ virNetDevWaitDadFinish(virSocketAddrPtr *addrs, size_t count) VIR_FREE(resp); } /* Check timeout. */ - ret = dad ? -1 : 0; + if (dad) { + virReportError(VIR_ERR_SYSTEM_ERROR, + _("Duplicate Address Detection " + "not finished in %d seconds"), VIR_DAD_WAIT_TIMEOUT); + } else { + ret = 0; + } cleanup: VIR_FREE(resp); -- 1.8.3.1

5 5

[libvirt] Again cgroups and isolcpus
by Henning Schild 29 Oct '15

29 Oct '15

Hi folks, i already started a discussion on the interaction of cgroups and isolcpus a while ago. But now i believe i have got a better understanding of how the two interact and i can describe problems that arise from that. The scenario: A machine that runs realtime tasks on pcpus reserved with isolcpus. It also runs VMs with the help of libvirt. It might also run realtime VMs with the help of libvirt. Moving a task into a new cgroup/cpuset and some modifications of the cpus in that set imply a setaffinity by the kernel. That affinity setting will ignore isolcpus. The result is possible "interference by" or "starvation of" these tasks. Now let me describe one scenario where that implicit setaffinity becomes a problem for our realtime system. libvirt creates a superset called the machine.slice and subsets called emulator and vpuX. By default the machine.slice inherits from the root which contains all pcpus, also the isolated ones. Now moving a task into that superset will place that task on isolcpus where it might interfere or simply starve. Turns out that a fresh qemu actually is put into that superset. That is a bug that should be fixed but let me address that one in another mail. My current point of view is that we need a strong mechanism to isolate cpus. isolcpus just is not good enough. The measure of choice probably is cpusets as well, and this time with the exclusive flag turned on. That will stop every other cpuset user from messing around with those cpus by accident. I am thinking of one or more cpusets where isolated cpus are parked and not used within this cgroup. Anyone wanting to use one of them will have to take it out there and explictily put it into their a new set. Now if libvirt makes the mistake to have tasks running in supersets these tasks will spread to the newly added rt-cpu. Or new tasks that run in supersets will end up on rt-cpus already in use. But at least we have containment in libvirt and the VMs it spawned. For alloc and free of rt-cpus i am planning to use libvirt hooks to begin with, from what i read they should enable me to do what i need. What do you guys think about the general idea to address the described problem? I will implement a prototype of the alloc-free of rt-cpus. My current hope is that libvirt hooks can be abused for that. I am thinking that at some point libvirt should be able to do that without hooks. It should get a notion of reserved ressources that are currently parked in other cgroups. My current suspicion is that the cpusets might just be the tip of the iceberg. -- for now i am running libvirt without cgroups to keep my isolcpus free cgroups/cpusets offer a switch to make a cpu exclusive to a set. That switch is great because it will act as an assert, a second line of defense. Having seen how cpusets and migration mess around with affinities i guess for realtime people have to insist on that second line of defense. Especially in times where cgroups are all over the place. In openstack one would actually say that a pcpu should be "dedicated". That will result in a vcpupin on exactly one pcpu. Unfortunately one meaning of "dedicated" gets lost in translation. It could otherwise be used by libvirtd to set cpuset.cpu_exclusive in the vcpu-cgroup. And i am bringing that up here because i do not think libvirt allows me to influence the cpu_exclusive flag for my vcpu cgroups. Henning

1 0