November 2015 - Devel - libvirt List Archives

[libvirt] Libvirt bite sized tasks
by Michal Privoznik 20 Mar '16

20 Mar '16

Dear lists, I've just started new wiki page which aim is to summarize small and trivial tasks, that starting contributors can take, investigate and implement. The aim is to give them something easy to start with while not scaring them out about complexity of our code. The page can be found here: http://wiki.libvirt.org/page/BiteSizedTasks The name is copied from qemu wiki: http://wiki.qemu.org/BiteSizedTasks Please, feel free to extend the list. I plan to use it in the future when interviewing GSoC candidates. Michal

3 4

[libvirt] [PATCH v2 0/5] Add virtio-gpu & virgl support
by Marc-André Lureau 18 Feb '16

18 Feb '16

The following series allow using virtio-gpu and virgl if qemu support it. Ths Spice bits are not yet in qemu 2.5, but should make it in 2.6 and I added the RFC patches that are expected to not change much. v1->v2: addressing Ján Tomko review - update accel2d/accel3d to be tristate and fix its usage - use a single QEMU_CAPS_DEVICE_VIRTIO_GPU cap for -device virtio-gpu-* & virtio-vga - remove qemu legacy, -vga support or command line parsing - add xml2xml tests - squash xml2argvtests with their respecitve feature addition commits - update libvirt version in doc - update commit messages Marc-André Lureau (5): Replace support{2d,3d} with accel{2d,3d} domain: replace bool accel{2d,3d} with a tristate qemu: add virtio video device qemu: add virtio-gpu virgl support RFC qemu: add spice opengl support docs/formatdomain.html.in | 11 +- docs/schemas/domaincommon.rng | 6 + src/conf/domain_conf.c | 84 +- src/conf/domain_conf.h | 6 +- src/qemu/qemu_capabilities.c | 12 + src/qemu/qemu_capabilities.h | 3 + src/qemu/qemu_command.c | 40 +- src/vbox/vbox_common.c | 18 +- src/vz/vz_sdk.c | 2 +- tests/qemucapabilitiesdata/caps_1.2.2-1.replies | 22 +- tests/qemucapabilitiesdata/caps_1.3.1-1.replies | 22 +- tests/qemucapabilitiesdata/caps_1.4.2-1.replies | 22 +- tests/qemucapabilitiesdata/caps_1.5.3-1.replies | 22 +- tests/qemucapabilitiesdata/caps_1.6.0-1.replies | 22 +- tests/qemucapabilitiesdata/caps_1.6.50-1.replies | 22 +- tests/qemucapabilitiesdata/caps_2.1.1-1.replies | 22 +- tests/qemucapabilitiesdata/caps_2.4.0-1.caps | 1 + tests/qemucapabilitiesdata/caps_2.4.0-1.replies | 95 +- tests/qemucapabilitiesdata/caps_2.5.0-1.caps | 168 + tests/qemucapabilitiesdata/caps_2.5.0-1.replies | 4043 ++++++++++++++++++++ tests/qemucapabilitiestest.c | 1 + .../qemuxml2argv-video-virtio-gpu-device.args | 23 + .../qemuxml2argv-video-virtio-gpu-device.xml | 31 + .../qemuxml2argv-video-virtio-gpu-spice-gl.args | 23 + .../qemuxml2argv-video-virtio-gpu-spice-gl.xml | 36 + .../qemuxml2argv-video-virtio-gpu-virgl.args | 23 + .../qemuxml2argv-video-virtio-gpu-virgl.xml | 33 + tests/qemuxml2argvtest.c | 14 + tests/qemuxml2xmltest.c | 4 + 29 files changed, 4732 insertions(+), 99 deletions(-) create mode 100644 tests/qemucapabilitiesdata/caps_2.5.0-1.caps create mode 100644 tests/qemucapabilitiesdata/caps_2.5.0-1.replies create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-video-virtio-gpu-device.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-video-virtio-gpu-device.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-video-virtio-gpu-spice-gl.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-video-virtio-gpu-spice-gl.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-video-virtio-gpu-virgl.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-video-virtio-gpu-virgl.xml -- 2.5.0

4 16

[libvirt] Found mem leak in livirt, need help to debug
by Piotr Rybicki 11 Feb '16

11 Feb '16

Hi. There is a mem leak in libvirt, when doing external snapshot (for backup purposes). My KVM domain uses raw storage images via libgfapi. I'm using latest 1.2.21 libvirt (although previous versions act the same). My bash script for snapshot backup uses series of shell commands (virsh connect to a remote libvirt host): * virsh domblklist KVM * qemu-img create -f qcow2 -o backing_file=gluster(...) - precreate backing file * virsh snapshot-create KVM SNAP.xml (...) - create snapshot from precreated XML snapshot file * cp main img file * virsh blockcommit KVM disk (...) Backup script works fine, however libvirtd process gets bigger and bigger each time I run this script. Some proof of memleak: 32017 - libvirtd pid When libvirt started: # ps p 32017 o vsz,rss VSZ RSS 585736 15220 When I start KVM via virsh start KVM # ps p 32017 o vsz,rss VSZ RSS 1327968 125956 When i start backup script, after snapshot is created (lots of mem allocated) # ps p 32017 o vsz,rss VSZ RSS 3264544 537632 After backup script finished # ps p 32017 o vsz,rss VSZ RSS 3715920 644940 When i start backup script for a second time, after snapshot is created # ps p 32017 o vsz,rss VSZ RSS 5521424 1056352 And so on, until libvirt spills 'Out of memory' when connecting, ane being really huge process. Now, I would like to diagnose it further, to provide detailed information about memleak. I tried to use valgrind, but unfortunatelly I'm on Opteron 6380 platform, and valgrind doesn't support XOP quitting witch SIGILL. If someone could provide me with detailed information on how to get some usefull debug info about this memleak, i'll be more than happy to do it, and share results here. Thanks in advance and best regards Piotr Rybicki

3 13

[libvirt] oVirt considers using macTableManager='libvirt'
by Ido Barkan 02 Feb '16

02 Feb '16

Hi guys, We, at oVirt, are considring using the automatic bridge management feature of libvirt for our hosts (macTableManager='libvirt'). I could find any discussion in the mailing list archives about the motivation for this feature. (was there?). If there wasn't I would like to start a new one, about the possible trade offs it would have in oVirt. Specifically I have a few questions: 1) The obvious performance motivation is clear: considering N hosts with M vms each connected to the same LAN, the first packet to any unknown yet host will flood all the vms in all N bridges. -- was there any other motivation that I do not understand (apart from slightly better security? 2) oVirt uses TC for port mirroring, in case this is requested by users, to mirror traffic from a chosen bridge to a chosen NIC in the host. I could not understand if macTableManager will interfere with it, or not. 3) Are there any drawbacks to enabling this feature? 4) We aim for rhel7.2. Will the feature be supported (or partially supported) for kernels older then 3.17? And if so, in what way? 5) oVirt currently builds its own bridges and tell libvirt about them. Does that have any affect on the functionality of that feature? 6) are there any plans to support OVS for this feature in the future? -- Thanks, Ido Barkan

4 7

[libvirt] [PATCH v3 0/6] admin: Introduce server listing API
by Erik Skultety 21 Jan '16

21 Jan '16

Since v2: - static names of daemons are passed to virNetServerNew directly, instead of creating an array of names like the previous version did - dropped client-side server identification through ID, only name is used - adjusted naming of some methods (prefixes again...) - converted the server listing example to virt-admin command (finally) Erik Skultety (6): rpc: Introduce new element 'name' to virnetserver structure virnetdaemon: Add post exec restart support for multiple servers admin: Move admin_server.{h,c} to admin.{h,c} admin: Introduce virAdmServer structure admin: Introduce adminDaemonConnectListServers API virt-admin: Introduce cmdSrvList daemon/Makefile.am | 6 +- daemon/admin.c | 181 +++++++++++++++++++++ daemon/admin.h | 36 ++++ daemon/admin_server.c | 121 +++++--------- daemon/admin_server.h | 23 ++- daemon/libvirtd.c | 4 +- include/libvirt/libvirt-admin.h | 11 ++ po/POTFILES.in | 2 +- src/admin/admin_protocol.x | 26 ++- src/admin_protocol-structs | 15 ++ src/datatypes.c | 36 ++++ src/datatypes.h | 34 ++++ src/libvirt-admin.c | 148 +++++++++++++++++ src/libvirt_admin_private.syms | 5 + src/libvirt_admin_public.syms | 3 + src/locking/lock_daemon.c | 3 +- src/logging/log_daemon.c | 3 +- src/lxc/lxc_controller.c | 2 +- src/rpc/virnetdaemon.c | 15 ++ src/rpc/virnetdaemon.h | 3 + src/rpc/virnetserver.c | 32 +++- src/rpc/virnetserver.h | 5 + .../input-data-admin-server-names.json | 128 +++++++++++++++ .../virnetdaemondata/output-data-admin-nomdns.json | 2 + .../output-data-admin-server-names.json | 128 +++++++++++++++ .../virnetdaemondata/output-data-anon-clients.json | 1 + .../output-data-initial-nomdns.json | 1 + tests/virnetdaemondata/output-data-initial.json | 1 + tests/virnetdaemontest.c | 40 ++--- tools/virt-admin.c | 62 +++++++ 30 files changed, 946 insertions(+), 131 deletions(-) create mode 100644 daemon/admin.c create mode 100644 daemon/admin.h create mode 100644 tests/virnetdaemondata/input-data-admin-server-names.json create mode 100644 tests/virnetdaemondata/output-data-admin-server-names.json -- 2.4.3

2 12

[libvirt] [PATCH V2 0/9] support multi-thread compress migration.
by ShaoHe Feng 18 Jan '16

18 Jan '16

These series patches support multi-thread compress during live migration. Eli Qiao (4): Add test cases for qemuMonitorJSONGetMigrationParameter remote: Add support for set and get multil thread migration parameters qemu_driver: Add support to set/get migration parameters. virsh: Add set and get multi-thread migration parameters commands ShaoHe Feng (5): qemu_migration: Add support for mutil-thread compressed migration enable qemu: Add monitor API for get/set migration parameters set multi-thread compress params for Migrate3 during live migration virsh: add multi-thread migration option for live migrate command Implement the public APIs for multi-thread compress parameters. .gnulib | 2 +- daemon/remote.c | 62 +++++++++++ include/libvirt/libvirt-domain.h | 31 ++++++ src/driver-hypervisor.h | 14 +++ src/libvirt-domain.c | 110 +++++++++++++++++++ src/libvirt_public.syms | 5 + src/qemu/qemu_domain.h | 3 + src/qemu/qemu_driver.c | 186 ++++++++++++++++++++++++++++++++ src/qemu/qemu_migration.c | 105 ++++++++++++++++++ src/qemu/qemu_migration.h | 32 ++++-- src/qemu/qemu_monitor.c | 40 ++++++- src/qemu/qemu_monitor.h | 11 ++ src/qemu/qemu_monitor_json.c | 93 ++++++++++++++++ src/qemu/qemu_monitor_json.h | 9 ++ src/qemu/qemu_monitor_text.c | 95 +++++++++++++++++ src/qemu/qemu_monitor_text.h | 10 ++ src/remote/remote_driver.c | 54 ++++++++++ src/remote/remote_protocol.x | 30 +++++- src/remote_protocol-structs | 26 +++++ tests/qemumonitorjsontest.c | 53 ++++++++++ tools/virsh-domain.c | 223 ++++++++++++++++++++++++++++++++++++++- tools/virsh.pod | 37 +++++-- 22 files changed, 1212 insertions(+), 19 deletions(-) -- 2.1.4

4 19

[libvirt] [PATCH 0/3] Misc fixes
by Cédric Bosdonnat 15 Jan '16

15 Jan '16

Hi all, Here are a few patches without strong connection together. The first one only allows us not to package virt-login-shell even with lxc driver enabled. The other ones are related to mounts security. I'm wondering if changing the default dropped capabilities in the lxc driver is acceptable... dropping sys_admin makes sense, but it can introduce incompatibilities for users needing it as they will need to explicitely enable it. Cédric Bosdonnat (3): Allow building lxc without virt-login-shell virt-aa-helper: don't deny writes to readonly mounts lxc: drop sys_admin caps by default configure.ac | 14 ++++++++++++++ src/lxc/lxc_container.c | 1 + src/security/virt-aa-helper.c | 5 ++++- tools/Makefile.am | 12 ++++++------ 4 files changed, 25 insertions(+), 7 deletions(-) -- 2.1.4

5 8

[libvirt] [RFC] memory settings interface for containers
by Nikolay Shirokovskiy 14 Jan '16

14 Jan '16

Hi, everyone. I plan to add means to configure vz containers memory setting and have trouble getting it done thru libvirt interface. Looks like current interface fits good for vm memory managment but its not clear how to use it with containers. First let's take aside memory hotplugging which is obviously not suitable for containers. Then memory interface is represented by 2 parameters: total_memory and cur_balloon. For VMs total_memory can't be changed at runtime, cur_ballon can't be greater than total_memory. But for containers memory model is different. We have only one parameter and it can be changed for running domains. So question is how to map this model to existing interface (it is unlikely to have a new interface for this case). I plan to make both parameters to have same meaning and be equal for containers and update virsh, API and xml model documentation accordingly. I'd be happy to hear core developers opinions on this topic.

4 10

[libvirt] Bug in RPC code causes failure to start LXC container using virDomainCreateXMLWithFiles
by Ben Gray 12 Jan '16

12 Jan '16

Hi, Occasionally when trying to start LXC containers with fds I get the following error: virNetMessageDupFD:562 : Unable to duplicate FD -1: Bad file descriptor I tracked it down to the code that handles EAGAIN errors from recvfd. In such cases the virNetMessageDecodeNumFDs function may be called multiple times from virNetServerClientDispatchRead and each time it overwrites the msg->fds array. In the best case (when msg->donefds == 0) this results in a memory leak, in the worse case it will leak any fd's already in msg->fds and cause subsequent failures when dup is called. A very similar problem is mention here: https://www.redhat.com/archives/libvir-list/2012-December/msg01306.html Below is my patch to fix the issue. --- a/src/rpc/virnetserverclient.c 2015-01-23 11:46:24.000000000 +0000 +++ b/src/rpc/virnetserverclient.c 2015-11-26 15:30:51.214462290 +0000 @@ -1107,36 +1107,40 @@ /* Now figure out if we need to read more data to get some * file descriptors */ - if (msg->header.type == VIR_NET_CALL_WITH_FDS && - virNetMessageDecodeNumFDs(msg) < 0) { - virNetMessageQueueServe(&client->rx); - virNetMessageFree(msg); - client->wantClose = true; - return; /* Error */ - } + if (msg->header.type == VIR_NET_CALL_WITH_FDS) { + size_t i; - /* Try getting the file descriptors (may fail if blocking) */ - for (i = msg->donefds; i < msg->nfds; i++) { - int rv; - if ((rv = virNetSocketRecvFD(client->sock, &(msg->fds[i]))) < 0) { + if (msg->nfds == 0 && + virNetMessageDecodeNumFDs(msg) < 0) { virNetMessageQueueServe(&client->rx); virNetMessageFree(msg); client->wantClose = true; - return; + return; /* Error */ } - if (rv == 0) /* Blocking */ - break; - msg->donefds++; - } - /* Need to poll() until FDs arrive */ - if (msg->donefds < msg->nfds) { - /* Because DecodeHeader/NumFDs reset bufferOffset, we - * put it back to what it was, so everything works - * again next time we run this method - */ - client->rx->bufferOffset = client->rx->bufferLength; - return; + /* Try getting the file descriptors (may fail if blocking) */ + for (i = msg->donefds; i < msg->nfds; i++) { + int rv; + if ((rv = virNetSocketRecvFD(client->sock, &(msg->fds[i]))) < 0) { + virNetMessageQueueServe(&client->rx); + virNetMessageFree(msg); + client->wantClose = true; + return; + } + if (rv == 0) /* Blocking */ + break; + msg->donefds++; + } + + /* Need to poll() until FDs arrive */ + if (msg->donefds < msg->nfds) { + /* Because DecodeHeader/NumFDs reset bufferOffset, we + * put it back to what it was, so everything works + * again next time we run this method + */ + client->rx->bufferOffset = client->rx->bufferLength; + return; + } } /* Definitely finished reading, so remove from queue */

2 1

[libvirt] [PATCH 0/5] auto-add USB2 controller set for Q35
by Laine Stump 11 Jan '16

11 Jan '16

For just about every other machinetype, libvirt automatically adds a USB controller if there is no controller (including "type='none'") specified in the config. It doesn't do this for the Q35 machinetype, because Q35 hardware would have a USB2 controller, USB2 controllers come in sets of multiple devices, and the code that auto-adds the USB controller was really setup to just add a single controller. Expanding that to adding a set of related controllers was beyond the amount of time I had when putting in the initial Q35 support, so I left it "for later", and then forgot about it until someone reminded me in the hall at KVM Forum this summer. I find the practice of auto-adding devices that aren't required for operation of the virtual machine to be a bit odd, but this does make the Q35 machinetype more consistent with all the others, and it is still possible to force no USB controllers by specifying: <controller type='usb' model='none'/> Since the USB controllers on a real Q35 machine are on bus 0 slot 0x1D, there is also a patch here to attempt to use that address for the first set of USB controllers (and 0x1A for the 2nd set). Finally, patch 1 is a bugfix for a problem that hadn't been noticed before, because nobody had tried to connect a USB controller to a pcie-root-port (which has a single slot that is numbered 0). Laine Stump (5): qemu: don't assume slot 0 is unused/reserved. qemu: prefer 00:1D.x and 00:1A.x for USB2 controllers on Q35 conf: add virDomainDefAddController() qemu: define virDomainDevAddUSBController() qemu: auto-add a USB2 controller set for Q35 machines src/conf/domain_conf.c | 104 +++++++++++++++++---- src/conf/domain_conf.h | 2 + src/libvirt_private.syms | 1 + src/qemu/qemu_command.c | 57 ++++++++++- src/qemu/qemu_domain.c | 14 ++- .../qemuxml2argv-q35-usb2-multi.args | 40 ++++++++ .../qemuxml2argv-q35-usb2-multi.xml | 47 ++++++++++ .../qemuxml2argv-q35-usb2-reorder.args | 40 ++++++++ .../qemuxml2argv-q35-usb2-reorder.xml | 47 ++++++++++ tests/qemuxml2argvdata/qemuxml2argv-q35-usb2.args | 30 ++++++ tests/qemuxml2argvdata/qemuxml2argv-q35-usb2.xml | 39 ++++++++ tests/qemuxml2argvdata/qemuxml2argv-q35.args | 5 + tests/qemuxml2argvtest.c | 22 +++++ .../qemuxml2xmlout-q35-usb2-multi.xml | 66 +++++++++++++ .../qemuxml2xmlout-q35-usb2-reorder.xml | 66 +++++++++++++ .../qemuxml2xmloutdata/qemuxml2xmlout-q35-usb2.xml | 46 +++++++++ tests/qemuxml2xmltest.c | 3 + 17 files changed, 606 insertions(+), 23 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-usb2-multi.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-usb2-multi.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-usb2-reorder.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-usb2-reorder.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-usb2.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-usb2.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-usb2-multi.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-usb2-reorder.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-usb2.xml -- 2.4.3

2 14