- Devel - libvirt List Archives

[libvirt] [PATCH] bhyve: fix virtio disk addresses
by Roman Bogorodskiy 07 Feb '17

07 Feb '17

Like it usually happens, I fixed one thing and broke another: in 803966c76 address allocation was fixed for SATA disks, but broke that for virtio disks, because it dropped disk address assignment completely. It's not needed for SATA disks anymore, but still needed for the virtio ones. Bring that back and add a couple of tests to make sure it won't happen again. --- src/bhyve/bhyve_device.c | 16 +++++++++ .../bhyvexml2argv-addr-multiple-virtio-disks.args | 11 ++++++ ...bhyvexml2argv-addr-multiple-virtio-disks.ldargs | 3 ++ .../bhyvexml2argv-addr-multiple-virtio-disks.xml | 32 +++++++++++++++++ .../bhyvexml2argv-addr-single-virtio-disk.args | 9 +++++ .../bhyvexml2argv-addr-single-virtio-disk.ldargs | 3 ++ .../bhyvexml2argv-addr-single-virtio-disk.xml | 22 ++++++++++++ tests/bhyvexml2argvtest.c | 2 ++ .../bhyvexml2xmlout-addr-multiple-virtio-disks.xml | 42 ++++++++++++++++++++++ .../bhyvexml2xmlout-addr-single-virtio-disk.xml | 30 ++++++++++++++++ tests/bhyvexml2xmltest.c | 2 ++ 11 files changed, 172 insertions(+) create mode 100644 tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.args create mode 100644 tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.ldargs create mode 100644 tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.xml create mode 100644 tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.args create mode 100644 tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.ldargs create mode 100644 tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.xml create mode 100644 tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-multiple-virtio-disks.xml create mode 100644 tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-single-virtio-disk.xml diff --git a/src/bhyve/bhyve_device.c b/src/bhyve/bhyve_device.c index 29528230f..55ce631ec 100644 --- a/src/bhyve/bhyve_device.c +++ b/src/bhyve/bhyve_device.c @@ -129,6 +129,22 @@ bhyveAssignDevicePCISlots(virDomainDefPtr def, goto error; } + for (i = 0; i < def->ndisks; i++) { + /* We only handle virtio disk addresses as SATA disks are + * attached to a controller and don't have their own PCI + * addresses */ + if (def->disks[i]->bus != VIR_DOMAIN_DISK_BUS_VIRTIO) + continue; + + if (def->disks[i]->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI && + !virPCIDeviceAddressIsEmpty(&def->disks[i]->info.addr.pci)) + continue; + if (virDomainPCIAddressReserveNextAddr(addrs, &def->disks[i]->info, + VIR_PCI_CONNECT_TYPE_PCI_DEVICE, + -1) < 0) + goto error; + } + return 0; error: diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.args b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.args new file mode 100644 index 000000000..8cc166894 --- /dev/null +++ b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.args @@ -0,0 +1,11 @@ +/usr/sbin/bhyve \ +-c 1 \ +-m 214 \ +-u \ +-H \ +-P \ +-s 0:0,hostbridge \ +-s 3:0,virtio-net,faketapdev,mac=52:54:00:bc:85:fe \ +-s 2:0,virtio-blk,/tmp/freebsd.img \ +-s 4:0,virtio-blk,/tmp/test.img \ +-s 5:0,virtio-blk,/tmp/test2.img bhyve diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.ldargs b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.ldargs new file mode 100644 index 000000000..32538b558 --- /dev/null +++ b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.ldargs @@ -0,0 +1,3 @@ +/usr/sbin/bhyveload \ +-m 214 \ +-d /tmp/freebsd.img bhyve diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.xml new file mode 100644 index 000000000..9bcd0a629 --- /dev/null +++ b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-multiple-virtio-disks.xml @@ -0,0 +1,32 @@ +<domain type='bhyve'> + <name>bhyve</name> + <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid> + <memory>219136</memory> + <vcpu>1</vcpu> + <os> + <type>hvm</type> + </os> + <devices> + <disk type='file' device='disk'> + <driver name='file' type='raw'/> + <source file='/tmp/freebsd.img'/> + <target dev='vda' bus='virtio'/> + </disk> + <disk type='file' device='disk'> + <driver name='file' type='raw'/> + <source file='/tmp/test.img'/> + <target dev='vdb' bus='virtio'/> + </disk> + <disk type='file' device='disk'> + <driver name='file' type='raw'/> + <source file='/tmp/test2.img'/> + <target dev='vdc' bus='virtio'/> + </disk> + <interface type='bridge'> + <mac address='52:54:00:bc:85:fe'/> + <model type='virtio'/> + <source bridge="virbr0"/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + </devices> +</domain> diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.args b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.args new file mode 100644 index 000000000..4dcc40404 --- /dev/null +++ b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.args @@ -0,0 +1,9 @@ +/usr/sbin/bhyve \ +-c 1 \ +-m 214 \ +-u \ +-H \ +-P \ +-s 0:0,hostbridge \ +-s 3:0,virtio-net,faketapdev,mac=52:54:00:bc:85:fe \ +-s 2:0,virtio-blk,/tmp/freebsd.img bhyve diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.ldargs b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.ldargs new file mode 100644 index 000000000..32538b558 --- /dev/null +++ b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.ldargs @@ -0,0 +1,3 @@ +/usr/sbin/bhyveload \ +-m 214 \ +-d /tmp/freebsd.img bhyve diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.xml new file mode 100644 index 000000000..6be9ae134 --- /dev/null +++ b/tests/bhyvexml2argvdata/bhyvexml2argv-addr-single-virtio-disk.xml @@ -0,0 +1,22 @@ +<domain type='bhyve'> + <name>bhyve</name> + <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid> + <memory>219136</memory> + <vcpu>1</vcpu> + <os> + <type>hvm</type> + </os> + <devices> + <disk type='file'> + <driver name='file' type='raw'/> + <source file='/tmp/freebsd.img'/> + <target dev='vda' bus='virtio'/> + </disk> + <interface type='bridge'> + <mac address='52:54:00:bc:85:fe'/> + <model type='virtio'/> + <source bridge="virbr0"/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + </devices> +</domain> diff --git a/tests/bhyvexml2argvtest.c b/tests/bhyvexml2argvtest.c index e80705780..c36b55a0a 100644 --- a/tests/bhyvexml2argvtest.c +++ b/tests/bhyvexml2argvtest.c @@ -190,6 +190,8 @@ mymain(void) DO_TEST("addr-single-sata-disk"); DO_TEST("addr-multiple-sata-disks"); DO_TEST("addr-more-than-32-sata-disks"); + DO_TEST("addr-single-virtio-disk"); + DO_TEST("addr-multiple-virtio-disks"); /* The same without 32 devs per controller support */ driver.bhyvecaps ^= BHYVE_CAP_AHCI32SLOT; diff --git a/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-multiple-virtio-disks.xml b/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-multiple-virtio-disks.xml new file mode 100644 index 000000000..542bff121 --- /dev/null +++ b/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-multiple-virtio-disks.xml @@ -0,0 +1,42 @@ +<domain type='bhyve'> + <name>bhyve</name> + <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <disk type='file' device='disk'> + <driver name='file' type='raw'/> + <source file='/tmp/freebsd.img'/> + <target dev='vda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </disk> + <disk type='file' device='disk'> + <driver name='file' type='raw'/> + <source file='/tmp/test.img'/> + <target dev='vdb' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <disk type='file' device='disk'> + <driver name='file' type='raw'/> + <source file='/tmp/test2.img'/> + <target dev='vdc' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </disk> + <controller type='pci' index='0' model='pci-root'/> + <interface type='bridge'> + <mac address='52:54:00:bc:85:fe'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + </devices> +</domain> diff --git a/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-single-virtio-disk.xml b/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-single-virtio-disk.xml new file mode 100644 index 000000000..d7abb5abc --- /dev/null +++ b/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-addr-single-virtio-disk.xml @@ -0,0 +1,30 @@ +<domain type='bhyve'> + <name>bhyve</name> + <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <disk type='file' device='disk'> + <driver name='file' type='raw'/> + <source file='/tmp/freebsd.img'/> + <target dev='vda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </disk> + <controller type='pci' index='0' model='pci-root'/> + <interface type='bridge'> + <mac address='52:54:00:bc:85:fe'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + </devices> +</domain> diff --git a/tests/bhyvexml2xmltest.c b/tests/bhyvexml2xmltest.c index 004afda14..ba9af2996 100644 --- a/tests/bhyvexml2xmltest.c +++ b/tests/bhyvexml2xmltest.c @@ -109,6 +109,8 @@ mymain(void) DO_TEST_DIFFERENT("addr-single-sata-disk"); DO_TEST_DIFFERENT("addr-multiple-sata-disks"); DO_TEST_DIFFERENT("addr-more-than-32-sata-disks"); + DO_TEST_DIFFERENT("addr-single-virtio-disk"); + DO_TEST_DIFFERENT("addr-multiple-virtio-disks"); /* The same without 32 devs per controller support */ driver.bhyvecaps ^= BHYVE_CAP_AHCI32SLOT; -- 2.11.0

2 2

[libvirt] [PATCH 00/10] Another set of qemu namespace fixes
by Michal Privoznik 07 Feb '17

07 Feb '17

The major problem was with symlinks. Imagine the following chain of symlinks: /dev/my_awesome_disk -> /home/user/blaah -> /dev/disk/by-uuid/$uuid -> /dev/sda We really need to create all those /dev/* symlinks and /dev/sda device. Also, some other (less critical) bugs are fixed. Michal Privoznik (10): virProcessRunInMountNamespace: Report errors from child util: Introduce virFileReadLink qemuDomainPrepareDisk: Fix ordering qemuSecurityRestoreAllLabel: Don't use transactions qemu_security: Use more transactions qemuDomain{Attach,Detach}Device NS helpers: Don't relabel devices qemuDomainCreateDevice: Properly deal with symlinks qemuDomainCreateDevice: Don't loop endlessly qemuDomainAttachDeviceMknod: Deal with symlinks qemuDomainAttachDeviceMknod: Don't loop endlessly src/libvirt_private.syms | 1 + src/qemu/qemu_domain.c | 438 +++++++++++++++++++++++++++-------------------- src/qemu/qemu_hotplug.c | 20 +-- src/qemu/qemu_security.c | 137 +++++++++------ src/util/virfile.c | 12 ++ src/util/virfile.h | 2 + src/util/virprocess.c | 8 +- 7 files changed, 374 insertions(+), 244 deletions(-) -- 2.11.0

4 34

[libvirt] [PATCH] qemu: Forbid <memoryBacking><locked> without <memtune><hard_limit>
by Andrea Bolognani 07 Feb '17

07 Feb '17

In order for memory locking to work, the hard limit on memory locking (and usage) has to be set appropriately by the user. The documentation mentions the requirement already: with this patch, it's going to be enforced by runtime checks as well. Note that this will make existing guests that don't satisfy the requirement disappear; that said, such guests have never been able to start in the first place. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1316774 --- src/qemu/qemu_domain.c | 20 ++++++++++++++++++++ tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml | 3 +++ ....xml => qemuxml2argv-mlock-without-hardlimit.xml} | 0 tests/qemuxml2argvtest.c | 1 + 4 files changed, 24 insertions(+) copy tests/qemuxml2argvdata/{qemuxml2argv-mlock-on.xml => qemuxml2argv-mlock-without-hardlimit.xml} (100%) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index c6ce090..bb29cfe 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2629,6 +2629,23 @@ qemuDomainDefVcpusPostParse(virDomainDefPtr def) static int +qemuDomainDefMemoryLockingPostParse(virDomainDefPtr def) +{ + /* Memory locking can only work properly if the memory locking limit + * for the QEMU process has been raised appropriately: the default one + * is extrememly low, so there's no way the guest will fit in there */ + if (def->mem.locked && !virMemoryLimitIsSet(def->mem.hard_limit)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Setting <memoryBacking><locked> requires " + "<memtune><hard_limit> to be set as well")); + return -1; + } + + return 0; +} + + +static int qemuDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps, unsigned int parseFlags, @@ -2692,6 +2709,9 @@ qemuDomainDefPostParse(virDomainDefPtr def, if (qemuDomainDefVcpusPostParse(def) < 0) goto cleanup; + if (qemuDomainDefMemoryLockingPostParse(def) < 0) + goto cleanup; + ret = 0; cleanup: virObjectUnref(qemuCaps); diff --git a/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml b/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml index 20a5eaa..2046663 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml @@ -3,6 +3,9 @@ <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> <memory unit='KiB'>219136</memory> <currentMemory unit='KiB'>219136</currentMemory> + <memtune> + <hard_limit unit='KiB'>256000</hard_limit> + </memtune> <memoryBacking> <locked/> </memoryBacking> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml b/tests/qemuxml2argvdata/qemuxml2argv-mlock-without-hardlimit.xml similarity index 100% copy from tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml copy to tests/qemuxml2argvdata/qemuxml2argv-mlock-without-hardlimit.xml diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 3532cb5..9b2fec5 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -2129,6 +2129,7 @@ mymain(void) DO_TEST_FAILURE("mlock-on", NONE); DO_TEST("mlock-off", QEMU_CAPS_REALTIME_MLOCK); DO_TEST("mlock-unsupported", NONE); + DO_TEST_PARSE_ERROR("mlock-without-hardlimit", NONE); DO_TEST_PARSE_ERROR("pci-bridge-negative-index-invalid", QEMU_CAPS_DEVICE_PCI_BRIDGE); -- 2.7.4

3 3

[libvirt] [PATCH v2] qemu: Forbid <memoryBacking><locked> without <memtune><hard_limit>
by Andrea Bolognani 07 Feb '17

07 Feb '17

In order for memory locking to work, the hard limit on memory locking (and usage) has to be set appropriately by the user. The documentation mentions the requirement already: with this patch, it's going to be enforced by runtime checks as well, by forbidding a non-compliant guest from starting at all. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1316774 --- Changes from [v1] * Address review feeback: - check in BuildCommandLine rather than in PostParse, so that non-compliant guests will merely fail to start rather than disappear completely. [v1] https://www.redhat.com/archives/libvir-list/2017-February/msg00180.html src/qemu/qemu_command.c | 9 +++++++++ tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml | 3 +++ 2 files changed, 12 insertions(+) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 1396661..ca3bcdc 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -7340,6 +7340,15 @@ qemuBuildMemCommandLine(virCommandPtr cmd, qemuBuildMemPathStr(cfg, def, qemuCaps, cmd) < 0) return -1; + /* Memory locking can only work properly if the memory locking limit + * for the QEMU process has been raised appropriately: the default one + * is extrememly low, so there's no way the guest will fit in there */ + if (def->mem.locked && !virMemoryLimitIsSet(def->mem.hard_limit)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Setting <memoryBacking><locked> requires " + "<memtune><hard_limit> to be set as well")); + return -1; + } if (def->mem.locked && !virQEMUCapsGet(qemuCaps, QEMU_CAPS_REALTIME_MLOCK)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("memory locking not supported by QEMU binary")); diff --git a/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml b/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml index 20a5eaa..2046663 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-mlock-on.xml @@ -3,6 +3,9 @@ <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> <memory unit='KiB'>219136</memory> <currentMemory unit='KiB'>219136</currentMemory> + <memtune> + <hard_limit unit='KiB'>256000</hard_limit> + </memtune> <memoryBacking> <locked/> </memoryBacking> -- 2.7.4

2 1

[libvirt] [PATCH] docs: mention bhyve SATA address changes in news.xml
by Roman Bogorodskiy 06 Feb '17

06 Feb '17

--- docs/news.xml | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index f408293a1..9e3c4ec3d 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -56,6 +56,22 @@ a fabric name has been removed by making it optional. </description> </change> + <change> + <summary> + bhyve: change address allocation schema for SATA disks + </summary> + <description> + Previously, the bhyve driver assigned PCI addresses to SATA disks directly + rather than assigning that to a controller and using SATA addresses for disks. + It was implemented this way because bhyve has no notion of an explicit SATA + controller. However, this doesn't go inline with the internal libvirt model, + it was changed for the bhyve driver to follow the common schema and + have PCI addresses for SATA controllers and SATA addresses for disks. If you're having + issues because of this, it's recommended to edit the domain's XML and remove + <address type='xml'> from the <disk> elements with + <target bus="sata"/> and let libvirt regenerate it properly. + </description> + </change> </section> </release> <release version="v3.0.0" date="2017-01-17"> -- 2.11.0

2 2

Re: [libvirt] Issue with libvirtd
by Michal Privoznik 06 Feb '17

06 Feb '17

On 01/10/2017 07:48 AM, Faysal Ali wrote: > Hi Michal, [It is usually good idea to keep the list CCed - it may help others finding a solution to their problems] > > Well I have created my little python/libivrt app to manage my virtual > machines. I am using python socket to check libivrt port availability, and > the error End of file while reading data: Input/output error* only happens > whenever that python socket script is trigger. > > Here is the script of python socket > > import libvirt, socket, sys > > hostname='kvm09' > port = 16509 > > try: > socket_host = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > socket_host.settimeout(1) > socket_host.connect((hostname, port)) > socket_host.close() > print 'true' > except Exception as err: > print err Ah, I haven't expected this. Of course your are seeing the error message. Libvirt has its own protocol on the top of TCP and since you are connecting and dying immediately - without sending any valid libvirt packet, the daemon logs an error. This is perfectly expected. Also, this is *not* how you connect to libvirt. You want to open a libvirt connection: conn = libvirt.open(uri) dom = conn.lookupByName(name) ... Michal

2 2

[libvirt] [PATCH] char: drop data written to a disconnected pty
by Ed Swierk 06 Feb '17

06 Feb '17

When a serial port writes data to a pty that's disconnected, drop the data and return the length dropped. This avoids triggering pointless retries in callers like the 16550A serial_xmit(), and causes qemu_chr_fe_write() to write all data to the log file, rather than logging only while a pty client like virsh console happens to be connected. Signed-off-by: Ed Swierk <eswierk(a)skyportsystems.com> --- qemu-char.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qemu-char.c b/qemu-char.c index 676944a..ccb6923 100644 --- a/qemu-char.c +++ b/qemu-char.c @@ -1528,7 +1528,7 @@ static int pty_chr_write(CharDriverState *chr, const uint8_t *buf, int len) /* guest sends data, check for (re-)connect */ pty_chr_update_read_handler_locked(chr); if (!s->connected) { - return 0; + return len; } } return io_channel_send(s->ioc, buf, len); -- 1.9.1

3 5

[libvirt] [PATCH v2 0/3] add per cpu stats to all domain stats
by Nikolay Shirokovskiy 06 Feb '17

06 Feb '17

Info provided in virDomainGetCPUStats is currently missed in all domain stats. This patch removes this discrepancy. You may need not yet merged patch 'qemu: fix crash on getting block stats for empty cdrom' to test this series. diff from v1: ================ 1. reuse code (patches 1, 2) 2. move per cpu stats to a distinct stats group 3. add documentation Nikolay Shirokovskiy (3): cgroup: extract interface part from virCgroupGetPercpuStats cgroup: reuse virCgroupGetPercpuTime in virCgroupGetPercpuVcpuSum add per cpu stats to all domain stats docs/news.xml | 9 ++ include/libvirt/libvirt-domain.h | 1 + src/libvirt-domain.c | 7 ++ src/libvirt_private.syms | 2 + src/qemu/qemu_driver.c | 64 ++++++++++++++ src/util/vircgroup.c | 178 +++++++++++++++++++++++++++------------ src/util/vircgroup.h | 17 ++++ tools/virsh-domain-monitor.c | 7 ++ tools/virsh.pod | 11 ++- 9 files changed, 238 insertions(+), 58 deletions(-) -- 1.8.3.1

2 7

[libvirt] OSX Homebrew support note ;)
by Justin Clift 06 Feb '17

06 Feb '17

Hi all, Compiling Libvirt directly from latest git master head is now supported on OSX Homebrew. It's as simple as: $ brew install --HEAD libvirt ... $ virsh -v 3.0.0 This should make development of Libvirt and related virtualisation things much easier on OSX. :) The credit for this goes to Roman Bogorodskiy and Homebrew developer "ilovezfs" (CC-d), for adding a functional rpcgen to Homebrew so we can generate the needed RPC bindings during the build (previously missing). Thanks heaps! :) Regards and best wishes, Justin Clift

4 7

[libvirt] [PATCH] libxl: fix dom0 autoballooning with Xen 4.8
by Jim Fehlig 03 Feb '17

03 Feb '17

xen.git commit 57f8b13c changed several of the libxl memory get/set functions to take 64 bit parameters. The libvirt libxl driver still uses uint32_t variables for these various parameters, which is particularly problematic for the libxl_set_memory_target() function. When dom0 autoballooning is enabled, libvirt (like xl) determines the memory needed to start a domain and the memory available. If memory available is less than memory needed, dom0 is ballooned down by passing a negative value to libxl_set_memory_target() 'target_memkb' parameter. Prior to xen.git commit 57f8b13c, 'target_memkb' was an int32_t. Subtracting a larger uint32 from a smaller uint32 and assigning it to int32 resulted in a negative number. After commit 57f8b13c, the same subtraction is widened to a int64, resulting in a large positive number. The simple fix taken by this patch is to assign the difference of the uint32 values to a temporary int32 variable, which is then passed to 'target_memkb' parameter of libxl_set_memory_target(). Note that it is undesirable to change libvirt to use 64 bit variables since it requires setting LIBXL_API_VERSION to 0x040800. Currently libvirt supports LIBXL_API_VERSION >= 0x040400, essentially Xen >= 4.4. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- src/libxl/libxl_domain.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index a5314b0..ed73cd2 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -909,6 +909,7 @@ libxlDomainFreeMem(libxl_ctx *ctx, libxl_domain_config *d_config) { uint32_t needed_mem; uint32_t free_mem; + int32_t target_mem; int tries = 3; int wait_secs = 10; @@ -922,7 +923,8 @@ libxlDomainFreeMem(libxl_ctx *ctx, libxl_domain_config *d_config) if (free_mem >= needed_mem) return 0; - if (libxl_set_memory_target(ctx, 0, free_mem - needed_mem, + target_mem = free_mem - needed_mem; + if (libxl_set_memory_target(ctx, 0, target_mem, /* relative */ 1, 0) < 0) goto error; -- 2.9.2

3 4

[libvirt] [RFC] Vhost-user backends cross-version migration support
by Maxime Coquelin 03 Feb '17

03 Feb '17

Hi, Few months ago, Michael reported a problem about migrating VMs relying on vhost-user between hosts supporting different backend versions: - Message-Id: <20161011173526-mutt-send-email-mst(a)kernel.org> - https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg03026.html The goal of this thread is to draft a proposal based on the outcomes of discussions with contributors of the different parties (DPDK/OVS /libvirt/...). Problem statement: ================== When migrating a VM from one host to another, the interfaces exposed by QEMU must stay unchanged in order to guarantee a successful migration. In the case of vhost-user interface, parameters like supported Virtio feature set, max number of queues, max vring sizes,... must remain compatible. Indeed, the frontend not being re-initialized, no renegotiation happens at migration time. For example, we have a VM that runs on host A, which has its vhost-user backend advertising VIRTIO_F_RING_INDIRECT_DESC feature. Since the Guest also support this feature, it is successfully negotiated, and guest transmit packets using indirect descriptor tables, that the backend knows to handle. At some point, the VM is being migrated to host B, which runs an older version of the backend not supporting this VIRTIO_F_RING_INDIRECT_DESC feature. The migration would break, because the Guest still have the VIRTIO_F_RING_INDIRECT_DESC bit sets, and the virtqueue contains some decriptors pointing to indirect tables, that backend B doesn't know to handle. This is just an example about Virtio features compatibility, but other backend implementation details could cause other failures. What we need is to be able to query the destination host's backend to ensure migration is possible. Also, we would need to query this statically, even before the VM is started, to be sure it could be migrated elsewhere for any reason. Solution 1: Libvirt queries DPDK vhost lib: *KO* ================================================ Initial idea was to have the management tool (libvirt) to query DPDK vhost lib and get key/value pairs and check whether migration is possible. This solution doesn't work for several reasons: 1. Vhost lib API provide a way for the application to disable features at runtime (understand, not at build time). So coming back to previous example, DPDK v16.11 supports indirect descriptor features, but it could be disabled by OVS. We had a look at whether this API was mandatory, and it turns out to be, as TSO feature is supported on DPDK but not in OVS. So we cannot rely on DPDK only. 2. Some parameter may be not only DPDK specific, such as the maximum number of queues for example. Solution 2: Libvirt queries OVS for vhost backend key/value pairs: *KO* ======================================================================= Second idea was for OVS to expose its vhost backend implementation parameters as key/value pairs, for example in the DB or by a dedicated tool. For example, you could have this kind of information: - virtio-features: 0x12045694 - max-rx-queues: 1024 - max-rx-ring-size: 512 Doing this, Libvirt has the information to take decision whether migration is possible or not. The problem is that Libvirt doesn't know (and want) to interpret these values (should it be equal/lower/greater/...?), and each time a new key is introduced in OVS, Libvirt will have to be updated to handle it, adding an unwanted synchronization constraint between the projects. Solution 3: Libvirt queries OVS for vhost backend version string: *OK* ====================================================================== The idea is to have a table of supported versions, associated to key/value pairs. Libvirt could query the list of supported versions strings for each hosts, and select the first common one among all hosts. Then, libvirt would ask OVS to probe the vhost-user interfaces in the selected version (compatibility mode). For example host A runs OVS-2.7, and host B OVS-2.6. Host A's OVS-2.7 has an OVS-2.6 compatibility mode (e.g. with indirect descriptors disabled), which should be selected at vhost-user interface probe time. Advantage of doing so is that libvirt does not need any update if new keys are introduced (i.e. it does not need to know how the new keys have to be handled), all these checks remain in OVS's vhost-user implementation. Ideally, we would support per vhost-user interface compatibility mode, which may have an impact also on DPDK API, as the Virtio feature update API is global, and not per port. - Implementation: ----------------- Goal here is just to illustrate this proposal, I'm sure you will have good suggestion to improve it. In OVS vhost-user library, we would introduce a new structure, for example (neither compiled nor tested): struct vhostuser_compat { char *version; uint64_t virtio_features; uint32_t max_rx_queue_sz; uint32_t max_nr_queues; }; *version* field is the compatibility version string. It could be something like: "upstream.ovs-dpdk.v2.6" In case for example Fedora adds some more patches to its package that would break migration to upstream version, it could have a dedicated compatibility string: "fc26.ovs-dpdk.v2.6". In case OVS-v2.7 does not break compatibility with previous OVS-v2.6 version, then no need to create a new compatibility entry, just keep v2.6 one. *virtio_features* field is the Virtio features set for a given compatibility version. When an OVS tag is to be created, it would be associated to a DPDK version. The Virtio features for these version would be stored in this field. It would allow to upgrade the DPDK package for example from v16.07 to v16.11 without breaking migration. In case the distribution wants to benefit from latests Virtio features, it would have to create a new entry to ensure migration won't be broken. *max_rx_queue_sz* *max_nr_queues* fields are just here for example, don't think this is needed today. I just want to illustrate that we have to anticipate other parameters than the Virtio feature set, even if not necessary at the moment. We create a table with different compatibility versions in OVS vhost-user lib: static struct vhostuser_compat vu_compat[] = { { .version = "upstream.ovs-dpdk.v2.7", .virtio_features = 0x12045694, .max_rx_queue_sz = 512, }, { .version = "upstream.ovs-dpdk.v2.6", .virtio_features = 0x10045694, .max_rx_queue_sz = 1024, }, } At some time during installation, or system init, the table would be parsed, and compatibility version strings would be stored into the OVS database, or a new tool would be created to list these strings. Before launching the VM, libvirt will query the version strings for each hosts using for example the JSON RPC API of OVS (maybe not the best solution, looking forward for your comments on this). Libvirt would then select the first common supported version, and insert this string into the vhost-user interfaces parameters in the OVS DBs of each host. When the vhost-user connection is initiated, OVS would know in which compatibility mode to init the interface, for example by restricting the support Virtio features of the interface. Do you think this is reasonable? Or maybe you have alternative ideas that would be best fit to ensure successful migration? Cheers, Maxime

6 29

[libvirt] [PATCH] HACKING: Update after recent change of the html file
by Peter Krempa 03 Feb '17

03 Feb '17

--- Pushed as trivial. HACKING | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/HACKING b/HACKING index ff545a8fa..fff003b9e 100644 --- a/HACKING +++ b/HACKING @@ -231,7 +231,8 @@ feature or changing the output of a program. such as adding new XML elements or fixing all but the most obscure bugs, must be (briefly) described in a release notes entry; changes that are only relevant to other libvirt developers, such as code refactoring, don't belong -in the release notes. +in the release notes. Note that "docs/news.xml" should be updated in its own +commit not to get in the way of backports. There is more on this subject, including lots of links to background reading on the subject, on Richard Jones' guide to working with open source projects -- 2.11.0

2 1

[libvirt] [PATCH] virtlockd: fix systemd unit file dependancies
by Daniel P. Berrange 03 Feb '17

03 Feb '17

After deploying virtlogd by default we identified a number of mistakes in the systemd unit file. virtlockd's relationship to libvirtd is the same as virtlogd, so we must apply the same unit file fixes to virtlockd Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- daemon/libvirtd.service.in | 1 + src/locking/virtlockd.service.in | 1 + src/locking/virtlockd.socket.in | 1 + 3 files changed, 3 insertions(+) diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in index bbf27da..c72dde5 100644 --- a/daemon/libvirtd.service.in +++ b/daemon/libvirtd.service.in @@ -6,6 +6,7 @@ [Unit] Description=Virtualization daemon Requires=virtlogd.socket +Requires=virtlockd.socket Before=libvirt-guests.service After=network.target After=dbus.service diff --git a/src/locking/virtlockd.service.in b/src/locking/virtlockd.service.in index 57089b0..69b568f 100644 --- a/src/locking/virtlockd.service.in +++ b/src/locking/virtlockd.service.in @@ -1,6 +1,7 @@ [Unit] Description=Virtual machine lock manager Requires=virtlockd.socket +Before=libvirtd.service Documentation=man:virtlockd(8) Documentation=http://libvirt.org diff --git a/src/locking/virtlockd.socket.in b/src/locking/virtlockd.socket.in index 9808bbb..45e0f20 100644 --- a/src/locking/virtlockd.socket.in +++ b/src/locking/virtlockd.socket.in @@ -1,5 +1,6 @@ [Unit] Description=Virtual machine lock manager socket +Before=libvirtd.service [Socket] ListenStream=@localstatedir@/run/libvirt/virtlockd-sock -- 2.9.3

3 2

[libvirt] [PATCH] docs: Release notes should be updated in a separate commit
by Andrea Bolognani 03 Feb '17

03 Feb '17

Updating docs/news.xml in the same commit that performs the documented change makes backports needlessly complicated, both for mainteinance branches and downstream distributions, because it introduces additional potential for merge conflicts. Document in the contributor guidelines that the release notes should be updated in a separate commit instead, so that it's easy to backport just the code change. --- docs/hacking.html.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/hacking.html.in b/docs/hacking.html.in index c7bbcbd..b1bb149 100644 --- a/docs/hacking.html.in +++ b/docs/hacking.html.in @@ -302,7 +302,8 @@ or fixing all but the most obscure bugs, must be (briefly) described in a release notes entry; changes that are only relevant to other libvirt developers, such as code refactoring, don't belong in the - release notes.</p> + release notes. Note that <code>docs/news.xml</code> should be updated + in its own commit not to get in the way of backports.</p> </li> </ol> -- 2.7.4

2 1

[libvirt] [V2 0/5] Support cache tune in libvirt
by Eli Qiao 03 Feb '17

03 Feb '17

This series patches are for supportting CAT featues, which also called cache tune in libvirt. First to expose cache information which could be tuned in capabilites XML. Then add new domain xml element support to add cacahe bank which will apply on this libvirt domain. This series patches add a util file `resctrl.c/h`, an interface to talk with linux kernel's sys fs. There are some TODO work such as logic of supportting l3code/l3data, expose new public interface to get free cache information. Some discussion about this feature support can be found from: https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html Eli Qiao (5): Resctrl: Add some utils functions Resctrl: expose cache information to capabilities Resctrl: Add new xml element to support cache tune Resctrl: Add private interface to set cachebanks Qemu: Set cache banks docs/schemas/domaincommon.rng | 41 ++ include/libvirt/virterror.h | 1 + src/Makefile.am | 1 + src/conf/capabilities.c | 56 +++ src/conf/capabilities.h | 23 + src/conf/domain_conf.c | 134 ++++++ src/conf/domain_conf.h | 18 + src/libvirt_private.syms | 10 + src/qemu/qemu_capabilities.c | 68 +++ src/qemu/qemu_driver.c | 4 + src/qemu/qemu_process.c | 10 + src/util/virerror.c | 1 + src/util/virresctrl.c | 994 ++++++++++++++++++++++++++++++++++++++++++ src/util/virresctrl.h | 135 ++++++ 14 files changed, 1496 insertions(+) create mode 100644 src/util/virresctrl.c create mode 100644 src/util/virresctrl.h -- 1.9.1

1 5

[libvirt] [PATCH 0/2] Fix random crash in qemuhotplugtest
by Peter Krempa 02 Feb '17

02 Feb '17

First patch makes the bug 100%[1] reproducible. *[1]: Race conditions may apply. Peter Krempa (2): DON'T APPLY: Reproducer to serialize threads "correctly" tests: qemuhotplug: Don't free the monitor object as part of @vm tests/qemuhotplugtest.c | 10 +++++++++- tests/qemumonitortestutils.c | 2 ++ 2 files changed, 11 insertions(+), 1 deletion(-) -- 2.11.0

2 4

[libvirt] [PATCH] tests: qemuhotplug: Fix memory leaks after cpu hotplug testing patches
by Peter Krempa 02 Feb '17

02 Feb '17

testQemuHotplugCpuDataFree leaked @data always and testQemuHotplugCpuPrepare leaked @prefix on success --- Pushed as trivial. tests/qemuhotplugtest.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemuhotplugtest.c b/tests/qemuhotplugtest.c index 9a203bd44..8cceb883e 100644 --- a/tests/qemuhotplugtest.c +++ b/tests/qemuhotplugtest.c @@ -377,6 +377,7 @@ testQemuHotplugCpuDataFree(struct testQemuHotplugCpuData *data) virObjectUnref(data->vm); qemuMonitorTestFree(data->mon); + VIR_FREE(data); } @@ -434,6 +435,8 @@ testQemuHotplugCpuPrepare(const char *test, if (qemuDomainRefreshVcpuInfo(&driver, data->vm, 0, false) < 0) goto error; + VIR_FREE(prefix); + return data; error: -- 2.11.0

1 0

[libvirt] [PATCH] qemu: turn on virtlockd by default
by Daniel P. Berrange 02 Feb '17

02 Feb '17

The virtlockd daemon has existed for years now, but we have never turned it on by default, requiring explicit user opt-in. This leaves users unprotected against accidents out of the box. By turning it on by default, users will at least be protected for mistakes involving local files, and files on shared filesystems that support fcntl() (eg NFS). In turning it on the various services files are updated to have the same dependancies for virtlockd as we have for virtlogd now, since turning the latter on exposed some gaps. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- daemon/libvirtd.service.in | 1 + src/locking/virtlockd.service.in | 1 + src/locking/virtlockd.socket.in | 1 + src/qemu/qemu.conf | 2 +- src/qemu/qemu_conf.c | 3 +++ 5 files changed, 7 insertions(+), 1 deletion(-) diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in index bbf27da..c72dde5 100644 --- a/daemon/libvirtd.service.in +++ b/daemon/libvirtd.service.in @@ -6,6 +6,7 @@ [Unit] Description=Virtualization daemon Requires=virtlogd.socket +Requires=virtlockd.socket Before=libvirt-guests.service After=network.target After=dbus.service diff --git a/src/locking/virtlockd.service.in b/src/locking/virtlockd.service.in index 57089b0..69b568f 100644 --- a/src/locking/virtlockd.service.in +++ b/src/locking/virtlockd.service.in @@ -1,6 +1,7 @@ [Unit] Description=Virtual machine lock manager Requires=virtlockd.socket +Before=libvirtd.service Documentation=man:virtlockd(8) Documentation=http://libvirt.org diff --git a/src/locking/virtlockd.socket.in b/src/locking/virtlockd.socket.in index 9808bbb..45e0f20 100644 --- a/src/locking/virtlockd.socket.in +++ b/src/locking/virtlockd.socket.in @@ -1,5 +1,6 @@ [Unit] Description=Virtual machine lock manager socket +Before=libvirtd.service [Socket] ListenStream=@localstatedir@/run/libvirt/virtlockd-sock diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index a8cd369..3239f7b 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -535,7 +535,7 @@ # share one writable disk, libvirt offers two approaches for # locking files. The first one is sanlock, the other one, # virtlockd, is then our own implementation. Accepted values -# are "sanlock" and "lockd". +# are "sanlock", "lockd", "nop". The default is "lockd". # #lock_manager = "lockd" diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 6613d59..d4c6cdc 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -314,6 +314,9 @@ virQEMUDriverConfigPtr virQEMUDriverConfigNew(bool privileged) cfg->glusterDebugLevel = 4; cfg->stdioLogD = true; + if (VIR_STRDUP(cfg->lockManagerName, "lockd") < 0) + goto error; + if (!(cfg->namespaces = virBitmapNew(QEMU_DOMAIN_NS_LAST))) goto error; -- 2.9.3

3 9

[libvirt] [PATCH] qemu: enforce maximum ports value for nec-xhci
by Ján Tomko 02 Feb '17

02 Feb '17

This controller only allows up to 15 ports. https://bugzilla.redhat.com/show_bug.cgi?id=1375417 --- src/qemu/qemu_domain.c | 8 ++++++++ .../qemuxml2argv-usb-controller-xhci-limit.xml | 16 ++++++++++++++++ tests/qemuxml2argvtest.c | 3 +++ 3 files changed, 27 insertions(+) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-usb-controller-xhci-limit.xml diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index c6ce090..61489e0 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3147,6 +3147,14 @@ qemuDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, if (dev->type == VIR_DOMAIN_DEVICE_CONTROLLER) { virDomainControllerDefPtr cont = dev->data.controller; + if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_USB && + cont->model == VIR_DOMAIN_CONTROLLER_MODEL_USB_NEC_XHCI && + cont->opts.usbopts.ports > 15) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("nec-xhci controller only supports up to 15 ports")); + goto cleanup; + } + if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_PCI) { if (cont->model == VIR_DOMAIN_CONTROLLER_MODEL_PCI_EXPANDER_BUS && !qemuDomainMachineIsI440FX(def)) { diff --git a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-xhci-limit.xml b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-xhci-limit.xml new file mode 100644 index 0000000..5ff2de7 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-xhci-limit.xml @@ -0,0 +1,16 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <devices> + <emulator>/usr/bin/qemu</emulator> + <controller type='usb' index='0' model='nec-xhci' ports='16'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 3532cb5..78cb609 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1415,6 +1415,9 @@ mymain(void) QEMU_CAPS_CHARDEV, QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_PIIX3_USB_UHCI, QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_NEC_USB_XHCI_PORTS, QEMU_CAPS_USB_HUB); + DO_TEST_PARSE_ERROR("usb-controller-xhci-limit", + QEMU_CAPS_CHARDEV, QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_PIIX3_USB_UHCI, + QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_NEC_USB_XHCI_PORTS); DO_TEST("smbios", QEMU_CAPS_SMBIOS_TYPE); DO_TEST_PARSE_ERROR("smbios-date", QEMU_CAPS_SMBIOS_TYPE); -- 2.10.2

2 1

[libvirt] [PATCH v3] Add support for Veritas HyperScale (VxHS) block device protocol
by Ashish Mittal 01 Feb '17

01 Feb '17

Sample XML for a vxhs vdisk is as follows: <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source protocol='vxhs' name='eb90327c-8302-4725-9e1b-4e85ed4dc251'> <host name='192.168.0.1' port='9999'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> <serial>eb90327c-8302-4725-9e1b-4e85ed4dc251</serial> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> Signed-off-by: Ashish Mittal <Ashish.Mittal(a)veritas.com> --- v2 changelog: (1) Added code for JSON parsing of a VxHS vdisk. (2) Added test case to verify JSON parsing. (3) Added missing switch-case checks for VIR_STORAGE_NET_PROTOCOL_VXHS. (4) Fixed line wrap in qemuxml2argv-disk-drive-network-vxhs.args. v3 changelog: (1) Implemented the modern syntax for VxHS disk specification. (2) Changed qemuxml2argvdata VxHS test case to verify the new syntax. (3) Added a negative test case to check failure when multiple hosts are specified for a VxHS disk. docs/formatdomain.html.in | 15 ++++- docs/schemas/domaincommon.rng | 1 + src/libxl/libxl_conf.c | 1 + src/qemu/qemu_command.c | 68 ++++++++++++++++++++++ src/qemu/qemu_driver.c | 3 + src/qemu/qemu_parse_command.c | 26 +++++++++ src/util/virstoragefile.c | 63 +++++++++++++++++++- src/util/virstoragefile.h | 1 + src/xenconfig/xen_xl.c | 1 + ...xml2argv-disk-drive-network-vxhs-multi-host.xml | 35 +++++++++++ .../qemuxml2argv-disk-drive-network-vxhs.args | 25 ++++++++ .../qemuxml2argv-disk-drive-network-vxhs.xml | 34 +++++++++++ tests/qemuxml2argvtest.c | 2 + tests/virstoragetest.c | 16 +++++ 14 files changed, 287 insertions(+), 4 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs-multi-host.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.xml diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index f7bef51..2a071c9 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -2319,9 +2319,9 @@ <dd> The <code>protocol</code> attribute specifies the protocol to access to the requested image. Possible values are "nbd", - "iscsi", "rbd", "sheepdog" or "gluster". If the - <code>protocol</code> attribute is "rbd", "sheepdog" or - "gluster", an additional attribute <code>name</code> is + "iscsi", "rbd", "sheepdog", "gluster" or "vxhs". If the + <code>protocol</code> attribute is "rbd", "sheepdog", "gluster" + or "vxhs", an additional attribute <code>name</code> is mandatory to specify which volume/image will be used. For "nbd", the <code>name</code> attribute is optional. For "iscsi" (<span class="since">since 1.0.4</span>), the <code>name</code> @@ -2329,6 +2329,9 @@ target's name by a slash (e.g., <code>iqn.2013-07.com.example:iscsi-pool/1</code>). If not specified, the default LUN is zero. + For "vxhs" (<span class="since">since 2.5.0</span>), the + <code>name</code> is the UUID of the volume, assigned by the + HyperScale sever. <span class="since">Since 0.8.7</span> </dd> <dt><code>volume</code></dt> @@ -2431,6 +2434,12 @@ <td> one or more (<span class="since">Since 2.1.0</span>), just one prior to that </td> <td> 24007 </td> </tr> + <tr> + <td> vxhs </td> + <td> a server running Veritas HyperScale daemon </td> + <td> only one </td> + <td> 9999 </td> + </tr> </table> <p> gluster supports "tcp", "rdma", "unix" as valid values for the diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 4d76315..cdc39ca 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -1470,6 +1470,7 @@ <value>ftp</value> <value>ftps</value> <value>tftp</value> + <value>vxhs</value> </choice> </attribute> <optional> diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index b569dda..7e12d32 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -604,6 +604,7 @@ libxlMakeNetworkDiskSrcStr(virStorageSourcePtr src, case VIR_STORAGE_NET_PROTOCOL_GLUSTER: case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: case VIR_STORAGE_NET_PROTOCOL_SSH: + case VIR_STORAGE_NET_PROTOCOL_VXHS: case VIR_STORAGE_NET_PROTOCOL_LAST: case VIR_STORAGE_NET_PROTOCOL_NONE: virReportError(VIR_ERR_NO_SUPPORT, diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index f8e48d2..08bad8e 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -491,6 +491,9 @@ qemuNetworkDriveGetPort(int protocol, /* no default port specified */ return 0; + case VIR_STORAGE_NET_PROTOCOL_VXHS: + return 9999; + case VIR_STORAGE_NET_PROTOCOL_RBD: case VIR_STORAGE_NET_PROTOCOL_LAST: case VIR_STORAGE_NET_PROTOCOL_NONE: @@ -899,6 +902,65 @@ qemuBuildGlusterDriveJSON(virStorageSourcePtr src) } +#define QEMU_DEFAULT_VXHS_PORT "9999" + +/* Build the VxHS host object */ +static virJSONValuePtr +qemuBuildVxHSDriveJSONHost(virStorageSourcePtr src) +{ + virJSONValuePtr server = NULL; + virStorageNetHostDefPtr host; + const char *portstr; + + if (src->nhosts != 1) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("VxHS supports only one server")); + goto cleanup; + } + + host = src->hosts; + portstr = host->port; + + if (!portstr) + portstr = QEMU_DEFAULT_VXHS_PORT; + + if (virJSONValueObjectCreate(&server, + "s:host", host->name, + "s:port", portstr, + NULL) < 0) + server = NULL; + + cleanup: + return server; +} + + +static virJSONValuePtr +qemuBuildVxHSDriveJSON(virStorageSourcePtr src) +{ + const char *protocol = virStorageNetProtocolTypeToString(src->protocol); + virJSONValuePtr server = NULL; + virJSONValuePtr ret = NULL; + + if (!(server = qemuBuildVxHSDriveJSONHost(src))) + return NULL; + + /* VxHS disk sepecification example: + * { driver:"vxhs", + * vdisk-id:"eb90327c-8302-4725-4e85ed4dc251", + * server.host:"1.2.3.4", + * server.port:1234} + */ + if (virJSONValueObjectCreate(&ret, + "s:driver", protocol, + "s:vdisk-id", src->path, + "a:server", server, NULL) < 0) + virJSONValueFree(server); + + return ret; +} + + static char * qemuBuildNetworkDriveURI(virStorageSourcePtr src, qemuDomainSecretInfoPtr secinfo) @@ -1034,6 +1096,7 @@ qemuBuildNetworkDriveStr(virStorageSourcePtr src, case VIR_STORAGE_NET_PROTOCOL_TFTP: case VIR_STORAGE_NET_PROTOCOL_ISCSI: case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_VXHS: ret = qemuBuildNetworkDriveURI(src, secinfo); break; @@ -1148,6 +1211,11 @@ qemuGetDriveSourceProps(virStorageSourcePtr src, if (!(fileprops = qemuBuildGlusterDriveJSON(src))) return -1; } + + if (src->protocol == VIR_STORAGE_NET_PROTOCOL_VXHS) { + if (!(fileprops = qemuBuildVxHSDriveJSON(src))) + return -1; + } break; } diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index b359e77..c76e9d4 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -13788,6 +13788,7 @@ qemuDomainSnapshotPrepareDiskExternalBackingInactive(virDomainDiskDefPtr disk) case VIR_STORAGE_NET_PROTOCOL_FTPS: case VIR_STORAGE_NET_PROTOCOL_TFTP: case VIR_STORAGE_NET_PROTOCOL_SSH: + case VIR_STORAGE_NET_PROTOCOL_VXHS: case VIR_STORAGE_NET_PROTOCOL_LAST: virReportError(VIR_ERR_INTERNAL_ERROR, _("external inactive snapshots are not supported on " @@ -13851,6 +13852,7 @@ qemuDomainSnapshotPrepareDiskExternalOverlayActive(virDomainSnapshotDiskDefPtr d case VIR_STORAGE_NET_PROTOCOL_FTPS: case VIR_STORAGE_NET_PROTOCOL_TFTP: case VIR_STORAGE_NET_PROTOCOL_SSH: + case VIR_STORAGE_NET_PROTOCOL_VXHS: case VIR_STORAGE_NET_PROTOCOL_LAST: virReportError(VIR_ERR_INTERNAL_ERROR, _("external active snapshots are not supported on " @@ -13996,6 +13998,7 @@ qemuDomainSnapshotPrepareDiskInternal(virConnectPtr conn, case VIR_STORAGE_NET_PROTOCOL_FTPS: case VIR_STORAGE_NET_PROTOCOL_TFTP: case VIR_STORAGE_NET_PROTOCOL_SSH: + case VIR_STORAGE_NET_PROTOCOL_VXHS: case VIR_STORAGE_NET_PROTOCOL_LAST: virReportError(VIR_ERR_INTERNAL_ERROR, _("internal inactive snapshots are not supported on " diff --git a/src/qemu/qemu_parse_command.c b/src/qemu/qemu_parse_command.c index 405e655..aab287a 100644 --- a/src/qemu/qemu_parse_command.c +++ b/src/qemu/qemu_parse_command.c @@ -263,6 +263,17 @@ qemuParseNBDString(virDomainDiskDefPtr disk) return -1; } +static int +qemuParseVxHSString(virDomainDiskDefPtr def) +{ + virURIPtr uri = NULL; + + if (!(uri = virURIParse(def->src->path))) + return -1; + + return qemuParseDriveURIString(def, uri, "vxhs"); +} + /* * This method takes a string representing a QEMU command line ARGV set @@ -737,6 +748,12 @@ qemuParseCommandLineDisk(virDomainXMLOptionPtr xmlopt, if (VIR_STRDUP(def->src->path, vdi) < 0) goto error; } + } else if (STRPREFIX(def->src->path, "vxhs:")) { + def->src->type = VIR_STORAGE_TYPE_NETWORK; + def->src->protocol = VIR_STORAGE_NET_PROTOCOL_VXHS; + + if (qemuParseVxHSString(def) < 0) + goto error; } else { def->src->type = VIR_STORAGE_TYPE_FILE; } @@ -1947,6 +1964,10 @@ qemuParseCommandLine(virCapsPtr caps, disk->src->type = VIR_STORAGE_TYPE_NETWORK; disk->src->protocol = VIR_STORAGE_NET_PROTOCOL_SHEEPDOG; val += strlen("sheepdog:"); + } else if (STRPREFIX(val, "vxhs:")) { + disk->src->type = VIR_STORAGE_TYPE_NETWORK; + disk->src->protocol = VIR_STORAGE_NET_PROTOCOL_VXHS; + val += strlen("vxhs:"); } else { disk->src->type = VIR_STORAGE_TYPE_FILE; } @@ -2023,6 +2044,11 @@ qemuParseCommandLine(virCapsPtr caps, goto error; break; + case VIR_STORAGE_NET_PROTOCOL_VXHS: + if (qemuParseVxHSString(disk) < 0) + goto error; + + break; case VIR_STORAGE_NET_PROTOCOL_HTTP: case VIR_STORAGE_NET_PROTOCOL_HTTPS: case VIR_STORAGE_NET_PROTOCOL_FTP: diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c index ce6d213..e62f4e6 100644 --- a/src/util/virstoragefile.c +++ b/src/util/virstoragefile.c @@ -85,7 +85,8 @@ VIR_ENUM_IMPL(virStorageNetProtocol, VIR_STORAGE_NET_PROTOCOL_LAST, "ftp", "ftps", "tftp", - "ssh") + "ssh", + "vxhs") VIR_ENUM_IMPL(virStorageNetHostTransport, VIR_STORAGE_NET_HOST_TRANS_LAST, "tcp", @@ -2633,6 +2634,7 @@ virStorageSourceParseBackingColon(virStorageSourcePtr src, case VIR_STORAGE_NET_PROTOCOL_ISCSI: case VIR_STORAGE_NET_PROTOCOL_GLUSTER: case VIR_STORAGE_NET_PROTOCOL_SSH: + case VIR_STORAGE_NET_PROTOCOL_VXHS: virReportError(VIR_ERR_INTERNAL_ERROR, _("malformed backing store path for protocol %s"), protocol); @@ -2964,6 +2966,64 @@ virStorageSourceParseBackingJSONRBD(virStorageSourcePtr src, } +static int +virStorageSourceParseBackingJSONVXHS(virStorageSourcePtr src, + virJSONValuePtr json, + int opaque ATTRIBUTE_UNUSED) +{ + virJSONValuePtr server; + const char *hostname, *port; + const char *uri = virJSONValueObjectGetString(json, "filename"); + const char *vdisk_id = virJSONValueObjectGetString(json, "vdisk-id"); + + src->type = VIR_STORAGE_TYPE_NETWORK; + src->protocol = VIR_STORAGE_NET_PROTOCOL_VXHS; + + /* legacy URI based syntax passed via 'filename' option */ + if (uri) + return virStorageSourceParseBackingJSONUriStr(src, uri, + VIR_STORAGE_NET_PROTOCOL_VXHS); + + server = virJSONValueObjectGetObject(json, "server"); + hostname = virJSONValueObjectGetString(server, "host"); + port = virJSONValueObjectGetString(server, "port"); + + if (!vdisk_id) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("missing 'vdisk-id' attribute in " + "JSON backing definition for VxHS volume")); + return -1; + } + if (!server) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("missing 'server' attribute in " + "JSON backing definition for VxHS volume")); + return -1; + } + if (!hostname) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("missing hostname for tcp backing server in " + "JSON backing definition for VxHS volume")); + return -1; + } + + + if (VIR_STRDUP(src->path, vdisk_id) < 0) + return -1; + + if (VIR_ALLOC_N(src->hosts, 1) < 0) + return -1; + src->nhosts = 1; + + src->hosts[0].transport = VIR_STORAGE_NET_HOST_TRANS_TCP; + + if (VIR_STRDUP(src->hosts[0].name, hostname) < 0 || + VIR_STRDUP(src->hosts[0].port, port) < 0) + return -1; + + return 0; +} + struct virStorageSourceJSONDriverParser { const char *drvname; int (*func)(virStorageSourcePtr src, virJSONValuePtr json, int opaque); @@ -2985,6 +3045,7 @@ static const struct virStorageSourceJSONDriverParser jsonParsers[] = { {"sheepdog", virStorageSourceParseBackingJSONSheepdog, 0}, {"ssh", virStorageSourceParseBackingJSONSSH, 0}, {"rbd", virStorageSourceParseBackingJSONRBD, 0}, + {"vxhs", virStorageSourceParseBackingJSONVXHS, 0}, }; diff --git a/src/util/virstoragefile.h b/src/util/virstoragefile.h index 1f62244..e60b6e9 100644 --- a/src/util/virstoragefile.h +++ b/src/util/virstoragefile.h @@ -134,6 +134,7 @@ typedef enum { VIR_STORAGE_NET_PROTOCOL_FTPS, VIR_STORAGE_NET_PROTOCOL_TFTP, VIR_STORAGE_NET_PROTOCOL_SSH, + VIR_STORAGE_NET_PROTOCOL_VXHS, VIR_STORAGE_NET_PROTOCOL_LAST } virStorageNetProtocol; diff --git a/src/xenconfig/xen_xl.c b/src/xenconfig/xen_xl.c index 18d9fe3..9fe24d6 100644 --- a/src/xenconfig/xen_xl.c +++ b/src/xenconfig/xen_xl.c @@ -942,6 +942,7 @@ xenFormatXLDiskSrcNet(virStorageSourcePtr src) case VIR_STORAGE_NET_PROTOCOL_GLUSTER: case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: case VIR_STORAGE_NET_PROTOCOL_SSH: + case VIR_STORAGE_NET_PROTOCOL_VXHS: case VIR_STORAGE_NET_PROTOCOL_LAST: case VIR_STORAGE_NET_PROTOCOL_NONE: virReportError(VIR_ERR_NO_SUPPORT, diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs-multi-host.xml b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs-multi-host.xml new file mode 100644 index 0000000..660bcde --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs-multi-host.xml @@ -0,0 +1,35 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <disk type='network' device='disk'> + <driver name='qemu' type='raw' cache='none'/> + <source protocol='vxhs' name='eb90327c-8302-4725-9e1b-4e85ed4dc251'> + <host name='192.168.0.1' port='9999'/> + <host name='192.168.0.2' port='9999'/> + </source> + <backingStore/> + <target dev='vda' bus='virtio'/> + <serial>eb90327c-8302-4725-9e1b-4e85ed4dc251</serial> + <alias name='virtio-disk0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.args b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.args new file mode 100644 index 0000000..23045e1 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.args @@ -0,0 +1,25 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/libexec/qemu-kvm \ +-name QEMUGuest1 \ +-S \ +-M pc \ +-cpu qemu32 \ +-m 214 \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-nographic \ +-nodefaults \ +-monitor unix:/tmp/lib/domain--1-QEMUGuest1/monitor.sock,server,nowait \ +-no-acpi \ +-boot c \ +-usb \ +-drive file.driver=vxhs,file.vdisk-id=eb90327c-8302-4725-9e1b-4e85ed4dc251,\ +file.server.host=192.168.0.1,file.server.port=9999,format=raw,if=none,\ +id=drive-virtio-disk0,cache=none \ +-device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,\ +id=virtio-disk0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.xml b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.xml new file mode 100644 index 0000000..45c807f --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.xml @@ -0,0 +1,34 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <disk type='network' device='disk'> + <driver name='qemu' type='raw' cache='none'/> + <source protocol='vxhs' name='eb90327c-8302-4725-9e1b-4e85ed4dc251'> + <host name='192.168.0.1' port='9999'/> + </source> + <backingStore/> + <target dev='vda' bus='virtio'/> + <serial>eb90327c-8302-4725-9e1b-4e85ed4dc251</serial> + <alias name='virtio-disk0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index ab3ad08..f751ec9 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -892,6 +892,8 @@ mymain(void) # endif DO_TEST("disk-drive-network-rbd-ipv6", NONE); DO_TEST_FAILURE("disk-drive-network-rbd-no-colon", NONE); + DO_TEST("disk-drive-network-vxhs", NONE); + DO_TEST_FAILURE("disk-drive-network-vxhs-multi-host", NONE); DO_TEST("disk-drive-no-boot", QEMU_CAPS_BOOTINDEX); DO_TEST_PARSE_ERROR("disk-device-lun-type-invalid", diff --git a/tests/virstoragetest.c b/tests/virstoragetest.c index f766df1..a28fbd9 100644 --- a/tests/virstoragetest.c +++ b/tests/virstoragetest.c @@ -1492,6 +1492,22 @@ mymain(void) "<source protocol='rbd' name='testshare'>\n" " <host name='example.com'/>\n" "</source>\n"); + TEST_BACKING_PARSE("json:{\"file.driver\":\"vxhs\"," + "\"file.filename\":\"vxhs://192.168.0.1:9999/c6718f6b-0401-441d-a8c3-1f0064d75ee0\"" + "}", + "<source protocol='vxhs' name='c6718f6b-0401-441d-a8c3-1f0064d75ee0'>\n" + " <host name='192.168.0.1' port='9999'/>\n" + "</source>\n"); + TEST_BACKING_PARSE("json:{\"file\":{\"driver\":\"vxhs\"," + "\"vdisk-id\":\"c6718f6b-0401-441d-a8c3-1f0064d75ee0\"," + "\"server\": { \"host\":\"example.com\"," + "\"port\":\"1234\"" + "}" + "}" + "}", + "<source protocol='vxhs' name='c6718f6b-0401-441d-a8c3-1f0064d75ee0'>\n" + " <host name='example.com' port='1234'/>\n" + "</source>\n"); cleanup: /* Final cleanup */ -- 2.5.5

4 6

[libvirt] [PATCH] qemu: add host per cpu info to all domain stats
by Nikolay Shirokovskiy 01 Feb '17

01 Feb '17

Info provided in virDomainGetCPUStats is now missed in all domain stats. This patch removes this discrepancy. Output example: cpu.count=2 cpu.0.time=536163399467 cpu.1.time=453846564946 cpu.0.vtime=530053197220 cpu.1.vtime=446078894510 --- src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++- src/util/vircgroup.c | 2 +- src/util/vircgroup.h | 5 ++++ 4 files changed, 77 insertions(+), 2 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index a2866a3..40f284c 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1343,6 +1343,7 @@ virCgroupGetMemoryUsage; virCgroupGetMemSwapHardLimit; virCgroupGetMemSwapUsage; virCgroupGetPercpuStats; +virCgroupGetPercpuVcpuSum; virCgroupHasController; virCgroupHasEmptyTasks; virCgroupKill; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 16b435a..61ae60e 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -18768,7 +18768,16 @@ qemuDomainGetStatsCpu(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, unsigned long long cpu_time = 0; unsigned long long user_time = 0; unsigned long long sys_time = 0; + unsigned long long *sum_cpu_time = NULL; + virBitmapPtr cpumap = NULL; + virBitmapPtr guestvcpus = NULL; + char *buf = NULL; + char *pos; + char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; + int ncpu; int err = 0; + int ret = -1; + size_t i; if (!priv->cgroup) return 0; @@ -18795,7 +18804,67 @@ qemuDomainGetStatsCpu(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, sys_time) < 0) return -1; - return 0; + if ((cpumap = virHostCPUGetPresentBitmap())) { + ncpu = virBitmapSize(cpumap); + + if (virTypedParamsAddULLong(&record->params, + &record->nparams, + maxparams, + "cpu.count", + ncpu) < 0) + goto cleanup; + + if (!virCgroupGetCpuacctPercpuUsage(priv->cgroup, &buf)) { + pos = buf; + for (i = 0; i < ncpu; i++) { + unsigned long long time = 0; + + if (virBitmapIsBitSet(cpumap, i) && + virStrToLong_ull(pos, &pos, 10, &time) < 0) { + VIR_WARN("cpuacct parse error"); + continue; + } + + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.%zu.time", i); + if (virTypedParamsAddULLong(&record->params, + &record->nparams, + maxparams, + param_name, + time) < 0) + goto cleanup; + } + } + + if (qemuDomainHasVcpuPids(dom) && + (guestvcpus = virDomainDefGetOnlineVcpumap(dom->def)) && + !VIR_ALLOC_N(sum_cpu_time, ncpu) && + !virCgroupGetPercpuVcpuSum(priv->cgroup, guestvcpus, + sum_cpu_time, ncpu, cpumap)) { + for (i = 0; i < ncpu; i++) { + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.%zu.vtime", i); + if (virTypedParamsAddULLong(&record->params, + &record->nparams, + maxparams, + param_name, + sum_cpu_time[i]) < 0) + goto cleanup; + } + } + } + + ret = 0; + + cleanup: + if (!ret && virGetLastError()) + virResetLastError(); + virBitmapFree(cpumap); + virBitmapFree(guestvcpus); + VIR_FREE(sum_cpu_time); + VIR_FREE(buf); + + return ret; } static int diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index 5aa1db5..0b36609 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -3122,7 +3122,7 @@ virCgroupDenyDevicePath(virCgroupPtr group, * s2 = t02 + t12 * s3 = t03 + t13 */ -static int +int virCgroupGetPercpuVcpuSum(virCgroupPtr group, virBitmapPtr guestvcpus, unsigned long long *sum_cpu_time, diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index 2de1bf2..5f71618 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -297,4 +297,9 @@ int virCgroupSetOwner(virCgroupPtr cgroup, int virCgroupHasEmptyTasks(virCgroupPtr cgroup, int controller); bool virCgroupControllerAvailable(int controller); +int virCgroupGetPercpuVcpuSum(virCgroupPtr group, + virBitmapPtr guestvcpus, + unsigned long long *sum_cpu_time, + size_t nsum, + virBitmapPtr cpumap); #endif /* __VIR_CGROUP_H__ */ -- 1.8.3.1

2 2

[libvirt] [PATCH] qemu: Fix indentation in qemu_interface.h
by Nitesh Konkar 01 Feb '17

01 Feb '17

Signed-off-by: Nitesh Konkar <nitkon12(a)linux.vnet.ibm.com> --- src/qemu/qemu_interface.h | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_interface.h b/src/qemu/qemu_interface.h index 66ead2a..a7faa0b 100644 --- a/src/qemu/qemu_interface.h +++ b/src/qemu/qemu_interface.h @@ -35,16 +35,17 @@ int qemuInterfaceStopDevice(virDomainNetDefPtr net); int qemuInterfaceStopDevices(virDomainDefPtr def); int qemuInterfaceDirectConnect(virDomainDefPtr def, - virQEMUDriverPtr driver, - virDomainNetDefPtr net, - int *tapfd, - size_t tapfdSize, - virNetDevVPortProfileOp vmop); -int qemuInterfaceEthernetConnect(virDomainDefPtr def, virQEMUDriverPtr driver, virDomainNetDefPtr net, int *tapfd, - size_t tapfdSize); + size_t tapfdSize, + virNetDevVPortProfileOp vmop); + +int qemuInterfaceEthernetConnect(virDomainDefPtr def, + virQEMUDriverPtr driver, + virDomainNetDefPtr net, + int *tapfd, + size_t tapfdSize); int qemuInterfaceBridgeConnect(virDomainDefPtr def, virQEMUDriverPtr driver, -- 2.1.0

2 1

[libvirt] char: Logging serial pty output when disconnected
by Ed Swierk 31 Jan '17

31 Jan '17

Interactive access to a guest serial console can be enabled by hooking the serial device to a pty backend, e.g. -device isa-serial,chardev=cs0 -chardev pty,id=cs0. With libvirt this can be configured via <console type='pty'><target type='serial' port='0'/></console>. Output from the same serial device can also be logged to a file by adding logfile=/somefile to the -chardev option (<log file=/somefile/> in libvirt). Unfortunately output gets logged only when a client like virsh console is connected to the pty; otherwise qemu drops it on the floor. This makes chardev logging much less useful than it could be for debugging guest problems after the fact. Currently qemu_chr_fe_write() calls qemu_chr_fe_write_log() only for data consumed by the backend chr_write function. With the pty backend, pty_chr_write() returns 0 indicating that the data was not consumed when the pty is disconnected. Simply changing it to return len instead of 0 tricks the caller into logging the data even when the pty is disconnected. I don't know what problems this might cause, but one data point is that tcp_chr_write() already happens to work this way. Alternatively, qemu_chr_fe_write() could be modified to log everything passed to it, regardless of how much data chr_write claims to have consumed. The trouble is that the serial device retries writing unconsumed data, so when the pty is disconnected you'd see every character duplicated 4 times in the log file. Any opinions on either approach, or other suggestions? If there are no objections to the first one, I'll prepare a patch. --Ed

3 5

[libvirt] CfP 12th Virtualization in High-Performance Cloud Computing Workshop (VHPC '17)
by VHPC 17 31 Jan '17

31 Jan '17

==================================================================== CALL FOR PAPERS 12th Workshop on Virtualization in High-Performance Cloud Computing (VHPC '17) held in conjunction with the International Supercomputing Conference - High Performance, June 18-22, 2017, Frankfurt, Germany. (Springer LNCS Proceedings) ==================================================================== Date: June 22, 2017 Workshop URL: http://vhpc.org Abstract Submission Deadline: February 28, 2017 Paper Submission Deadline: April 25, 2017 (Springer LNCS) Abstract/Paper Submission Link: https://edas.info/newPaper.php?c=23179 Call for Papers Virtualization technologies constitute a key enabling factor for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to manage complex infrastructures in a seamless fashion to support the highly dynamic and heterogeneous workloads and hosted applications customers deploy. Similarly, HPC environments have been increasingly adopting techniques that enable flexible management of vast computing and networking resources, close to marginal provisioning cost, which is unprecedented in the history of scientific and commercial computing. Various virtualization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple underutilized servers with heterogeneous software and operating systems (OSes), and its capability to live-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-level virtualization (i.e., containerization), with its capability to isolate multiple user-space environments and to allow for their coexistence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; I/O Virtualization allows physical NICs/HBAs to take traffic from multiple VMs or containers; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology and IP addressing, provides the fundamental ground on top of which evolved network services can be realized with an unprecedented level of dynamicity and flexibility; the increasingly adopted paradigm of Software-Defined Networking (SDN) promises to extend this flexibility to the control and data planes of network paths. Publication Accepted papers will be published in a Springer LNCS proceedings volume. Topics of Interest The VHPC program committee solicits original, high-quality submissions related to virtualization across the entire software stack with a special focus on the intersection of HPC and the cloud. Major Topics - Virtualization in supercomputing environments, HPC clusters, HPC in the cloud and grids - OS-level virtualization and containers (Docker, rkt, Singularity, Shifter, i.a.) - Lightweight/specialized operating systems, unikernels - Optimizations of virtual machine monitor platforms and hypervisors - Hypervisor support for heterogenous resources (GPUs, co-processors, FPGAs, etc.) - Virtualization support for emerging memory technologies - Virtualization in enterprise HPC and microvisors - Software defined networks and network virtualization - Management, deployment of virtualized environments and orchestration (Kubernetes i.a.), - Workflow-pipeline container-based composability - Performance measurement, modelling and monitoring of virtualized/cloud workloads - Virtualization in data intensive computing and Big Data processing - HPC convergence - Adaptation of HPC technologies in the cloud (high performance networks, RDMA, etc.) - ARM-based hypervisors, ARM virtualization extensions - I/O virtualization and cloud based storage systems - GPU, FPGA and many-core accelerator virtualization - Job scheduling/control/policy and container placement in virtualized environments - Cloud reliability, fault-tolerance and high-availability - QoS and SLA in virtualized environments - IaaS platforms, cloud frameworks and APIs - Large-scale virtualization in domains such as finance and government - Energy-efficient and power-aware virtualization - Container security - Configuration management tools for containers (including CFEngine, Puppet, i.a.) - Emerging topics including multi-kernel approaches and,NUMA in hypervisors The Workshop on Virtualization in High-Performance Cloud Computing (VHPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, plus lightning talks that are limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. Important Dates February 28, 2017 - Abstract Submission Deadline April 25, 2017 - Paper submission deadline May 30, 2017 - Acceptance notification June 22, 2017 - Workshop Day June 25, 2017 - Camera-ready version due Chair Michael Alexander (chair), scaledinfra technologies, Austria Anastassios Nanos (co-chair), NTUA, Greece Balazs Gerofi (co-chair), RIKEN Advanced Institute for Computational Science, Japan Program committee Stergios Anastasiadis, University of Ioannina, Greece Jakob Blomer, CERN, Europe Ron Brightwell, Sandia National Laboratories, USA Eduardo César, Universidad Autonoma de Barcelona, Spain Julian Chesterfield, OnApp, UK Stephen Crago, USC ISI, USA Christoffer Dall, Columbia University, USA Patrick Dreher, MIT, USA Robert Futrick, Cycle Computing, USA Maria Girone, CERN, Europe Kyle Hale, Northwestern University, USA Romeo Kinzler, IBM, Switzerland Brian Kocoloski, University of Pittsburgh, USA Nectarios Koziris, National Technical University of Athens, Greece John Lange, University of Pittsburgh, USA Che-Rung Lee, National Tsing Hua University, Taiwan Giuseppe Lettieri, University of Pisa, Italy Qing Liu, Oak Ridge National Laboratory, USA Nikos Parlavantzas, IRISA, France Kevin Pedretti, Sandia National Laboratories, USA Amer Qouneh, University of Florida, USA Carlos Reaño, Technical University of Valencia, Spain Thomas Ryd, CFEngine, Norway Josh Simons, VMWare, USA Borja Sotomayor, University of Chicago, USA Craig Stewart, Indiana University, USA Anata Tiwari, San Diego Supercomputer Center, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden Yasuhiro Watashiba, Osaka University, Japan Nicholas Wright, Lawrence Berkeley National Laboratory, USA Chao-Tung Yang, Tunghai University, Taiwan Paper Submission-Publication Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, key words, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. Accepted papers will be published in a Springer LNCS volume. . The format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Format Guidelines: ftp://ftp.springer.de/pub/tex/latex/llncs/latex2e/llncs2e.zip Abstract, Paper Submission Link: https://edas.info/newPaper.php?c=23179 Lightning Talks Lightning Talks are non-paper track, synoptical in nature and are strictly limited to 5 minutes. They can be used to gain early feedback on ongoing research, for demonstrations, to present research results, early research ideas, perspectives and positions of interest to the community. Submit abstract via the main submission link. General Information The workshop is one day in length and will be held in conjunction with the International Supercomputing Conference - High Performance (ISC) 2017, June 18-22, Frankfurt, Germany.

1 0

[libvirt] [PATCH] util: Fix domain object leaks on closecallbacks
by John Ferlan 31 Jan '17

31 Jan '17

Originally/discovered proposed by "Wang King <king.wang(a)huawei.com>" When the virCloseCallbacksSet is first called, it increments the refcnt on the domain object to ensure it doesn't get deleted before the callback is called. The refcnt would be decremented in virCloseCallbacksUnset once the entry is removed from the closeCallbacks has table. When (mostly) normal shutdown occurs, the qemuProcessStop will end up calling qemuProcessAutoDestroyRemove and will remove the callback from the list and hash table normally and decrement the refcnt. However, when qemuConnectClose calls virCloseCallbacksRun, it will scan the (locked) closeCallbacks list for matching domain and callback function. If an entry is found, it will be removed from the closeCallbacks list and placed into a lookaside list to be processed when the closeCallbacks lock is dropped. The callback function (e.g. qemuProcessAutoDestroy) is called and will run qemuProcessStop. That code will fail to find the callback in the list when qemuProcessAutoDestroyRemove is called and thus not decrement the domain refcnt. Instead since the entry isn't found the code will just return (mostly) harmlessly. This patch will resolve the issue by taking another ref during the search UUID process during virCloseCallackRun, decrementing the refcnt taken by virCloseCallbacksSet, calling the callback routine and returning overwriting the vm (since it could return NULL). Finally, it will call the virDomainObjEndAPI to lower the refcnt and remove the lock taken during the search UUID processing. This may cause the vm to be destroyed. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- src/util/virclosecallbacks.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/src/util/virclosecallbacks.c b/src/util/virclosecallbacks.c index 2f430cf..4db50e8 100644 --- a/src/util/virclosecallbacks.c +++ b/src/util/virclosecallbacks.c @@ -346,17 +346,24 @@ virCloseCallbacksRun(virCloseCallbacksPtr closeCallbacks, for (i = 0; i < list->nentries; i++) { virDomainObjPtr vm; - if (!(vm = virDomainObjListFindByUUID(domains, - list->entries[i].uuid))) { + /* Grab a ref and lock to the vm */ + if (!(vm = virDomainObjListFindByUUIDRef(domains, + list->entries[i].uuid))) { char uuidstr[VIR_UUID_STRING_BUFLEN]; virUUIDFormat(list->entries[i].uuid, uuidstr); VIR_DEBUG("No domain object with UUID %s", uuidstr); continue; } - vm = list->entries[i].callback(vm, conn, opaque); - if (vm) - virObjectUnlock(vm); + /* Remove the ref taken out during virCloseCallbacksSet since + * we're about to call the callback function and we have another + * ref anyway (so it cannot be deleted). + * + * Call the callback function, ignoring the return since it might be + * NULL. Once we're done with the object, then end the API usage. */ + virObjectUnref(vm); + ignore_value(list->entries[i].callback(vm, conn, opaque)); + virDomainObjEndAPI(&vm); } VIR_FREE(list->entries); VIR_FREE(list); -- 2.7.4

2 1