- Devel - libvirt List Archives

[libvirt] [RFC v2 00/20] qmp: Report bus information on 'query-machines'
by Eduardo Habkost 29 Nov '16

29 Nov '16

Changes v1 -> v2: * v1 series subject was: "qmp: Report supported device types on 'query-machines'" * Now we return additional bus information: bus ID, bus type, and the list of accepted device types on each bus * Now hotplug can be covered because accepted-device-types can be set by individual bus instances if necessary * Now the new field is optional, and every machine having the new field are now validated in "strict mode" on test code (meaning all buses must be present on the list) * TODO: only PC was converted, machines from other architectures don't include the field yet * Legacy-PCI vs PCIe is now handled: * PCIDeviceClass::is_express field was removed * Defined INTERFACE_LEGACY_PCI_DEVICE and INTERFACE_PCIE_DEVICE, buses and devices now report appropriate interface names * Now PCIe buses can return the right information because each bus instance can set its own accepted-device-types list * TODO: PCIe pci-bridge classes need to be changed to not include INTERFACE_LEGACY_PCI_DEVICE * TODO: replace q35 hack with appropriate code that set Legacy-PCI rules in PCI code * Removed patches from v1: * pc: Initialize default bus lists * s390x: Initialize default bus lists * arm: Initialize default bus lists * mips: Initialize default bus lists * ppc: Initialize default bus lists * qdev: Add device_class_set_bus_type() function (validation is more difficult with the PCI/PCIe rules, plan is to try to remove DeviceClass::bus_type field, se "Limitations" below) * See individual patches for additional details The Problem =========== Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized. Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types. Example: floppy support on q35 vs i440fx ---------------------------------------- There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default. With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not. Example: Legacy PCI vs vs PCIe devices -------------------------------------- Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus. Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names. Example: spapr and PCIe root bus -------------------------------- See the thread at: Subject: [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default If we make new spapr machine-type versions create a PCIe root bus, management software will need a way to find out: 1) The type of the default bus for the machine type; 2) The ID of the default bus for the machine type. Otherwise, management software will have to hardcode it based on machine-type version. The proposed interface should solve this problem. The Proposed Interface ====================== Bus info on query-machines -------------------------- This series adds a new field to the output of 'query-machines': 'always-available-buses'. It will contain a a list of MachineBusInfo structs. MachineBusInfo will contain: * bus-id: The bus ID * bus-type: The bus type * accepted-device-types: A list of accepted device types for the bus. bus-id can be used to find out what's the right "bus" argument to be used when adding a device to the machine configuration (e.g. when using -device and device_add). accepted-device-types can be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the bus. accepted-device-types property on bus objects --------------------------------------------- This series also adds a 'accepted-device-types' property to bus objects, so management software can check which kinds of devices can be plugged at runtime. Example output -------------- TODO: update it. Considered alternatives ======================= Indirect mapping (machine => bus => device) ------------------------------------------- This RFC implements a mechanism to implement ax machine-type => accepted-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => accepted-device-types. But some buses have no correct bus-type => accepted-device-types mapping. PCIe buses, for example, may or may not accept legacy PCI buses, depending on the machine and the bus topology. imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface. The Implementation ================== This add a new field to MachineClass: always_available_buses, a new field to BusState: accepted_device_types, and a new field to BusClass: device_type. On most cases, the normal QOM type hierarchy is used to define the set of accepted devices for a bus. On the case of PCI buses, INTERFACE_PCIE_DEVICE and INTERFACE_LEGACY_PCI_DEVICE interface names were introduced, to indicate the set of devices accepted by PCI buses. That means we are duplicating information: MachineClass::always_available_buses duplicates knowledge that is already encoded in the machine init function. To make sure the information is correct, a qmp-machine-info.py test case is added, that will validate the supported-device-types field based on the buses created by the machine. Test Code --------- qdev-based test code for the new field was implemented in a Python script. Some extra support was implemented in tests/Makefile.include, scripts/qemu.py and scripts/qtest.py to allow the test to be implemented. Limitations =========== TODO: "slots" vs "buses" ------------------------ The current interface is returning information about "available buses". Future versions might return more detailed information about "slots" where devices can be plugged. This would make the proposed mechanism more flexible, and cover cases like CPU hotplug (in other words, the new interface could replace query-hotpluggable-cpus in a generic way). "default defaults" vs "-nodefault defaults" ------------------------------------------- libvirt use -nodefaults when starting QEMU, so knowing which buses are available when using -nodefaults is more interesting for them. Other software, on the other hand, might be interested in the results without -nodefaults. The new 'always-available-bus' field won't cover the more general cases, where "-nodefaults" is not used, or when there are extra QEMU options that create more buses (e.g. "-machine usb=on", or explicit -device options that would create additional buses). Additional mechanisms may be added to let management software to ask which buses are created when extra options are used, but maybe thoses cases would be more easily solved if management software checks the bus list at runtime (after the machine was actually created). TYPE_SYS_BUS_DEVICE is too generic ---------------------------------- Currently all machines have a TYPE_SYS_BUS bus, and all TYPE_SYS_BUS_DEVICE subclasses are reported as supported by that bus. On the other hand, omitting sysbus would make query-machines lie about supporting sysbus devices, and would require management software to add special cases for sysbus devices. For this reason, sysbus is being included in the output of query-machines, although its accepted-device-types info is less useful than the information about other buses. Future versions of QEMU can keep the same interface, but make it include more useful information, e.g., return more specific bus info, with type names that exclude sysbus devices that can't possibly work with the machine. PCI vs PCIe ----------- This was a limitation on v1, but I believe the problem was addressed on v2. This series defines two new interface names: INTERFACE_LEGACY_PCI_DEVICE and INTERFACE_PCIE_DEVICE. * Legacy PCI devices will implement INTERFACE_LEGACY_PCI_DEVICE. * PCIe devices will implement INTERFACE_PCIE_DEVICE. * Hybrid devices that support both will implement both * interfaces. Buses will have accepted-device-types set accordingly: * Legacy PCI buses will report INTERFACE_LEGACY_PCI_DEVICE on accepted-device-types. * PCIe buses that accept only PCIe devices will report INTERFACE_PCIE_DEVICE on accepted-device-types. * PCIe buses that accept both PCIe and legacy PCI devices will report both INTERFACE_PCIE_DEVICE and INTERFACE_LEGACY_PCI_DEVICE on accepted-device-types. The Q35 root bus is the only existing example of a hybrid bus. See the next section about the hack where this is implemented. Q35 root bus hack ----------------- This series includes a hack on Q35 code to add INTERFACE_LEGACY_PCI_DEVICE to accepted-device-types on the root PCIe bus. Suggestions on where this information should be encoded are welcome. Only PC is covered ------------------ This series sets always-available-buses only on the PC machine-types, by now. Future versions might have the field added to other machines. sPAPR is an obvious candidate, as there are plans to change new spapr machine-type versions to use PCIe as the root bus. Out of scope: Configurable buses -------------------------------- There's no way to map machine options like "usb=on|off" to device-types or buses. This is not on the scope of this series. Additional mechanisms may be added to let management software to ask which buses are created when extra options are used, but maybe thoses cases would be more easily solved if management software checks the bus list at runtime (after the machine was actually created). Existing DeviceClass::bus_type field ------------------------------------ The new BusClass::device_type and BusState::accepted_device_types fields duplicate knowledge that is already encoded in DeviceClass::bus_type. v1 of this series included code that validated both fields to ensure they are consistent with each other. But the PCI/PCIe hierarchy makes validation more difficult, so the validation code was removed on v2. The DeviceClass::bus_type abstraction breaks in the case of hybrid PCI buses, because only a bus type name is not enough information to find the right bus to plug a device. This needs to be addressed somehow (maybe DeviceClass::bus_type can be completely removed), but I didn't look at the device_add code to find out the best way to do that. --- Cc: libvir-list(a)redhat.com Cc: Laine Stump <laine(a)redhat.com> Cc: Alexey Kardashevskiy <aik(a)ozlabs.ru> Cc: Greg Kurz <groug(a)kaod.org> Cc: David Gibson <david(a)gibson.dropbear.id.au> Cc: Andrea Bolognani <abologna(a)redhat.com>, Cc: Eric Blake <eblake(a)redhat.com> Cc: Cornelia Huck <cornelia.huck(a)de.ibm.com> Cc: David Hildenbrand <david(a)redhat.com> Eduardo Habkost (20): qemu.py: Make logging optional qtest.py: Support QTEST_LOG environment variable qtest.py: make logging optional qtest.py: Make 'binary' parameter optional tests: Add rules to non-gtester qtest test cases qdev: Add 'accepted-device-types' property to BusClass qmp: Add 'always-available-buses' field to 'query-machines' virtio-pci: Set PCIDeviceClass::is_express=1 vmxnet3: Set PCIDeviceClass::is_express=1 pvscsi: Set PCIDeviceClass::is_express=1 pci: INTERFACE_LEGACY_PCI_DEVICE and INTERFACE_PCIE_DEVICE interfaces pci: Replace is_express with INTERFACE_PCIE_DEVICE [trivial] edu: Move edu_info outside function [automated] Add INTERFACE_LEGACY_PCI_DEVICE to all PCI device subclasses eepro100: Add INTERFACE_LEGACY_PCI_DEVICE [incomplete] remove INTERFACE_LEGACY_PCI_DEVICE from PCIe-only devices pci: Set device_type on bus classes q35: Hack to make root bus accept legacy PCI devices pci: validate interfaces on base_class_init pc: Initialize default bus lists hw/acpi/piix4.c | 1 + hw/audio/ac97.c | 4 + hw/audio/es1370.c | 4 + hw/audio/intel-hda.c | 11 +++ hw/block/fdc.c | 15 +++- hw/block/nvme.c | 5 +- hw/char/serial-pci.c | 12 +++ hw/char/virtio-serial-bus.c | 1 + hw/core/bus.c | 21 +++++ hw/core/machine.c | 33 ++++++- hw/core/sysbus.c | 1 + hw/display/cirrus_vga.c | 4 + hw/display/qxl.c | 4 + hw/display/vga-pci.c | 4 + hw/display/vmware_vga.c | 4 + hw/i2c/core.c | 7 ++ hw/i2c/smbus_ich9.c | 4 + hw/i386/amd_iommu.c | 4 + hw/i386/kvm/pci-assign.c | 4 + hw/i386/pc.c | 1 + hw/i386/pc_piix.c | 29 ++++++ hw/i386/pc_q35.c | 21 +++++ hw/i386/xen/xen_platform.c | 4 + hw/i386/xen/xen_pvdevice.c | 4 + hw/ide/ich.c | 4 + hw/ide/pci.c | 4 + hw/ide/qdev.c | 1 + hw/input/adb.c | 7 ++ hw/ipack/ipack.c | 7 ++ hw/ipack/tpci200.c | 4 + hw/isa/i82378.c | 4 + hw/isa/isa-bus.c | 1 + hw/isa/lpc_ich9.c | 1 + hw/isa/piix4.c | 4 + hw/isa/vt82c686.c | 16 ++++ hw/mips/gt64xxx_pci.c | 4 + hw/misc/auxbus.c | 1 + hw/misc/edu.c | 20 +++-- hw/misc/ivshmem.c | 4 + hw/misc/macio/macio.c | 4 + hw/misc/pci-testdev.c | 4 + hw/net/e1000.c | 4 + hw/net/e1000e.c | 5 +- hw/net/eepro100.c | 5 ++ hw/net/ne2000.c | 4 + hw/net/pcnet-pci.c | 4 + hw/net/rocker/rocker.c | 4 + hw/net/rtl8139.c | 4 + hw/net/vmxnet3.c | 36 ++------ hw/pci-bridge/dec.c | 4 + hw/pci-bridge/ioh3420.c | 6 +- hw/pci-bridge/pci_expander_bridge.c | 8 ++ hw/pci-bridge/xio3130_downstream.c | 6 +- hw/pci-bridge/xio3130_upstream.c | 6 +- hw/pci-host/apb.c | 4 + hw/pci-host/bonito.c | 4 + hw/pci-host/gpex.c | 4 + hw/pci-host/grackle.c | 4 + hw/pci-host/piix.c | 8 ++ hw/pci-host/ppce500.c | 4 + hw/pci-host/prep.c | 4 + hw/pci-host/q35.c | 14 +++ hw/pci-host/uninorth.c | 16 ++++ hw/pci-host/versatile.c | 4 + hw/pci/pci.c | 41 ++++++++- hw/pci/pci_bridge.c | 4 + hw/ppc/ppc4xx_pci.c | 4 + hw/ppc/spapr_vio.c | 1 + hw/s390x/css-bridge.c | 2 + hw/s390x/event-facility.c | 1 + hw/s390x/s390-pci-bus.c | 7 ++ hw/scsi/esp-pci.c | 4 + hw/scsi/lsi53c895a.c | 4 + hw/scsi/megasas.c | 12 ++- hw/scsi/mptsas.c | 4 + hw/scsi/scsi-bus.c | 1 + hw/scsi/vmw_pvscsi.c | 29 ++---- hw/sd/core.c | 7 ++ hw/sd/sdhci.c | 4 + hw/sh4/sh_pci.c | 4 + hw/sparc64/sun4u.c | 4 + hw/ssi/ssi.c | 7 ++ hw/usb/bus.c | 1 + hw/usb/dev-smartcard-reader.c | 7 ++ hw/usb/hcd-ehci-pci.c | 4 + hw/usb/hcd-ohci.c | 4 + hw/usb/hcd-uhci.c | 4 + hw/usb/hcd-xhci.c | 5 +- hw/vfio/pci-quirks.c | 4 + hw/vfio/pci.c | 6 +- hw/virtio/virtio-bus.c | 1 + hw/virtio/virtio-pci.c | 27 +++--- hw/watchdog/wdt_i6300esb.c | 4 + hw/xen/xen_pt.c | 4 + include/hw/boards.h | 7 ++ include/hw/pci/pci.h | 9 +- include/hw/qdev-core.h | 8 ++ qapi-schema.json | 37 +++++++- scripts/qemu.py | 25 ++++-- scripts/qtest.py | 15 +++- tests/Makefile.include | 39 +++++++- tests/qmp-machine-info.py | 173 ++++++++++++++++++++++++++++++++++++ vl.c | 6 ++ 103 files changed, 882 insertions(+), 102 deletions(-) create mode 100755 tests/qmp-machine-info.py -- 2.7.4

1 2

[libvirt] [PATCH 0/3] Better documentation of migration APIs and flags
by Jiri Denemark 29 Nov '16

29 Nov '16

We have 6 public APIs for migration: virDomainMigrate, virDomainMigrate2, virDomainMigrate3, virDomainMigrateToURI, virDomainMigrateToURI2, and virDomainMigrateToURI3. Each of them was documented separately, but the documentation was not consistent. Some APIs documented individual flags, some didn't do that. Some notes about specific behavior were not added to all APIs. And so on. Since the newer versions provide a superset of the functionality of the older ones mainly adding new parameters, it doesn't really make sense to keep 6 (ideally) almost identical copies of the same documentation. To get out of this mess this patch set moves documentation of individual migration flags directly to libvirt-domain.h, where the flags are defined. And the documentation of virDomainMigrate{,ToURI}{,2} is reduced to point to the *3 APIs with a description of how old parameters are mapped to the new ones. Jiri Denemark (3): Enhance documentation of virDomainMigrateFlags Consolidate documentation of virDomainMigrate{,ToURI}{,2,3} docs: Update news docs/news.html.in | 2 + include/libvirt/libvirt-domain.h | 153 ++++++++++++++--- src/libvirt-domain.c | 348 ++++----------------------------------- 3 files changed, 170 insertions(+), 333 deletions(-) -- 2.11.0.rc2

2 6

[libvirt] [RFC PATCH 1/1] gluster: cache glfs connection object per volume
by Prasanna Kumar Kalever 29 Nov '16

29 Nov '16

This patch offer, 1. Optimizing the calls to glfs_init() and friends 2. Temporarily reduce the memory leak appear in libvirt process account, even if the root cause for the leaks are glfs_fini() (7 - 10MB per object) [Hopefully gluster should address this in its future releases, not very near] Currently, a start of a VM will call 2 glfs_new/glfs_init (which will create glfs object, once for stat, read headers and next to chown) and then will fork qemu process which will call once again (for actual read write IO). Not that all, in case if we are have 4 extra attached disks, then the total calls to glfs_init() and friends will be (4+1)*2 in libvirt and (4+1)*1 in qemu space i.e 15 calls. Since we don't have control over qemu process as that executes in a different process environment, lets do not bother much about it. This patch shrinks these 10 calls (i.e objects from above example) to just one, by maintaining a cache of glfs objects. The glfs object is shared across other only if volume name and all the volfile servers match. In case of hit glfs object takes a ref and only on close unref happens. Thanks to 'Peter Krempa' for the discussion. Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever(a)redhat.com> --- WORK IN PROGRESS: (WIP) ---------------- While initially caching the glfs object, i.e. in virStorageBackendGlusterSetPreopened() I have took a ref=2, so on the following virStorageBackendGlusterClosePreopened() --ref will make ref=1, which will help not cleaning up the object which has to be shared with next coming disks (if conditions meat). Given some context, the idea is that on time-out (or after all disks are initiallized) some one should call virStorageBackendGlusterClosePreopened() which will ideally make ref=0, hence cached object will be cleaned/deleted calling glfs_fini() I had a thought of doing the time-out cleanup call in virSecurityManagerSetAllLabel() or similar, but that looks too odd for me? Some help ? Thanks in advance. --- src/storage/storage_backend_gluster.c | 136 ++++++++++++++++++++++++++++++++-- src/storage/storage_backend_gluster.h | 33 ++++++++- 2 files changed, 160 insertions(+), 9 deletions(-) diff --git a/src/storage/storage_backend_gluster.c b/src/storage/storage_backend_gluster.c index 8e86704..4f53ebc 100644 --- a/src/storage/storage_backend_gluster.c +++ b/src/storage/storage_backend_gluster.c @@ -47,19 +47,132 @@ struct _virStorageBackendGlusterState { char *dir; /* dir from URI, or "/"; always starts and ends in '/' */ }; +virGlusterDefPtr ConnCache = {0,}; + typedef struct _virStorageBackendGlusterState virStorageBackendGlusterState; typedef virStorageBackendGlusterState *virStorageBackendGlusterStatePtr; +void +virStorageBackendGlusterSetPreopened(virStorageSourcePtr src, glfs_t *fs) +{ + size_t i; + virStorageBackendGlusterStatePtrPreopened entry = NULL; + + if (ConnCache == NULL && (VIR_ALLOC(ConnCache) < 0)) + return; + + for (i = 0; i < ConnCache->nConn; i++) { + if (STREQ(ConnCache->Conn[i]->volname, src->volume)) + return; + } + + if (VIR_ALLOC(entry) < 0) + goto L1; + + if (VIR_STRDUP(entry->volname, src->volume) < 0) + goto L1; + + entry->nhosts = src->nhosts; + for (i = 0; i < src->nhosts; i++) { + if (VIR_ALLOC_N(entry->hosts[i], strlen(src->hosts[i].name) + 1) < 0) + goto L2; + strcpy(entry->hosts[i], src->hosts[i].name); + } + + entry->fs = fs; + entry->ref = 2; /* persist glfs obj per volume until a final timeout + virStorageBackendGlusterClosePreopened() is called */ + + if (VIR_INSERT_ELEMENT(ConnCache->Conn, -1, ConnCache->nConn, entry) < 0) + goto L2; + + return; + +L2: + for (i = 0; i < entry->nhosts; i++) + VIR_FREE(entry->hosts[i]); +L1: + if(ConnCache->nConn == 0) + VIR_FREE(ConnCache); + VIR_FREE(entry->volname); + VIR_FREE(entry); +} + +glfs_t * +virStorageBackendGlusterFindPreopened(virStorageSourcePtr src) +{ + size_t i, j, k, ret = 0; + size_t min, max; + + if (ConnCache == NULL) + return NULL; + + virStorageBackendGlusterStatePtrPreopened entry; + + for (i = 0; i < ConnCache->nConn; i++) { + entry = ConnCache->Conn[i]; + if (STREQ(entry->volname, src->volume)) { + min = entry->nhosts < src->nhosts ? entry->nhosts : src->nhosts; + max = entry->nhosts >= src->nhosts ? entry->nhosts : src->nhosts; + for (j = 0; j< min; j++) { + if (entry->nhosts == min) { + for (k = 0; k < max; k++) { + if (STREQ(entry->hosts[j], src->hosts[k].name)) { + ret = 1; + break; + } + } + if (!ret) + return NULL; + } else { + for (k = 0; k < max; k++) { + if (STREQ(src->hosts[j].name, entry->hosts[k])) { + ret = 1; + break; + } + } + if (!ret) + return NULL; + } + } + entry->ref++; + return entry->fs; + } + } + return NULL; +} + +int +virStorageBackendGlusterClosePreopened(glfs_t *fs) +{ + size_t i; + int ret = 0; + + if (fs == NULL) + return ret; + + for (i = 0; i < ConnCache->nConn; i++) { + if (ConnCache->Conn[i]->fs == fs) { + if (--ConnCache->Conn[i]->ref) + return ret; + + ret = glfs_fini(ConnCache->Conn[i]->fs); + VIR_FREE(ConnCache->Conn[i]->volname); + VIR_FREE(ConnCache->Conn[i]); + + VIR_DELETE_ELEMENT(ConnCache->Conn, i, ConnCache->nConn); + } + } + return ret; +} + static void virStorageBackendGlusterClose(virStorageBackendGlusterStatePtr state) { if (!state) return; - /* Yuck - glusterfs-api-3.4.1 appears to always return -1 for - * glfs_fini, with errno containing random data, so there's no way - * to tell if it succeeded. 3.4.2 is supposed to fix this.*/ - if (state->vol && glfs_fini(state->vol) < 0) + if (state->vol && virStorageBackendGlusterClosePreopened(state->vol) < 0) VIR_DEBUG("shutdown of gluster volume %s failed with errno %d", state->volname, errno); @@ -556,8 +669,7 @@ virStorageFileBackendGlusterDeinit(virStorageSourcePtr src) src, src->hosts->name, src->hosts->port ? src->hosts->port : "0", src->volume, src->path); - if (priv->vol) - glfs_fini(priv->vol); + virStorageBackendGlusterClosePreopened(priv->vol); VIR_FREE(priv->canonpath); VIR_FREE(priv); @@ -630,11 +742,20 @@ virStorageFileBackendGlusterInit(virStorageSourcePtr src) src, priv, src->volume, src->path, (unsigned int)src->drv->uid, (unsigned int)src->drv->gid); + + priv->vol = virStorageBackendGlusterFindPreopened(src); + if (priv->vol) { + src->drv->priv = priv; + return 0; + } + if (!(priv->vol = glfs_new(src->volume))) { virReportOOMError(); goto error; } + virStorageBackendGlusterSetPreopened(src, priv->vol); + for (i = 0; i < src->nhosts; i++) { if (virStorageFileBackendGlusterInitServer(priv, src->hosts + i) < 0) goto error; @@ -652,8 +773,7 @@ virStorageFileBackendGlusterInit(virStorageSourcePtr src) return 0; error: - if (priv->vol) - glfs_fini(priv->vol); + virStorageBackendGlusterClosePreopened(priv->vol); VIR_FREE(priv); return -1; diff --git a/src/storage/storage_backend_gluster.h b/src/storage/storage_backend_gluster.h index 6796016..a0326aa 100644 --- a/src/storage/storage_backend_gluster.h +++ b/src/storage/storage_backend_gluster.h @@ -22,9 +22,40 @@ #ifndef __VIR_STORAGE_BACKEND_GLUSTER_H__ # define __VIR_STORAGE_BACKEND_GLUSTER_H__ -# include "storage_backend.h" +#include "storage_backend.h" +#include <glusterfs/api/glfs.h> extern virStorageBackend virStorageBackendGluster; extern virStorageFileBackend virStorageFileBackendGluster; + +struct _virStorageBackendGlusterStatePreopened { + char *volname; + size_t nhosts; + char *hosts[1024]; /* FIXME: 1024 ? */ + glfs_t *fs; + int ref; +}; + +typedef struct _virStorageBackendGlusterStatePreopened virStorageBackendGlusterStatePreopened; +typedef virStorageBackendGlusterStatePreopened *virStorageBackendGlusterStatePtrPreopened; + +struct _virGlusterDef { + size_t nConn; + virStorageBackendGlusterStatePtrPreopened *Conn; +}; + +typedef struct _virGlusterDef virGlusterDef; +typedef virGlusterDef *virGlusterDefPtr; + +extern virGlusterDefPtr ConnCache; + +void +virStorageBackendGlusterSetPreopened(virStorageSourcePtr src, glfs_t *fs); + +glfs_t* +virStorageBackendGlusterFindPreopened(virStorageSourcePtr src); + +int +virStorageBackendGlusterClosePreopened(glfs_t *fs); #endif /* __VIR_STORAGE_BACKEND_GLUSTER_H__ */ -- 2.7.4

3 2

[libvirt] [PATCH 0/2] qemu: error out on USB ports out of range
by Ján Tomko 29 Nov '16

29 Nov '16

https://bugzilla.redhat.com/show_bug.cgi?id=1399260 Ján Tomko (2): tests: Fix USB ports in usb-redir-filter qemu: error out on USB ports out of range src/conf/domain_addr.c | 7 ++++++ .../qemuxml2argv-usb-ports-out-of-range.xml | 25 ++++++++++++++++++++++ .../qemuxml2argv-usb-redir-filter-version.args | 4 ++-- .../qemuxml2argv-usb-redir-filter-version.xml | 4 ++-- tests/qemuxml2argvtest.c | 3 +++ .../qemuxml2xmlout-usb-redir-filter-version.xml | 4 ++-- 6 files changed, 41 insertions(+), 6 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-usb-ports-out-of-range.xml -- 2.7.3

2 3

[libvirt] [PATCH] qemuProcessReconnect: Avoid relabeling images after migration
by Jiri Denemark 29 Nov '16

29 Nov '16

Restarting libvirtd on the source host at the end of migration when a domain is already running on the destination would cause image labels to be reset effectively killing the domain. Commit e8d0166e1d fixed similar issue on the destination host, but kept the source always resetting the labels, which was mostly correct except for the specific case handled by this patch. https://bugzilla.redhat.com/show_bug.cgi?id=1343858 Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_process.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index f8f379a58..01c9259b6 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -2955,7 +2955,8 @@ qemuProcessRecoverMigrationOut(virQEMUDriverPtr driver, virConnectPtr conn, qemuMigrationJobPhase phase, virDomainState state, - int reason) + int reason, + unsigned int *stopFlags) { bool postcopy = state == VIR_DOMAIN_PAUSED && (reason == VIR_DOMAIN_PAUSED_POSTCOPY || @@ -3019,6 +3020,7 @@ qemuProcessRecoverMigrationOut(virQEMUDriverPtr driver, case QEMU_MIGRATION_PHASE_CONFIRM3: /* migration completed, we need to kill the domain here */ + *stopFlags |= VIR_QEMU_PROCESS_STOP_MIGRATED; return -1; } @@ -3044,7 +3046,8 @@ static int qemuProcessRecoverJob(virQEMUDriverPtr driver, virDomainObjPtr vm, virConnectPtr conn, - const struct qemuDomainJobObj *job) + const struct qemuDomainJobObj *job, + unsigned int *stopFlags) { qemuDomainObjPrivatePtr priv = vm->privateData; virDomainState state; @@ -3055,7 +3058,7 @@ qemuProcessRecoverJob(virQEMUDriverPtr driver, switch (job->asyncJob) { case QEMU_ASYNC_JOB_MIGRATION_OUT: if (qemuProcessRecoverMigrationOut(driver, vm, conn, job->phase, - state, reason) < 0) + state, reason, stopFlags) < 0) return -1; break; @@ -3402,7 +3405,7 @@ qemuProcessReconnect(void *opaque) if (qemuProcessRefreshBalloonState(driver, obj, QEMU_ASYNC_JOB_NONE) < 0) goto error; - if (qemuProcessRecoverJob(driver, obj, conn, &oldjob) < 0) + if (qemuProcessRecoverJob(driver, obj, conn, &oldjob, &stopFlags) < 0) goto error; if (qemuProcessUpdateDevices(driver, obj) < 0) -- 2.11.0.rc2

2 1

[libvirt] [PATCH] qemu: Report tunnelled post-copy migration as unsupported
by Jiri Denemark 29 Nov '16

29 Nov '16

Post-copy migration needs bi-directional communication between the source and the destination QEMU processes, which is not supported by tunnelled migration. https://bugzilla.redhat.com/show_bug.cgi?id=1371358 Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_migration.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index d4a55d8f7..26b2e6d25 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3167,6 +3167,12 @@ qemuMigrationBeginPhase(virQEMUDriverPtr driver, goto cleanup; } + if (flags & VIR_MIGRATE_POSTCOPY && flags & VIR_MIGRATE_TUNNELLED) { + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s", + _("post-copy is not supported with tunnelled migration")); + goto cleanup; + } + if (flags & (VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SHARED_INC)) { bool has_drive_mirror = virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_DRIVE_MIRROR); @@ -3645,6 +3651,12 @@ qemuMigrationPrepareAny(virQEMUDriverPtr driver, goto cleanup; } + if (flags & VIR_MIGRATE_POSTCOPY && flags & VIR_MIGRATE_TUNNELLED) { + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s", + _("post-copy is not supported with tunnelled migration")); + goto cleanup; + } + if (!(caps = virQEMUDriverGetCapabilities(driver, false))) goto cleanup; -- 2.11.0.rc2

2 1

[libvirt] [PATCH 0/2] Fix case with non-root domain and hugepages
by Michal Privoznik 29 Nov '16

29 Nov '16

Yet another bug found due to my work on containerizing qemu. Michal Privoznik (2): qemu: Create hugepage path on per domain basis security: Implement virSecurityManagerSetHugepages src/qemu/qemu_command.c | 4 +- src/qemu/qemu_conf.c | 45 ++++++++++++++++------ src/qemu/qemu_conf.h | 16 +++++--- src/qemu/qemu_driver.c | 19 +++------ src/qemu/qemu_process.c | 25 +++++++++++- src/security/security_dac.c | 11 ++++++ src/security/security_selinux.c | 10 +++++ .../qemuxml2argv-hugepages-numa.args | 4 +- .../qemuxml2argv-hugepages-pages.args | 14 +++---- .../qemuxml2argv-hugepages-pages2.args | 2 +- .../qemuxml2argv-hugepages-pages3.args | 2 +- .../qemuxml2argv-hugepages-pages5.args | 2 +- .../qemuxml2argv-hugepages-shared.args | 12 +++--- tests/qemuxml2argvdata/qemuxml2argv-hugepages.args | 2 +- .../qemuxml2argv-memory-hotplug-dimm-addr.args | 4 +- .../qemuxml2argv-memory-hotplug-dimm.args | 4 +- 16 files changed, 118 insertions(+), 58 deletions(-) -- 2.8.4

4 8

[libvirt] [PATCH v2 00/31] qemu: Add support for unavailable-features
by Jiri Denemark 28 Nov '16

28 Nov '16

QEMU 2.8.0 adds support for unavailable-features in query-cpu-definitions reply. The unavailable-features array lists CPU features which prevent a corresponding CPU model from being usable on current host. It can only be used when all the unavailable features are disabled. Empty array means the CPU model can be used without modifications. Changes in v2: - many; the way we probe QEMU was rewritten so that we probe both KVM and TCG capabilities and store them separately whenever we expect them to be different Big thanks to Andrea Bolognani for providing the updated replies data from aarch64 and ppc64le machines. Jiri Denemark (31): qemu: Make QMP probing process reusable qemu: Use -machine when probing capabilities via QMP qemu: Probe KVM state earlier qemucapsprobe: Ignore all greetings except the first one qemu: Enable KVM when probing capabilities qemu: Discard caps cache when KVM availability changes qemu: Use saner defaults for domain capabilities qemu: Don't return unusable virttype in domain capabilities qemu: Refactor virQEMUCapsCacheLookup qemu: Refresh caps in virQEMUCapsCacheLookupByArch qemu: Introduce virQEMUCapsLoadCPUModels qemu: Introduce virQEMUCapsFormatCPUModels qemu: Probe CPU models for KVM and TCG tests: Update capabilities for QEMU 1.2.2 tests: Update capabilities for QEMU 1.3.1 tests: Update capabilities for QEMU 1.4.2 tests: Update capabilities for QEMU 1.5.3 tests: Update capabilities for QEMU 1.6.0 tests: Update capabilities for QEMU 1.7.0 tests: Update capabilities for QEMU 2.1.1 tests: Update capabilities for QEMU 2.4.0 tests: Update capabilities for QEMU 2.5.0 tests: Update capabilities for QEMU 2.6.0 tests: Update capabilities for QEMU 2.6.0 (aarch64, GICv2) qemu: Avoid reporting "host" as a supported CPU model tests: Update capabilities for QEMU 2.6.0 (aarch64, GICv3) tests: Update capabilities for QEMU 2.6.0 (ppc64le) tests: Update capabilities for QEMU 2.7.0 tests: Add QEMU 2.8.0 capabilities data tests: Add QEMU 2.8.0 domain capabilities tests qemu: Add support for unavailable-features src/conf/domain_capabilities.c | 6 +- src/conf/domain_capabilities.h | 3 +- src/qemu/qemu_capabilities.c | 684 +- src/qemu/qemu_capabilities.h | 15 +- src/qemu/qemu_capspriv.h | 8 + src/qemu/qemu_driver.c | 27 +- src/qemu/qemu_monitor.h | 1 + src/qemu/qemu_monitor_json.c | 18 + src/qemu/qemu_process.c | 3 +- tests/domaincapsschemadata/qemu_2.6.0.ppc64le.xml | 1 + .../domaincapsschemadata/qemu_2.8.0-tcg.x86_64.xml | 116 + tests/domaincapsschemadata/qemu_2.8.0.x86_64.xml | 116 + tests/domaincapstest.c | 8 + .../qemucapabilitiesdata/caps_1.2.2.x86_64.replies | 216 +- tests/qemucapabilitiesdata/caps_1.2.2.x86_64.xml | 68 +- .../qemucapabilitiesdata/caps_1.3.1.x86_64.replies | 172 +- tests/qemucapabilitiesdata/caps_1.3.1.x86_64.xml | 74 +- .../qemucapabilitiesdata/caps_1.4.2.x86_64.replies | 170 +- tests/qemucapabilitiesdata/caps_1.4.2.x86_64.xml | 74 +- .../qemucapabilitiesdata/caps_1.5.3.x86_64.replies | 170 +- tests/qemucapabilitiesdata/caps_1.5.3.x86_64.xml | 74 +- .../qemucapabilitiesdata/caps_1.6.0.x86_64.replies | 170 +- tests/qemucapabilitiesdata/caps_1.6.0.x86_64.xml | 74 +- .../qemucapabilitiesdata/caps_1.7.0.x86_64.replies | 170 +- tests/qemucapabilitiesdata/caps_1.7.0.x86_64.xml | 74 +- .../qemucapabilitiesdata/caps_2.1.1.x86_64.replies | 173 +- tests/qemucapabilitiesdata/caps_2.1.1.x86_64.xml | 77 +- .../qemucapabilitiesdata/caps_2.4.0.x86_64.replies | 182 +- tests/qemucapabilitiesdata/caps_2.4.0.x86_64.xml | 86 +- .../qemucapabilitiesdata/caps_2.5.0.x86_64.replies | 182 +- tests/qemucapabilitiesdata/caps_2.5.0.x86_64.xml | 86 +- .../caps_2.6.0-gicv2.aarch64.replies | 194 +- .../caps_2.6.0-gicv2.aarch64.xml | 93 +- .../caps_2.6.0-gicv3.aarch64.replies | 207 +- .../caps_2.6.0-gicv3.aarch64.xml | 95 +- .../caps_2.6.0.ppc64le.replies | 1406 +- tests/qemucapabilitiesdata/caps_2.6.0.ppc64le.xml | 1296 +- .../qemucapabilitiesdata/caps_2.6.0.x86_64.replies | 182 +- tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 86 +- .../qemucapabilitiesdata/caps_2.7.0.x86_64.replies | 185 +- tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 89 +- .../qemucapabilitiesdata/caps_2.8.0.x86_64.replies | 14246 +++++++++++++++++++ tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 293 + tests/qemucapabilitiestest.c | 6 + tests/qemucapsprobemock.c | 9 +- tests/qemumonitorjsontest.c | 27 +- tests/qemuxml2argvtest.c | 39 +- 47 files changed, 20114 insertions(+), 1637 deletions(-) create mode 100644 tests/domaincapsschemadata/qemu_2.8.0-tcg.x86_64.xml create mode 100644 tests/domaincapsschemadata/qemu_2.8.0.x86_64.xml create mode 100644 tests/qemucapabilitiesdata/caps_2.8.0.x86_64.replies create mode 100644 tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml -- 2.10.2

3 72

[libvirt] Wiki account
by Eric Farman 28 Nov '16

28 Nov '16

I believe this is still the preferred method... Could I please have an account "farman" for the libvirt wiki, in order to contribute some of the cover-letter "howto" information from the recent vhost-scsi series? Thanks, - Eric

2 1

[libvirt] [PATCH v4] storage_backend_rbd: check the return value of rados_conf_set
by Chen Hanxiao 28 Nov '16

28 Nov '16

From: Chen Hanxiao <chenhanxiao(a)gmail.com> We had a lot of rados_conf_set and check works. Use helper virStorageBackendRBDRADOSConfSet for them. Signed-off-by: Chen Hanxiao <chenhanxiao(a)gmail.com> --- v4: introduce helper virStorageBackendRBDRADOSConfSet v3: fix a copy-paste error v2: add another missing return value check src/storage/storage_backend_rbd.c | 72 +++++++++++++++++++++++---------------- 1 file changed, 42 insertions(+), 30 deletions(-) diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c index 718c4d6..b1c51ab 100644 --- a/src/storage/storage_backend_rbd.c +++ b/src/storage/storage_backend_rbd.c @@ -52,6 +52,23 @@ typedef struct _virStorageBackendRBDState virStorageBackendRBDState; typedef virStorageBackendRBDState *virStorageBackendRBDStatePtr; static int +virStorageBackendRBDRADOSConfSet(rados_t cluster, + const char *option, + const char *value) +{ + VIR_DEBUG("Setting RADOS option '%s' to '%s'", + option, value); + if (rados_conf_set(cluster, option, value) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("failed to set RADOS option: %s"), + option); + return -1; + } + + return 0; +} + +static int virStorageBackendRBDOpenRADOSConn(virStorageBackendRBDStatePtr ptr, virConnectPtr conn, virStoragePoolSourcePtr source) @@ -93,20 +110,13 @@ virStorageBackendRBDOpenRADOSConn(virStorageBackendRBDStatePtr ptr, if (!(rados_key = virStringEncodeBase64(secret_value, secret_value_size))) goto cleanup; - VIR_DEBUG("Found cephx key: %s", rados_key); - if (rados_conf_set(ptr->cluster, "key", rados_key) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("failed to set RADOS option: %s"), - "rados_key"); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "key", rados_key) < 0) goto cleanup; - } - if (rados_conf_set(ptr->cluster, "auth_supported", "cephx") < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("failed to set RADOS option: %s"), - "auth_supported"); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "auth_supported", "cephx") < 0) goto cleanup; - } } else { VIR_DEBUG("Not using cephx authorization"); if (rados_create(&ptr->cluster, NULL) < 0) { @@ -114,12 +124,9 @@ virStorageBackendRBDOpenRADOSConn(virStorageBackendRBDStatePtr ptr, _("failed to create the RADOS cluster")); goto cleanup; } - if (rados_conf_set(ptr->cluster, "auth_supported", "none") < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("failed to set RADOS option: %s"), - "auth_supported"); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "auth_supported", "none") < 0) goto cleanup; - } } VIR_DEBUG("Found %zu RADOS cluster monitors in the pool configuration", @@ -145,35 +152,40 @@ virStorageBackendRBDOpenRADOSConn(virStorageBackendRBDStatePtr ptr, goto cleanup; mon_buff = virBufferContentAndReset(&mon_host); - VIR_DEBUG("RADOS mon_host has been set to: %s", mon_buff); - if (rados_conf_set(ptr->cluster, "mon_host", mon_buff) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("failed to set RADOS option: %s"), - "mon_host"); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "mon_host", + mon_buff) < 0) goto cleanup; - } /* * Set timeout options for librados. * In case the Ceph cluster is down libvirt won't block forever. * Operations in librados will return -ETIMEDOUT when the timeout is reached. */ - VIR_DEBUG("Setting RADOS option client_mount_timeout to %s", client_mount_timeout); - rados_conf_set(ptr->cluster, "client_mount_timeout", client_mount_timeout); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "client_mount_timeout", + client_mount_timeout) < 0) + goto cleanup; - VIR_DEBUG("Setting RADOS option rados_mon_op_timeout to %s", mon_op_timeout); - rados_conf_set(ptr->cluster, "rados_mon_op_timeout", mon_op_timeout); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "rados_mon_op_timeout", + mon_op_timeout) < 0) + goto cleanup; - VIR_DEBUG("Setting RADOS option rados_osd_op_timeout to %s", osd_op_timeout); - rados_conf_set(ptr->cluster, "rados_osd_op_timeout", osd_op_timeout); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "rados_osd_op_timeout", + osd_op_timeout) < 0) + goto cleanup; /* * Librbd supports creating RBD format 2 images. We no longer have to invoke * rbd_create3(), we can tell librbd to default to format 2. * This leaves us to simply use rbd_create() and use the default behavior of librbd */ - VIR_DEBUG("Setting RADOS option rbd_default_format to %s", rbd_default_format); - rados_conf_set(ptr->cluster, "rbd_default_format", rbd_default_format); + if (virStorageBackendRBDRADOSConfSet(ptr->cluster, + "rbd_default_format", + rbd_default_format) < 0) + goto cleanup; ptr->starttime = time(0); if ((r = rados_connect(ptr->cluster)) < 0) { -- 2.7.4

2 1