October 2016 - Devel - libvirt List Archives

[libvirt] [PATCH RESEND v2 0/5] Implementation of QEMU vhost-scsi
by Eric Farman 05 Oct '16

05 Oct '16

[Resending after the release of 2.2.0; no changes other than a rebase to current master and the associated tweaking to the capabilities patch] This patch series provides a libvirt implementation of the vhost-scsi interface in QEMU. As near as I can see, this was discussed upstream in July 2014[1], and ended in a desire to replace a vhost-scsi controller in favor of a hostdev element instead[2]. There is no capability check in this series for vhost-scsi in the underlying QEMU. Using a recent QEMU built with --disable-vhost-scsi fails with "not a valid device model name." Host Filesystem Example: # ls /sys/kernel/config/target/vhost/ discovery_auth naa.5001405df3e54061 version # ls /sys/kernel/config/target/vhost/naa.5001405df3e54061/tpgt_1/lun/ lun_0 QEMU Example (snippet): -device vhost-scsi-ccw,wwpn=naa.5001405df3e54061,devno=fe.0.1000 Libvirt Example (snippet): <hostdev mode='subsystem' type='scsi_host'> <source protocol='vhost' wwpn='naa.5001405df3e54061'/> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x1000'/> </hostdev> Guest Viewpoint: # lsscsi [1:0:1:0] disk LIO-ORG disk0 4.0 /dev/sda # dmesg | grep 1: [ 6.065735] scsi host1: Virtio SCSI HBA [ 6.093892] scsi 1:0:1:0: Direct-Access LIO-ORG disk0 4.0 PQ: 0 ANSI: 5 [ 6.313615] sd 1:0:1:0: Attached scsi generic sg0 type 0 [ 6.314981] sd 1:0:1:0: [sda] 29360128 512-byte logical blocks: (15.0 GB/14.0 GiB) [ 6.317290] sd 1:0:1:0: [sda] Write Protect is off [ 6.317566] sd 1:0:1:0: [sda] Mode Sense: 43 00 10 08 [ 6.317853] sd 1:0:1:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 6.352722] sd 1:0:1:0: [sda] Attached SCSI disk Changelog: v2->v2.1: - Rebased to current master (6 September) v1->v2: https://www.redhat.com/archives/libvir-list/2016-August/msg01028.html - Rebase - Applies to current master (20 August) - Added a capability check for QEMU 2.7 - Fixed the qemuxml2argv tests as the -smp options had changed - Reworked ccwaddrs parameter in virDomainCCWAddressAssign call - Comments - Squashed documentation, XML schema, XML parsing, and infrastructure patches into a single patch - Switched from "hostdev type='scsi'" to "hostdev type='scsi_host'"; this removes the refactoring patches since we're not shoe-horning a new SCSI protocol into the existing code - Reworked the handling of the fd's such that we send them to qemu after any possible errors could occur and cause us to back out - s/qemuBuildSCSIVhostHostdevDevStr/qemuBuildHostHostdevDevStr/ - Added virBufferCheckError, and an error message for vhostfdSize > 1, in qemuBuildHostHostdevDevStr - Added qemuBuildDeviceAddressStr in qemuBuildHostHostdevDevStr, thus superceding the last patch in the v1 series - Other - Simplified the vhostfd logic to just be a single int, rather than an alloc'd array (left the vhostfdSize described above as an identifier for if QEMU ever supports multiple vhostfds) - Replaced "qemuMonitorAddDevice" with "qemuMonitorAddDeviceWithFd" in hotplug routine v1: https://www.redhat.com/archives/libvir-list/2016-July/msg01004.html [1] http://www.redhat.com/archives/libvir-list/2014-July/msg01235.html [2] http://www.redhat.com/archives/libvir-list/2014-July/msg01390.html Eric Farman (5): Introduce a "scsi_host" hostdev type qemu: Introduce vhost-scsi capability qemu: Add vhost-scsi string for -device parameter qemu: Allow hotplug of vhost-scsi device tests: Introduce basic vhost-scsi test docs/formatdomain.html.in | 24 ++++ docs/schemas/domaincommon.rng | 23 ++++ src/conf/domain_audit.c | 2 + src/conf/domain_conf.c | 62 ++++++++- src/conf/domain_conf.h | 17 +++ src/libvirt_private.syms | 1 + src/qemu/qemu_capabilities.c | 3 + src/qemu/qemu_capabilities.h | 3 + src/qemu/qemu_command.c | 80 +++++++++++ src/qemu/qemu_command.h | 6 + src/qemu/qemu_domain_address.c | 10 ++ src/qemu/qemu_hotplug.c | 149 +++++++++++++++++++++ src/security/security_dac.c | 2 + src/util/virscsi.c | 26 ++++ src/util/virscsi.h | 1 + tests/domaincapsschemadata/full.xml | 1 + tests/qemucapabilitiesdata/caps_1.5.3.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_1.6.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_1.7.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.1.1.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.4.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.5.0.x86_64.xml | 1 + .../caps_2.6.0-gicv2.aarch64.xml | 1 + .../caps_2.6.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64le.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 1 + .../qemuxml2argv-hostdev-scsi-vhost-scsi.args | 24 ++++ .../qemuxml2argv-hostdev-scsi-vhost-scsi.xml | 33 +++++ tests/qemuxml2argvmock.c | 12 ++ tests/qemuxml2argvtest.c | 3 + 31 files changed, 491 insertions(+), 2 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-scsi-vhost-scsi.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-scsi-vhost-scsi.xml -- 1.9.1

2 18

[libvirt] [PATCH 0/3] Cleanup/unify RBD backend context setup
by John Ferlan 05 Oct '16

05 Oct '16

While working on a way to create a luks volume on an RBD backend - I figured I'd clean up the initialization of the environment a bit. Rather than a lot of copy/paste code - use a common initializer. John Ferlan (3): rbd: Change virStorageBackendRBDCloseRADOSConn to be static void rbd: Change from static to alloc contexts rbd: Move the encryption check in build src/storage/storage_backend_rbd.c | 150 +++++++++++++++++++------------------- 1 file changed, 73 insertions(+), 77 deletions(-) -- 2.7.4

2 6

[libvirt] [PATCH 0/2] Add support for preallocated fd memory
by Jaroslav Safka 05 Oct '16

05 Oct '16

Hi the xml element to be extended with additional children is the memoryBacking element. We would like to introduce 3 new elements source,access and allocation <memoryBacking> <source type="file|anonymous" path='/path/to/qemu/'/> <access Mode="shared|private"/> <allocation mode="immediate|ondemand"/> </memoryBacking> If allocation is immediate then -mem-prealloc should be added to the qemu commanline. If source is file then -object memory-backend-file,id=mem,size=1024M,mem-path=/var/lib/libvirt/qemu -numa node,memdev=mem Should be added to the qemu commandline If access is shared then the share=on parameter should be added to the memory-backend-file e.g. -object memory-backend-file,id=mem,size=1024M,mem-path=/var/lib/libvirt/qemu,share=on Jaroslav Safka (2): Add support for preallocated fd memory Add support for preallocated fd memory docs/schemas/domaincommon.rng | 37 +++++ src/conf/domain_conf.c | 149 +++++++++++++++----- src/conf/domain_conf.h | 34 +++++ src/qemu/qemu_command.c | 156 ++++++++++++++++----- src/qemu/qemu_command.h | 4 + .../qemuxml2argv-fd-memory-no-numa-topology.args | 34 +++++ .../qemuxml2argv-fd-memory-no-numa-topology.xml | 96 +++++++++++++ .../qemuxml2argv-fd-memory-numa-topology.args | 34 +++++ .../qemuxml2argv-fd-memory-numa-topology.xml | 99 +++++++++++++ .../qemuxml2argv-memorybacking-set.xml | 32 +++++ .../qemuxml2argv-memorybacking-unset.xml | 32 +++++ tests/qemuxml2argvtest.c | 24 ++++ .../qemuxml2xmlout-memorybacking-set.xml | 40 ++++++ .../qemuxml2xmlout-memorybacking-unset.xml | 40 ++++++ tests/qemuxml2xmltest.c | 3 + 15 files changed, 748 insertions(+), 66 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-fd-memory-no-numa-topology.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-fd-memory-no-numa-topology.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-fd-memory-numa-topology.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-fd-memory-numa-topology.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memorybacking-set.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memorybacking-unset.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-memorybacking-set.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-memorybacking-unset.xml -- 2.5.5 -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

4 5

[libvirt] 2 major proposals to enhance libvirt and its attractivity among developers & users
by jean-christophe Manciot 05 Oct '16

05 Oct '16

Hello everyone. Going straight to the point: 1) *add connection type: Docker* This should not induce a lot of development since there is already an LXC connection type. 2) add the ability to *choose the bridge type of any virtual network*: - Linux bridge + VPP : Cisco has recently open-sourced the virtual switch/router <https://wiki.fd.io/view/VPP/What_is_VPP%3F> which it uses with DPDK as a core part of some of its commercial virtual products. Its performance should be unprecedented as compared to the current Linux bridge or even OvS. "VPP is also applicable to many architectures (x86, ARM, and PowerPC) and deployment environments (bare metal, VM, container)", according to Simon Dredge in FD.io Takes Over VPP and Unites with DPDK to Accelerate NFV Data Planes to Outright Nutty Speeds <http://www.metaswitch.com/the-switch/fd.io-takes-over-vpp>. + OvS: not a priority IMHO. Regards. -- *Jean-Christophe Manciot* *[image: Architecte réseaux et Sécurité] <https://learningnetwork.cisco.com/people/manciot.jeanchristophe/content> *[image: Network & Security Architect] <https://fr.linkedin.com/in/jeanchristophemanciot/en> <https://twitter.com/jc_manciot> <https://plus.google.com/u/0/+jeanchristopheManciot-IT/posts>

2 2

[libvirt] Entering freeze for libvirt-2.3.0
by Daniel Veillard 05 Oct '16

05 Oct '16

Since nobody complained about my earlier message with the release plan, I tagged libvirt-2.3.0 candidate release 1 in git and pushed signed tarball and rpms to the usual place: ftp://libvirt.org/libvirt/ As usual my limited testing is really not sufficient so please give it a try, I enjoy the view of a completely green https://ci.centos.org/view/libvirt-project/ :-) but that doesn't test portability to other platforms for example ! Then I will try to push rc2 on Thursday, that way the final release can happen during the week-end or on Monday if all goes well, thanks for testing it ! Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

4 7

[libvirt] [PATCH 0/3] introduce pull backups
by Nikolay Shirokovskiy 05 Oct '16

05 Oct '16

Hi, everyone. Please take a look at pull backup API and its qemu implementation. There is no any documentation inside yet so I put it here. Backup is described in xml like next: <domainbackup> <address type="ip" host="0.0.0.0" port="7000"/> <disks> <disk name='sda'> <source file="/root/backup.hdd"/> </disk> <disk name='sdb' present='no'/> </disks> </domainbackup> <disks>, <disk>, <source> elements and 'present' attribute are optional. The export address is bare minimum to export all non read only disks with inserted media. This xml description is close to what we have for snapshots. Implementation depends on experimental qemu command 'x-blockdev-del' and 'blockdev-add' which is still described as work-in-progress even it has no 'x-' prefix. I failed to find the information whether client reading disks being backed up is able to detect all failures in the process or not so some efforts are done to track back up blockjobs failures and report them on pull backup stop. At the same time other clean up errors on stop are ignored so that client can know the essence was the backup operation successful or not. There is a work in progress to provide dirty bitmap information thru nbd to be able to make incremental backups in pull backup scheme. Thus the design should be future proof in this aspect. I see next changes to xml description: <domainbackup> <address type="ip" host="0.0.0.0" port="7000"/> <branch name="backup1"/> <disks> <disk name='sda' type='incremental'> <source file="/root/backup.hdd"/> <bitmap granularity='32Kib'/> </disk> </disks> </domainbackup> Branch name essentially becames dirty backup name in terms of qemu. This option makes possible to have independent branches of incremental backups with possibly different schedule and granularity. Stop operation in case of incremental backups should be specified further to finish/cancel the operation - we have flags for that. Nikolay Shirokovskiy (3): qemu: store guest visible disk size from qemu monitor block info qemu: special error code in case of no job on cancel block job introduce pull backup examples/object-events/event-test.c | 3 + include/libvirt/libvirt-domain-backup.h | 45 +++ include/libvirt/libvirt-domain.h | 3 + include/libvirt/libvirt.h | 1 + include/libvirt/virterror.h | 1 + po/POTFILES.in | 2 + src/Makefile.am | 3 + src/access/viraccessperm.c | 3 +- src/access/viraccessperm.h | 6 + src/conf/backup_conf.c | 295 ++++++++++++++ src/conf/backup_conf.h | 85 ++++ src/conf/domain_conf.c | 2 +- src/driver-hypervisor.h | 11 + src/libvirt-domain-backup.c | 86 ++++ src/libvirt_private.syms | 6 + src/libvirt_public.syms | 2 + src/qemu/qemu_blockjob.c | 2 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_domain.h | 5 + src/qemu/qemu_driver.c | 684 ++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor.c | 37 ++ src/qemu/qemu_monitor.h | 12 + src/qemu/qemu_monitor_json.c | 133 ++++++- src/qemu/qemu_monitor_json.h | 16 + src/remote/remote_driver.c | 2 + src/remote/remote_protocol.x | 33 +- src/util/virerror.c | 1 + tests/qemumonitorjsontest.c | 36 ++ tools/Makefile.am | 1 + tools/virsh-backup.c | 150 +++++++ tools/virsh-backup.h | 28 ++ tools/virsh-domain.c | 3 +- tools/virsh.c | 2 + tools/virsh.h | 1 + 34 files changed, 1693 insertions(+), 8 deletions(-) create mode 100644 include/libvirt/libvirt-domain-backup.h create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/libvirt-domain-backup.c create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 1.8.3.1

3 21

[libvirt] [PATCH v2 1/2] NSS: Add explicit check to not report expired lease
by Nehal J Wani 05 Oct '16

05 Oct '16

The NSS module shouldn't rely on custom leases database to not have entries for leases which have expired. Change-Id: Ic3e043003d33ded0da74696a1d27ed4967ddbfb8 --- tools/nss/libvirt_nss.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tools/nss/libvirt_nss.c b/tools/nss/libvirt_nss.c index 54c4a2a..cb3edf5 100644 --- a/tools/nss/libvirt_nss.c +++ b/tools/nss/libvirt_nss.c @@ -42,6 +42,7 @@ #include "virlease.h" #include "viralloc.h" #include "virfile.h" +#include "virtime.h" #include "virerror.h" #include "virstring.h" #include "virsocketaddr.h" @@ -114,6 +115,8 @@ findLease(const char *name, ssize_t i, nleases; leaseAddress *tmpAddress = NULL; size_t ntmpAddress = 0; + time_t currtime; + long long expirytime; *address = NULL; *naddress = 0; @@ -161,6 +164,11 @@ findLease(const char *name, nleases = virJSONValueArraySize(leases_array); DEBUG("Read %zd leases", nleases); + if ((currtime = time(NULL)) == (time_t) - 1) { + ERROR("Failed to get current system time"); + goto cleanup; + } + for (i = 0; i < nleases; i++) { virJSONValuePtr lease; const char *lease_name; @@ -181,6 +189,16 @@ findLease(const char *name, if (STRNEQ_NULLABLE(name, lease_name)) continue; + if (virJSONValueObjectGetNumberLong(lease, "expiry-time", &expirytime) < 0) { + /* A lease cannot be present without expiry-time */ + ERROR("expiry-time field missing for %s", name); + goto cleanup; + } + + /* Do not report expired lease */ + if (expirytime < (long long) currtime) + continue; + DEBUG("Found record for %s", lease_name); *found = true; -- 2.4.11

2 6

[libvirt] QMP stubs: how to return "not implemented" errors?
by Eduardo Habkost 05 Oct '16

05 Oct '16

Hi, When adding new QMP commands that are implemented by arch-specific code, we have been adding stubs that report QERR_UNSUPPORTED (see stubs/arch-query-cpu-model-expansion.c for an example). But we are using GenericError for that, and this prevents clients from reliably checking if the command is really implemented by the QEMU binary. What should be the right solution for this? Some of the options I have considered are: 1) Using CommandNotFound as the error class in the stubs. This sounds wrong because the command exists (it is present in query-commands and in the QAPI schema). 2) Creating a CommandNotImplemented error class. Simple to do, but it would require clients to make two separate checks, before concluding that the command is available (checking query-commands or query-qmp-schema, and then checking for CommandNotImplemented errors). 3.1) Removing the command from query-commands and from the QAPI schema on binaries that don't implement the command. Needlessly complex? 3.2) Removing the unimplemented command from query-commands only (by calling qmp_disable_command()), but keeping it on the QAPI schema. I am not sure it's OK to do that. If it is, this sounds like the simplest solution. Any ideas? -- Eduardo

3 5

[libvirt] Release of libvirt-2.3.0
by Daniel Veillard 04 Oct '16

04 Oct '16

We are a little late but that allowed us to catch a regression before it impacted the release :) I have tagged the release in git, pushed signed tarball and rpms to the usual place: ftp://libvirt.org/libvirt/ I also pushed the associated python bindings release available at: ftp://libvirt.org/libvirt/python/ This is a release of moderate size, around 250 commits, with a balance of features bugs fixes and improvements: Features: - Improvements on CPU models (Jiri Denemark) - More performance metrics and events (Qiaowei Ren, John Ferlan) Documentation: - docs: correct version requirements for <kvm><hidden='on'/></kvm> (Laine Stump) - doc: fix note about Xen credit scheduler (Jim Fehlig) - Fix various code comment typos (Nitesh Konkar) - cpu: Document missing parameters for cpuCompare* (Jiri Denemark) - libvirt-nodedev.c:fix a typo (Nitesh Konkar) - libvirt-admin.c:fix a typo (Nitesh Konkar) - driver.c:fix a typo (Nitesh Konkar) - domain_conf.c:fix the comment. (Nitesh Konkar) - domain_conf.c:fix a typo (Nitesh Konkar) - domain_conf.c:fix a typo (Nitesh Konkar) - stream.c: fix a typo (Nitesh Konkar) - config-post.h:fix a typo (Nitesh Konkar) - virsh: man: Do not document our old deprecated nodedev-dettach command (Erik Skultety) - qemu_process: fix a typo (Chen Hanxiao) - docs: expand docs on user x509 cert locations (Daniel P. Berrange) - virt-admin: Fix some English grammar in the manpage (Erik Skultety) - virt-admin: Document srv-list command in virt-admin's man page (Erik Skultety) - docs: Add libvirt-go Go bindings to binding page (Roman Mohr) Portability: - mingw: Package cputypes.rng for mingw32 too (Jiri Denemark) - mingw: Package cputypes.rng (Jiri Denemark) - Add missing 'extern' for VIR_EXPORT_VAR macro (Daniel P. Berrange) - Make sure sys/types.h is included after sys/sysmacros.h (Michal Privoznik) Bug Fixes: - qemu: vcpu: Clear vcpu order information rather than making it invalid (Peter Krempa) - qemu: Fix coldplug of vcpus (Peter Krempa) - qemu: process: Enforce 'vcpu' order range to <1,maxvcpus> (Peter Krempa) - qemu: process: Fix off-by-one in vcpu order duplicate error message (Peter Krempa) - sanlock: Properly init io_timeout (Michal Privoznik) - m4: Check for sanlock_write_lockspace (Michal Privoznik) - qemu: Only use memory-backend-file with NUMA if needed (Martin Kletzander) - vsh: Write out history on "quit" or "exit" in interactive mode (John Ferlan) - libxl: fix param assignment in domainGetSchedulerParameters (Jim Fehlig) - qemu: Fix crash in qemucapsprobe (Jiri Denemark) - qemuDomainUpdateDeviceConfig: Allow full disk update (Michal Privoznik) - libxl: find virDomainObj in libxlDomainShutdownThread (Jim Fehlig) - qemu: Update guest CPU def in live XML (Jiri Denemark) - cpu: Set nfeatures_max correctly in x86Decode (Jiri Denemark) - cpu: Drop false support for ARM cpu-model (Jiri Denemark) - migration: Document we don't copy storage during offline migration (Michal Privoznik) - qemu: driver: Don't return automatic NUMA emulator pinning data for persistentDef (Peter Krempa) - qemu: driver: Don't return automatic NUMA vCPU pinning data for persistentDef (Peter Krempa) - qemuDomainOpenGraphics: Start job early (Jiri Denemark) - virtlogd: Don't stop or restart along with libvirtd (Andrea Bolognani) - tests: fix use of fixedcontent variable (Laine Stump) - qemuBuildMemoryBackendStr: Don't crash if no hugetlbfs is mounted (Michal Privoznik) - qemu: map "virtio" video model to "virt" machtype correctly (arm/aarch64) (Laszlo Ersek) - util: Expose virHostCPUStatsAssign on non-Linux (Martin Kletzander) - qemu: domain: Don't infer vcpu state (Peter Krempa) - virDomainGetBlockJobInfo: Fix corner case when qemu reports no info (Michal Privoznik) - tests: use a fixed chardev TLS path (Daniel P. Berrange) - numa: Rename virNumaGetHostNodeset and make it return only nodes with memory (Peter Krempa) - configure: Check for major() more strictly (Michal Privoznik) - tests: fix incorrect status handling by virsh-self-test (Erik Skultety) - conf: allow hotplugging "legacy PCI" device to manually addressed PCIe slot (Laine Stump) - qemu: fix improper initialization of cgroupControllers bitmap (Laine Stump) - audit: Audit information about shmem devices (Martin Kletzander) - storage: Need to properly read the crypt offset value (John Ferlan) - log: Fix reporting OOM error incorrectly when defining a logging filter (Nikolay Shirokovskiy) - qemu: Don't use query-migrate on destination (Jiri Denemark) - util: Quiet the logging if perf file doesn't exist (John Ferlan) - qemu: hotplug: Don't wait if cdrom tray is opened forcibly (Peter Krempa) - qemu: Remove stale transient def when migration fails (Jiri Denemark) - security: Fixing wrong label in virt-aa-helper.c. (Julio Faracco) - Avoid segfault in virt-aa-helper when handling read-only filesystems (Rufo Dogav) - storage: Delete extra wrap after vol-resize error (Yanqiu Zhang) - qemu: process: Fix start with unpluggable vcpus with NUMA pinning (Peter Krempa) - virt-admin: Output srv-clients-set data as unsigned int rather than signed (Erik Skultety) - util: fix crash in virClassIsDerivedFrom for CloseCallbacks objects (Maxim Nestratov) - util: hostcpu: improve CPU freq code for FreeBSD (Roman Bogorodskiy) - util: storage: Properly set protocol type when parsing gluster json string (Peter Krempa) - conf: Fix initialization value of 'multi' in PCI address (Xian Han Yu) Improvements: - qemu: process: Don't use shifted indexes for vcpu order verification (Peter Krempa) - lock_driver_sanlock: Avoid global driver variable whenever possible (Michal Privoznik) - bhyve: chase cpuCompareXML rename (Roman Bogorodskiy) - xlconfigtest: add test for channel conversion (Joao Martins) - xenconfig: channels conversion support (Joao Martins) - libxl: channels support (Joao Martins) - conf: add xen type for channels (Joao Martins) - spec: Package cputypes.rng (Jiri Denemark) - qemu: make qemuGetCompressionProgram return int not an enum (Daniel P. Berrange) - Fix coding style issues. (Nitesh Konkar) - conf: Skip post parse callbacks when creating copy (Michal Privoznik) - domain_conf: Introduce VIR_DOMAIN_DEF_PARSE_SKIP_POST_PARSE (Michal Privoznik) - qemuDomainDefAssignAddresses: Fetch caps from domain object (Michal Privoznik) - qemuDomainDeviceDefPostParse: Fetch caps from domain object (Michal Privoznik) - qemuDomainDefPostParse: Fetch qemuCaps from domain object (Michal Privoznik) - conf: Extend virDomainDefAssignAddressesCallback for parseOpaque (Michal Privoznik) - conf: Extend virDomainDeviceDefPostParse for parseOpaque (Michal Privoznik) - virDomainDefCopy: Introduce @parseOpaque argument (Michal Privoznik) - virDomainDefParse{File,String}: Introduce @parseOpaque argument (Michal Privoznik) - virDomainDefParseNode: Introduce @parseOpaque argument (Michal Privoznik) - virDomainDefPostParse: Introduce @parseOpaque argument (Michal Privoznik) - storage_backend_rbd: remove unnessary translated message marker (Chen Hanxiao) - qemu: Get/return compressedpath program (John Ferlan) - qemu: Remove qemuCompressProgramAvailable (John Ferlan) - qemu: Use qemuGetCompressionProgram for error paths (John Ferlan) - qemu: Alter qemuGetCompressionProgram warning message (John Ferlan) - qemu: Remove getCompressionType (John Ferlan) - qemu: Introduce helper qemuGetCompressionProgram (John Ferlan) - qemu: Adjust doCoreDump to call getCompressionType (John Ferlan) - qemu: Move getCompressionType (John Ferlan) - apparmor: move qemu-bridge-helper to libvirtd profile (Cédric Bosdonnat) - libxl: increase usbdevice list only when finding such an input device (Cédric Bosdonnat) - qemuBuildHostNetStr: Realign (Michal Privoznik) - qemuhelptest: regenerate data for qemu-kvm-1.2.0 (Pavel Hrdina) - qemuhelptest: regenerate data for qemu-kvm-0.13.0 (Pavel Hrdina) - qemuhelptest: regenerate data for qemu-kvm-0.12.3 (Pavel Hrdina) - qemuhelptest: regenerate data for qemu-1.2.0 (Pavel Hrdina) - qemuhelptest: regenerate data for qemu-1.1.0 (Pavel Hrdina) - qemuhelptest: regenerate data for qemu-1.0 (Pavel Hrdina) - qemuhelptest: regenerate data for qemu-0.12.1 (Pavel Hrdina) - qemuhelptest: remove downstream test data (Pavel Hrdina) - qemu: Fix improper indention (John Ferlan) - Fix Multiple Typos (Nitesh Konkar) - Move CMT feature filtering to QEMU driver (Jiri Denemark) - cpu: Rework cpuCompare* APIs (Jiri Denemark) - cpu: Introduce virCPUCheckFeature (Jiri Denemark) - cpu: Rework virCPUDataCheckFeature (Jiri Denemark) - cpu: Rename cpuHasFeature to virCPUDataCheckFeature (Jiri Denemark) - cpu: Introduce virCPUTranslate (Jiri Denemark) - cpu: Rework cpuUpdate (Jiri Denemark) - cpu: Add x86FeatureInData (Jiri Denemark) - cpu: Report error for unknown features in x86HasFeature (Jiri Denemark) - cpu: Make x86ModelFromCPU a bit smarter (Jiri Denemark) - cpu: Make x86ModelFromCPU easier to read (Jiri Denemark) - qemu: Introduce virQEMUCapsIsCPUModeSupported (Jiri Denemark) - qemu: Introduce virQEMUCapsGetHostModel (Jiri Denemark) - Show host model in domain capabilities (Jiri Denemark) - qemu: Store host-model CPU in qemu capabilities (Jiri Denemark) - conf: Introduce virCPUDefCopyModelFilter (Jiri Denemark) - conf: Introduce virCPUDefStealModel (Jiri Denemark) - conf: Introduce virCPUDefCopyWithoutModel (Jiri Denemark) - qemu: Propagate virCapsPtr to virQEMUCapsNewForBinaryInternal (Jiri Denemark) - schema: Separate CPU related definitions into cputypes.rng (Jiri Denemark) - domcaps: Add CPU usable flag (Jiri Denemark) - domcaps: Show only CPU models supported by libvirt (Jiri Denemark) - cpu: Don't overwrite errors in cpuGetModels (Jiri Denemark) - cpu: Special case models == NULL in cpuGetModels (Jiri Denemark) - qemu: Fill in CPU domain capabilities (Jiri Denemark) - qemu: Introduce virQEMUCapsGuestIsNative (Jiri Denemark) - qemuxml2argvtest: Reorder CPU features (Jiri Denemark) - qemuxml2argvtest: Set correct architecture for KVM guests (Jiri Denemark) - qemuxml2argvtest: Properly setup CPU models in qemuCaps (Jiri Denemark) - qemu: Separate guest CPU validation from command line creation (Jiri Denemark) - testutilsqemu: Add default CPU for PPC64 architectures (Jiri Denemark) - testutilsqemu: Helpers for changing host CPU and arch (Jiri Denemark) - qemuxml2argvtest: Properly initialize qemuCaps->arch (Jiri Denemark) - qemuxml2argvtest: Update qemuCaps after parsing domain XML (Jiri Denemark) - qemuxml2argvtest: Reorder functions (Jiri Denemark) - qemuxml2argvtest: Get rid of testCompareXMLToArgvHelper (Jiri Denemark) - qemuxml2argvtest: Rename "out" labels as "cleanup" (Jiri Denemark) - qemuxml2argvtest: Rename extraFlags as qemuCaps (Jiri Denemark) - qemu: Use virDomainCapsCPUModels for cpuDefinitions (Jiri Denemark) - domcaps: Add support for listing supported CPU models (Jiri Denemark) - cpuGetModels: Switch to virArch (Jiri Denemark) - libxl: support VIR_MIGRATE_PERSIST_DEST migration flag (Jim Fehlig) - qemu_process: move graphics validation into separate function (Pavel Hrdina) - qemu_process: show shutoff reasons when debug log disabled (Chen Hanxiao) - qemu: domain: Add macro to simplify access to vm private data (Peter Krempa) - conf: Introduce virDomainObjGetOneDefState (Peter Krempa) - qemu: Ignore graphics cookie if port == 0 (Jiri Denemark) - qemu: agent: give better error messages whe agent monitor is down (Nikolay Shirokovskiy) - qemu: agent: reissue sync on garbage sync reply (Nikolay Shirokovskiy) - qemu: agent: skip outdated sync replies (Nikolay Shirokovskiy) - qemu: agent: ignore garbage while waiting for sync reply (Nikolay Shirokovskiy) - qemu: agent: ignore delayed reply when unsynced (Nikolay Shirokovskiy) - conf: Add some shmem helpers for future use (Martin Kletzander) - qemu: Add qemuAssignDeviceShmemAlias and use it (Martin Kletzander) - qemu: Rename qemuBuildShmemBackendStr to qemuBuildShmemBackendChrStr (Martin Kletzander) - qemu: Abstract shmem socket path preparation (Martin Kletzander) - qemu: Reorder shmem params nicely (Martin Kletzander) - qemu: Move common checks outside qemuBuildShmemDevLegacyStr (Martin Kletzander) - qemu: Rename qemuBuildShmemDevStr to qemuBuildShmemDevLegacyStr (Martin Kletzander) - qemu: Make qemuBuildShmemDevStr static (Martin Kletzander) - virt-admin: Add some command aliases to provide syntax sugar over ugly commands (Erik Skultety) - virt-admin: Tweak command parsing logic so that aliases point to new commands (Erik Skultety) - vsh: discard vshCmddefOptFill and move its body to vshCmddefOptParse (Erik Skultety) - vsh: Extract vshCmddefCheckInternals from vshCmddefOptParse (Erik Skultety) - vsh: vshCmddefHelp: Drop the unnecessary 'else' branch (Erik Skultety) - vsh: vshCmddefHelp: Drop unnecessary variable 'help' (Erik Skultety) - vsh: Enforce checking for missing help string (Erik Skultety) - test: update capability replies for qemu-2.7.0 (Pavel Hrdina) - Revert "configure: Check for major() more strictly" (Eric Blake) - build: update to latest gnulib (Eric Blake) - qemu: Introduce qemuGetHupageMemPath (Michal Privoznik) - qemu_agent|monitor: use different log on hangup event (Chen Hanxiao) - conf: Allow copying of shmem defs (Martin Kletzander) - schema: Allow shmem to have alias (Martin Kletzander) - qemuhotplugtest: Only read result_filename if used (Martin Kletzander) - test driver: Implement testNodeGetFreePages. (Tomáš Ryšavý) - test driver: added pages sizes into XML into test driver. (Tomáš Ryšavý) - test driver: Implement testNodeGetFreeMemory (Tomáš Ryšavý) - test driver: Implement testNodeGetCPUStats (Tomáš Ryšavý) - test driver: Implement virConnectGetType. (Tomáš Ryšavý) - test driver: Implement virConnectGetSysinfo. (Tomáš Ryšavý) - test_driver: Replace magic constant (Tomáš Ryšavý) - virhostcpu: Expose virHostCPUStatsAssign (Tomáš Ryšavý) - virsh: use virConnectGetDomainCapabilities with maxvcpus (Shivaprasad G Bhat) - tests: Use fixedcontent variable (Martin Kletzander) - hyperv: additional server 2008 wmi classes (Jason Miesionczek) - tests: Self test virt-admin (Michal Privoznik) - virsh: Move cmdSelfTest to vsh (Michal Privoznik) - qemu: monitor: Add vcpu state information to monitor data (Peter Krempa) - qemu: monitor: qemuMonitorGetCPUInfoHotplug: Add iterator 'anycpu' (Peter Krempa) - qemu: monitor: Use a more obvious iterator name (Peter Krempa) - qemuDomainGetBlockJobInfo: Move info translation into separate func (Michal Privoznik) - util: numa: Remove impossible error handling (Peter Krempa) - qemu: Add missing 'p' to qemuCgrouEmulatorAllNodesRestore (Peter Krempa) - virsh: add --keep-nvram option to undefine command (Nikolay Shirokovskiy) - qemu: add VIR_DOMAIN_UNDEFINE_KEEP_NVRAM support (Nikolay Shirokovskiy) - api: add VIR_DOMAIN_UNDEFINE_KEEP_NVRAM flag (Nikolay Shirokovskiy) - qemu: avoid parameter named 'listen' (Daniel P. Berrange) - storage: Need to refresh secret for luks volume after volume refresh (John Ferlan) - libxl: implement virConnectBaselineCPU (Joao Martins) - libxl: implement virConnectCompareCPU (Joao Martins) - libxl: describe host cpu features based on hwcaps (Joao Martins) - libxl: describe host topology in capabilities (Joao Martins) - qemu: domain: Clear startup policy for dropped removable media (Peter Krempa) - qemu: Implement virtio-net rx_queue_size (Michal Privoznik) - qemu_capabilities: Introduce virtio-net-*.rx_queue_size (Michal Privoznik) - conf: Add support for virtio-net.rx_queue_size (Michal Privoznik) - conf: Add new secret type "tls" (John Ferlan) - qemu: Add the ability to hotplug the TLS X.509 environment (John Ferlan) - qemu: Add support for TLS X.509 path to TCP chardev backend (John Ferlan) - conf: Introduce chartcp_tls_x509_cert_dir (John Ferlan) - conf: Add new default TLS X.509 certificate default directory (John Ferlan) - Add helper for removing transient definition (Jiri Denemark) - virsh: Fix *-event error string (Christophe Fergeau) - qemu: driver: Remove unnecessary condition (Peter Krempa) - qemu: cgroup: Extract temporary relaxing of cgroup setting for vcpu hotplug (Peter Krempa) - qemu: guest agent: introduce new error code VIR_ERR_AGENT_UNSYNCED (Yuri Pudgorodskiy) - tools: Pass opaque data in vshCompleter and introduce autoCompleteOpaque (Nishith Shah) - libvirt-guests.service: Add Requires=libvirtd.service (Andrea Bolognani) - libvirt-guests.service: Split After= relationship (Andrea Bolognani) - libvirt-guests.service: Improve description (Andrea Bolognani) - virtlogd.socket: Tie lifecycle to libvirtd.service (Andrea Bolognani) - tests: add missing data files for core config (Daniel P. Berrange) - qemu: Don't warn about missing device in DEVICE_DELETED event (Jiri Denemark) - qemu: allow turning off QEMU guest RAM dump globally (Daniel P. Berrange) - qemu: add a max_core setting to qemu.conf for core dump size (Daniel P. Berrange) - util: storage: Add json pseudo protocol support for legacy RBD strings (Peter Krempa) - virsh: Complete multiple options when any one option requires data (Nishith Shah) - virsh: Allow data or argument options to be completed as well (Nishith Shah) - virsh: Introduce usage of option completers to auto-complete arguments (Nishith Shah) - perf: add more perf events support (Qiaowei Ren) - perf: Adjust the perf initialization (Qiaowei Ren) - util: Move virPerfNew and virPerfFree (John Ferlan) - virsh: Rework the perf event names into a table. (John Ferlan) - virsh: Add a forward reference to perf command from domstats --perf (John Ferlan) - util: Add some comment details for virPerfEventType (Qiaowei Ren) - perf: Remove the switch from qemuDomainGetStatsPerf (Qiaowei Ren) - perf: rename qemuDomainGetStatsPerfRdt() (Qiaowei Ren) - qemu: Filter cur_balloon ABI check for certain transactions (Nikolay Shirokovskiy) - xmconfigdata: drop tests for multi serial (Bob Liu) - xlconfigdata: add tests for multi serial (Bob Liu) - xenconfig: rm format/parse multi serial for xen-xm (Bob Liu) - libxl: support serial list (Bob Liu) - virpci: support driver_override sysfs interface (Jim Fehlig) - libxl: allow vendor/product addressing for USB hostdevs (Cédric Bosdonnat) - Add virHostdevFindUSBDevice to private symbols (Cédric Bosdonnat) Thanks everybody for your help with this release, be it with patches, bug reports, documentation, patch reviews, etc ! Enjoy the release :-) Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 2

[libvirt] [PATCH] qemu_command: Count the cpus correctly when the <vcpus> are absent
by Shivaprasad 04 Oct '16

04 Oct '16

qemuBuildSmpCommandLine - is ending up generating "-smp 0," after first/subsequent restarts of the guest and we see "error: internal error: qemu reported thread id for inactive vcpu 'X'" The PostParse logic which changes the hotplug = ABSENT to NO for each vcpus is not called with virDomainDefCopy(because of SKIP_POST_PARSE) after first/subsequent restarts of the guest. Skipping post-parse is fine for all devices as the changes done as part of previous post-parse are captured in virDomainDefFormat and effectively copied during virDomainDefCopy. With cpu-hotplug, the def->individualvcpus is what determines whether the <vcpus> are to be formatted or not in the xml. Since individualvcpus is not set, the copies will have hotpluggable as ABSENT. The fix here is qemuBuildSmpCommandLine to check if the individualvcpus are set OR check the ABSENT flag. We dont want to set the individualvcpus in post-parse as that would disallow migration backwards to qemus not supporting hotplug. So, set check the ABSENT flag too when counting the vcpus instead. Signed-off-by: Shivaprasad <sbhat(a)linux.vnet.ibm.com> --- src/qemu/qemu_command.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 529bcb1..0a3c9ed 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -7113,7 +7113,8 @@ qemuBuildSmpCommandLine(virCommandPtr cmd, * in a different way */ for (i = 0; i < maxvcpus; i++) { vcpu = virDomainDefGetVcpu(def, i); - if (vcpu->online && vcpu->hotpluggable == VIR_TRISTATE_BOOL_NO) + if (vcpu->online && ((vcpu->hotpluggable == VIR_TRISTATE_BOOL_NO) || + (vcpu->hotpluggable == VIR_TRISTATE_BOOL_ABSENT))) nvcpus++; }

2 1