- Devel - libvirt List Archives

[libvirt] [RFC PATCH 0/4] qemu: Forbid NBD migration with TLS
by Peter Krempa 30 Apr '18

30 Apr '18

Currently we can't use TLS for NBD so allowing it if TLS is requested creates a security problem. Reject it by refusing to migrate disks and setup TLS on destination since that is easy enough. Note: That I've did not test this yet since my TLS setup was broken. I'll fix it later today and reprot the findings. Peter Krempa (4): qemu: caps: Add capability for TLS transport in the NBD server qemu: monitor: Add 'tls-creds' parameter to 'nbd-server-start' command qemu: migration: Use TLS environment for NBD server if requested qemu: migration: Forbid 'nbd' migration of non-shared storage if TLS is requested src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_migration.c | 28 +++++++++++++++++++--- src/qemu/qemu_monitor.c | 7 +++--- src/qemu/qemu_monitor.h | 3 ++- src/qemu/qemu_monitor_json.c | 4 +++- src/qemu/qemu_monitor_json.h | 3 ++- tests/qemucapabilitiesdata/caps_2.10.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.x86_64.xml | 1 + tests/qemumonitorjsontest.c | 2 +- 27 files changed, 59 insertions(+), 10 deletions(-) -- 2.16.2

3 12

[libvirt] [PATCH v2] qemu: migration: Don't crash on access to 'current' job
by Peter Krempa 30 Apr '18

30 Apr '18

When a VM is destroyed while being migrated (waiting in qemuMigrationSrcWaitForCompletion) the private object cleanup code frees the 'current' job info. Since the migration code attempts to setup various aspects of the current job even on failure this results into a crash. Job data is cleared in qemuDomainObjPrivateDataClear since commit 888aa4b6b9db Fix this by skipping all of the code which requires the qemu process to be alive if the VM is not active any more. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/qemu/qemu_migration.c | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index b913e99928..1a857329ec 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3561,27 +3561,28 @@ qemuMigrationSrcRun(virQEMUDriverPtr driver, error: orig_err = virSaveLastError(); - if (cancel && - priv->job.current->status != QEMU_DOMAIN_JOB_STATUS_QEMU_COMPLETED && - virDomainObjIsActive(vm) && - qemuDomainObjEnterMonitorAsync(driver, vm, - QEMU_ASYNC_JOB_MIGRATION_OUT) == 0) { - qemuMonitorMigrateCancel(priv->mon); - ignore_value(qemuDomainObjExitMonitor(driver, vm)); - } + if (virDomainObjIsActive(vm)) { + if (cancel && + priv->job.current->status != QEMU_DOMAIN_JOB_STATUS_QEMU_COMPLETED && + qemuDomainObjEnterMonitorAsync(driver, vm, + QEMU_ASYNC_JOB_MIGRATION_OUT) == 0) { + qemuMonitorMigrateCancel(priv->mon); + ignore_value(qemuDomainObjExitMonitor(driver, vm)); + } - /* cancel any outstanding NBD jobs */ - if (mig && mig->nbd) - qemuMigrationSrcCancelDriveMirror(driver, vm, false, - QEMU_ASYNC_JOB_MIGRATION_OUT, - dconn); + /* cancel any outstanding NBD jobs */ + if (mig && mig->nbd) + qemuMigrationSrcCancelDriveMirror(driver, vm, false, + QEMU_ASYNC_JOB_MIGRATION_OUT, + dconn); + + if (priv->job.current->status != QEMU_DOMAIN_JOB_STATUS_CANCELED) + priv->job.current->status = QEMU_DOMAIN_JOB_STATUS_FAILED; + } if (iothread) qemuMigrationSrcStopTunnel(iothread, true); - if (priv->job.current->status != QEMU_DOMAIN_JOB_STATUS_CANCELED) - priv->job.current->status = QEMU_DOMAIN_JOB_STATUS_FAILED; - goto cleanup; exit_monitor: -- 2.16.2

2 1

[libvirt] [PATCH] qemu: migration: Don't crash on access to 'current' job
by Peter Krempa 30 Apr '18

30 Apr '18

When a VM is destroyed while being migrated (waiting in qemuMigrationSrcWaitForCompletion) the private object cleanup code frees the 'current' job info. Since the migration code attempts to setup various aspects of the current job even on failure this results into a crash. Job data is cleared in qemuDomainObjPrivateDataClear since commit 888aa4b6b9db Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- I'm not certain that there isn't any other bigger root cause for this, but if the job is not present due to the VM being destroyed, it does not make much sense to set any data. src/qemu/qemu_migration.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 88b8253fa9..9f4de08d46 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3552,6 +3552,7 @@ qemuMigrationSrcRun(virQEMUDriverPtr driver, orig_err = virSaveLastError(); if (cancel && + priv->job.current && priv->job.current->status != QEMU_DOMAIN_JOB_STATUS_QEMU_COMPLETED && virDomainObjIsActive(vm) && qemuDomainObjEnterMonitorAsync(driver, vm, @@ -3569,7 +3570,8 @@ qemuMigrationSrcRun(virQEMUDriverPtr driver, if (iothread) qemuMigrationSrcStopTunnel(iothread, true); - if (priv->job.current->status != QEMU_DOMAIN_JOB_STATUS_CANCELED) + if (priv->job.current && + priv->job.current->status != QEMU_DOMAIN_JOB_STATUS_CANCELED) priv->job.current->status = QEMU_DOMAIN_JOB_STATUS_FAILED; goto cleanup; -- 2.16.2

2 1

[libvirt] Entering freeze for libvirt-4.3.0
by Daniel Veillard 29 Apr '18

29 Apr '18

As planned I just tagged the release candidate 1 in git and pushed signed tarball and rpms to the usual place: ftp://libvirt.org/libvirt/ Seems all fine with my limited testing and jenkins is all green: https://ci.centos.org/view/libvirt/ so so far thing looks good but that won't replace individual testing especially on other arches and OSes ! If thing go well I will push the RC2 over the week-end and target the final 4.3.0 release on Tues or Wed, in the meantime, plase give it some testing ! Thanks, Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 1

[libvirt] [PATCH v2] nwfilter: increase pcap buffer size to be compatible with TPACKET_V3
by Laine Stump 27 Apr '18

27 Apr '18

When an nwfilter rule sets the parameter CTRL_IP_LEARNING to "dhcp", this turns on the "dhcpsnoop" thread, which uses libpcap to monitor traffic on the domain's tap device and extract the IP address from the DHCP response. If libpcap on the host is built with HAVE_TPACKET3 defined (to enable support for TPACKET_V3), the dhcpsnoop code's initialization of the libpcap socket would fail with the following error: virNWFilterSnoopDHCPOpen:1134 : internal error: pcap_setfilter: can't remove kernel filter: Bad file descriptor It turns out that this was because TPACKET_V3 requires a larger buffer size than libvirt was setting (we were setting it to 128k). Changing the buffer size to 256k eliminates the error, and the dhcpsnoop thread once again works properly. A fuller explanation of why TPACKET_V3 requires such a large buffer, for future git spelunkers: libpcap calls setsockopt(... SOL_PACKET, PACKET_RX_RING...) to setup a ring buffer for receiving packets; two of the attributes sent to this API are called tp_frame_size, and tp_frame_nr. If libpcap was built with HAVE_TPACKET3 defined, tp_trame_size is set to MAXIMUM_SNAPLEN (defined in libpcap sources as 262144) and tp_frame_nr is set to: [the buffer size we set, i.e. PCAP_BUFFERSIZE i.e. 262144] / tp_frame_size. So if PCAP_BUFFERSIZE < MAXIMUM_SNAPLEN, then tp_frame_nr (the number of frames in the ring buffer) is 0, which is nonsensical. This same value is later used as a multiplier to determine the size for a call to malloc() (which would also fail). (NB: if HAVE_TPACKET3 is *not* defined, then tp_frame_size is set to the snaplen set by the user (in our case 576) plus a small amount to account for ethernet headers, so 256k is far more than adequate) Since the TPACKET_V3 code in libpcap actually reads multiple packets into each frame, it's not a problem to have only a single frame (especially when we are monitoring such infrequent traffic), so it's okay to set this relatively small buffer size (in comparison to the default, which is 2MB), which is important since every guest using dhcp snooping in a nwfilter rule will hold 2 of these buffers for the entire life of the guest. Thanks to Christian Ehrhardt <paelzer(a)gmail.com> for discovering that buffer size was the problem (this was not at all obvious from the error that was logged!) Resolves: https://bugzilla.redhat.com/1547237 Fixes: https://bugs.launchpad.net/libvirt/+bug/1758037 Signed-off-by: Laine Stump <laine(a)laine.org> --- change from V1: you asked for "big comment", you got it! If you think this is overkill, or underkill, let me know. I decided that the details about proper buffersize were better sitting right next to the definition of PCAP_BUFFERSIZE, but that there should also be a comment down in the code where any potential future error would occur, as that would more easily catch the eye of the hapless keyboard jockey who gets to rediscover this problem the next time libpcap changes. Also, I'm planning to request that the libpcap developers check for a viable buffersize during the pcap_set_buffer_size() call so that in the future this error is reported immediately rather than just having some random-seeming failure at a later time when pcap_setfilter() is called. src/nwfilter/nwfilter_dhcpsnoop.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/src/nwfilter/nwfilter_dhcpsnoop.c b/src/nwfilter/nwfilter_dhcpsnoop.c index 6069e70460..50cfb944a2 100644 --- a/src/nwfilter/nwfilter_dhcpsnoop.c +++ b/src/nwfilter/nwfilter_dhcpsnoop.c @@ -256,10 +256,21 @@ struct _virNWFilterDHCPDecodeJob { # define DHCP_BURST_INTERVAL_S 10 /* sec */ /* - * libpcap 1.5 requires a 128kb buffer - * 128 kb is bigger than (DHCP_PKT_BURST * PCAP_PBUFSIZE / 2) + * NB: Any libpcap built with HAVE_TPACKET3 will require + * PCAP_BUFFERSIZE to be at least 262144 (although + * pcap_set_buffer_size() with a lower value will succeed, and the + * error will only show up later when pcap_setfilter() is called). + * + * It is possible that in the future libpcap could increase the + * minimum size even further, but due to the fact that each guest + * using dhcp snooping keeps 2 pcap sockets open (and thus 2 buffers + * allocated) for the life of the guest, we want to minimize the + * length of the buffer, so instead of leaving it at the default size + * (2MB), we are setting it to the minimum viable size and including + * this clue in the source to help quickly resolve the problem when/if + * it reoccurs. */ -# define PCAP_BUFFERSIZE (128 * 1024) +# define PCAP_BUFFERSIZE (256 * 1024) # define MAX_QUEUED_JOBS (DHCP_PKT_BURST + 2 * DHCP_PKT_RATE) @@ -1114,6 +1125,11 @@ virNWFilterSnoopDHCPOpen(const char *ifname, virMacAddr *mac, goto cleanup_nohandle; } + /* IMPORTANT: If there is any failure of *any* pcap_* function + * during setup of the socket, look to the comment where + * PCAP_BUFFERSIZE is defined. It may be too small, even if the + * generated error doesn't imply that. + */ if (pcap_set_snaplen(handle, PCAP_PBUFSIZE) < 0 || pcap_set_buffer_size(handle, PCAP_BUFFERSIZE) < 0 || pcap_activate(handle) < 0) { -- 2.14.3

2 1

[libvirt] ignore_value
by gaosheng cui 27 Apr '18

27 Apr '18

Hi,using ignore_value in libvirt source code to do function return value processing,but I can’t understand about it,can you give me some tips?thanks very much!

2 1

[libvirt] [PATCH v2 0/6] Add support for TPM emulator and CRB interface
by Stefan Berger 27 Apr '18

27 Apr '18

This series of patches add support for the new TPM CRB interface in QEMU that will become available with QEMU 2.12. The rest of the patches add support for the TPM emulator backend that is available in QEMU and based on swtpm + libtpms. It allows to attach a TPM 1.2 or 2 to a QEMU VM. sVirt labels are used for labeling the swtpm process, its UnixIO socket, and log file with the same label that the QEMU process gets. Besides that swtpm is added to the emulator cgroup. The device XML can be changed from a TPM 1.2 to a TPM 2 and back to a TPM 1.2. The device state is not removed during those changes but only when the domain is undefined. I must admit that the first swtpm support patch is quite big. So far I only broke out the test cases into a subsequent patch. Regards, Stefan v1->v2: - reorganized directories where files are written to - all directories and files are chown'ed before swtpm is started - much refactoring Stefan Berger (6): tpm: Enable TPM CRB interface tpm: Add support for external swtpm TPM emulator tpm: Add test cases for external swtpm TPM emulator tpm: Label the external swtpm with SELinux labels tpm: Add support for choosing emulation of a TPM 2 tpm: Add swtpm to emulator cgroup docs/formatdomain.html.in | 47 ++ docs/schemas/domaincommon.rng | 23 +- src/conf/domain_audit.c | 2 + src/conf/domain_conf.c | 73 ++- src/conf/domain_conf.h | 15 + src/libvirt_private.syms | 9 + src/qemu/Makefile.inc.am | 2 + src/qemu/libvirtd_qemu.aug | 3 + src/qemu/qemu.conf | 7 + src/qemu/qemu_capabilities.c | 10 + src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_cgroup.c | 54 ++ src/qemu/qemu_cgroup.h | 1 + src/qemu/qemu_command.c | 52 +- src/qemu/qemu_conf.c | 35 +- src/qemu/qemu_conf.h | 5 + src/qemu/qemu_domain.c | 4 + src/qemu/qemu_driver.c | 7 + src/qemu/qemu_extdevice.c | 303 +++++++++++ src/qemu/qemu_extdevice.h | 44 ++ src/qemu/qemu_process.c | 16 + src/qemu/test_libvirtd_qemu.aug.in | 1 + src/security/security_dac.c | 6 + src/security/security_driver.h | 4 + src/security/security_manager.c | 18 + src/security/security_manager.h | 3 + src/security/security_selinux.c | 75 +++ src/security/security_stack.c | 19 + src/util/vircgroup.c | 42 ++ src/util/vircgroup.h | 1 + src/util/virfile.c | 60 +++ src/util/virfile.h | 2 + src/util/virtpm.c | 596 ++++++++++++++++++++- src/util/virtpm.h | 25 +- tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml | 2 + tests/qemuxml2argvdata/tpm-emulator-tpm2.args | 24 + tests/qemuxml2argvdata/tpm-emulator-tpm2.xml | 30 ++ tests/qemuxml2argvdata/tpm-emulator.args | 24 + tests/qemuxml2argvdata/tpm-emulator.xml | 30 ++ tests/qemuxml2argvdata/tpm-passthrough-crb.args | 24 + tests/qemuxml2argvdata/tpm-passthrough-crb.xml | 32 ++ tests/qemuxml2argvtest.c | 20 + tests/qemuxml2xmloutdata/tpm-emulator-tpm2.xml | 34 ++ tests/qemuxml2xmloutdata/tpm-emulator.xml | 34 ++ tests/qemuxml2xmloutdata/tpm-passthrough-crb.xml | 36 ++ tests/qemuxml2xmltest.c | 1 + 50 files changed, 1842 insertions(+), 19 deletions(-) create mode 100644 src/qemu/qemu_extdevice.c create mode 100644 src/qemu/qemu_extdevice.h create mode 100644 tests/qemuxml2argvdata/tpm-emulator-tpm2.args create mode 100644 tests/qemuxml2argvdata/tpm-emulator-tpm2.xml create mode 100644 tests/qemuxml2argvdata/tpm-emulator.args create mode 100644 tests/qemuxml2argvdata/tpm-emulator.xml create mode 100644 tests/qemuxml2argvdata/tpm-passthrough-crb.args create mode 100644 tests/qemuxml2argvdata/tpm-passthrough-crb.xml create mode 100644 tests/qemuxml2xmloutdata/tpm-emulator-tpm2.xml create mode 100644 tests/qemuxml2xmloutdata/tpm-emulator.xml create mode 100644 tests/qemuxml2xmloutdata/tpm-passthrough-crb.xml -- 2.5.5

2 18

[libvirt] [dbus PATCH 0/6] More DomainAPIs
by Katerina Koukiou 27 Apr '18

27 Apr '18

Katerina Koukiou (6): Implement BlockCopy method for Domain Interface Implement InterfaceAddresses method for Domain Interface Implement PinEmulator method for Domain Interface Implement PinIOThread method for Domain Interface Implement PinVcpu method for Domain Interface Implement PMWakeup method for Domain Interface data/org.libvirt.Domain.xml | 40 +++++++ src/domain.c | 269 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 309 insertions(+) -- 2.15.0

2 12

[libvirt] [dbus PATCH v3] Implement GetIOThreadInfo method for Domain Interface
by Katerina Koukiou 27 Apr '18

27 Apr '18

Signed-off-by: Katerina Koukiou <kkoukiou(a)redhat.com> --- Use VIR_CPU_USED to parse cpumap data/org.libvirt.Domain.xml | 6 ++++ src/domain.c | 68 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) diff --git a/data/org.libvirt.Domain.xml b/data/org.libvirt.Domain.xml index 1cd79f8..8be4c52 100644 --- a/data/org.libvirt.Domain.xml +++ b/data/org.libvirt.Domain.xml @@ -222,6 +222,12 @@ <arg name="flags" type="u" direction="in"/> <arg name="interfaceParameters" type="a{sv}" direction="out"/> </method> + <method name="GetIOThreadInfo"> + <annotation name="org.gtk.GDBus.DocString" + value="See https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetIOThreadIn…"/> + <arg name="flags" type="u" direction="in"/> + <arg name="ioThreadInfo" type="a(uab)" direction="out"/> + </method> <method name="GetJobInfo"> <annotation name="org.gtk.GDBus.DocString" value="See https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetJobInfo"/> diff --git a/src/domain.c b/src/domain.c index ec5b99c..9f6ad1b 100644 --- a/src/domain.c +++ b/src/domain.c @@ -85,6 +85,25 @@ virtDBusDomainFSInfoListClear(virtDBusDomainFSInfoList *info) G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(virtDBusDomainFSInfoList, virtDBusDomainFSInfoListClear); +struct _virtDBusDomainIOThreadInfoList { + virDomainIOThreadInfoPtr *info; + gint count; +}; + +typedef struct _virtDBusDomainIOThreadInfoList virtDBusDomainIOThreadInfoList; + +static void +virtDBusDomainIOThreadInfoListClear(virtDBusDomainIOThreadInfoList *info) +{ + for (gint i = 0; i < info->count; i++) + virDomainIOThreadInfoFree(info->info[i]); + + g_free(info->info); +} + +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(virtDBusDomainIOThreadInfoList, + virtDBusDomainIOThreadInfoListClear); + static GVariant * virtDBusDomainMemoryStatsToGVariant(virDomainMemoryStatPtr stats, gint nr_stats) @@ -1218,6 +1237,54 @@ virtDBusDomainGetInterfaceParameters(GVariant *inArgs, *outArgs = g_variant_new_tuple(&grecords, 1); } +static void +virtDBusDomainGetIOThreadInfo(GVariant *inArgs, + GUnixFDList *inFDs G_GNUC_UNUSED, + const gchar *objectPath, + gpointer userData, + GVariant **outArgs, + GUnixFDList **outFDs G_GNUC_UNUSED, + GError **error) +{ + virtDBusConnect *connect = userData; + g_autoptr(virDomain) domain = NULL; + g_auto(virtDBusDomainIOThreadInfoList) info = { 0 }; + GVariantBuilder builder; + guint flags; + gint cpuCount; + GVariant *gret; + + g_variant_get(inArgs, "(u)", &flags); + + domain = virtDBusDomainGetVirDomain(connect, objectPath, error); + if (!domain) + return; + + info.count = virDomainGetIOThreadInfo(domain, &info.info, flags); + if (info.count < 0) + return virtDBusUtilSetLastVirtError(error); + + cpuCount = virNodeGetCPUMap(virDomainGetConnect(domain), NULL, NULL, 0); + if (cpuCount < 0) + return virtDBusUtilSetLastVirtError(error); + + g_variant_builder_init(&builder, G_VARIANT_TYPE("a(uab)")); + + for (gint i = 0; i < info.count; i++) { + g_variant_builder_open(&builder, G_VARIANT_TYPE("(uab)")); + g_variant_builder_add(&builder, "u", info.info[i]->iothread_id); + g_variant_builder_open(&builder, G_VARIANT_TYPE("ab")); + + for (gint j = 0; j < cpuCount; j++) + g_variant_builder_add(&builder, "b", VIR_CPU_USED(info.info[i]->cpumap, j)); + g_variant_builder_close(&builder); + g_variant_builder_close(&builder); + } + gret = g_variant_builder_end(&builder); + + *outArgs = g_variant_new_tuple(&gret, 1); +} + static void virtDBusDomainGetJobInfo(GVariant *inArgs G_GNUC_UNUSED, GUnixFDList *inFDs G_GNUC_UNUSED, @@ -2651,6 +2718,7 @@ static virtDBusGDBusMethodTable virtDBusDomainMethodTable[] = { { "GetGuestVcpus", virtDBusDomainGetGuestVcpus }, { "GetHostname", virtDBusDomainGetHostname }, { "GetInterfaceParameters", virtDBusDomainGetInterfaceParameters }, + { "GetIOThreadInfo", virtDBusDomainGetIOThreadInfo }, { "GetJobInfo", virtDBusDomainGetJobInfo }, { "GetJobStats", virtDBusDomainGetJobStats }, { "GetMemoryParameters", virtDBusDomainGetMemoryParameters }, -- 2.15.0

2 1

[libvirt] [PATCH 0/2] URGENT: Fix TLS migration
by Peter Krempa 27 Apr '18

27 Apr '18

TLS would not be enabled for migration even when requested. This is a regression introduced in the migration params refactoring series pushed in this devel cycle. Peter Krempa (2): qemu: migration: Move and unexport qemuMigrationParamsSetString qemu: migration: Set the 'set' boolean in qemuMigrationParamsSetString src/qemu/qemu_migration_params.c | 41 +++++++++++++++++++++++++--------------- src/qemu/qemu_migration_params.h | 5 ----- 2 files changed, 26 insertions(+), 20 deletions(-) -- 2.16.2

3 5

[libvirt] [dbus PATCH] Revert "Implement MigrateGetMaxDowntime method for Domain Interface"
by Katerina Koukiou 27 Apr '18

27 Apr '18

This reverts commit d4a4fd4e9e0ab50f473c24192abc34644cd8213e. This API was introduced in libvirt 3.7.0 and no APIs from version > 3.0.0 should be included until next release. --- data/org.libvirt.Domain.xml | 6 ------ src/domain.c | 31 ------------------------------- 2 files changed, 37 deletions(-) diff --git a/data/org.libvirt.Domain.xml b/data/org.libvirt.Domain.xml index 1cd79f8..6f919da 100644 --- a/data/org.libvirt.Domain.xml +++ b/data/org.libvirt.Domain.xml @@ -332,12 +332,6 @@ <arg name="flags" type="u" direction="in"/> <arg name="cacheSize" type="t" direction="out"/> </method> - <method name="MigrateGetMaxDowntime"> - <annotation name="org.gtk.GDBus.DocString" - value="See https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainMigrateGetMax…"/> - <arg name="flags" type="u" direction="in"/> - <arg name="downtime" type="t" direction="out"/> - </method> <method name="MigrateGetMaxSpeed"> <annotation name="org.gtk.GDBus.DocString" value="See https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainMigrateGetMax…"/> diff --git a/src/domain.c b/src/domain.c index ec5b99c..06cad6f 100644 --- a/src/domain.c +++ b/src/domain.c @@ -1793,36 +1793,6 @@ virtDBusDomainMigrateGetCompressionCache(GVariant *inArgs, *outArgs = g_variant_new("(t)", cacheSize); } -static void -virtDBusDomainMigrateGetMaxDowntime(GVariant *inArgs, - GUnixFDList *inFDs G_GNUC_UNUSED, - const gchar *objectPath, - gpointer userData, - GVariant **outArgs, - GUnixFDList **outFDs G_GNUC_UNUSED, - GError **error) -{ - virtDBusConnect *connect = userData; - g_autoptr(virDomain) domain = NULL; - gulong downtime; - guint flags; - gint ret; - - g_variant_get(inArgs, "(u)", &flags); - - domain = virtDBusDomainGetVirDomain(connect, objectPath, error); - if (!domain) - return; - - ret = virDomainMigrateGetMaxDowntime(domain, - (unsigned long long *)&downtime, - flags); - if (ret < 0) - return virtDBusUtilSetLastVirtError(error); - - *outArgs = g_variant_new("(t)", downtime); -} - static void virtDBusDomainMigrateGetMaxSpeed(GVariant *inArgs, GUnixFDList *inFDs G_GNUC_UNUSED, @@ -2669,7 +2639,6 @@ static virtDBusGDBusMethodTable virtDBusDomainMethodTable[] = { { "MemoryPeek", virtDBusDomainMemoryPeek }, { "MemoryStats", virtDBusDomainMemoryStats }, { "MigrateGetCompressionCache", virtDBusDomainMigrateGetCompressionCache }, - { "MigrateGetMaxDowntime", virtDBusDomainMigrateGetMaxDowntime }, { "MigrateGetMaxSpeed", virtDBusDomainMigrateGetMaxSpeed }, { "MigrateSetCompressionCache", virtDBusDomainMigrateSetCompressionCache }, { "MigrateSetMaxDowntime", virtDBusDomainMigrateSetMaxDowntime }, -- 2.15.0

2 1

[libvirt] [dbus PATCH 00/10] NodeAPIs for Connect Interface
by Katerina Koukiou 27 Apr '18

27 Apr '18

Katerina Koukiou (10): Implement NodeGetCellsFreeMemory method for Connect Interface Implement NodeGetCPUStats method for Connect Interface Implement NodeGetFreeMemory method for Connect Interface Implement NodeGetInfo method for Connect Interface Implement NodeGetMemoryParameters method in Connect Interface Implement NodeGetMemoryStats method for Connect Interface Implement NodeGetSecurityModel method for Domain Interface Implement NodeSetMemoryParameters method for Connect Interface Implement NodeSuspendForDuration method for Connect Interface Implement NodeGetCPUMap method for Connect Interface data/org.libvirt.Connect.xml | 63 ++++++++ src/connect.c | 342 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 405 insertions(+) -- 2.15.0

2 16

[libvirt] [GSoC] Introduction
by Sukrit Bhatnagar 27 Apr '18

27 Apr '18

Hi, I am Sukrit Bhatnagar, selected in GSoC '18 for the project "Automatic freeing of resources". I will be modifying the libvirt core C library to add functionality for GCC cleanup attribute. Over the next couple of months, I will be discussing the general code design in this list. I had started a similar discussion a few weeks ago [1]. I will also be continuing the addition of virQEMUBuildBufferEscapeComma in qemu_command.c which I had been working on as a BiteSizedTask. Looking forward to work with the libvirt community. [1]: https://www.redhat.com/archives/libvir-list/2018-March/msg01532.html -- Thanks, Sukrit Bhatnagar

1 0

[libvirt] [jenkins-ci PATCH 0/5] guests: Enable ccache transparently
by Andrea Bolognani 27 Apr '18

27 Apr '18

This allows us to drop the ugly workaround from the Go and MinGW templates; as a nice side-effect, we also remove the need to hardcode most OS-dependent paths in the inventory. Andrea Bolognani (5): guests: Reorder tasks guests: Figure out OS-dependent paths automatically guests: Figure out more OS-dependent paths ahead of time guests: Create ccache symlink farm jobs: Remove $CC override guests/group_vars/all/main.yml | 7 +-- guests/host_vars/libvirt-freebsd-10/main.yml | 5 -- guests/host_vars/libvirt-freebsd-11/main.yml | 5 -- guests/host_vars/libvirt-freebsd-current/main.yml | 5 -- guests/site.yml | 17 +++--- guests/tasks/paths.yml | 69 +++++++++++++++++++++++ guests/tasks/users.yml | 33 ++++++++++- guests/templates/bashrc | 5 +- jobs/defaults.yaml | 6 -- jobs/go.yaml | 8 --- 10 files changed, 111 insertions(+), 49 deletions(-) create mode 100644 guests/tasks/paths.yml -- 2.14.3

3 9

[libvirt] [PATCH v4 0/5] Enable memory-backend-file.discard-data
by Michal Privoznik 26 Apr '18

26 Apr '18

v4 of: https://www.redhat.com/archives/libvir-list/2018-April/msg01897.html diff to v3: - patches 1 and 2 from the original set are pushed, - patch 3 is split into two (1 and 2 in this series), - Introduced new QEMU_CAPS_QOM_LIST_PROPERTIES - virQEMUCapsProbeQMPGenericProps is called iff the capability is set Michal Privoznik (5): qemuMonitorJSONGetDeviceProps: Separate props processing qemu_monitor: Introduce qemuMonitorGetObjectProps qemu_capabilities: Introduce QEMU_CAPS_QOM_LIST_PROPERTIES qemu_capabilities: Introduce QEMU_CAPS_OBJECT_MEMORY_FILE_DISCARD qemu: Introduce memoryBacking/discard docs/formatdomain.html.in | 34 +- docs/schemas/cputypes.rng | 5 + docs/schemas/domaincommon.rng | 10 + src/conf/domain_conf.c | 39 +- src/conf/domain_conf.h | 3 + src/conf/numa_conf.c | 27 + src/conf/numa_conf.h | 3 + src/libvirt_private.syms | 1 + src/qemu/qemu_capabilities.c | 20 + src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_command.c | 27 +- src/qemu/qemu_monitor.c | 13 + src/qemu/qemu_monitor.h | 3 + src/qemu/qemu_monitor_json.c | 89 +- src/qemu/qemu_monitor_json.h | 4 + .../caps_2.12.0.aarch64.replies | 416 +- tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml | 4 +- .../qemucapabilitiesdata/caps_2.12.0.ppc64.replies | 4454 ++++++++++---------- tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml | 4 +- .../qemucapabilitiesdata/caps_2.12.0.s390x.replies | 78 +- tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml | 2 + .../caps_2.12.0.x86_64.replies | 804 ++-- tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml | 4 +- tests/qemuxml2argvdata/hugepages-pages7.args | 3 +- tests/qemuxml2argvdata/hugepages-pages7.xml | 4 +- tests/qemuxml2argvtest.c | 3 +- 26 files changed, 3252 insertions(+), 2804 deletions(-) -- 2.16.1

3 13

[libvirt] [PATCH] nwfilter: increase pcap buffer size to be compatible with TPACKET_V3
by Laine Stump 26 Apr '18

26 Apr '18

When an nwfilter rule sets the parameter CTRL_IP_LEARNING to "dhcp", this turns on the "dhcpsnoop" thread, which uses libpcap to monitor traffic on the domain's tap device and extract the IP address from the DHCP response. If libpcap on the host is built with TPACKET_V3 defined, the dhcpsnoop code's initialization of the libpcap socket fails with the following error: virNWFilterSnoopDHCPOpen:1134 : internal error: pcap_setfilter: can't remove kernel filter: Bad file descriptor It turns out that this was because libpcap with TPACKET_V3 defined requires a larger buffer size than libvirt was setting (we were setting it to 128k). Changing the buffer size to 256k eliminates the error, and the dhcpsnoop thread once again works properly. Thanks to Christian Ehrhardt <paelzer(a)gmail.com> for discovering that buffer size was the problem. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1547237 Signed-off-by: Laine Stump <laine(a)laine.org> --- src/nwfilter/nwfilter_dhcpsnoop.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/nwfilter/nwfilter_dhcpsnoop.c b/src/nwfilter/nwfilter_dhcpsnoop.c index 6069e70460..62eb617515 100644 --- a/src/nwfilter/nwfilter_dhcpsnoop.c +++ b/src/nwfilter/nwfilter_dhcpsnoop.c @@ -259,7 +259,7 @@ struct _virNWFilterDHCPDecodeJob { * libpcap 1.5 requires a 128kb buffer * 128 kb is bigger than (DHCP_PKT_BURST * PCAP_PBUFSIZE / 2) */ -# define PCAP_BUFFERSIZE (128 * 1024) +# define PCAP_BUFFERSIZE (256 * 1024) # define MAX_QUEUED_JOBS (DHCP_PKT_BURST + 2 * DHCP_PKT_RATE) -- 2.14.3

3 6

[libvirt] [dbus PATCH 0/6] More DomainSet* APIs
by Katerina Koukiou 26 Apr '18

26 Apr '18

Katerina Koukiou (6): Implement SetBlkioParameters method for Domain Interface Implement SetBlockIOTune method for Domain Interface Implement SetInterfaceParameters method for Domain Interface Implement SetMemoryParameters method for Domain Interface Implement SetNumaParameters method for Domain Interface Implement SetSchedulerParameters method for Domain Interface data/org.libvirt.Domain.xml | 37 ++++++++ src/domain.c | 200 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 237 insertions(+) -- 2.15.0

2 12

[libvirt] [dbus PATCH v2] Implement GetIOThreadInfo method for Domain Interface
by Katerina Koukiou 26 Apr '18

26 Apr '18

Signed-off-by: Katerina Koukiou <kkoukiou(a)redhat.com> --- data/org.libvirt.Domain.xml | 6 +++++ src/domain.c | 63 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/data/org.libvirt.Domain.xml b/data/org.libvirt.Domain.xml index 96b9220..531cdb5 100644 --- a/data/org.libvirt.Domain.xml +++ b/data/org.libvirt.Domain.xml @@ -222,6 +222,12 @@ <arg name="flags" type="u" direction="in"/> <arg name="interfaceParameters" type="a{sv}" direction="out"/> </method> + <method name="GetIOThreadInfo"> + <annotation name="org.gtk.GDBus.DocString" + value="See https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetIOThreadIn…"/> + <arg name="flags" type="u" direction="in"/> + <arg name="ioThreadInfo" type="a(uab)" direction="out"/> + </method> <method name="GetJobInfo"> <annotation name="org.gtk.GDBus.DocString" value="See https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetJobInfo"/> diff --git a/src/domain.c b/src/domain.c index 76eec1d..e7b4b24 100644 --- a/src/domain.c +++ b/src/domain.c @@ -85,6 +85,25 @@ virtDBusDomainFSInfoListClear(virtDBusDomainFSInfoList *info) G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(virtDBusDomainFSInfoList, virtDBusDomainFSInfoListClear); +struct _virtDBusDomainIOThreadInfoList { + virDomainIOThreadInfoPtr *info; + gint count; +}; + +typedef struct _virtDBusDomainIOThreadInfoList virtDBusDomainIOThreadInfoList; + +static void +virtDBusDomainIOThreadInfoListClear(virtDBusDomainIOThreadInfoList *info) +{ + for (gint i = 0; i < info->count; i++) + virDomainIOThreadInfoFree(info->info[i]); + + g_free(info->info); +} + +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(virtDBusDomainIOThreadInfoList, + virtDBusDomainIOThreadInfoListClear); + static GVariant * virtDBusDomainMemoryStatsToGVariant(virDomainMemoryStatPtr stats, gint nr_stats) @@ -1218,6 +1237,49 @@ virtDBusDomainGetInterfaceParameters(GVariant *inArgs, *outArgs = g_variant_new_tuple(&grecords, 1); } +static void +virtDBusDomainGetIOThreadInfo(GVariant *inArgs, + GUnixFDList *inFDs G_GNUC_UNUSED, + const gchar *objectPath, + gpointer userData, + GVariant **outArgs, + GUnixFDList **outFDs G_GNUC_UNUSED, + GError **error) +{ + virtDBusConnect *connect = userData; + g_autoptr(virDomain) domain = NULL; + g_auto(virtDBusDomainIOThreadInfoList) info = { 0 }; + GVariantBuilder builder; + guint flags; + GVariant *gret; + + g_variant_get(inArgs, "(u)", &flags); + + domain = virtDBusDomainGetVirDomain(connect, objectPath, error); + if (!domain) + return; + + info.count = virDomainGetIOThreadInfo(domain, &info.info, flags); + if (info.count < 0) + return virtDBusUtilSetLastVirtError(error); + + g_variant_builder_init(&builder, G_VARIANT_TYPE("a(uab)")); + + for (gint i = 0; i < info.count; i++) { + g_variant_builder_open(&builder, G_VARIANT_TYPE("(uab)")); + g_variant_builder_add(&builder, "u", info.info[i]->iothread_id); + + g_variant_builder_open(&builder, G_VARIANT_TYPE("ab")); + for (gint j = 0; j < info.info[i]->cpumaplen; j++) + g_variant_builder_add(&builder, "b", info.info[i]->cpumap[j]); + g_variant_builder_close(&builder); + g_variant_builder_close(&builder); + } + gret = g_variant_builder_end(&builder); + + *outArgs = g_variant_new_tuple(&gret, 1); +} + static void virtDBusDomainGetJobInfo(GVariant *inArgs G_GNUC_UNUSED, GUnixFDList *inFDs G_GNUC_UNUSED, @@ -2457,6 +2519,7 @@ static virtDBusGDBusMethodTable virtDBusDomainMethodTable[] = { { "GetGuestVcpus", virtDBusDomainGetGuestVcpus }, { "GetHostname", virtDBusDomainGetHostname }, { "GetInterfaceParameters", virtDBusDomainGetInterfaceParameters }, + { "GetIOThreadInfo", virtDBusDomainGetIOThreadInfo }, { "GetJobInfo", virtDBusDomainGetJobInfo }, { "GetJobStats", virtDBusDomainGetJobStats }, { "GetMemoryParameters", virtDBusDomainGetMemoryParameters }, -- 2.15.0

2 1

[libvirt] [PATCH] syntax-check: pass srcdir to group-qemu-caps.pl
by Ján Tomko 26 Apr '18

26 Apr '18

The script assumed to be run in the source directory. Pass top_srcdir as the argument to fix VPATH builds. My commit 81a7571 broke this. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- Pushed as a build-breaker fix. cfg.mk | 2 +- tests/group-qemu-caps.pl | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/cfg.mk b/cfg.mk index 158d4cbd73..c3d18279ce 100644 --- a/cfg.mk +++ b/cfg.mk @@ -1120,7 +1120,7 @@ test-wrap-argv: $(PERL) $(top_srcdir)/tests/test-wrap-argv.pl --check $$files group-qemu-caps: - $(PERL) $(top_srcdir)/tests/group-qemu-caps.pl --check + $(PERL) $(top_srcdir)/tests/group-qemu-caps.pl --check $(top_srcdir)/ # sc_po_check can fail if generated files are not built first sc_po_check: \ diff --git a/tests/group-qemu-caps.pl b/tests/group-qemu-caps.pl index 847e360254..514764f47f 100755 --- a/tests/group-qemu-caps.pl +++ b/tests/group-qemu-caps.pl @@ -31,15 +31,21 @@ if (defined $ARGV[0] && $ARGV[0] eq "--check") { shift @ARGV; } +my $prefix = ''; +if (defined $ARGV[0]) { + $prefix = $ARGV[0]; + shift @ARGV; +} + my $ret = 0; -if (&regroup_caps('src/qemu/qemu_capabilities.c', +if (&regroup_caps($prefix . 'src/qemu/qemu_capabilities.c', '^VIR_ENUM_IMPL$virQEMUCaps,', '$;', 0, " ") < 0) { $ret = 1; } -if (&regroup_caps('src/qemu/qemu_capabilities.h', +if (&regroup_caps($prefix . 'src/qemu/qemu_capabilities.h', 'virQEMUCapsFlags grouping marker', 'QEMU_CAPS_LAST \/\* this must', 1, -- 2.16.1

1 0

[libvirt] [PATCH v2] logging: Don't inhibit shutdown in system daemon
by Martin Kletzander 26 Apr '18

26 Apr '18

That is a job of libvirtd and virtlogd has a dependency on it, so that will prevent it properly. Doing it one extra time in virtlogd might also cause AVC denials because it is not allowed to call that dbus method. Caused by commit df34363d58bb. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1547250 Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- I have an idea how to make this way nicer, but it requires a rework and maybe people will not like it as much, so for now this is the preferred fix. v2: - Don't revert the shutdown inhibition for all instances, only for the system one. v1: - https://www.redhat.com/archives/libvir-list/2018-April/msg02381.html src/logging/log_daemon.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/logging/log_daemon.c b/src/logging/log_daemon.c index 35d7ebb6d2bb..91bd9d0b9052 100644 --- a/src/logging/log_daemon.c +++ b/src/logging/log_daemon.c @@ -128,6 +128,12 @@ virLogDaemonInhibitor(bool inhibit, void *opaque) { virLogDaemonPtr dmn = opaque; + /* virtlogd uses inhibition only to stop session daemon being killed after + * the specified timeout, for the system daemon this is taken care of by + * libvirtd and the dependencies between the services. */ + if (virNetDaemonIsPrivileged(dmn->dmn)) + return; + if (inhibit) virNetDaemonAddShutdownInhibition(dmn->dmn); else -- 2.17.0

2 1

[libvirt] [PATCH] logging: Don't inhibit shutdown
by Martin Kletzander 26 Apr '18

26 Apr '18

That is a job of libvirtd and virtlogd has a dependency on it, so that will prevent it properly. Doing it one extra time in virtlogd might also cause AVC denials because it is not allowed to call that dbus method. This basically reverts df34363d58bbf424d5c8170a93d3ad5dcd4afb26, but manually due to the numerous changes since that was merged. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1547250 Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- src/logging/log_daemon.c | 19 ++----------------- src/logging/log_handler.c | 28 ++++------------------------ src/logging/log_handler.h | 11 ++--------- 3 files changed, 8 insertions(+), 50 deletions(-) diff --git a/src/logging/log_daemon.c b/src/logging/log_daemon.c index 35d7ebb6d2bb..9cf3fd4dbb5f 100644 --- a/src/logging/log_daemon.c +++ b/src/logging/log_daemon.c @@ -123,17 +123,6 @@ virLogDaemonFree(virLogDaemonPtr logd) } -static void -virLogDaemonInhibitor(bool inhibit, void *opaque) -{ - virLogDaemonPtr dmn = opaque; - - if (inhibit) - virNetDaemonAddShutdownInhibition(dmn->dmn); - else - virNetDaemonRemoveShutdownInhibition(dmn->dmn); -} - static virLogDaemonPtr virLogDaemonNew(virLogDaemonConfigPtr config, bool privileged) { @@ -185,9 +174,7 @@ virLogDaemonNew(virLogDaemonConfigPtr config, bool privileged) if (!(logd->handler = virLogHandlerNew(privileged, config->max_size, - config->max_backups, - virLogDaemonInhibitor, - logd))) + config->max_backups))) goto error; return logd; @@ -277,9 +264,7 @@ virLogDaemonNewPostExecRestart(virJSONValuePtr object, bool privileged, if (!(logd->handler = virLogHandlerNewPostExecRestart(child, privileged, config->max_size, - config->max_backups, - virLogDaemonInhibitor, - logd))) + config->max_backups))) goto error; return logd; diff --git a/src/logging/log_handler.c b/src/logging/log_handler.c index 40dfa8ecaebe..6d4e539f078e 100644 --- a/src/logging/log_handler.c +++ b/src/logging/log_handler.c @@ -65,9 +65,6 @@ struct _virLogHandler { virLogHandlerLogFilePtr *files; size_t nfiles; - - virLogHandlerShutdownInhibitor inhibitor; - void *opaque; }; static virClassPtr virLogHandlerClass; @@ -174,7 +171,6 @@ virLogHandlerDomainLogFileEvent(int watch, return; error: - handler->inhibitor(false, handler->opaque); virLogHandlerLogFileClose(handler, logfile); virObjectUnlock(handler); } @@ -183,9 +179,7 @@ virLogHandlerDomainLogFileEvent(int watch, virLogHandlerPtr virLogHandlerNew(bool privileged, size_t max_size, - size_t max_backups, - virLogHandlerShutdownInhibitor inhibitor, - void *opaque) + size_t max_backups) { virLogHandlerPtr handler; @@ -198,8 +192,6 @@ virLogHandlerNew(bool privileged, handler->privileged = privileged; handler->max_size = max_size; handler->max_backups = max_backups; - handler->inhibitor = inhibitor; - handler->opaque = opaque; return handler; @@ -220,8 +212,6 @@ virLogHandlerLogFilePostExecRestart(virLogHandlerPtr handler, if (VIR_ALLOC(file) < 0) return NULL; - handler->inhibitor(true, handler->opaque); - if ((path = virJSONValueObjectGetString(object, "path")) == NULL) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Missing 'path' field in JSON document")); @@ -276,7 +266,6 @@ virLogHandlerLogFilePostExecRestart(virLogHandlerPtr handler, return file; error: - handler->inhibitor(false, handler->opaque); virLogHandlerLogFileFree(file); return NULL; } @@ -286,9 +275,7 @@ virLogHandlerPtr virLogHandlerNewPostExecRestart(virJSONValuePtr object, bool privileged, size_t max_size, - size_t max_backups, - virLogHandlerShutdownInhibitor inhibitor, - void *opaque) + size_t max_backups) { virLogHandlerPtr handler; virJSONValuePtr files; @@ -297,9 +284,7 @@ virLogHandlerNewPostExecRestart(virJSONValuePtr object, if (!(handler = virLogHandlerNew(privileged, max_size, - max_backups, - inhibitor, - opaque))) + max_backups))) return NULL; if (!(files = virJSONValueObjectGet(object, "files"))) { @@ -349,10 +334,8 @@ virLogHandlerDispose(void *obj) virLogHandlerPtr handler = obj; size_t i; - for (i = 0; i < handler->nfiles; i++) { - handler->inhibitor(false, handler->opaque); + for (i = 0; i < handler->nfiles; i++) virLogHandlerLogFileFree(handler->files[i]); - } VIR_FREE(handler->files); } @@ -373,8 +356,6 @@ virLogHandlerDomainOpenLogFile(virLogHandlerPtr handler, virObjectLock(handler); - handler->inhibitor(true, handler->opaque); - for (i = 0; i < handler->nfiles; i++) { if (STREQ(virRotatingFileWriterGetPath(handler->files[i]->file), path)) { @@ -429,7 +410,6 @@ virLogHandlerDomainOpenLogFile(virLogHandlerPtr handler, error: VIR_FORCE_CLOSE(pipefd[0]); VIR_FORCE_CLOSE(pipefd[1]); - handler->inhibitor(false, handler->opaque); virLogHandlerLogFileFree(file); virObjectUnlock(handler); return -1; diff --git a/src/logging/log_handler.h b/src/logging/log_handler.h index 70be567ccc37..76df55c55e57 100644 --- a/src/logging/log_handler.h +++ b/src/logging/log_handler.h @@ -30,20 +30,13 @@ typedef struct _virLogHandler virLogHandler; typedef virLogHandler *virLogHandlerPtr; -typedef void (*virLogHandlerShutdownInhibitor)(bool inhibit, - void *opaque); - virLogHandlerPtr virLogHandlerNew(bool privileged, size_t max_size, - size_t max_backups, - virLogHandlerShutdownInhibitor inhibitor, - void *opaque); + size_t max_backups); virLogHandlerPtr virLogHandlerNewPostExecRestart(virJSONValuePtr child, bool privileged, size_t max_size, - size_t max_backups, - virLogHandlerShutdownInhibitor inhibitor, - void *opaque); + size_t max_backups); void virLogHandlerFree(virLogHandlerPtr handler); -- 2.17.0

2 2

[libvirt] [PATCH] syntax-check: check QEMU caps grouping
by Ján Tomko 25 Apr '18

25 Apr '18

Introduce a perl script that is able to regroup both the QEMU_CAPS constants and the capability strings. Check correct grouping as a part of syntax check. For in-place regrouping after a rebase, just run: tests/group-qemu-caps.pl without any parameters. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- cfg.mk | 5 +- src/qemu/qemu_capabilities.h | 2 +- tests/group-qemu-caps.pl | 118 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 123 insertions(+), 2 deletions(-) create mode 100755 tests/group-qemu-caps.pl diff --git a/cfg.mk b/cfg.mk index 4078bc2c6..7782ad6b3 100644 --- a/cfg.mk +++ b/cfg.mk @@ -1093,7 +1093,7 @@ _autogen_error: ifneq ($(_gl-Makefile),) syntax-check: spacing-check test-wrap-argv \ - prohibit-duplicate-header mock-noinline + prohibit-duplicate-header mock-noinline group-qemu-caps endif # Don't include duplicate header in the source (either *.c or *.h) @@ -1114,6 +1114,9 @@ test-wrap-argv: $(AM_V_GEN)files=`$(VC_LIST) | grep -E '\.(ldargs|args)'`; \ $(PERL) $(top_srcdir)/tests/test-wrap-argv.pl --check $$files +group-qemu-caps: + $(PERL) $(top_srcdir)/tests/group-qemu-caps.pl --check + # sc_po_check can fail if generated files are not built first sc_po_check: \ $(srcdir)/src/remote/remote_daemon_dispatch_stubs.h \ diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index bec28cae9..00c76fdd1 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -46,7 +46,7 @@ * longer be used in code. Periodically we can then purge all the * X_ flags and re-group what's left. */ -typedef enum { +typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ /* 0 */ X_QEMU_CAPS_KQEMU, /* Whether KQEMU is compiled in */ X_QEMU_CAPS_VNC_COLON, /* VNC takes or address + display */ diff --git a/tests/group-qemu-caps.pl b/tests/group-qemu-caps.pl new file mode 100755 index 000000000..847e36025 --- /dev/null +++ b/tests/group-qemu-caps.pl @@ -0,0 +1,118 @@ +#!/usr/bin/env perl +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. +# +# +# Regroup array values into smaller groups separated by numbered comments. +# +# If --check is the first parameter, the script will return +# a non-zero value if a file is not grouped correctly. +# Otherwise the files are regrouped in place. + +use strict; +use warnings; + +my $check = 0; + +if (defined $ARGV[0] && $ARGV[0] eq "--check") { + $check = 1; + shift @ARGV; +} + +my $ret = 0; +if (&regroup_caps('src/qemu/qemu_capabilities.c', + '^VIR_ENUM_IMPL$virQEMUCaps,', + '$;', + 0, + " ") < 0) { + $ret = 1; +} +if (&regroup_caps('src/qemu/qemu_capabilities.h', + 'virQEMUCapsFlags grouping marker', + 'QEMU_CAPS_LAST \/\* this must', + 1, + " ") < 0) { + $ret = 1; +} + +exit $ret; + +sub regroup_caps { + my $filename = shift; + my $start_regex = shift; + my $end_regex = shift; + my $trailing_newline = shift; + my $counter_prefix = shift; + my $step = 5; + + open FILE, '<', $filename or die "cannot open $filename: $!"; + my @original = <FILE>; + close FILE; + + my @fixed; + my $game_on = 0; + my $counter = 0; + foreach (@original) { + if ($game_on) { + next if ($_ =~ '/\* [0-9]+ \*/'); + next if (/^\s+$/); + if ($counter % $step == 0) { + if ($counter != 0) { + push @fixed, "\n"; + } + push @fixed, "$counter_prefix/* $counter */\n"; + } + if (!($_ =~ '/\*' && !($_ =~ '\*/'))) { + # count two-line comments as one line + $counter++; + } + } + if (/$start_regex/) { + $game_on = 1; + } elsif ($game_on && $_ =~ /$end_regex/) { + if (($counter -1) % $step == 0) { + pop @fixed; # /* $counter */ + if ($counter != 1) { + pop @fixed; # \n + } + } + if ($trailing_newline) { + push @fixed, "\n"; + } + $game_on = 0; + } + push @fixed, $_; + } + + if ($check) { + my $nl = join('', @fixed); + my $ol = join('', @original); + unless ($nl eq $ol) { + open DIFF, "| diff -u $filename -" or die "cannot run diff: $!"; + print DIFF $nl; + close DIFF; + + print STDERR "Incorrect array grouping in $filename\n"; + print STDERR "Use group-qemu-caps.pl to group long array members\n"; + return -1; + } + } else { + open FILE, '>', $filename or die "cannot open $filename: $!"; + foreach my $line (@fixed) { + print FILE $line; + } + close FILE; + } +} -- 2.13.6

2 1

[libvirt] [RFC v2] external (pull) backup API
by Nikolay Shirokovskiy 25 Apr '18

25 Apr '18

Hi, all. This is another RFC on pull backup API. This API provides means to read domain disks in a snapshotted state so that client can back them up as well as means to write domain disks to revert them to backed up state. The previous version of RFC is [1]. I'll also describe the API implementation details to shed light on misc qemu dirty bitmap commands usage. This API does not use existent disks snapshots. Instead it introduces snapshots provided by qemu's blockdev-backup command. The reason is we need snapshotted disk state only temporarily for duration of backup operation and newly introduced snapshots can be easily discarded at the end of operation without block commit operation. Technically difference is next. On usual snapshot we create new image backed by original and all new data goes to the new image thus original image stays in a snapshotted state. In temporary snapshots we create new image backed by original and all new data still goes to the original image but before new data is written old data to be overwritten is popped out to the new image thus we get snapshotted state thru new image. Disks snapshots as well as disks itself are avaiable to read/write thru qemu NBD server. Here is typical actions on domain backup: - create temporary snapshot of domain disks of interest - export snaphots thru NBD - back them up - remove disks from export - delete temporary snapshot and typical actions on domain restore: - start domain in paused state - export domain disks of interest thru NBD for write - restore them - remove disks from export - resume or destroy domain Now let's write down API in more details. There are minor changes in comparison with previous version [1]. *Temporary snapshot API* In previous version it is called 'Fleece API' after qemu terms and I'll still use BlockSnapshot prefix for commands as in previous RFC instead of TmpSnapshots which I inclined more now. virDomainBlockSnapshotPtr virDomainBlockSnapshotCreateXML(virDomainPtr domain, const char *xmlDesc, unsigned int flags); virDomainBlockSnapshotDelete(virDomainBlockSnapshotPtr snapshot, unsigned int flags); virDomainBlockSnapshotList(virDomainPtr domain, virDomainBlockSnapshotPtr **snaps, unsigned int flags); virDomainBlockSnapshotGetXMLDesc(virDomainBlockSnapshotPtr snapshot, unsigned int flags); virDomainBlockSnapshotPtr virDomainBlockSnapshotLookupByName(virDomainPtr domain, const char *name, unsigned int flags); Here is an example of snapshot xml description: <domainblocksnapshot> <name>d068765e-8b50-4d74-9b72-1e55c663cbf8</name> <disk name='sda' type="file"> <fleece file="/tmp/snapshot-a.hdd"/> </disk> <disk name='sdb' type="file"> <fleece file="/tmp/snapshot-b.hdd"/> </disk> </domainblocksnapshot> Temporary snapshots are indepentent thus they are not organized in tree structure as usual snapshots, so the 'list snapshots' and 'lookup' function will suffice. Qemu can track what disk's blocks are changed from snapshotted state so on next backup client can backup only changed blocks. virDomainBlockSnapshotCreateXML accepts VIR_DOMAIN_BLOCK_SNAPSHOT_CREATE_CHECKPOINT flag to turn this option for snapshot which means to track changes from this particular snapshot. I used checkpoint term and not [dirty] bitmap because many qemu dirty bitmaps are used to provide changed blocks from the given checkpoint to current snapshot in current implementation (see *Implemenation* section for more details). Also bitmap keeps block changes and thus itself changes in time and checkpoint is a more statical terms means you can query changes from that moment in time. Checkpoints are visible in active domain xml: <disk type='file' device='disk'> .. <target dev='sda' bus='scsi'/> <alias name='scsi0-0-0-0'/> <checkpoint name="93a5c045-6457-2c09-e56c-927cdf34e178"> <checkpoint name="5768a388-c1c4-414c-ac4e-eab216ba7c0c"> .. </disk> Every checkpoint requires qemu dirty bitmap which eats 16MiB of RAM with default dirty block size of 64KiB for 1TiB disk and the same amount of disk space is used. So client need to manage checkpoints and delete unused. Thus next API function: int virDomainBlockCheckpointRemove(virDomainPtr domain, const char *name, unsigned int flags); *Block export API* I guess it is natural to treat qemu NBD server as a domain device. So we can use virDomainAttachDeviceFlags/virDomainDetachDeviceFlags API to start/stop NBD server and virDomainUpdateDeviceFlags to add/delete disks to be exported. While I'm have no doubts about start/stop operations using virDomainUpdateDeviceFlags looks a bit inconvinient so I decided to add a pair of API functions just to add/delete disks to be exported: int virDomainBlockExportStart(virDomainPtr domain, const char *xmlDesc, unsigned int flags); int virDomainBlockExportStop(virDomainPtr domain, const char *xmlDesc, unsigned int flags); I guess more appropriate names are virDomainBlockExportAdd and virDomainBlockExportRemove but as I already have a patch series implementing pull backups with these names I would like to keep these names now. These names also reflect that in the implementation I decided to start/stop NBD server in a lazy manner. While it is a bit innovative for libvirt API I guess it is convinient because to refer NBD server to add/remove disks to we need to identify it thru it's parameters like type, address etc until we introduce some device id (which does not looks consistent with current libvirt design). So it looks like we have all parameters to start/stop server in the frame of these calls so why have extra API calls just to start/stop server manually. If we later need to have NBD server without disks we can perfectly support virDomainAttachDeviceFlags/virDomainDetachDeviceFlags. Here is example of xml to add/remove disks (specifying checkpoint attribute is not needed for removing disks of course): <domainblockexport type="nbd"> <address type="ip" host="0.0.0.0" port="8000"/> <disk name="sda" snapshot="0044757e-1a2d-4c2c-b92f-bb403309bb17" checkpoint="d068765e-8b50-4d74-9b72-1e55c663cbf8"/> <disk name="sdb" snapshot="0044757e-1a2d-4c2c-b92f-bb403309bb17" checkpoint="d068765e-8b50-4d74-9b72-1e55c663cbf8"/> </domainblockexport> And this is how this NBD server will be exposed in domain xml: <devices> ... <blockexport type="nbd"> <address type="ip" host="0.0.0.0" port="8000"/> <disk name="sda" snapshot="0044757e-1a2d-4c2c-b92f-bb403309bb17" checkpoint="d068765e-8b50-4d74-9b72-1e55c663cbf8" exportname="sda-0044757e-1a2d-4c2c-b92f-bb403309bb17"/> <disk name="sdb" snapshot="0044757e-1a2d-4c2c-b92f-bb403309bb17" checkpoint="d068765e-8b50-4d74-9b72-1e55c663cbf8 exportname="sdb-0044757e-1a2d-4c2c-b92f-bb403309bb17"/> </blockexport> ... </devices> *Implementation details from qemu-libvirt interactions POV* 1. Temporary snapshot - create snapshot - add fleece blockdev backed by disk of interest - start fleece blockjob which will pop out data to be overwritten to fleece blockdev { "execute": "blockdev-add" "arguments": { "backing": "drive-scsi0-0-0-0", "driver": "qcow2", "file": { "driver": "file", "filename": "/tmp/snapshot-a.hdd" }, "node-name": "snapshot-scsi0-0-0-0" }, } { "execute": "transaction" "arguments": { "actions": [ { "type": "blockdev-backup" "data": { "device": "drive-scsi0-0-0-0", "target": "snapshot-scsi0-0-0-0" "sync": "none", }, } ] }, } - delete snapshot - cancel fleece blockjob - delete fleece blockdev { "execute": "block-job-cancel" "arguments": { "device": "drive-scsi0-0-0-0" }, } { "execute": "blockdev-del" "arguments": { "node-name": "snapshot-scsi0-0-0-0" }, } 2. Block export - add disks to export - start NBD server if it is not started - add disks { "execute": "nbd-server-start" "arguments": { "addr": { "type": "inet" "data": { "host": "0.0.0.0", "port": "49300" }, } }, } { "execute": "nbd-server-add" "arguments": { "device": "snapshot-scsi0-0-0-0", "name": "sda-d068765e-8b50-4d74-9b72-1e55c663cbf8", "writable": false }, } - remove disks from export - remove disks - stop NBD server if there are no disks left { "arguments": { "mode": "hard", "name": "sda-d068765e-8b50-4d74-9b72-1e55c663cbf8" }, "execute": "nbd-server-remove" } { "execute": "nbd-server-stop" } 3. Checkpoints (the most interesting part) First a few facts about qemu dirty bitmaps. Bitmap can be either in active or disable state. In disabled state it does not get changed on guest writes. And oppositely in active state it tracks guest writes. This implementation uses approach with only one active bitmap at a time. This should reduce guest write penalties in the presence of checkpoints. So on first snapshot we create bitmap B_1. Now it tracks changes from the snapshot 1. On second snapshot we create bitmap B_2 and disable bitmap B1 and so on. Now bitmap B1 keep changes from snaphost 1 to snapshot 2, B2 - changes from snaphot 2 to snapshot 3 and so on. Last bitmap is active and gets most disk change after latest snapshot. Getting changed blocks bitmap from some checkpoint in past till current snapshot is quite simple in this scheme. For example if the last snapshot is 7 then to get changes from snapshot 3 to latest snapshot we need to merge bitmaps B3, B4, B4 and B6. Merge is just logical OR on bitmap bits. Deleting a checkpoint somewhere in the middle of checkpoint sequence requires merging correspondent bitmap to the previous bitmap in this scheme. We use persitent bitmaps in the implementation. This means upon qemu process termination bitmaps are saved in disks images metadata and restored back on qemu process start. This makes checkpoint a persistent property that is we keep them across domain start/stops. Qemu does not try hard to keep bitmaps. If upon save something goes wrong bitmap is dropped. The same is applied to the migration process too. For backup process it is not critical. If we don't discover a checkpoint we always can make a full backup. Also qemu provides no special means to track order of bitmaps. These facts are critical for implementation with one active bitmap at a time. We need right order of bitmaps upon merge - for snapshot N and block changes from snanpshot K, K < N to N we need to merge bitmaps B_{K}, ..., B_{N-1}. Also if one of the bitmaps to be merged is missing we can't calculate desired block changes too. So the implementation encode bitmap order in their names. For snapshot A1, bitmap name will be A1, for snapshot A2 bitmap name will be A2^A1 and so on. Using this naming encoding upon domain start we can find out bitmap order and check for missing ones. This complicates a bit bitmap removing though. For example removing a bitmap somewhere in the middle looks like this: - removing bitmap K (bitmap name is NAME_{K}^NAME_{K-1} - create new bitmap named NAME_{K+1}^NAME_{K-1} ---. - disable new bitmap | This is effectively renaming - merge bitmap NAME_{K+1}^NAME_{K} to the new bitmap | of bitmap K+1 to comply the naming scheme - remove bitmap NAME_{K+1}^NAME_{K} ___/ - merge bitmap NAME_{K}^NAME_{K-1} to NAME_{K-1}^NAME_{K-2} - remove bitmap NAME_{K}^NAME_{K-1} As you can see we need to change name for bitmap K+1 to keep our bitmap naming scheme. This is done creating new K+1 bitmap with appropriate name and copying old K+1 bitmap into new. So while it is possible to have only one active bitmap at a time it costs some exersices at managment layer. To me it looks like qemu itself is a better place to track bitmaps chain order and consistency. Now how exporting bitmaps looks like. - add to export disk snapshot N with changes from checkpoint K - add fleece blockdev to NBD exports - create new bitmap T - disable bitmap T - merge bitmaps K, K+1, .. N-1 into T - add bitmap to T to nbd export - remove disk snapshot from export - remove fleece blockdev from NBD exports - remove bitmap T Here is qemu commands examples for operation with checkpoints, I'll make several snapshots with checkpoints for purpuse of better illustration. - create snapshot d068765e-8b50-4d74-9b72-1e55c663cbf8 with checkpoint - same as without checkpoint but additionally add bitmap on fleece blockjob start ... { "execute": "transaction" "arguments": { "actions": [ { "type": "blockdev-backup" "data": { "device": "drive-scsi0-0-0-0", "sync": "none", "target": "snapshot-scsi0-0-0-0" }, }, { "type": "block-dirty-bitmap-add" "data": { "name": "libvirt-d068765e-8b50-4d74-9b72-1e55c663cbf8", "node": "drive-scsi0-0-0-0", "persistent": true }, } ] }, } - delete snapshot d068765e-8b50-4d74-9b72-1e55c663cbf8 - same as without checkpoints - create snapshot 0044757e-1a2d-4c2c-b92f-bb403309bb17 with checkpoint - same actions as for the first snapshot, but additionally disable the first bitmap ... { "execute": "transaction" "arguments": { "actions": [ { "type": "blockdev-backup" "data": { "device": "drive-scsi0-0-0-0", "sync": "none", "target": "snapshot-scsi0-0-0-0" }, }, { "type": "x-vz-block-dirty-bitmap-disable" "data": { "name": "libvirt-d068765e-8b50-4d74-9b72-1e55c663cbf8", "node": "drive-scsi0-0-0-0" }, }, { "type": "block-dirty-bitmap-add" "data": { "name": "libvirt-0044757e-1a2d-4c2c-b92f-bb403309bb17^d068765e-8b50-4d74-9b72-1e55c663cbf8", "node": "drive-scsi0-0-0-0", "persistent": true }, } ] }, } - delete snapshot 0044757e-1a2d-4c2c-b92f-bb403309bb17 - create snapshot 8fc02db3-166f-4de7-b7aa-1f7303e6162b with checkpoint - add disk snapshot 8fc02db3-166f-4de7-b7aa-1f7303e6162b to export and bitmap with changes from checkpoint d068765e-8b50-4d74-9b72-1e55c663cbf8 - same as add export without checkpoint, but aditionally - form result bitmap - add bitmap to NBD export ... { "execute": "transaction" "arguments": { "actions": [ { "type": "block-dirty-bitmap-add" "data": { "node": "drive-scsi0-0-0-0", "name": "libvirt-__export_temporary__", "persistent": false }, }, { "type": "x-vz-block-dirty-bitmap-disable" "data": { "node": "drive-scsi0-0-0-0" "name": "libvirt-__export_temporary__", }, }, { "type": "x-vz-block-dirty-bitmap-merge" "data": { "node": "drive-scsi0-0-0-0", "src_name": "libvirt-d068765e-8b50-4d74-9b72-1e55c663cbf8" "dst_name": "libvirt-__export_temporary__", }, }, { "type": "x-vz-block-dirty-bitmap-merge" "data": { "node": "drive-scsi0-0-0-0", "src_name": "libvirt-0044757e-1a2d-4c2c-b92f-bb403309bb17^d068765e-8b50-4d74-9b72-1e55c663cbf# "dst_name": "libvirt-__export_temporary__", }, } ] }, } { "execute": "x-vz-nbd-server-add-bitmap" "arguments": { "name": "sda-8fc02db3-166f-4de7-b7aa-1f7303e6162b" "bitmap": "libvirt-__export_temporary__", "bitmap-export-name": "d068765e-8b50-4d74-9b72-1e55c663cbf8", }, } - remove snapshot 8fc02db3-166f-4de7-b7aa-1f7303e6162b from export - same as without checkpoint but additionally remove temporary bitmap ... { "arguments": { "name": "libvirt-__export_temporary__", "node": "drive-scsi0-0-0-0" }, "execute": "block-dirty-bitmap-remove" } - delete checkpoint 0044757e-1a2d-4c2c-b92f-bb403309bb17 (similar operation is described in the section about naming scheme for bitmaps, with difference that K+1 is N here and thus new bitmap should not be disabled) { "arguments": { "actions": [ { "type": "block-dirty-bitmap-add" "data": { "node": "drive-scsi0-0-0-0", "name": "libvirt-8fc02db3-166f-4de7-b7aa-1f7303e6162b^d068765e-8b50-4d74-9b72-1e55c663cbf8", "persistent": true }, }, { "type": "x-vz-block-dirty-bitmap-merge" "data": { "node": "drive-scsi0-0-0-0", "src_name": "libvirt-0044757e-1a2d-4c2c-b92f-bb403309bb17^d068765e-8b50-4d74-9b72-1e55c663cbf# "dst_name": "libvirt-d068765e-8b50-4d74-9b72-1e55c663cbf8", }, }, { "type": "x-vz-block-dirty-bitmap-merge" "data": { "node": "drive-scsi0-0-0-0", "src_name": "libvirt-8fc02db3-166f-4de7-b7aa-1f7303e6162b^0044757e-1a2d-4c2c-b92f-bb403309bb1# "dst_name": "libvirt-8fc02db3-166f-4de7-b7aa-1f7303e6162b^d068765e-8b50-4d74-9b72-1e55c663cbf# }, }, ] }, "execute": "transaction" } { "execute": "x-vz-block-dirty-bitmap-remove" "arguments": { "node": "drive-scsi0-0-0-0" "name": "libvirt-8fc02db3-166f-4de7-b7aa-1f7303e6162b^0044757e-1a2d-4c2c-b92f-bb403309bb17", }, }, { "execute": "x-vz-block-dirty-bitmap-remove" "arguments": { "node": "drive-scsi0-0-0-0" "name": "libvirt-0044757e-1a2d-4c2c-b92f-bb403309bb17^d068765e-8b50-4d74-9b72-1e55c663cbf8", }, } Here is a list of bitmap commands used in implementation but not yet in upstream (AFAIK). x-vz-block-dirty-bitmap-remove x-vz-block-dirty-bitmap-merge x-vz-block-dirty-bitmap-disable x-vz-block-dirty-bitmap-enable (not in the examples; used when removing most recent checkpoint) x-vz-nbd-server-add-bitmap *Restore operation nuances* As it was written above to restore a domain one needs to start it in paused state, export domain's disks and write them from backup. However qemu currently does not let export disks for write even for a domain that never starts guests CPU. We have an experimental qemu command option -x-vz-nbd-restore (passed together with -incoming option) to fix it. *Links* [1] Previous version of RFC https://www.redhat.com/archives/libvir-list/2017-November/msg00514.html

5 51

[libvirt] [dbus PATCH 00/15] Some more get APIs for Domain Interface
by Katerina Koukiou 25 Apr '18

25 Apr '18

Katerina Koukiou (15): Implement GetTime method for Domain Interface Implement MigrateGetCompressionCache method for Domain Interface Implement GetBlockIoTune method for Domain Interface Implement GetBlockJobInfo method for Domain Interface Implement GetDiskErrors method for Domain Interface Impement GetFSInfo method for Domain Interface Implement GetGuestVcpus method for Domain interface Implement GetHostname method for Domain Interface Implement GetInterfaceParameters method for Domain Interface Implement GetIOThreadInfo method for Domain Interface Implement GetJobStats method for Domain Interface Implement GetMetadata method for Domain Interface Implement GetNumaParameters method for Domain Interface Implement GetPerfEvents method for Domain Interface Implement GetCPUStats method for Domain Interface data/org.libvirt.Domain.xml | 98 +++++++ src/domain.c | 665 ++++++++++++++++++++++++++++++++++++++++++++ tests/test_domain.py | 6 + 3 files changed, 769 insertions(+) -- 2.15.0

4 36

[libvirt] [PATCH v2] qemu: Add I/O thread support info into domain capabilities
by Martin Kletzander 25 Apr '18

25 Apr '18

Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- docs/schemas/domaincaps.rng | 10 ++++++++++ src/conf/domain_capabilities.c | 3 +++ src/conf/domain_capabilities.h | 1 + src/qemu/qemu_capabilities.c | 11 +++++++++++ tests/domaincapsschemadata/basic.xml | 1 + tests/domaincapsschemadata/full.xml | 1 + tests/domaincapsschemadata/qemu_1.7.0.x86_64.xml | 1 + .../domaincapsschemadata/qemu_2.12.0-virt.aarch64.xml | 1 + tests/domaincapsschemadata/qemu_2.12.0.ppc64.xml | 1 + tests/domaincapsschemadata/qemu_2.12.0.s390x.xml | 1 + tests/domaincapsschemadata/qemu_2.12.0.x86_64.xml | 1 + .../domaincapsschemadata/qemu_2.6.0-virt.aarch64.xml | 1 + tests/domaincapsschemadata/qemu_2.6.0.aarch64.xml | 1 + tests/domaincapsschemadata/qemu_2.6.0.ppc64.xml | 1 + tests/domaincapsschemadata/qemu_2.6.0.x86_64.xml | 1 + tests/domaincapsschemadata/qemu_2.7.0.s390x.xml | 1 + tests/domaincapsschemadata/qemu_2.8.0-tcg.x86_64.xml | 1 + tests/domaincapsschemadata/qemu_2.8.0.s390x.xml | 1 + tests/domaincapsschemadata/qemu_2.8.0.x86_64.xml | 1 + tests/domaincapsschemadata/qemu_2.9.0-q35.x86_64.xml | 1 + tests/domaincapsschemadata/qemu_2.9.0-tcg.x86_64.xml | 1 + tests/domaincapsschemadata/qemu_2.9.0.x86_64.xml | 1 + 22 files changed, 43 insertions(+) diff --git a/docs/schemas/domaincaps.rng b/docs/schemas/domaincaps.rng index 39053181eb9a..049b2ae7b64f 100644 --- a/docs/schemas/domaincaps.rng +++ b/docs/schemas/domaincaps.rng @@ -28,6 +28,9 @@ <optional> <ref name='vcpu'/> </optional> + <optional> + <ref name='iothreads'/> + </optional> <optional> <ref name='os'/> </optional> @@ -53,6 +56,13 @@ </element> </define> + <define name='iothreads'> + <element name='iothreads'> + <ref name='supported'/> + <empty/> + </element> + </define> + <define name='loader'> <element name='loader'> <ref name='supported'/> diff --git a/src/conf/domain_capabilities.c b/src/conf/domain_capabilities.c index bebbaf44d00e..f18bea99d63f 100644 --- a/src/conf/domain_capabilities.c +++ b/src/conf/domain_capabilities.c @@ -562,6 +562,9 @@ virDomainCapsFormat(virDomainCapsPtr const caps) if (caps->maxvcpus) virBufferAsprintf(&buf, "<vcpu max='%d'/>\n", caps->maxvcpus); + virBufferAsprintf(&buf, "<iothreads supported='%s'/>\n", + caps->iothreads ? "yes" : "no"); + virDomainCapsOSFormat(&buf, &caps->os); virDomainCapsCPUFormat(&buf, &caps->cpu); diff --git a/src/conf/domain_capabilities.h b/src/conf/domain_capabilities.h index fa4c1e442f57..3b5ce214d636 100644 --- a/src/conf/domain_capabilities.h +++ b/src/conf/domain_capabilities.h @@ -147,6 +147,7 @@ struct _virDomainCaps { /* Some machine specific info */ int maxvcpus; + bool iothreads; /* Whether I/O threads are supported or not. */ virDomainCapsOS os; virDomainCapsCPU cpu; diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 833c75514c25..00634abb1649 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -4632,6 +4632,16 @@ virQEMUCapsFillDomainCPUCaps(virCapsPtr caps, } +static int +virQEMUCapsFillDomainIOThreadCaps(virQEMUCapsPtr qemuCaps, + virDomainCapsPtr domCaps) +{ + domCaps->iothreads = virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_IOTHREAD); + + return 0; +} + + static int virQEMUCapsFillDomainDeviceDiskCaps(virQEMUCapsPtr qemuCaps, const char *machine, @@ -4866,6 +4876,7 @@ virQEMUCapsFillDomainCaps(virCapsPtr caps, if (virQEMUCapsFillDomainOSCaps(os, firmwares, nfirmwares) < 0 || virQEMUCapsFillDomainCPUCaps(caps, qemuCaps, domCaps) < 0 || + virQEMUCapsFillDomainIOThreadCaps(qemuCaps, domCaps) < 0 || virQEMUCapsFillDomainDeviceDiskCaps(qemuCaps, domCaps->machine, disk) < 0 || virQEMUCapsFillDomainDeviceGraphicsCaps(qemuCaps, graphics) < 0 || diff --git a/tests/domaincapsschemadata/basic.xml b/tests/domaincapsschemadata/basic.xml index 6b788d9144c8..7f9582430a7a 100644 --- a/tests/domaincapsschemadata/basic.xml +++ b/tests/domaincapsschemadata/basic.xml @@ -3,6 +3,7 @@ <domain>uml</domain> <machine>my-machine-type</machine> <arch>x86_64</arch> + <iothreads supported='no'/> <os supported='no'/> <cpu> <mode name='host-passthrough' supported='no'/> diff --git a/tests/domaincapsschemadata/full.xml b/tests/domaincapsschemadata/full.xml index ab6ef9f2ef17..b97bc883262a 100644 --- a/tests/domaincapsschemadata/full.xml +++ b/tests/domaincapsschemadata/full.xml @@ -4,6 +4,7 @@ <machine>my-machine-type</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='no'/> <os supported='yes'> <loader supported='yes'> <value>/foo/bar</value> diff --git a/tests/domaincapsschemadata/qemu_1.7.0.x86_64.xml b/tests/domaincapsschemadata/qemu_1.7.0.x86_64.xml index 8d1ad865703e..d71d948477ef 100644 --- a/tests/domaincapsschemadata/qemu_1.7.0.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_1.7.0.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-i440fx-1.7</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='no'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.12.0-virt.aarch64.xml b/tests/domaincapsschemadata/qemu_2.12.0-virt.aarch64.xml index 9cba942fbf8e..9feceeea74c6 100644 --- a/tests/domaincapsschemadata/qemu_2.12.0-virt.aarch64.xml +++ b/tests/domaincapsschemadata/qemu_2.12.0-virt.aarch64.xml @@ -4,6 +4,7 @@ <machine>virt-2.12</machine> <arch>aarch64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.12.0.ppc64.xml b/tests/domaincapsschemadata/qemu_2.12.0.ppc64.xml index 4029e9e386a9..62f2d4f78a6e 100644 --- a/tests/domaincapsschemadata/qemu_2.12.0.ppc64.xml +++ b/tests/domaincapsschemadata/qemu_2.12.0.ppc64.xml @@ -4,6 +4,7 @@ <machine>pseries-2.12</machine> <arch>ppc64</arch> <vcpu max='1024'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.12.0.s390x.xml b/tests/domaincapsschemadata/qemu_2.12.0.s390x.xml index 50680c1ef8d2..be505ee331a4 100644 --- a/tests/domaincapsschemadata/qemu_2.12.0.s390x.xml +++ b/tests/domaincapsschemadata/qemu_2.12.0.s390x.xml @@ -4,6 +4,7 @@ <machine>s390-ccw-virtio-2.12</machine> <arch>s390x</arch> <vcpu max='248'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.12.0.x86_64.xml b/tests/domaincapsschemadata/qemu_2.12.0.x86_64.xml index 851d914bc9c6..84a83551ad42 100644 --- a/tests/domaincapsschemadata/qemu_2.12.0.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_2.12.0.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-i440fx-2.12</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.6.0-virt.aarch64.xml b/tests/domaincapsschemadata/qemu_2.6.0-virt.aarch64.xml index 7c019b230868..b3dc0dd8004b 100644 --- a/tests/domaincapsschemadata/qemu_2.6.0-virt.aarch64.xml +++ b/tests/domaincapsschemadata/qemu_2.6.0-virt.aarch64.xml @@ -4,6 +4,7 @@ <machine>virt-2.6</machine> <arch>aarch64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.6.0.aarch64.xml b/tests/domaincapsschemadata/qemu_2.6.0.aarch64.xml index 3b142806217b..13bb19198d38 100644 --- a/tests/domaincapsschemadata/qemu_2.6.0.aarch64.xml +++ b/tests/domaincapsschemadata/qemu_2.6.0.aarch64.xml @@ -4,6 +4,7 @@ <machine>integratorcp</machine> <arch>aarch64</arch> <vcpu max='1'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.6.0.ppc64.xml b/tests/domaincapsschemadata/qemu_2.6.0.ppc64.xml index 98c72d36de04..d9a243d9d573 100644 --- a/tests/domaincapsschemadata/qemu_2.6.0.ppc64.xml +++ b/tests/domaincapsschemadata/qemu_2.6.0.ppc64.xml @@ -4,6 +4,7 @@ <machine>pseries-2.6</machine> <arch>ppc64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.6.0.x86_64.xml b/tests/domaincapsschemadata/qemu_2.6.0.x86_64.xml index de81886237ee..7eabf6756b89 100644 --- a/tests/domaincapsschemadata/qemu_2.6.0.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_2.6.0.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-i440fx-2.6</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.7.0.s390x.xml b/tests/domaincapsschemadata/qemu_2.7.0.s390x.xml index b93d00ece197..c2045263530f 100644 --- a/tests/domaincapsschemadata/qemu_2.7.0.s390x.xml +++ b/tests/domaincapsschemadata/qemu_2.7.0.s390x.xml @@ -4,6 +4,7 @@ <machine>s390-ccw-virtio-2.7</machine> <arch>s390x</arch> <vcpu max='248'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.8.0-tcg.x86_64.xml b/tests/domaincapsschemadata/qemu_2.8.0-tcg.x86_64.xml index d860cd833fed..93e0a610f6ce 100644 --- a/tests/domaincapsschemadata/qemu_2.8.0-tcg.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_2.8.0-tcg.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-i440fx-2.8</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.8.0.s390x.xml b/tests/domaincapsschemadata/qemu_2.8.0.s390x.xml index ee40d1e84ec3..6555e99c68a3 100644 --- a/tests/domaincapsschemadata/qemu_2.8.0.s390x.xml +++ b/tests/domaincapsschemadata/qemu_2.8.0.s390x.xml @@ -4,6 +4,7 @@ <machine>s390-ccw-virtio-2.8</machine> <arch>s390x</arch> <vcpu max='248'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.8.0.x86_64.xml b/tests/domaincapsschemadata/qemu_2.8.0.x86_64.xml index 33161f7b8234..2a6edd9a0f80 100644 --- a/tests/domaincapsschemadata/qemu_2.8.0.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_2.8.0.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-i440fx-2.8</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.9.0-q35.x86_64.xml b/tests/domaincapsschemadata/qemu_2.9.0-q35.x86_64.xml index 07e1db641a76..aef8a90f3930 100644 --- a/tests/domaincapsschemadata/qemu_2.9.0-q35.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_2.9.0-q35.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-q35-2.9</machine> <arch>x86_64</arch> <vcpu max='288'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.9.0-tcg.x86_64.xml b/tests/domaincapsschemadata/qemu_2.9.0-tcg.x86_64.xml index 6c2bbac190c7..e4afdeedc4e6 100644 --- a/tests/domaincapsschemadata/qemu_2.9.0-tcg.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_2.9.0-tcg.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-i440fx-2.9</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> diff --git a/tests/domaincapsschemadata/qemu_2.9.0.x86_64.xml b/tests/domaincapsschemadata/qemu_2.9.0.x86_64.xml index 2fc3b721389d..8f0f4fb7369b 100644 --- a/tests/domaincapsschemadata/qemu_2.9.0.x86_64.xml +++ b/tests/domaincapsschemadata/qemu_2.9.0.x86_64.xml @@ -4,6 +4,7 @@ <machine>pc-i440fx-2.9</machine> <arch>x86_64</arch> <vcpu max='255'/> + <iothreads supported='yes'/> <os supported='yes'> <loader supported='yes'> <value>/usr/share/AAVMF/AAVMF_CODE.fd</value> -- 2.17.0

2 1