- Devel - libvirt List Archives

[PATCH 00/30] Support more 'curl' and 'ssh' driver options (mostly for unbreaking libguestfs)
by Peter Krempa 13 Mar '20

13 Mar '20

With blockdev we need to interpret the fields in order to actually use them in qemu. Unfortunately we didn't do that for some and libguestfs was using them in backing store strings. Model all known curl driver options and pass-through ssh driver options as we don't model the ssh driver. Changes can be fetched from: git fetch https://gitlab.com/pipo.sk/libvirt.git json-props https://gitlab.com/pipo.sk/libvirt/-/commits/json-props Peter Krempa (30): qemuBlockStorageSourceDetachPrepare: Get rid of cleanup section qemu: Don't take double pointer in qemuDomainSecretInfoFree qemuMigrationParamsResetTLS: Adapt to modern memory management qemuMigrationParamsResetTLS: Fix comment qemu: domain: Split out encryption of secret object data qemu: Introduce another helper for creating alias for a 'secret' object qemuDomainSecretStorageSourcePrepare: Fix naming of alias variables qemuDomainDeviceDiskDefPostParseRestoreSecAlias: Hardcode restored aliases qemu: Split out initialization of secrets for 'iscsi' hostdevs qemuDomainSecretAESSetupFromSecret: Use 'qemuAliasForSecret' qemuDomainSecretStorageSourcePrepare: Change aliases for disk secrets qemuDomainGetSecretAESAlias: Replace outstanding uses with qemuAliasForSecret conf: Add support for modifying ssl validation for https/ftps disks conf: Add support for cookies for HTTP based disks conf: Add support for setting timeout and readahead size for network disks qemuDomainValidateStorageSource: Validate new network storage parameters qemuxml2argvtest: Add test case for disks with http(s) source qemu: block: Implement ssl verification configuration qemu: domain: Store data for 'secret' object representing http cookies qemuDomainSecretStorageSourcePrepare: Setup secret for http cookies qemu: Handle hotplug and commandline for secret objects for http cookies qemu: block: Add support for HTTP cookies qemu: block: Implement readahead and timeout properties for 'curl' driver virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout' virStorageSourceParseBackingJSONUri: Handle undocumented value 'off' for sslverify qemublocktest: Load QMP schema earlier qemublocktest: Extract schema root for blockdev-add validation qemublocktest: XMLjsonXML: Test formatting/parsing of modern JSON qemublocktest: Add JSON->JSON test cases for block device backends qemu: Pass through arguments of 'ssh' block driver used by libguestfs docs/formatdomain.html.in | 35 ++ docs/schemas/domaincommon.rng | 98 ++++- src/conf/domain_conf.c | 119 +++++++ src/libvirt_private.syms | 1 + src/qemu/qemu_alias.c | 32 +- src/qemu/qemu_alias.h | 4 +- src/qemu/qemu_block.c | 50 ++- src/qemu/qemu_block.h | 3 + src/qemu/qemu_command.c | 5 + src/qemu/qemu_domain.c | 337 +++++++++++------- src/qemu/qemu_domain.h | 7 +- src/qemu/qemu_hotplug.c | 4 +- src/qemu/qemu_migration_params.c | 16 +- src/util/virstoragefile.c | 229 +++++++++++- src/util/virstoragefile.h | 24 ++ .../disk-network-http.xml | 19 + tests/qemublocktest.c | 136 ++++++- .../jsontojson/curl-libguestfs-in.json | 1 + .../jsontojson/curl-libguestfs-out.json | 9 + .../ssh-passthrough-libguestfs-in.json | 1 + .../ssh-passthrough-libguestfs-out.json | 14 + tests/qemustatusxml2xmldata/modern-in.xml | 1 + ...-backing-chains-noindex.x86_64-2.12.0.args | 4 +- ...-backing-chains-noindex.x86_64-latest.args | 6 +- ...sk-hostdev-scsi-virtio-iscsi-auth-AES.args | 6 +- .../disk-network-http.x86_64-latest.args | 67 ++++ tests/qemuxml2argvdata/disk-network-http.xml | 61 ++++ .../disk-network-iscsi.x86_64-2.12.0.args | 12 +- .../disk-network-iscsi.x86_64-latest.args | 8 +- .../disk-network-rbd.x86_64-2.12.0.args | 4 +- .../disk-network-rbd.x86_64-latest.args | 4 +- ...isk-network-source-auth.x86_64-2.12.0.args | 10 +- ...isk-network-source-auth.x86_64-latest.args | 8 +- .../disk-nvme.x86_64-latest.args | 4 +- .../encrypted-disk-usage.args | 4 +- tests/qemuxml2argvdata/encrypted-disk.args | 4 +- .../luks-disks-source-qcow2.args | 24 +- ...luks-disks-source-qcow2.x86_64-latest.args | 32 +- tests/qemuxml2argvdata/luks-disks-source.args | 26 +- tests/qemuxml2argvdata/luks-disks.args | 10 +- tests/qemuxml2argvdata/user-aliases.args | 4 +- tests/qemuxml2argvtest.c | 1 + tests/virstoragetest.c | 30 ++ 43 files changed, 1197 insertions(+), 277 deletions(-) create mode 100644 tests/qemublocktestdata/jsontojson/curl-libguestfs-in.json create mode 100644 tests/qemublocktestdata/jsontojson/curl-libguestfs-out.json create mode 100644 tests/qemublocktestdata/jsontojson/ssh-passthrough-libguestfs-in.json create mode 100644 tests/qemublocktestdata/jsontojson/ssh-passthrough-libguestfs-out.json create mode 100644 tests/qemuxml2argvdata/disk-network-http.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-http.xml -- 2.24.1

3 61

[PATCH] virQEMUCaps: Drop unused usedQMP member
by Michal Privoznik 12 Mar '20

12 Mar '20

The virQEMUCaps structure has usedQMP member which in the past used to tell if qemu we are dealing with is capable of QMP. Well, we don't support HMP anymore (minus a few HMP passthrough commands, which are wrapped into QMP anyways) and the member is not used really. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_capabilities.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index a75ca0574d..1395549b2b 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -638,7 +638,6 @@ static void virQEMUDomainCapsCacheDispose(void *obj) struct _virQEMUCaps { virObject parent; - bool usedQMP; bool kvmSupportsNesting; char *binary; @@ -1825,7 +1824,6 @@ virQEMUCapsPtr virQEMUCapsNewCopy(virQEMUCapsPtr qemuCaps) return NULL; ret->invalidation = qemuCaps->invalidation; - ret->usedQMP = qemuCaps->usedQMP; ret->kvmSupportsNesting = qemuCaps->kvmSupportsNesting; ret->ctime = qemuCaps->ctime; @@ -4980,7 +4978,6 @@ virQEMUCapsInitQMPMonitor(virQEMUCapsPtr qemuCaps, qemuCaps->version = major * 1000000 + minor * 1000 + micro; qemuCaps->package = g_steal_pointer(&package); - qemuCaps->usedQMP = true; if (virQEMUCapsInitQMPArch(qemuCaps, mon) < 0) return -1; -- 2.24.1

2 1

[RFCv2] qemu: convert DomainLogContext class to use GObject
by Gaurav Agrawal 12 Mar '20

12 Mar '20

--- src/qemu/qemu_domain.c | 35 +++++++++++++++++++---------------- src/qemu/qemu_domain.h | 6 +++--- src/qemu/qemu_process.c | 4 ++-- 3 files changed, 24 insertions(+), 21 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 4b467afa81..b3f17b8382 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -151,7 +151,7 @@ qemuDomainObjFromDomain(virDomainPtr domain) struct _qemuDomainLogContext { - virObject parent; + GObject parent; int writefd; int readfd; /* Only used if manager == NULL */ @@ -161,37 +161,46 @@ struct _qemuDomainLogContext { virLogManagerPtr manager; }; -static virClassPtr qemuDomainLogContextClass; +G_DEFINE_TYPE(qemuDomainLogContext, qemu_domain_log_context, G_TYPE_OBJECT); static virClassPtr qemuDomainSaveCookieClass; -static void qemuDomainLogContextDispose(void *obj); +static void qemuDomainLogContextFinalize(GObject *obj); static void qemuDomainSaveCookieDispose(void *obj); static int qemuDomainOnceInit(void) { - if (!VIR_CLASS_NEW(qemuDomainLogContext, virClassForObject())) - return -1; - if (!VIR_CLASS_NEW(qemuDomainSaveCookie, virClassForObject())) return -1; return 0; } +static void qemu_domain_log_context_init(qemuDomainLogContext *logctxt G_GNUC_UNUSED) +{ +} + +static void qemu_domain_log_context_class_init(qemuDomainLogContextClass *klass) +{ + GObjectClass *obj = G_OBJECT_CLASS(klass); + + obj->finalize = qemuDomainLogContextFinalize; +} + VIR_ONCE_GLOBAL_INIT(qemuDomain); static void -qemuDomainLogContextDispose(void *obj) +qemuDomainLogContextFinalize(GObject *object) { - qemuDomainLogContextPtr ctxt = obj; + qemuDomainLogContextPtr ctxt = QEMU_DOMAIN_LOG_CONTEXT(object); VIR_DEBUG("ctxt=%p", ctxt); virLogManagerFree(ctxt->manager); VIR_FREE(ctxt->path); VIR_FORCE_CLOSE(ctxt->writefd); VIR_FORCE_CLOSE(ctxt->readfd); + G_OBJECT_CLASS(qemu_domain_log_context_parent_class)->finalize(object); } const char * @@ -10591,13 +10600,7 @@ qemuDomainLogContextPtr qemuDomainLogContextNew(virQEMUDriverPtr driver, qemuDomainLogContextMode mode) { g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); - qemuDomainLogContextPtr ctxt = NULL; - - if (qemuDomainInitialize() < 0) - return NULL; - - if (!(ctxt = virObjectNew(qemuDomainLogContextClass))) - return NULL; + qemuDomainLogContextPtr ctxt = QEMU_DOMAIN_LOG_CONTEXT(g_object_new(QEMU_TYPE_DOMAIN_LOG_CONTEXT, NULL)); VIR_DEBUG("Context new %p stdioLogD=%d", ctxt, cfg->stdioLogD); ctxt->writefd = -1; @@ -10666,7 +10669,7 @@ qemuDomainLogContextPtr qemuDomainLogContextNew(virQEMUDriverPtr driver, return ctxt; error: - virObjectUnref(ctxt); + g_object_unref(ctxt); return NULL; } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 099ee59772..9e2cec5779 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -20,7 +20,7 @@ */ #pragma once - +#include <glib-object.h> #include "virthread.h" #include "vircgroup.h" #include "virperf.h" @@ -601,9 +601,9 @@ struct qemuProcessEvent { void qemuProcessEventFree(struct qemuProcessEvent *event); -typedef struct _qemuDomainLogContext qemuDomainLogContext; +#define QEMU_TYPE_DOMAIN_LOG_CONTEXT qemu_domain_log_context_get_type() +G_DECLARE_FINAL_TYPE(qemuDomainLogContext, qemu_domain_log_context, QEMU, DOMAIN_LOG_CONTEXT, GObject); typedef qemuDomainLogContext *qemuDomainLogContextPtr; -G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuDomainLogContext, virObjectUnref); typedef struct _qemuDomainSaveCookie qemuDomainSaveCookie; typedef qemuDomainSaveCookie *qemuDomainSaveCookiePtr; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 67dad9841a..e7f5991b23 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1929,7 +1929,7 @@ static void qemuProcessMonitorLogFree(void *opaque) { qemuDomainLogContextPtr logCtxt = opaque; - virObjectUnref(logCtxt); + g_clear_object(&logCtxt); } @@ -1983,7 +1983,7 @@ qemuConnectMonitor(virQEMUDriverPtr driver, virDomainObjPtr vm, int asyncJob, driver); if (mon && logCtxt) { - virObjectRef(logCtxt); + g_object_ref(logCtxt); qemuMonitorSetDomainLog(mon, qemuProcessMonitorReportLogError, logCtxt, -- 2.24.1

1 0

[PATCH 0/2] qemu: agent: sync once if qemu has serial port event
by Nikolay Shirokovskiy 12 Mar '20

12 Mar '20

The second patch is unrelated through. Nikolay Shirokovskiy (2): qemu: agent: sync once if qemu has serial port event qemu: remove redundant needReply argument of qemuAgentCommand src/qemu/qemu_agent.c | 60 ++++++++++++++++++++---------------- src/qemu/qemu_agent.h | 3 +- src/qemu/qemu_process.c | 3 +- tests/qemumonitortestutils.c | 3 +- 4 files changed, 40 insertions(+), 29 deletions(-) -- 2.23.0

2 7

Re: [PATCH V4 0/5] Introduce Advanced Watch Dog module
by Zhang, Chen 12 Mar '20

12 Mar '20

On 2/12/2020 10:56 AM, Jason Wang wrote: > On 2020/2/11 下午4:58, Zhang, Chen wrote: >>> -----Original Message----- >>> From: Jason Wang<jasowang(a)redhat.com> >>> Sent: Monday, January 20, 2020 10:57 AM >>> To: Zhang, Chen<chen.zhang(a)intel.com>; Paolo Bonzini >>> <pbonzini(a)redhat.com>; Philippe Mathieu-Daudé<philmd(a)redhat.com>; >>> qemu-dev<qemu-devel(a)nongnu.org> >>> Cc: Zhang Chen<zhangckid(a)gmail.com> >>> Subject: Re: [PATCH V4 0/5] Introduce Advanced Watch Dog module >>> >>> >>> On 2020/1/19 下午5:10, Zhang, Chen wrote: >>>> Hi~ >>>> >>>> Anyone have comments about this module? >>> Hi Chen: >>> >>> I will take a look at this series. >> Sorry for slow reply due to CNY and extend leave. >> OK, waiting your comments~ Thanks~ >> >>> Two general questions: >>> >>> - if it can detect more than network stall, it should not belong to /net >> This module use network connection status to detect all the issue(Host to Guest/Host to Host/Host to Admin...). >> The target is more than network but all use network way. So it is looks a tricky problem. > > Ok. > > >>> - need to convince libvirt guys for this proposal, since usually it's the duty of >>> upper layer instead of qemu itself >>> >> Yes, It looks a upper layer responsibility, but In the cover latter I have explained the reason why we need this in Qemu. >> try to make this module as simple as possible. This module give upper layer software a new way to connect/monitoring Qemu. >> And due to all the COLO code implement in Qemu side, Many customer want to use this FT solution without other dependencies, >> it is very easy to integrated to real product. >> >> Thanks >> Zhang Chen > > I would like to hear from libvirt about such design. Hi Jason, OK. I add the libvirt mailing list in this thread. The full mail discussion and patches: https://lists.nongnu.org/archive/html/qemu-devel/2020-02/msg02611.html By the way, I noticed Eric is libvirt maintianer. Hi Eric and Paolo, Can you give some comments about this series? Thanks Zhang Chen > > Thanks >

2 4

Re: [PATCHv2 1/3] Share Dup Daemon Function *SetupLogging
by LAN BAI 12 Mar '20

12 Mar '20

On Mar 11, 2020 9:30 AM, Ján Tomko <jtomko(a)redhat.com> wrote: On a Sunday in 2020, Lan wrote: >One of the BiteSizedTasks > >Introduce src/util/virdaemon.c/h files > >Introduce a new function virDaemonSetupLogging (src/util/virdaemon.c) >for shared code in >virLockDaemonSetupLogging (src/locking/lock_daemon.c) >virLogDaemonSetupLogging (src/logging/log_daemon.c) >daemonSetupLogging (src/remote/remote_daemon.c) > >Introduce virDaemonLogConfig struct for config->log_* variables in >virLockDaemonConfig/virLogDaemonConfig/daemonConfig struct > >Signed-off-by: Lan Bai <lbai(a)wisc.edu> >--- > src/libvirt_private.syms | 2 + > src/locking/lock_daemon.c | 58 ++--------------------------- > src/logging/log_daemon.c | 50 ++----------------------- > src/remote/remote_daemon.c | 57 ++--------------------------- > src/util/Makefile.inc.am | 4 +- > src/util/virdaemon.c | 75 ++++++++++++++++++++++++++++++++++++++ > src/util/virdaemon.h | 35 ++++++++++++++++++ > 7 files changed, 126 insertions(+), 155 deletions(-) > create mode 100644 src/util/virdaemon.c > create mode 100644 src/util/virdaemon.h > >diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms >index de0c7a3133..50cbd6d7af 100644 >--- a/src/libvirt_private.syms >+++ b/src/libvirt_private.syms >@@ -1906,6 +1906,8 @@ virCryptoHashBuf; > virCryptoHashString; > virCryptoHaveCipher; > >+# util/virdaemon.h >+virDaemonSetupLogging; > > # util/virdbus.h > virDBusCallMethod; >diff --git a/src/locking/lock_daemon.c b/src/locking/lock_daemon.c >index 5e5a0c1089..5ba851cb55 100644 >--- a/src/locking/lock_daemon.c >+++ b/src/locking/lock_daemon.c >@@ -46,6 +46,7 @@ > #include "virstring.h" > #include "virgettext.h" > #include "virenum.h" >+#include "virdaemon.h" > > #include "locking/lock_daemon_dispatch.h" > #include "locking/lock_protocol.h" >@@ -477,59 +478,6 @@ virLockDaemonErrorHandler(void *opaque G_GNUC_UNUSED, > } > > >-/* >- * Set up the logging environment >- * By default if daemonized all errors go to the logfile libvirtd.log, >- * but if verbose or error debugging is asked for then also output >- * informational and debug messages. Default size if 64 kB. >- */ >-static int >-virLockDaemonSetupLogging(virLockDaemonConfigPtr config, >- bool privileged, >- bool verbose, >- bool godaemon) >-{ >- virLogReset(); >- >- /* >- * Libvirtd's order of precedence is: >- * cmdline > environment > config >- * >- * Given the precedence, we must process the variables in the opposite >- * order, each one overriding the previous. >- */ >- if (config->log_level != 0) >- virLogSetDefaultPriority(config->log_level); >- >- /* In case the config is empty, both filters and outputs will become empty, >- * however we can't start with empty outputs, thus we'll need to define and >- * setup a default one. >- */ >- ignore_value(virLogSetFilters(config->log_filters)); >- ignore_value(virLogSetOutputs(config->log_outputs)); >- >- /* If there are some environment variables defined, use those instead */ >- virLogSetFromEnv(); >- >- /* >- * Command line override for --verbose >- */ >- if ((verbose) && (virLogGetDefaultPriority() > VIR_LOG_INFO)) >- virLogSetDefaultPriority(VIR_LOG_INFO); >- >- /* Define the default output. This is only applied if there was no setting >- * from either the config or the environment. >- */ >- virLogSetDefaultOutput("virtlockd", godaemon, privileged); >- >- if (virLogGetNbOutputs() == 0) >- virLogSetOutputs(virLogGetDefaultOutput()); >- >- return 0; >-} >- >- >- > /* Display version information. */ > static void > virLockDaemonVersion(const char *argv0) >@@ -1186,7 +1134,9 @@ int main(int argc, char **argv) { > } > VIR_FREE(remote_config_file); > >- if (virLockDaemonSetupLogging(config, privileged, verbose, godaemon) < 0) { >+ if (virDaemonSetupLogging((virDaemonLogConfigPtr)(&(config->log_level)), This still does not compile for me: ../../src/locking/lock_daemon.c:1131:31: error: cast from 'unsigned int *' to 'virDaemonLogConfigPtr' (aka 'struct _virDaemonLogConfig *') increases required alignment from 4 to 8 [-Werror,-Wcast-align] if (virDaemonSetupLogging((virDaemonLogConfigPtr)(&(config->log_level)), ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is modified in following commits. If you apply the whole patch it shouldn't be a problem. The first step along with introducing the virDaemonLogConfig structure is moving the three log_parameters into this structure. So struct _virLockDaemonConfig { unsigned int log_level; char *log_filters; char *log_outputs; unsigned int max_clients; unsigned int admin_max_clients; }; would become something like struct _virLockDaemonConfig { virDaemonLogConfig log_config; unsigned int max_clients; unsigned int admin_max_clients; }; And a function like: virDaemonLogConfigLoadOptions(virDaemonLogConfigPtr log_config, virConfPtr conf) could be used to replace the three lines in every daemon's config loader. How do you solve the problem of different virConfPtr type. I have an initializer for virDaemonLogConfig in following commits. Jano

2 1

[PATCHv2 0/5] update tls files without restarting libvirtd
by Zhang Bo 12 Mar '20

12 Mar '20

v1: https://www.redhat.com/archives/libvir-list/2020-February/msg00370.html v2: according to Dienial's suggestion: * update each tls file one time -> update all of them at one time * forced to re-create the credentials object, rather than allowing to append to the original ones. Zhang Bo (5): virnetserver: Introduce virNetServerUpdateTlsFiles tls: Add a mutex lock on 'tlsCtxt' admin: Introduce virAdmServerUpdateTlsFiles virt-admin: Introduce command srv-update-tls docs: update virt-admin.rst for server-update-tls docs/manpages/virt-admin.rst | 16 +++++++ include/libvirt/libvirt-admin.h | 3 ++ src/admin/admin_protocol.x | 12 +++++- src/admin/admin_server.c | 9 ++++ src/admin/admin_server.h | 3 ++ src/admin/libvirt-admin.c | 30 +++++++++++++ src/admin/libvirt_admin_private.syms | 1 + src/admin/libvirt_admin_public.syms | 1 + src/libvirt_remote.syms | 1 + src/rpc/virnetserver.c | 51 ++++++++++++++++++++++ src/rpc/virnetserver.h | 2 + src/rpc/virnetserverclient.c | 4 ++ src/rpc/virnettlscontext.c | 46 ++++++++++++++++++++ src/rpc/virnettlscontext.h | 3 ++ tools/virt-admin.c | 64 ++++++++++++++++++++++++++++ 15 files changed, 245 insertions(+), 1 deletion(-) -- 2.23.0.windows.1

3 9

[PATCH] conf: qemu: add support for io_uring
by Zhenyu Ye 12 Mar '20

12 Mar '20

QEMU has added support for io_uring IO mode, see: https://git.qemu.org/git/qemu.git/ adcd6e93. This patch add support for io_uring in libvirt. Signed-off-by: Zhenyu Ye <yezhenyu2(a)huawei.com> --- src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + 2 files changed, 2 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index d2d97daf80..5ced2f3b6b 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -361,6 +361,7 @@ VIR_ENUM_IMPL(virDomainDiskIo, "default", "native", "threads", + "io_uring", ); VIR_ENUM_IMPL(virDomainDeviceSGIO, diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 91b776c28a..905fdecab3 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -421,6 +421,7 @@ typedef enum { VIR_DOMAIN_DISK_IO_DEFAULT = 0, VIR_DOMAIN_DISK_IO_NATIVE, VIR_DOMAIN_DISK_IO_THREADS, + VIR_DOMAIN_DISK_IO_URING, VIR_DOMAIN_DISK_IO_LAST } virDomainDiskIo; -- 2.19.1

2 1

[PATCH] qemu: Create multipath targets for PRs
by Michal Privoznik 12 Mar '20

12 Mar '20

If a disk has persistent reservations enabled, qemu-pr-helper might open not only /dev/mapper/control but also individual targets of the multipath device. We are already querying for them in CGroups, but now we have to create them in the namespace too. This was brought up in [1]. 1: https://bugzilla.redhat.com/show_bug.cgi?id=1711045#c61 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_domain.c | 64 ++++++++++------ src/util/virdevmapper.h | 4 +- src/util/virutil.h | 2 +- tests/qemuhotplugmock.c | 75 +++++++++++++++++++ tests/qemuhotplugtest.c | 13 ++++ .../qemuhotplug-disk-scsi-multipath.xml | 8 ++ ...uhotplug-base-live+disk-scsi-multipath.xml | 62 +++++++++++++++ 7 files changed, 204 insertions(+), 24 deletions(-) create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-disk-scsi-multipath.xml create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+disk-scsi-multipath.xml diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 33c2158eb5..c8e6be7fae 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -62,6 +62,7 @@ #include "virdomaincheckpointobjlist.h" #include "backup_conf.h" #include "virutil.h" +#include "virdevmapper.h" #ifdef __linux__ # include <sys/sysmacros.h> @@ -14574,6 +14575,9 @@ qemuDomainSetupDisk(virQEMUDriverConfigPtr cfg G_GNUC_UNUSED, bool hasNVMe = false; for (next = disk->src; virStorageSourceIsBacking(next); next = next->backingStore) { + VIR_AUTOSTRINGLIST targetPaths = NULL; + size_t i; + if (next->type == VIR_STORAGE_TYPE_NVME) { g_autofree char *nvmePath = NULL; @@ -14592,6 +14596,19 @@ qemuDomainSetupDisk(virQEMUDriverConfigPtr cfg G_GNUC_UNUSED, if (qemuDomainCreateDevice(next->path, data, false) < 0) return -1; + + if (virDevMapperGetTargets(next->path, &targetPaths) < 0 && + errno != ENOSYS && errno != EBADF) { + virReportSystemError(errno, + _("Unable to get devmapper targets for %s"), + next->path); + return -1; + } + + for (i = 0; targetPaths && targetPaths[i]; i++) { + if (qemuDomainCreateDevice(targetPaths[i], data, false) < 0) + return -1; + } } } @@ -15607,21 +15624,19 @@ qemuDomainNamespaceSetupDisk(virDomainObjPtr vm, virStorageSourcePtr src) { virStorageSourcePtr next; - char **paths = NULL; + VIR_AUTOSTRINGLIST paths = NULL; size_t npaths = 0; bool hasNVMe = false; - g_autofree char *dmPath = NULL; - g_autofree char *vfioPath = NULL; - int ret = -1; for (next = src; virStorageSourceIsBacking(next); next = next->backingStore) { + VIR_AUTOSTRINGLIST targetPaths = NULL; g_autofree char *tmpPath = NULL; if (next->type == VIR_STORAGE_TYPE_NVME) { hasNVMe = true; if (!(tmpPath = virPCIDeviceAddressGetIOMMUGroupDev(&next->nvme->pciAddr))) - goto cleanup; + return -1; } else { if (virStorageSourceIsEmpty(next) || !virStorageSourceIsLocalStorage(next)) { @@ -15632,30 +15647,35 @@ qemuDomainNamespaceSetupDisk(virDomainObjPtr vm, tmpPath = g_strdup(next->path); } - if (VIR_APPEND_ELEMENT(paths, npaths, tmpPath) < 0) - goto cleanup; + if (virStringListAdd(&paths, tmpPath) < 0) + return -1; + + if (virDevMapperGetTargets(next->path, &targetPaths) < 0 && + errno != ENOSYS && errno != EBADF) { + virReportSystemError(errno, + _("Unable to get devmapper targets for %s"), + next->path); + return -1; + } + + if (virStringListMerge(&paths, &targetPaths) < 0) + return -1; } /* qemu-pr-helper might require access to /dev/mapper/control. */ - if (src->pr) { - dmPath = g_strdup(QEMU_DEVICE_MAPPER_CONTROL_PATH); - if (VIR_APPEND_ELEMENT_COPY(paths, npaths, dmPath) < 0) - goto cleanup; - } + if (src->pr && + virStringListAdd(&paths, QEMU_DEVICE_MAPPER_CONTROL_PATH) < 0) + return -1; - if (hasNVMe) { - vfioPath = g_strdup(QEMU_DEV_VFIO); - if (VIR_APPEND_ELEMENT(paths, npaths, vfioPath) < 0) - goto cleanup; - } + if (hasNVMe && + virStringListAdd(&paths, QEMU_DEV_VFIO) < 0) + return -1; + npaths = virStringListLength((const char **) paths); if (qemuDomainNamespaceMknodPaths(vm, (const char **) paths, npaths) < 0) - goto cleanup; + return -1; - ret = 0; - cleanup: - virStringListFreeCount(paths, npaths); - return ret; + return 0; } diff --git a/src/util/virdevmapper.h b/src/util/virdevmapper.h index e576d2bf7e..87bbc63cfd 100644 --- a/src/util/virdevmapper.h +++ b/src/util/virdevmapper.h @@ -20,6 +20,8 @@ #pragma once +#include "internal.h" + int virDevMapperGetTargets(const char *path, - char ***devPaths); + char ***devPaths) G_GNUC_NO_INLINE; diff --git a/src/util/virutil.h b/src/util/virutil.h index cc6b82a177..ee23f0c1f4 100644 --- a/src/util/virutil.h +++ b/src/util/virutil.h @@ -120,7 +120,7 @@ bool virValidateWWN(const char *wwn); int virGetDeviceID(const char *path, int *maj, - int *min); + int *min) G_GNUC_NO_INLINE; int virSetDeviceUnprivSGIO(const char *path, const char *sysfs_dir, int unpriv_sgio); diff --git a/tests/qemuhotplugmock.c b/tests/qemuhotplugmock.c index 43a9d79051..8e5b07788d 100644 --- a/tests/qemuhotplugmock.c +++ b/tests/qemuhotplugmock.c @@ -19,7 +19,24 @@ #include <config.h> #include "qemu/qemu_hotplug.h" +#include "qemu/qemu_process.h" #include "conf/domain_conf.h" +#include "virdevmapper.h" +#include "virutil.h" +#include "virmock.h" + +static int (*real_virGetDeviceID)(const char *path, int *maj, int *min); +static bool (*real_virFileExists)(const char *path); + +static void +init_syms(void) +{ + if (real_virFileExists) + return; + + VIR_MOCK_REAL_INIT(virGetDeviceID); + VIR_MOCK_REAL_INIT(virFileExists); +} unsigned long long qemuDomainGetUnplugTimeout(virDomainObjPtr vm G_GNUC_UNUSED) @@ -31,3 +48,61 @@ qemuDomainGetUnplugTimeout(virDomainObjPtr vm G_GNUC_UNUSED) return 200; return 100; } + + +int +virDevMapperGetTargets(const char *path, + char ***devPaths) +{ + *devPaths = NULL; + + if (STREQ(path, "/dev/mapper/virt")) { + *devPaths = g_new(char *, 4); + (*devPaths)[0] = g_strdup("/dev/block/8:0"); /* /dev/sda */ + (*devPaths)[1] = g_strdup("/dev/block/8:16"); /* /dev/sdb */ + (*devPaths)[2] = g_strdup("/dev/block/8:32"); /* /dev/sdc */ + (*devPaths)[3] = NULL; + } + + return 0; +} + + +int +virGetDeviceID(const char *path, int *maj, int *min) +{ + init_syms(); + + if (STREQ(path, "/dev/mapper/virt")) { + *maj = 254; + *min = 0; + return 0; + } + + return real_virGetDeviceID(path, maj, min); +} + + +bool +virFileExists(const char *path) +{ + init_syms(); + + if (STREQ(path, "/dev/mapper/virt")) + return true; + + return real_virFileExists(path); +} + + +int +qemuProcessStartManagedPRDaemon(virDomainObjPtr vm G_GNUC_UNUSED) +{ + return 0; +} + + +void +qemuProcessKillManagedPRDaemon(virDomainObjPtr vm G_GNUC_UNUSED) +{ +} diff --git a/tests/qemuhotplugtest.c b/tests/qemuhotplugtest.c index e008c1bf0d..8b411d63f0 100644 --- a/tests/qemuhotplugtest.c +++ b/tests/qemuhotplugtest.c @@ -87,6 +87,8 @@ qemuHotplugCreateObjects(virDomainXMLOptionPtr xmlopt, virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_VNC); virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_SPICE); virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_SPICE_FILE_XFER_DISABLE); + virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_PR_MANAGER_HELPER); + virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_SCSI_BLOCK); if (qemuTestCapsCacheInsert(driver.qemuCapsCache, priv->qemuCaps) < 0) return -1; @@ -750,6 +752,17 @@ mymain(void) "device_del", QMP_DEVICE_DELETED("scsi3-0-5-6") QMP_OK, "human-monitor-command", HMP("")); + DO_TEST_ATTACH("base-live", "disk-scsi-multipath", false, true, + "object-add", QMP_OK, + "human-monitor-command", HMP("OK\\r\\n"), + "device_add", QMP_OK); + DO_TEST_DETACH("base-live", "disk-scsi-multipath", true, true, + "device_del", QMP_OK, + "human-monitor-command", HMP("")); + DO_TEST_DETACH("base-live", "disk-scsi-multipath", false, false, + "device_del", QMP_DEVICE_DELETED("scsi0-0-0-0") QMP_OK, + "human-monitor-command", HMP("")); + DO_TEST_ATTACH("base-live", "qemu-agent", false, true, "chardev-add", QMP_OK, "device_add", QMP_OK); diff --git a/tests/qemuhotplugtestdevices/qemuhotplug-disk-scsi-multipath.xml b/tests/qemuhotplugtestdevices/qemuhotplug-disk-scsi-multipath.xml new file mode 100644 index 0000000000..5a6f955284 --- /dev/null +++ b/tests/qemuhotplugtestdevices/qemuhotplug-disk-scsi-multipath.xml @@ -0,0 +1,8 @@ +<disk type='block' device='lun'> + <driver name='qemu' type='raw'/> + <source dev='/dev/mapper/virt'> + <reservations managed='yes'/> + </source> + <target dev='sda' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> +</disk> diff --git a/tests/qemuhotplugtestdomains/qemuhotplug-base-live+disk-scsi-multipath.xml b/tests/qemuhotplugtestdomains/qemuhotplug-base-live+disk-scsi-multipath.xml new file mode 100644 index 0000000000..40af064d10 --- /dev/null +++ b/tests/qemuhotplugtestdomains/qemuhotplug-base-live+disk-scsi-multipath.xml @@ -0,0 +1,62 @@ +<domain type='kvm' id='7'> + <name>hotplug</name> + <uuid>d091ea82-29e6-2e34-3005-f02617b36e87</uuid> + <memory unit='KiB'>4194304</memory> + <currentMemory unit='KiB'>4194304</currentMemory> + <vcpu placement='static'>4</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <acpi/> + <apic/> + <pae/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <disk type='block' device='lun'> + <driver name='qemu' type='raw'/> + <source dev='/dev/mapper/virt'> + <reservations managed='yes'> + <source type='unix' path='/tmp/lib/domain-7-hotplug/pr-helper0.sock' mode='client'/> + </reservations> + </source> + <backingStore/> + <target dev='sda' bus='scsi'/> + <alias name='scsi0-0-0-0'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'> + <alias name='usb'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> + </controller> + <controller type='ide' index='0'> + <alias name='ide'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <controller type='scsi' index='0' model='virtio-scsi'> + <alias name='scsi0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </controller> + <controller type='pci' index='0' model='pci-root'> + <alias name='pci'/> + </controller> + <controller type='virtio-serial' index='0'> + <alias name='virtio-serial0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </controller> + <input type='mouse' bus='ps2'> + <alias name='input0'/> + </input> + <input type='keyboard' bus='ps2'> + <alias name='input1'/> + </input> + <memballoon model='none'/> + </devices> + <seclabel type='none' model='none'/> +</domain> -- 2.24.1

3 3

[PATCHv2 0/3] Share duplicated daemon code in *SetupLogging functions
by Lan 11 Mar '20

11 Mar '20

This patch is part of the bite sized task, Share duplicated daemon code. A new function is introduced for shared code in *SetupLogging functions. Commit 1: Share Dup Daemon Function *SetupLogging Introduce src/util/virdaemon.c/h files. Introduce new function virDaemonSetupLogging for shared code in *SetupLogging. Convert all callers of *SetupLogging functions to virDaemonSetupLogging. Introduce new struct virDaemonLogConfig for log configs (log_*) in *DeamonConfig structs. *DeamonConfig is an input to virDaemonSetupLogging. Commit 2: Alloc and Free virDaemonLogConfig Introduce functions to alloc and free the new struct virDaemonLogConfig. Commit 3: Passing virDaemonLogConfig to DaemonSetupLogging Convert all the callers that now pass virDaemonLogConfig to the new function virDaemonSetupLogging. Lan (3): Share Dup Daemon Function *SetupLogging Alloc and Free virDaemonLogConfig Passing virDaemonLogConfig to DaemonSetupLogging src/libvirt_private.syms | 4 ++ src/locking/lock_daemon.c | 66 +++++------------------- src/logging/log_daemon.c | 58 +++++---------------- src/remote/remote_daemon.c | 65 +++++------------------- src/util/Makefile.inc.am | 2 + src/util/virdaemon.c | 101 +++++++++++++++++++++++++++++++++++++ src/util/virdaemon.h | 43 ++++++++++++++++ 7 files changed, 186 insertions(+), 153 deletions(-) create mode 100644 src/util/virdaemon.c create mode 100644 src/util/virdaemon.h -- 2.17.1

2 4