- Devel - libvirt List Archives

[libvirt] Introducing Fabian Freyer (GSoC 2016 student)
by Roman Bogorodskiy 12 May '16

12 May '16

Hi, I'd like to introduce Fabian Freyer (CCed), he's taking part in Google Support of Code this year within FreeBSD organization and his project is called "Improving libvirt support for bhyve". Below I'm sharing a tentative plan we currently have. Or, to be more specific, it's a list of ideas for the bhyve driver with things were implementation is clear going first, followed by items were some additional research and working on approach needed. There's no goal to implement everything from this list though. This list was assembled by Fabian with some minor edits from me. Suggestions and ideas are very welcome. --- The primary aim of this project is to implement missing calls and functionality in the libvirt bhyve driver. According to the libvirt API Support Matrix [1], there are a large number of calls not yet implemented. While some missing API calls are not applicable to bhyve, a number of them are, among them the following calls: General calls: - virConnectDomainXMLFromNative This would mostly be an argument parser for a bhyve(8) and bhyvectl(8) command line. - virConnectGetCPUModelNames This needs a research: bhyve is not very flexible in configuration of what CPU model is exposed to the guest, need to figure out if that’s worth implementing now. Connection calls: Most of these include some form of authentication handling and are therefore not applicable. However, the following do apply to bhyve and are easy to implement: - virConnectGetType Trivially return “BHYVE” - virConnectIsAlive Trivially return 1, since “A connection will be classed as alive if it is [...] local” and /dev/vmm is local. - virConnectIsEncrypted Trivially return 0, since bhyve does not support encrypted interfaces. General Domain calls: - virDomainGetMaxMemory Since bhyve does not support memory ballooning, just return the amount of memory allocated here - virDomainGetMaxVcpus, virDomainGetVcpus Would use the approach described in this mailing list thread: “Until the vCPU state is exposed by bhyvectl, or we provide a sysctl, you can use heuristics: the number of vCPU threads for the bhyve process, or scan all vCPUs and only count those that have a non-zero RIP.” [2] - virDomainGetCPUStats - virDomainGetTime - virDomainInjectNMI Call bhyvectl --inject-nmi - virDomainReset Reset a bhyve VM with ‘bhyvectl --force-reset’ and then clean things up using bhyvectl --destroy; update bhyve monitor code to handle exit code 0 from bhyve(8) that’s corresponding to reset (0 - reset / reboot, 1 - shutdown, 2 - halt) to trigger re-starting of the VM. - virDomainReboot Block-Device level calls: These would implement access to the vdev block storage layer. The plan is implement support for both file-backed and zvol-backed virtual machines for the following API calls: - virDomainGetBlockInfo - virDomainBlockPeek - virDomainBlockCopy - virDomainBlockStats - virDomainBlockStatsFlags Going further, since zvols support snapshotting, I plan to implement the following for zvol-backed storage: - virDomainBlockCommit - virDomainBlockPull VirtFS layer I would like to create patches to support specifying filesystems when creating the domain as well as the following calls to be merged at a later time when VirtFS-9p support for bhyve becomes ready: - virDomainFSFreeze - virDomainFSThaw Memory inspection - virDomainMemoryPeek A guest’s memory space is exposed in /dev/vmm, so this call would have to read from that. - virDomainMemoryStats Virt-host-validate(1) bhyve support Add bhyve support to virt-host-validate: - Check support of CPU supports features required for bhyve - Check if the vmm module is available - Check if essential networking things are available (if_bridge(4), if_tap(4)) + nmdm(4) for console Networking support Currently the bhyve’s libvirt driver (as well as libxl/FreeBSD driver) only supports L2 interface bridging. There’s no support for upper level schemas like NAT for example. This is a huge task that involves research of what firefall is more applicable (ipfw or pf), designing of the firewall rules and the actual implemtation. PCI passthrough support I.e. the the "bhyve ... -s 7:0,passthru,4/0/0" thing. Probably that will involve the HAL nodedev driver modifications. --- PS For historic purposes, I've stashed the original proposal: https://people.freebsd.org/~novel/misc/FabianFreyer_GoogleSummerofCode2016p… 1: https://libvirt.org/hvsupport.html 2: https://lists.freebsd.org/pipermail/freebsd-virtualization/2014-April/00246… Roman Bogorodskiy

5 5

[libvirt] [PATCH] util: Fix virGetLastErrorMessage to return proper error when 'err' is NULL
by Erik Skultety 12 May '16

12 May '16

Both virGetLastError and virGetLastErrorMessage call virLastErrorObject method that returns a thread-local error object. However, if a direct call to malloc or pthread_setspecific (probably also due to malloc, since it sets ENOMEM) fail, virLastErrorObject returns NULL which, although incorrectly interpreted by virGetLastError as no error, still requires the caller to check for NULL pointer. This isn't the case with virGetLastErrorMessage that also treated it incorrectly as no error, but returned the literal "no error". This patch tweaks the checks in the virGetLastErrorMessage function, so that if virLastErrorObject failed, it returned "unknown error" which is equivalent to the current approach with virGetLastError and if it returned NULL, "unknown error" was set. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- src/util/virerror.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/util/virerror.c b/src/util/virerror.c index 5d875e3..1177570 100644 --- a/src/util/virerror.c +++ b/src/util/virerror.c @@ -281,9 +281,9 @@ const char * virGetLastErrorMessage(void) { virErrorPtr err = virLastErrorObject(); - if (!err || err->code == VIR_ERR_OK) + if (err && err->code == VIR_ERR_OK) return _("no error"); - if (err->message == NULL) + if (!err || !err->message) return _("unknown error"); return err->message; } -- 2.5.5

2 2

[libvirt] [PATCH v3 0/2] Add IV Secret Object support (for RBD only)
by John Ferlan 11 May '16

11 May '16

v2 here: http://www.redhat.com/archives/libvir-list/2016-May/msg00067.html First 6 patches from v2 were pushed since they were ACK'd I modified the 7th patch to add the ATTRIBUTE_UNUSED to the static functions qemuDomainSecretHaveEncypt and qemuDomainSecretIVSetup and merged in the HAVE_GNUTLS_CIPHER_ENCRYPT. This should address Michal's point in patch 7 review. Patch 2 thus becomes the "remainder" of the work necessary in order to get IV Secrets available for RBD volumes. It's most of the former patch 8 with removal if iSCSI oddities. Patch 8 will then remove the ATTRIBUTE_UNUSED from the 2 static functions now that qemuDomainSecretSetup exists to call it. The change here was to add the check for protocol == VIR_STORAGE_NET_PROTOCOL_RBD before "defaulting" to using the IV secret (e.g., the magic decision point). Since the command code didn't need the secret setup for iSCSI, it's been extracted to another safe place just in case it needs to be resurrected. John Ferlan (2): qemu: Introduce new Secret IV API's qemu: Utilize qemu secret objects for RBD auth/secret configure.ac | 1 + src/qemu/qemu_alias.c | 23 +++ src/qemu/qemu_alias.h | 2 + src/qemu/qemu_command.c | 130 +++++++++++++- src/qemu/qemu_domain.c | 189 ++++++++++++++++++++- ...emuxml2argv-disk-drive-network-rbd-auth-IV.args | 31 ++++ ...qemuxml2argv-disk-drive-network-rbd-auth-IV.xml | 42 +++++ tests/qemuxml2argvmock.c | 31 +++- tests/qemuxml2argvtest.c | 11 ++ 9 files changed, 451 insertions(+), 9 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-rbd-auth-IV.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-rbd-auth-IV.xml -- 2.5.5

3 8

[libvirt] [PATCH] tests: qemu: mock master key generation
by Peter Krempa 11 May '16

11 May '16

The master key generation is using host state to get the random bytes. Currently it's not used so it doesn't break any tests, but it may become a problem in the future. Fix it by mocking qemuDomainGenerateRandomKey. This is possible after exporting it and switching to the dynamically linked qemu driver object for tests. --- This also makes the test not call any gnutls function which crashed certain gnutls versions as we did not initialize it (gnutls_global_init()). It may be also worth adding the call to virtTestMain so that tests don't run into that problem, but I'm not totaly persuaded that it's a good idea. src/qemu/qemu_domain.c | 2 +- src/qemu/qemu_domain.h | 2 ++ tests/Makefile.am | 2 +- tests/qemuxml2argvmock.c | 21 +++++++++++++++++++++ 4 files changed, 25 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 3da0079..d955ed4 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -640,7 +640,7 @@ qemuDomainMasterKeyReadFile(qemuDomainObjPrivatePtr priv) * * Returns pointer memory containing key on success, NULL on failure */ -static uint8_t * +uint8_t * qemuDomainGenerateRandomKey(size_t nbytes) { uint8_t *key; diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index c711188..b116949 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -669,4 +669,6 @@ int qemuDomainSecretPrepare(virConnectPtr conn, virDomainObjPtr vm) int qemuDomainDefValidateDiskLunSource(const virStorageSource *src) ATTRIBUTE_NONNULL(1); +uint8_t *qemuDomainGenerateRandomKey(size_t nbytes); + #endif /* __QEMU_DOMAIN_H__ */ diff --git a/tests/Makefile.am b/tests/Makefile.am index 75efb90..ba14b8f 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -561,7 +561,7 @@ libqemutestdriver_la_LIBADD = $(qemu_LDADDS) qemuxml2argvtest_SOURCES = \ qemuxml2argvtest.c testutilsqemu.c testutilsqemu.h \ testutils.c testutils.h -qemuxml2argvtest_LDADD = $(qemu_LDADDS) $(LDADDS) $(LIBXML_LIBS) +qemuxml2argvtest_LDADD = libqemutestdriver.la $(LDADDS) $(LIBXML_LIBS) qemuxml2argvmock_la_SOURCES = \ qemuxml2argvmock.c diff --git a/tests/qemuxml2argvmock.c b/tests/qemuxml2argvmock.c index 1616eed..1bad3ae 100644 --- a/tests/qemuxml2argvmock.c +++ b/tests/qemuxml2argvmock.c @@ -30,6 +30,10 @@ #include "virstring.h" #include "virtpm.h" #include "virutil.h" +#include "viralloc.h" + +#include "qemu/qemu_domain.h" + #include <time.h> #include <unistd.h> @@ -145,3 +149,20 @@ virCommandPassFD(virCommandPtr cmd ATTRIBUTE_UNUSED, { /* nada */ } + + +uint8_t * +qemuDomainGenerateRandomKey(size_t nbytes) +{ + size_t i; + uint8_t seq = 0; + uint8_t *key; + + if (VIR_ALLOC_N(key, nbytes) < 0) + return NULL; + + for (i = 0; i < nbytes; i++) + key[i] = seq++; + + return key; +} -- 2.8.2

2 4

[libvirt] [PATCH] spec: Fix broken indentation reported by syntax-check
by Erik Skultety 11 May '16

11 May '16

syntax-check complained about broken indentation in libvirt.spec.in which was broken by commit 3694e038 Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- Pushed under trivial rule. Erik libvirt.spec.in | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index 886f5cf..d96917b 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -4,9 +4,9 @@ # or for RHEL 6 or newer. It may need some tweaks for other distros. # If neither fedora nor rhel was defined, try to guess them from dist %if (0%{?fedora} && 0%{?fedora} >= 20) || (0%{?rhel} && 0%{?rhel} >= 6) -%define supported_platform 1 + %define supported_platform 1 %else -%define supported_platform 0 + %define supported_platform 0 %endif # Default to skipping autoreconf. Distros can change just this one line -- 2.5.5

1 0

[libvirt] [PATCH REPOST 0/5] admin: Introduce APIs to get/set client processing
by Erik Skultety 11 May '16

11 May '16

This is just a rebased version of https://www.redhat.com/archives/libvir-list/2016-April/msg00466.html. Patches 1/7 and 2/7 of the original series were already pushed with another series. Erik Skultety (5): virnetserver: Introduce client processing controls getters admin: Introduce some public constants related to client processing controls admin: Introduce virAdmServerGetClientProcessingControls admin: Introduce virAdmServerSetClientProcessingControls virt-admin: Introduce commands srv-clients-info and srv-clients-set daemon/admin.c | 86 +++++++++++++++++++ daemon/admin_server.c | 76 +++++++++++++++++ daemon/admin_server.h | 10 +++ include/libvirt/libvirt-admin.h | 52 ++++++++++++ src/admin/admin_protocol.x | 30 ++++++- src/admin/admin_remote.c | 74 +++++++++++++++++ src/admin_protocol-structs | 20 +++++ src/libvirt-admin.c | 89 ++++++++++++++++++++ src/libvirt_admin_private.syms | 3 + src/libvirt_admin_public.syms | 2 + src/rpc/virnetserver.c | 83 +++++++++++++++++++ src/rpc/virnetserver.h | 9 ++ tools/virt-admin.c | 177 ++++++++++++++++++++++++++++++++++++++++ 13 files changed, 710 insertions(+), 1 deletion(-) -- 2.5.5

2 15

[libvirt] [PATCH 0/5] domaincaps: Report spice GL capabilities
by Cole Robinson 11 May '16

11 May '16

This extends domain capabilities to report some bits apps will want to determine if spice GL is supported: <domainCapabilities> <devices> <video supported='yes'> <enum name='modelType'/> <enum name='virtioAccel3D'/> </video> <graphics supported='yes'> <enum name='type'/> <enum name='spiceGL'/> </graphics> </devices> </domainCapabilities> Cole Robinson (5): domaincaps: Report graphics type enum domaincaps: Report video modelType domaincaps: Report graphics spiceGL qemu: command: unconditionally allow accel3d='no' domaincaps: Report graphics spiceGL docs/formatdomaincaps.html.in | 76 +++++++++++++++++++++- docs/schemas/domaincaps.rng | 16 +++++ src/conf/domain_capabilities.c | 28 ++++++++ src/conf/domain_capabilities.h | 21 ++++++ src/qemu/qemu_capabilities.c | 53 ++++++++++++++- src/qemu/qemu_command.c | 2 +- tests/domaincapsschemadata/domaincaps-basic.xml | 2 + tests/domaincapsschemadata/domaincaps-full.xml | 31 +++++++++ .../domaincaps-qemu_1.6.50-1.xml | 21 ++++++ .../domaincaps-qemu_2.6.0-1.xml | 24 +++++++ .../domaincaps-qemu_2.6.0-2.xml | 19 ++++++ .../domaincaps-qemu_2.6.0-3.xml | 19 ++++++ .../domaincaps-qemu_2.6.0-4.xml | 19 ++++++ .../domaincaps-qemu_2.6.0-5.xml | 19 ++++++ tests/domaincapstest.c | 10 +++ 15 files changed, 357 insertions(+), 3 deletions(-) -- 2.7.4

5 25

[libvirt] [PATCH 0/2] *** use virGetLastErrorMessage ***
by Jovanka Gulicoska 11 May '16

11 May '16

*** BLURB HERE *** Jovanka Gulicoska (2): tests: use virGetLastErrorMessage() Use virGetLastErrorMessage()

3 9

[libvirt] [PATCH 0/3] Patches for some device mapper multipath issues
by John Ferlan 11 May '16

11 May '16

Patch 1 just makes sure that prior to calling refreshPool we make sure to clear the objects; otherwise, the refresh code will duplicate. Text within patch 3 seems to indicate that entries were showing up more than once - although those may be bug related. In any case, although perhaps not necessary here - better safe than sorry. Patch 2 is a rework of changes that went into 1.3.2 related to how libvirt_parthelper generates the volume target path name that didn't quite get the algorithm right when the device source path ended with a non numeric value. As it turns out it seems the naming algorithm is much "simpler"; however, rather than just remove the attribute, I figured it should be kept "just in case" something *needed* to add that "p" it would be possible. It seems from output described here: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/ht… That it would be possible. Patch 3 was probably seen because Patch 2 testing occurs, so the same "sequence" used in other disk backend based testing was attempted. As it turns out, I don't believe the deletion for a multipath device ever really worked. The original patch could have been reverted except that it intermingled changes to support linking device-mapper with parthelper. John Ferlan (3): storage: Need to clear pool prior to calling the refreshPool storage: Fix algorithm generating path names for devmapper storage: Fix virStorageBackendDiskDeleteVol for device mapper configure.ac | 15 ++---- docs/formatstorage.html.in | 18 ++----- src/storage/parthelper.c | 13 +++-- src/storage/storage_backend_disk.c | 108 ++++++++++++++++++++++++------------- src/storage/storage_driver.c | 3 ++ 5 files changed, 89 insertions(+), 68 deletions(-) -- 2.5.5

2 6

[libvirt] [PATCH] libxl: fix vm lock overwritten bug
by Wang Yufei 11 May '16

11 May '16

In libxl driver we do virObjectRef in libxlDomainObjBeginJob, If virCondWaitUntil failed, it goes to error, do virObjectUnref, There's a chance that someone undefine the vm at the same time, and refs unref to zero, vm is freed in libxlDomainObjBeginJob. But the vm outside function is not Null, we do virObjectUnlock(vm). That's how we overwrite the vm memory after it's freed. Because the coding amount is much, I fix it partly in libxlDomainCreateWithFlags. If my opinion is right and there're no problems, I'll fix them all later. Signed-off-by: Wang Yufei <james.wangyufei(a)huawei.com> --- .gnulib | 1 - src/libxl/libxl_domain.c | 6 +----- src/libxl/libxl_domain.h | 2 +- src/libxl/libxl_driver.c | 5 ++--- 4 files changed, 4 insertions(+), 10 deletions(-) delete mode 160000 .gnulib diff --git a/.gnulib b/.gnulib deleted file mode 160000 index 6cc32c6..0000000 --- a/.gnulib +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 6cc32c63e80bc1a30c521b2f07f2b54909b59892 diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 14a900c..a90ce53 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -123,7 +123,6 @@ libxlDomainObjBeginJob(libxlDriverPrivatePtr driver ATTRIBUTE_UNUSED, return -1; then = now + LIBXL_JOB_WAIT_TIME; - virObjectRef(obj); while (priv->job.active) { VIR_DEBUG("Wait normal job condition for starting job: %s", @@ -157,7 +156,6 @@ libxlDomainObjBeginJob(libxlDriverPrivatePtr driver ATTRIBUTE_UNUSED, virReportSystemError(errno, "%s", _("cannot acquire job mutex")); - virObjectUnref(obj); return -1; } @@ -171,7 +169,7 @@ libxlDomainObjBeginJob(libxlDriverPrivatePtr driver ATTRIBUTE_UNUSED, * non-zero, false if the reference count has dropped to zero * and obj is disposed. */ -bool +void libxlDomainObjEndJob(libxlDriverPrivatePtr driver ATTRIBUTE_UNUSED, virDomainObjPtr obj) { @@ -183,8 +181,6 @@ libxlDomainObjEndJob(libxlDriverPrivatePtr driver ATTRIBUTE_UNUSED, libxlDomainObjResetJob(priv); virCondSignal(&priv->job.cond); - - return virObjectUnref(obj); } int diff --git a/src/libxl/libxl_domain.h b/src/libxl/libxl_domain.h index 1c1eba3..ce28944 100644 --- a/src/libxl/libxl_domain.h +++ b/src/libxl/libxl_domain.h @@ -85,7 +85,7 @@ libxlDomainObjBeginJob(libxlDriverPrivatePtr driver, enum libxlDomainJob job) ATTRIBUTE_RETURN_CHECK; -bool +void libxlDomainObjEndJob(libxlDriverPrivatePtr driver, virDomainObjPtr obj) ATTRIBUTE_RETURN_CHECK; diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index bf97c9c..a46994a 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -294,7 +294,7 @@ libxlDomObjFromDomain(virDomainPtr dom) libxlDriverPrivatePtr driver = dom->conn->privateData; char uuidstr[VIR_UUID_STRING_BUFLEN]; - vm = virDomainObjListFindByUUID(driver->domains, dom->uuid); + vm = virDomainObjListFindByUUIDRef(driver->domains, dom->uuid); if (!vm) { virUUIDFormat(dom->uuid, uuidstr); virReportError(VIR_ERR_NO_DOMAIN, @@ -2691,8 +2691,7 @@ libxlDomainCreateWithFlags(virDomainPtr dom, vm = NULL; cleanup: - if (vm) - virObjectUnlock(vm); + virDomainObjEndAPI(&vm); return ret; } -- 1.9.5.msysgit.1

3 3