- Devel - libvirt List Archives

[libvirt] [PATCH] conf: move 'generated' member from virMacAddr to virDomainNetDef
by Laine Stump 19 Feb '18

19 Feb '18

Commit 7e62c4cd26d (first appearing in libvirt-3.9.0 as a resolution to rhbz #1343919) added a "generated" attribute to virMacAddr that was set whenever a mac address was auto-generated by libvirt. This knowledge was used in a single place - when trying to match a NetDef from the domain to delete with user-provided XML. Since the XML parser always auto-generates a MAC address for NetDefs when none is provided, it was previously impossible to make a search where the MAC address wasn't significant, but the addition of the "generated" attribute made it possible for the search function to ignore auto-generated MACs. This implementation had a problem though - it was adding a field to a "low level" struct - virMacAddr - which is used in other places with the assumption that it contains exactly a 6 byte MAC address and nothing else. In particular, virNWFilterSnoopEthHdr uses virMacAddr as part of the definition of an ethernet packet header, whose layout must of course match an actual ethernet packet. Adding the extra bools into virNWFilterSnoopEthHdr caused the nwfilter driver's "IP discovery via DHCP packet snooping" functionality to mysteriously stop working. In order to fix that behavior, and prevent potential future similar odd behavior, this patch moves the "generated" member out of virMacAddr (so that it is again really just a MAC address) and into virDomainNetDef, and sets it only when virDomainNetGenerateMAC() is called from virDomainNetDefParseXML() (which is the only time we care about it). Resolves: https://bugzilla.redhat.com/1529338 (It should also be applied to any maintenance branch that applies commit 7e62c4cd26 and friends to resolve https://bugzilla.redhat.com/1343919) Signed-off-by: Laine Stump <laine(a)laine.org> --- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + src/util/virmacaddr.c | 5 ----- src/util/virmacaddr.h | 2 -- tests/bhyveargv2xmlmock.c | 1 - 5 files changed, 3 insertions(+), 9 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 3cfd6de5e0..7783a3dbef 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -11064,6 +11064,7 @@ virDomainNetDefParseXML(virDomainXMLOptionPtr xmlopt, } } else { virDomainNetGenerateMAC(xmlopt, &def->mac); + def->mac_generated = true; } if (devaddr) { @@ -16283,7 +16284,7 @@ virDomainNetFindIdx(virDomainDefPtr def, virDomainNetDefPtr net) size_t i; int matchidx = -1; char mac[VIR_MAC_STRING_BUFLEN]; - bool MACAddrSpecified = !net->mac.generated; + bool MACAddrSpecified = !net->mac_generated; bool PCIAddrSpecified = virDomainDeviceAddressIsValid(&net->info, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index e6212818aa..b0a175b4a4 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -966,6 +966,7 @@ struct _virDomainActualNetDef { struct _virDomainNetDef { virDomainNetType type; virMacAddr mac; + bool mac_generated; /* true if mac was *just now* auto-generated by libvirt */ char *model; union { struct { diff --git a/src/util/virmacaddr.c b/src/util/virmacaddr.c index 409fdc34d5..7afe032b9c 100644 --- a/src/util/virmacaddr.c +++ b/src/util/virmacaddr.c @@ -107,7 +107,6 @@ void virMacAddrSet(virMacAddrPtr dst, const virMacAddr *src) { memcpy(dst, src, sizeof(*src)); - dst->generated = false; } /** @@ -121,7 +120,6 @@ void virMacAddrSetRaw(virMacAddrPtr dst, const unsigned char src[VIR_MAC_BUFLEN]) { memcpy(dst->addr, src, VIR_MAC_BUFLEN); - dst->generated = false; } /** @@ -151,7 +149,6 @@ virMacAddrParse(const char* str, virMacAddrPtr addr) { size_t i; - addr->generated = false; errno = 0; for (i = 0; i < VIR_MAC_BUFLEN; i++) { char *end_ptr; @@ -220,7 +217,6 @@ virMacAddrParseHex(const char *str, virMacAddrPtr addr) str[VIR_MAC_HEXLEN]) return -1; - addr->generated = false; for (i = 0; i < VIR_MAC_BUFLEN; i++) addr->addr[i] = (virHexToBin(str[2 * i]) << 4 | virHexToBin(str[2 * i + 1])); @@ -236,7 +232,6 @@ void virMacAddrGenerate(const unsigned char prefix[VIR_MAC_PREFIX_BUFLEN], addr->addr[3] = virRandomBits(8); addr->addr[4] = virRandomBits(8); addr->addr[5] = virRandomBits(8); - addr->generated = true; } /* The low order bit of the first byte is the "multicast" bit. */ diff --git a/src/util/virmacaddr.h b/src/util/virmacaddr.h index ef4285d639..f4f5e2ce11 100644 --- a/src/util/virmacaddr.h +++ b/src/util/virmacaddr.h @@ -36,8 +36,6 @@ typedef virMacAddr *virMacAddrPtr; struct _virMacAddr { unsigned char addr[VIR_MAC_BUFLEN]; - bool generated; /* True if MAC address was autogenerated, - false otherwise. */ }; int virMacAddrCompare(const char *mac1, const char *mac2); diff --git a/tests/bhyveargv2xmlmock.c b/tests/bhyveargv2xmlmock.c index dd25f4e13a..1f08bebb7b 100644 --- a/tests/bhyveargv2xmlmock.c +++ b/tests/bhyveargv2xmlmock.c @@ -16,7 +16,6 @@ virMacAddrGenerate(const unsigned char prefix[VIR_MAC_PREFIX_BUFLEN], addr->addr[3] = 0; addr->addr[4] = 0; addr->addr[5] = 0; - addr->generated = true; } int -- 2.14.3

3 2

[libvirt] [PATCH v3 00/10] Cleanups in QEMU driver wrt virConnectPtr
by Daniel P. Berrangé 19 Feb '18

19 Feb '18

There are many places we can stop passing around virConnectPtr now and directly open secondary drivers where required instead. v3: - More migration function renames - Style fixes v2: - Now with working tests ! Daniel P. Berrangé (10): driver: allow override of connection for secondary drivers conf: reimplement virDomainNetResolveActualType in terms of public API qemu: stop passing virConnectPtr into qemuMonitorStartCPUs conf: stop passing virConnectPtr into virDomainDiskTranslateSourcePool qemu: don't pass virConnectPtr around for secrets qemu: stop passing in virConnectPtr for looking up networks qemu: remove virConnectPtr from some more startup code paths qemu: remove virConnectPtr in some migration methods qemu: don't export migration job APIs qemu: rename migration APIs to include Src or Dst in their name src/conf/domain_conf.c | 90 ++- src/conf/domain_conf.h | 14 +- src/driver.c | 184 ++++- src/driver.h | 7 + src/libvirt_private.syms | 6 + src/network/bridge_driver.c | 76 +-- src/qemu/qemu_conf.c | 3 +- src/qemu/qemu_conf.h | 3 +- src/qemu/qemu_domain.c | 111 ++- src/qemu/qemu_domain.h | 21 +- src/qemu/qemu_driver.c | 324 ++++----- src/qemu/qemu_hotplug.c | 66 +- src/qemu/qemu_hotplug.h | 15 +- src/qemu/qemu_migration.c | 1522 +++++++++++++++++++++--------------------- src/qemu/qemu_migration.h | 301 ++++----- src/qemu/qemu_monitor.c | 10 +- src/qemu/qemu_monitor.h | 11 +- src/qemu/qemu_monitor_json.c | 3 +- src/qemu/qemu_monitor_json.h | 3 +- src/qemu/qemu_monitor_text.c | 9 +- src/qemu/qemu_monitor_text.h | 3 +- src/qemu/qemu_process.c | 154 ++--- src/qemu/qemu_process.h | 12 +- tests/Makefile.am | 7 +- tests/qemuhotplugtest.c | 4 +- tests/qemumonitorjsontest.c | 2 +- tests/qemuxml2argvtest.c | 11 +- 27 files changed, 1517 insertions(+), 1455 deletions(-) -- 2.14.3

2 14

Re: [libvirt] Regarding libvirt patchset "Keep original security label"
by Michal Privoznik 19 Feb '18

19 Feb '18

[Adding libvirt-list as others might chime in or when somebody is solving similar issue they can find answers in the archive.] On 02/19/2018 10:09 AM, Toni Peltonen wrote: > Hi, > > Sorry to bother you with ages old stuff like this (https://www.redhat.com/archives/libvir-list/2014-September/msg00551.html) patch set. > > I did some proof of concept work on my laptop during the weekend with a couple of CentOS 7 virtual machines running shared GFS2 mount (with working dlm locking) and libvirtd + QEMU guests on top of that. I am running the latest libvirtd packages from CentOS upstream. > > While I managed to get dlm and global POSIX locking working as expected when I added virtlockd to the picture to actually hold locks for the VM images I ended up getting some gray hair realizing that this ages old security label issue is still present. The situation is basically still the same: > > - On a shared filesystem (like GFS2), despite virtlockd locks working as expected, libvirtd still (as expected with current code) tries to change the ownership and security labels of the target file (QCOW2 image) > > - This sudden change causes the virtual machine running on the other host to drop as read-only since SELinux starts preventing all write operations for it > > - With SELinux in Permissive mode on the host that runs the virtual machine everything work as expected So you have a disk that is shared between two domains? One way to avoid the problem is to have <shareable/> in disk definition so that libvirt doesn't restore disk labels. However, that still might not work for you because it means that libvirt still changes labels on domain start. So far the only way to prevent this from happening is using custom labels https://libvirt.org/formatdomain.html#elementsDisks . > > > I saw that the only code available (that I could easily find and understand) to really tackle this issue was your patch set from as early as 2014. It seems it never hit upstream though, at least I can't find any of the relevant parts in the upstream code. > > I am just reaching out to ask whether this patch set was left out intentionally (https://www.redhat.com/archives/libvir-list/2014-September/msg00551.html) just lost in time or if you are aware of any other work that might be in progress to tackle this issue in the upstream? Unfortunately, the patches were abandoned and the issue is even worse. I remember having a discussion on this topic lately (although can't recall where). Turns out, we firstly relabel the disks and only after that we try to obtain disk locks. So if the latter fails (e.g. because another domain holds the locks) the lables are changed anyway. The suggested solution was to have two locks: one for disk contents the other for disk metadata (like security labels). Anyway, I don't think there's anybody working on this actively. Sorry. Michal

1 0

[libvirt] [PATCH 0/2] Two blockjob fixes
by Peter Krempa 19 Feb '18

19 Feb '18

Peter Krempa (2): virsh: Fix internal naming of some blockjob commands qemu: blockcopy: Add check for bandwidth src/qemu/qemu_driver.c | 8 +++++++ tools/virsh-domain.c | 60 +++++++++++++++++++++++++------------------------- 2 files changed, 38 insertions(+), 30 deletions(-) -- 2.15.0

2 3

[libvirt] ANNOUNCE: ruby-libvirt 0.7.1
by Chris Lalancette 18 Feb '18

18 Feb '18

I'm pleased to release ruby-libvirt 0.7.1. ruby-libvirt is a ruby wrapper around the libvirt API. The changelog between 0.7.0 and 0.7.1 is: * Fix a bad bug in block_resize (Marius Rieder) * Fix up some problems pointed out by clang * Fix up the tests for small semantic differences in how libvirt works Version 0.7.1 is available from http://libvirt.org/ruby: Tarball: http://libvirt.org/ruby/download/ruby-libvirt-0.7.1.tgz Gem: http://libvirt.org/ruby/download/ruby-libvirt-0.7.1.gem It is also available from rubygems.org; to get the latest version, run: $ gem install ruby-libvirt Packages for it are also built for Fedora, and will be released shortly. As usual, if you run into questions, problems, or bugs, please feel free to mail me (clalancette at gmail.com) and the libvirt mailing list. Thanks to those who contributed to this release.

1 0

[libvirt] [jenkins-ci PATCH] projects: Don't run virt-manager's test suite on FreeBSD
by Andrea Bolognani 16 Feb '18

16 Feb '18

The libxml2 bindings are broken under Python 3.6, so we have to disable this for the time being. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- projects/virt-manager.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/projects/virt-manager.yaml b/projects/virt-manager.yaml index 8f3112e..e626cb0 100644 --- a/projects/virt-manager.yaml +++ b/projects/virt-manager.yaml @@ -20,6 +20,14 @@ - python-distutils-check-job: pyver: 3 parent_jobs: 'virt-manager-master-py{pyver}-build' + # libxml2's Python 3 bindings don't work properly on FreeBSD, + # so skip the test suite there for the time being. See + # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224902 + machines: + - libvirt-debian-9 + - libvirt-fedora-26 + - libvirt-fedora-27 + - libvirt-fedora-rawhide - python-distutils-rpm-job: pyver: 3 parent_jobs: 'virt-manager-master-py{pyver}-check' -- 2.14.3

2 1

[libvirt] [PATCH 0/6] Alternative way of turning NWFilterObj(list) into virObject
by Michal Privoznik 16 Feb '18

16 Feb '18

This is heavily based on John's work (except 1/6). The difference to his patches is in using virObjectRecursiveLockableNew() for recursive locks instead of RW locks and lock promoting. Also, in 6/6 I was more confident and removed driver lock from define/undefine APIs. John Ferlan (1): nwfilter: Remove need for nwfilterDriverLock in some API's Michal Privoznik (5): virObject: use virReportSystemError if applicable virObject: Introduce virObjectRecursiveLockableNew virNWFilterObj: Turn into virObjectLockable conf: Introduce and use virNWFilterObjEndAPI nwfilter: Convert _virNWFilterObjList to use virObjectRWLockable cfg.mk | 1 - src/conf/virdomainobjlist.c | 3 +- src/conf/virnwfilterobj.c | 504 +++++++++++++++++++++------------ src/conf/virnwfilterobj.h | 12 +- src/libvirt_private.syms | 5 +- src/nwfilter/nwfilter_driver.c | 65 ++--- src/nwfilter/nwfilter_gentech_driver.c | 11 +- src/util/virobject.c | 30 +- src/util/virobject.h | 4 + 9 files changed, 382 insertions(+), 253 deletions(-) -- 2.13.6

5 15

[libvirt] [PATCH v5 0/3] nwfilter common object adjustments
by John Ferlan 15 Feb '18

15 Feb '18

v4: https://www.redhat.com/archives/libvir-list/2017-December/msg00282.html But more recently reviewed: https://www.redhat.com/archives/libvir-list/2018-February/msg00022.html https://www.redhat.com/archives/libvir-list/2018-February/msg00015.html https://www.redhat.com/archives/libvir-list/2018-February/msg00016.html Changes since v4: Pushed former patch 1/4 since it was ACK'd... - Patch 2: * Return NULL on error in virNWFilterObjListFindInstantiateFilter instead of the somewhat obfuscated @obj which would be NULL as a result of the virNWFilterObjEndAPI * Use virNWFilterObjDemoteFromWrite prior to return obj when the path was returning a write locked object. - Patch 3: * Reword the comment prior to virNWFilterObjListFindByNameLocked in order to better explain what's happening * Ensure the write lock occurs while the nwfilters lock is held. * Fix the virNWFilterCountData setup * Fix the formatting for the @def in virNWFilterObjListExportCallback (there was an extra space) - Patch 4: * No change, still needs the official ACK/R-b though NB: Testing done... * Ran/passed the nwfilter concurrency test w/ the changes * Ran/passed the Avocado nwfilter_update_lock, nwfilter_update_vm_running, and virsh.nwfilter_undefine.error_test.acl_test tests - these were the ones with the most nwfilter "interaction" that would cause issues at various points in time during installation. John Ferlan (3): nwfilter: Convert _virNWFilterObj to use virObjectRWLockable nwfilter: Convert _virNWFilterObjList to use virObjectRWLockable nwfilter: Remove need for nwfilterDriverLock in some API's src/conf/virnwfilterobj.c | 540 +++++++++++++++++++++++---------- src/conf/virnwfilterobj.h | 11 +- src/libvirt_private.syms | 3 +- src/nwfilter/nwfilter_driver.c | 71 ++--- src/nwfilter/nwfilter_gentech_driver.c | 11 +- 5 files changed, 427 insertions(+), 209 deletions(-) -- 2.13.6

4 21

[libvirt] [PATCH 0/3] Fix trouble with <backingStore> in virsh detach-disk
by Peter Krempa 15 Feb '18

15 Feb '18

We've created malformed <auth> element for RBD images in backing store of a disk which failed parsing and broke detaching disks via virsh. Fix it by not parsing auth data at all from the disk backing images and remove the <backingStore> element when constructing the XML for disk detach. Peter Krempa (3): virsh: detach-disk: Add --print-xml switch util: storage: Remove detected authentication data for backing chains virsh: Remove <backingStore> sub-element in virshFindDisk src/util/virstoragefile.c | 8 ++++++++ tests/virstoragetest.c | 15 +++------------ tools/virsh-domain.c | 44 +++++++++++++++++++++++++++++++++----------- tools/virsh.pod | 4 ++++ 4 files changed, 48 insertions(+), 23 deletions(-) -- 2.15.0

2 4

[libvirt] [jenkins-ci PATCH] jobs: Set $PYTHONPATH for python-distutils jobs
by Andrea Bolognani 15 Feb '18

15 Feb '18

Since we install Python modules under $VIRT_PREFIX, we need to set $PYTHONPATH or the interpreter won't be able to locate them. This is currently being done per-worker in the Jenkins Web interface. However, now that we've introduced Python 3 builds, depending on the project and the OS, we might be using any of Python 2.7, 3.4, 3.5 and 3.6, which means we can't have a single $PYTHONPATH anymore: in particular, while it's okay to have non-esisting directories in $PYTHONPATH, we have to make sure that a Python 3 interpreter will never try to use a Python 2 module and vice versa. To solve the issue, we use a fairly large hammer: we set $PYTHONPATH at the job level, and include all reasonable minor versions for the Python major version (pyver) the job is using, even if they don't yet exist. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Another approach would be to use something like T=$VIRT_PREFIX/lib/python{pyver}.4/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages T=$T:$VIRT_PREFIX/lib/python{pyver}.5/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages T=$T:$VIRT_PREFIX/lib/python{pyver}.6/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages T=$T:$VIRT_PREFIX/lib/python{pyver}.7/site-packages T=$T:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages export PYTHONPATH=$T but that's just a different kind of ugly :/ jobs/python-distutils.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml index ff68c29..510769e 100644 --- a/jobs/python-distutils.yaml +++ b/jobs/python-distutils.yaml @@ -42,6 +42,7 @@ - shell: | {global_env} {local_env} + export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages {command_pre_build} python{pyver} ./setup.py build python{pyver} ./setup.py install --prefix=$VIRT_PREFIX @@ -83,6 +84,7 @@ - shell: | {global_env} {local_env} + export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages python{pyver} ./setup.py test publishers: - email: @@ -121,6 +123,7 @@ - shell: | {global_env} {local_env} + export PYTHONPATH=$VIRT_PREFIX/lib/python{pyver}.4/site-packages:$VIRT_PREFIX/lib64/python{pyver}.4/site-packages:$VIRT_PREFIX/lib/python{pyver}.5/site-packages:$VIRT_PREFIX/lib64/python{pyver}.5/site-packages:$VIRT_PREFIX/lib/python{pyver}.6/site-packages:$VIRT_PREFIX/lib64/python{pyver}.6/site-packages:$VIRT_PREFIX/lib/python{pyver}.7/site-packages:$VIRT_PREFIX/lib64/python{pyver}.7/site-packages sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in python{pyver} ./setup.py rpm publishers: -- 2.14.3

2 2