- Devel - libvirt List Archives

Re: [libvirt] [BUG] When PLUG a bridge interface to an active VM, the generated LIVE and CONFIG mac address are different
by wujing (O) 28 Aug '19

28 Aug '19

> >>> On 8/22/19 10:43 AM, Xu Yandong (Yandong Xu) wrote: > >> That possibly might be out of scope, but autofilling the mac address > >> as early as virDomainNetDefParseXML also is not ideal. > > > > We have pushed a patch bellow that can restore the situation to an older > state. > > > > Subject: [PATCH] qemu: use the same def when attaching device live and > > config > > > > diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index > > 617d7d5..eca54d0 100644 > > --- a/src/qemu/qemu_driver.c > > +++ b/src/qemu/qemu_driver.c > > @@ -8615,6 +8615,22 @@ > qemuDomainAttachDeviceLiveAndConfig(virDomainObjPtr vm, > > if (!(caps = virQEMUDriverGetCapabilities(driver, false))) > > goto cleanup; > > > > + if (flags & VIR_DOMAIN_AFFECT_CONFIG && > > + flags & VIR_DOMAIN_AFFECT_LIVE) { > > + /* If we are affecting both CONFIG and LIVE > > + * use the same xml of device preferentially > > + * to make the configuration consistent. > > + */ > > + devLive = virDomainDeviceDefParse(xml, vm->def, > > This use of vm->def prefers live XML. This means that for instance PCI > address are assigned based on current PCI layout in live XML which in general > is different to inactive XML. > > And it's not only PCI addresses, we autogenerate some out aspects of > <interface/> (e.g. model) and even more for other devices. Just take a look > at qemuDomainDeviceDefPostParse(). All functions there which take domain > def as an argument do so because they are basing autogenerated value on > domain definition. Plus there's more in parser code. > > While your patch might work for your use case, it can break others. > > BTW: have you read the original commit that caused this? I'm failing to see > how we would not re-introduce the problem with your patch. If you're using > live XML to validate/generate device addresses, how can we generate valid > address for inactive XML? > This patch is a little different with the original commit, since we only prefer live XML when using LIVE AND CONFIG flags at the same time. Still parse separate device def if attach with only one flag. So this patch is kinda a combination of 1e0534a7702 and 55ce6564634, just to narrow down the problem influence. And I verified as the steps in https://bugzilla.redhat.com/show_bug.cgi?id=1559867, work as expected. Wu Jing

2 1

[libvirt] [PATCH] Generate status of the backend implementation in storage.html
by Cedric Bosdonnat 28 Aug '19

28 Aug '19

Since it helps a user to know which of the storage backends support what operation, include an autogenerated matrix showing it in the docs. --- .gitignore | 1 + docs/Makefile.am | 9 +++-- docs/apibuild.py | 2 ++ docs/storage.html.in | 14 ++++++++ docs/storagebackendstatus.py | 64 ++++++++++++++++++++++++++++++++++++ 5 files changed, 88 insertions(+), 2 deletions(-) create mode 100644 docs/storagebackendstatus.py diff --git a/.gitignore b/.gitignore index 82495e8692..06875abebd 100644 --- a/.gitignore +++ b/.gitignore @@ -58,6 +58,7 @@ /configure.lineno /conftest.* /docs/aclperms.htmlinc +/docs/storagebackendstatus.htmlinc /docs/apibuild.py.stamp /docs/devhelp/libvirt.devhelp /docs/hvsupport.html.in diff --git a/docs/Makefile.am b/docs/Makefile.am index 1cdb584b0b..526821b1d3 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -195,6 +195,7 @@ EXTRA_DIST= \ $(kbase_html_in) $(kbase_html) \ aclperms.htmlinc \ hvsupport.pl \ + storagebackendstatus.py \ $(schema_DATA) acl_generated = aclperms.htmlinc @@ -209,7 +210,8 @@ MAINTAINERCLEANFILES = \ $(addprefix $(srcdir)/,$(devhelphtml)) \ $(addprefix $(srcdir)/,$(internals_html)) \ $(addprefix $(srcdir)/,$(kbase_html)) \ - $(srcdir)/hvsupport.html.in $(srcdir)/aclperms.htmlinc + $(srcdir)/hvsupport.html.in $(srcdir)/aclperms.htmlinc \ + $(srcdir)/storagebackendstatus.htmlinc timestamp="$(shell if test -n "$$SOURCE_DATE_EPOCH"; \ then \ @@ -230,6 +232,9 @@ web: $(dot_html) $(internals_html) $(kbase_html) \ hvsupport.html: $(srcdir)/hvsupport.html.in +storagebackendstatus.htmlinc: $(srcdir)/storagebackendstatus.py + $(PYTHON) $< >$@ + $(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \ $(top_srcdir)/src/libvirt_public.syms \ $(top_srcdir)/src/libvirt_qemu.syms $(top_srcdir)/src/libvirt_lxc.syms \ @@ -256,7 +261,7 @@ MAINTAINERCLEANFILES += \ convert -rotate 90 $< $@ %.html.tmp: %.html.in site.xsl subsite.xsl page.xsl \ - $(acl_generated) + $(acl_generated) storagebackendstatus.htmlinc $(AM_V_GEN)name=`echo $@ | sed -e 's/.tmp//'`; \ dir=`dirname $@` ; \ if test "$$dir" = "."; \ diff --git a/docs/apibuild.py b/docs/apibuild.py index dbdc1c95af..31944b8176 100755 --- a/docs/apibuild.py +++ b/docs/apibuild.py @@ -2010,6 +2010,8 @@ class docBuilder: self.includes = includes + list(lxc_included_files.keys()) elif name == "libvirt-admin": self.includes = includes + list(admin_included_files.keys()) + else: + self.includes = includes self.modules = {} self.headers = {} self.idx = index() diff --git a/docs/storage.html.in b/docs/storage.html.in index e0e4edec1e..2e4f662222 100644 --- a/docs/storage.html.in +++ b/docs/storage.html.in @@ -826,5 +826,19 @@ <h3>Valid volume format types</h3> <p>The valid volume types are the same as for the directory pool.</p> + + <h2>Storage Pool Types implementation status</h2> + + <p> + The storage backends have different level of support of the various pool and volume actions. + Here is a matrix of the current support for each of them. + </p> + + <p> + <strong>Note:</strong> some functions like Start and Stop will not trigger an exception when + called on a backend that doesn't implement them. + </p> + + <div id="include" filename="storagebackendstatus.htmlinc"/> </body> </html> diff --git a/docs/storagebackendstatus.py b/docs/storagebackendstatus.py new file mode 100644 index 0000000000..d105fea626 --- /dev/null +++ b/docs/storagebackendstatus.py @@ -0,0 +1,64 @@ +import os +import os.path +import re + +srcdir = os.path.abspath((os.environ.get("srcdir", os.path.join("..", "src")))) + +def get_allowed_functions(): + functions = [] + with open(os.path.join(srcdir, 'storage', 'storage_backend.h'), 'r') as handle: + content = ''.join(handle.readlines()) + definition = re.search('struct _virStorageBackend {([^}]+)}', content) + if definition is not None: + functions = re.findall('virStorageBackend[^ ]+ ([^;]+)', definition.group(1)) + return functions + +class Backend: + def __init__(self, name, code): + self.name = name + self.functions = [member[1:] for member in re.findall('.([^ ]+) = ', code) if member != '.type'] + +def get_backends(): + backends = [] + for root, dirs, files in os.walk(os.path.join(srcdir, 'storage')): + storage_impls = [os.path.join(root, f) for f in files if re.match('storage_backend_[^.]+.c', f)] + for impl in storage_impls: + handle = open(impl, 'r') + content = ''.join(handle.readlines()) + handle.close() + chunks = re.findall('virStorageBackend virStorageBackend([^ ]+) = {([^}]*)}', content) + backends.extend([Backend(chunk[0], chunk[1]) for chunk in chunks]) + return backends + +def main(): + functions = get_allowed_functions() + backends = get_backends() + + headers = '\n'.join(['<th>%s</th>' % backend.name for backend in backends]) + rows = [] + for func in functions: + cell_template = '<td style="text-align: center">%s</td>' + support = [cell_template % ('✔' if func in backend.functions else '') for backend in backends] + rows.append('\n'.join(['<tr>', '<td>%s</td>' % func] + support + ['</tr>'])) + + print('''<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> +<body> +<table class='top_table'> +<thead> +<tr> +<th> +</th> +%s +</tr> +</thead> +<tbody> +%s +</tbody> +</table> +</body> +</html>''' % (headers, '\n'.join(rows))) + +if __name__ == '__main__': + main() -- 2.22.0

2 3

[libvirt] [BUG] When PLUG a bridge interface to an active VM, the generated LIVE and CONFIG mac address are different
by Xu Yandong (Yandong Xu) 28 Aug '19

28 Aug '19

Hi, When plug a bridge interface to an active VM with both LIVE AND CONFIG flags, libvirt generate different mac address to LIVE and CONFIG instance, so After I reboot the VM, DHCP server doesn't assign the same IP address to the new bridge interface. Steps: 1. Having running VM ## virsh list Id Name State ------------------------------- 1 centos_7.6_64 running 2. Prepare a bridge interface xml ## cat interface.xml <interface type='bridge'> <source bridge='br0-int'/> <virtualport type='openvswitch'> </virtualport> <model type='virtio'/> <target dev='ovsvnet4'/> </interface> 3. Plug interface with --live and --config ##virsh attach-device centos_7.6_64 interface.xml --config --live Device attached successfully 4. Check LIVE XML ## cat /var/run/libvirt/qemu/centos_7.6_64.xml | grep ovsvnet4 -A 4 -B 6 <interface type='bridge'> <mac address='52:54:00:23:7d:fb'/> <source bridge='br0-int'/> <virtualport type='openvswitch'> <parameters interfaceid='c9f07f2b-3888-4f25-bd80-f5ba04d07bf8'/> </virtualport> <target dev='ovsvnet4'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </interface> 5. Check Config XML ## cat /etc/libvirt/qemu/centos_7.6_64.xml | grep ovsvnet4 -A 3 -B 6 <interface type='bridge'> <mac address='52:54:00:bb:d4:53'/> <source bridge='br0-int'/> <virtualport type='openvswitch'> <parameters interfaceid='189234ea-878e-4e8a-9f63-f43fac50aebd'/> </virtualport> <target dev='ovsvnet4'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </interface> The mac address and virtualport interfaceid does not match in LIVE and CONFIG XML. This phenomenon was introduced by the commit: commit 55ce65646348884656fd7bf3f109ebf8f7603494 Author: John Ferlan <jferlan(a)redhat.com> Date: Tue Jun 12 08:44:21 2018 -0400 qemu: Use the correct vm def on cold attach https://bugzilla.redhat.com/show_bug.cgi?id=1559867 When attaching a device to the domain we need to be sure to use the correct domain definition (vm->def or vm->newDef) when calling virDomainDeviceDefParse because the post parse processing algorithms that may assign an address for the device will use whatever domain definition was passed in.

4 5

[libvirt] [PATCH 0/4] remote: Enforce RPC limits when encoding typed params
by Michal Privoznik 27 Aug '19

27 Aug '19

Couple of bug fixes here. Michal Prívozník (4): remote_daemon_dispatch: Check for limit properly in remoteDispatchConnectGetAllDomainStats remote: Check for limits when encoding typed params remote_daemon_dispatch: Don't leak @ret on failure remote_daemon_dispatch: Don't open code xdr_free() src/admin/admin_remote.c | 2 + src/admin/admin_server_dispatch.c | 27 ++-------- src/remote/remote_daemon_dispatch.c | 81 ++++++++++++----------------- src/remote/remote_driver.c | 48 +++-------------- src/rpc/gendispatch.pl | 2 + src/util/virtypedparam.c | 13 ++++- src/util/virtypedparam.h | 1 + 7 files changed, 59 insertions(+), 115 deletions(-) -- 2.21.0

3 7

[libvirt] [jenkins-ci PATCH v2] guests: Introduce a var to enable shallow git clones for projects
by Erik Skultety 27 Aug '19

27 Aug '19

First of all, we don't need the full history for builds and we can save some time during the prepare phase. The disk footprint is about 5x smaller, but the most important thing is the repo clone duration difference e.g. in libvirt's case is quite significant: full history: real 3m45.236s user 1m53.074s sys 0m15.577s depth 1: real 0m14.868s user 0m2.215s sys 0m0.906s Since jenkins has a local copy and doesn't clone from the internet, this patch modifies only the configs for local VMs. --- guests/group_vars/all/main.yml | 2 ++ guests/playbooks/build/jobs/prepare.yml | 1 + 2 files changed, 3 insertions(+) diff --git a/guests/group_vars/all/main.yml b/guests/group_vars/all/main.yml index b73795e..6efcb52 100644 --- a/guests/group_vars/all/main.yml +++ b/guests/group_vars/all/main.yml @@ -5,3 +5,5 @@ ansible_ssh_pass: root jenkins_url: https://ci.centos.org/computer/{{ inventory_hostname }}/slave-agent.jnlp + +git_shallow_clone: true diff --git a/guests/playbooks/build/jobs/prepare.yml b/guests/playbooks/build/jobs/prepare.yml index 576afbc..477c169 100644 --- a/guests/playbooks/build/jobs/prepare.yml +++ b/guests/playbooks/build/jobs/prepare.yml @@ -5,6 +5,7 @@ version: '{{ git_branch }}' dest: '{{ name }}' force: yes + depth: '{{ git_shallow_clone | ternary(1, omit) }}' when: - inventory_hostname in machines -- 2.20.1

1 0

[libvirt] [PATCH v2 0/6] support boot from vfio-ccw mdev devices
by Boris Fiuczynski 27 Aug '19

27 Aug '19

Changes in v2: 1. moved refactoring into separate patch 2. removed line misalignment 3. moved valition from command line generator into domain xml validation 4. added tests for vfio-ap and vhost scsi Boris Fiuczynski (6): tests: add vfio-ap mdev tests tests: add vhost scsi hostdev boot unsupported test qemu: move hostdev boot validation into domain validation qemu: make error messages device specific qemu: refactor mdev validation method signatures qemu: support bootindex on vfio-ccw mdev devices src/qemu/qemu_command.c | 17 ++---- src/qemu/qemu_domain.c | 53 ++++++++++++++----- .../hostdev-scsi-vhost-scsi-pci-boot-fail.xml | 41 ++++++++++++++ .../hostdev-subsys-mdev-vfio-ap-boot-fail.xml | 23 ++++++++ .../hostdev-subsys-mdev-vfio-ap.args | 28 ++++++++++ .../hostdev-subsys-mdev-vfio-ap.xml | 22 ++++++++ .../hostdev-subsys-mdev-vfio-ccw-boot.args | 29 ++++++++++ .../hostdev-subsys-mdev-vfio-ccw-boot.xml | 23 ++++++++ tests/qemuxml2argvtest.c | 16 ++++++ .../hostdev-subsys-mdev-vfio-ap.xml | 27 ++++++++++ .../hostdev-subsys-mdev-vfio-ccw-boot.xml | 28 ++++++++++ tests/qemuxml2xmltest.c | 8 +++ 12 files changed, 291 insertions(+), 24 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pci-boot-fail.xml create mode 100644 tests/qemuxml2argvdata/hostdev-subsys-mdev-vfio-ap-boot-fail.xml create mode 100644 tests/qemuxml2argvdata/hostdev-subsys-mdev-vfio-ap.args create mode 100644 tests/qemuxml2argvdata/hostdev-subsys-mdev-vfio-ap.xml create mode 100644 tests/qemuxml2argvdata/hostdev-subsys-mdev-vfio-ccw-boot.args create mode 100644 tests/qemuxml2argvdata/hostdev-subsys-mdev-vfio-ccw-boot.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-subsys-mdev-vfio-ap.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-subsys-mdev-vfio-ccw-boot.xml -- 2.17.0

4 16

[libvirt] [PATCH] virsh: Add support for setting bandwidth in migrate
by Jim Fehlig 27 Aug '19

27 Aug '19

Commit f15789ec added support for setting postcopy migration bandwidth to the migrate subcommand. This change does the same for standard migration. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- I used the term 'standard migration' in the commit message but 'pre-copy' in the documentation. If one is preferred or more correct I can change the other. tools/virsh-domain.c | 13 +++++++++++++ tools/virsh.pod | 4 +++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index ccda71d7e0..8984c2f3b4 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -10587,6 +10587,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_INT, .help = N_("number of connections for parallel migration") }, + {.name = "precopy-bandwidth", + .type = VSH_OT_INT, + .help = N_("pre-copy migration bandwidth limit in MiB/s") + }, {.name = NULL} }; @@ -10801,6 +10805,15 @@ doMigrate(void *opaque) goto save_error; } + if ((rv = vshCommandOptULongLong(ctl, cmd, "precopy-bandwidth", &ullOpt)) < 0) { + goto out; + } else if (rv > 0) { + if (virTypedParamsAddULLong(&params, &nparams, &maxparams, + VIR_MIGRATE_PARAM_BANDWIDTH, + ullOpt) < 0) + goto save_error; + } + if (vshCommandOptBool(cmd, "live")) flags |= VIR_MIGRATE_LIVE; if (vshCommandOptBool(cmd, "p2p")) diff --git a/tools/virsh.pod b/tools/virsh.pod index af739bec24..aa0eb16f1a 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1927,6 +1927,7 @@ I<domain> I<desturi> [I<migrateuri>] [I<graphicsuri>] [I<listen-address>] [I<dna [I<auto-converge-increment>] [I<--persistent-xml> B<file>] [I<--tls>] [I<--postcopy-bandwidth> B<bandwidth>] [I<--parallel> [I<--parallel-connections> B<connections>]] +[I<--precopy-bandwidth> B<bandwidth>] Migrate domain to another host. Add I<--live> for live migration; <--p2p> for peer-2-peer migration; I<--direct> for direct migration; or I<--tunnelled> @@ -1958,7 +1959,8 @@ B<migrate-postcopy> command sent from another virsh instance or use I<--postcopy-after-precopy> along with I<--postcopy> to let libvirt automatically switch to post-copy after the first pass of pre-copy is finished. The maximum bandwidth consumed during the post-copy phase may be limited using -I<--postcopy-bandwidth>. +I<--postcopy-bandwidth>. The maximum bandwidth consumed during the pre-copy phase +may be limited using I<--precopy-bandwidth>. I<--auto-converge> forces convergence during live migration. The initial guest CPU throttling rate can be set with I<auto-converge-initial>. If the -- 2.22.0

2 1

[libvirt] [PATCH 0/9] tests: Couple of small improvements
by Michal Privoznik 27 Aug '19

27 Aug '19

*** BLURB HERE *** Michal Prívozník (9): tests: Always put a '\n' after each debug print tests: Always put '\n' at the end of VIR_TEST_VERBOSE virhashtest: Drop useless new line virhostdevtest: Check for integer retval in more verbose way virhostdevtest: Drop most of 'cleanup' and 'out' labels virhostdevtest: Drop useless VIR_TEST_DEBUG virhostdevtest: s/VIR_DEBUG/VIR_TEST_DEBUG/ virhostdevtest: Reset libvirt error on expected failure virhostdevtest: s/virReportError/fprintf/ tests/bhyveargv2xmltest.c | 6 +- tests/bhyvexml2argvtest.c | 4 +- tests/bhyvexml2xmltest.c | 2 +- tests/cputest.c | 16 +-- tests/qemublocktest.c | 10 +- tests/qemuhotplugtest.c | 10 +- tests/qemumonitorjsontest.c | 8 +- tests/qemuxml2argvtest.c | 2 +- tests/qemuxml2xmltest.c | 4 +- tests/securityselinuxlabeltest.c | 4 +- tests/storagepoolxml2argvtest.c | 8 +- tests/testutils.c | 2 +- tests/testutils.h | 8 +- tests/testutilsqemu.c | 11 +- tests/testutilsqemuschema.c | 10 +- tests/utiltest.c | 32 ++--- tests/virbuftest.c | 8 +- tests/virerrortest.c | 12 +- tests/virhashtest.c | 21 ++- tests/virhostcputest.c | 2 +- tests/virhostdevtest.c | 217 +++++++++++-------------------- tests/virjsontest.c | 80 ++++++------ tests/virlogtest.c | 14 +- tests/virnetdaemontest.c | 4 +- tests/virpcitest.c | 2 +- tests/virresctrltest.c | 2 +- 26 files changed, 221 insertions(+), 278 deletions(-) -- 2.21.0

2 20

[libvirt] [jenkins-ci PATCH] Use shallow git clones for projects
by Erik Skultety 27 Aug '19

27 Aug '19

First of all, we don't need the full history for builds and we can save some time during the prepare phase. The disk footprint is about 5x smaller, but the most important thing is the repo clone duration difference e.g. in libvirt's case is quite significant: full history: real 3m45.236s user 1m53.074s sys 0m15.577s depth 1: real 0m14.868s user 0m2.215s sys 0m0.906s --- guests/playbooks/build/jobs/prepare.yml | 1 + jenkins/jobs/autotools.yaml | 1 + jenkins/jobs/generic.yaml | 1 + jenkins/jobs/go.yaml | 1 + jenkins/jobs/perl-modulebuild.yaml | 1 + jenkins/jobs/python-distutils.yaml | 1 + 6 files changed, 6 insertions(+) diff --git a/guests/playbooks/build/jobs/prepare.yml b/guests/playbooks/build/jobs/prepare.yml index 576afbc..35ca9e4 100644 --- a/guests/playbooks/build/jobs/prepare.yml +++ b/guests/playbooks/build/jobs/prepare.yml @@ -5,6 +5,7 @@ version: '{{ git_branch }}' dest: '{{ name }}' force: yes + depth: 1 when: - inventory_hostname in machines diff --git a/jenkins/jobs/autotools.yaml b/jenkins/jobs/autotools.yaml index f04ec17..1a85781 100644 --- a/jenkins/jobs/autotools.yaml +++ b/jenkins/jobs/autotools.yaml @@ -27,6 +27,7 @@ after: true skip-tag: true wipe-workspace: false + shallow-clone: true triggers: - reverse: jobs: '{obj:parent_jobs}' diff --git a/jenkins/jobs/generic.yaml b/jenkins/jobs/generic.yaml index ca69513..f58945e 100644 --- a/jenkins/jobs/generic.yaml +++ b/jenkins/jobs/generic.yaml @@ -27,6 +27,7 @@ after: true skip-tag: true wipe-workspace: false + shallow-clone: true triggers: - reverse: jobs: '{obj:parent_jobs}' diff --git a/jenkins/jobs/go.yaml b/jenkins/jobs/go.yaml index 1bdcb5f..dd3c5d4 100644 --- a/jenkins/jobs/go.yaml +++ b/jenkins/jobs/go.yaml @@ -27,6 +27,7 @@ after: true skip-tag: true wipe-workspace: false + shallow-clone: true triggers: - reverse: jobs: '{obj:parent_jobs}' diff --git a/jenkins/jobs/perl-modulebuild.yaml b/jenkins/jobs/perl-modulebuild.yaml index 7173493..0ea271e 100644 --- a/jenkins/jobs/perl-modulebuild.yaml +++ b/jenkins/jobs/perl-modulebuild.yaml @@ -27,6 +27,7 @@ after: true skip-tag: true wipe-workspace: false + shallow-clone: true triggers: - reverse: jobs: '{obj:parent_jobs}' diff --git a/jenkins/jobs/python-distutils.yaml b/jenkins/jobs/python-distutils.yaml index f56794c..a3eb076 100644 --- a/jenkins/jobs/python-distutils.yaml +++ b/jenkins/jobs/python-distutils.yaml @@ -27,6 +27,7 @@ after: true skip-tag: true wipe-workspace: false + shallow-clone: true triggers: - reverse: jobs: '{obj:parent_jobs}' -- 2.20.1

2 3

[libvirt] [PATCH v3 00/48] Split the libvirtd daemon into per-driver daemons
by Daniel P. Berrangé 27 Aug '19

27 Aug '19

This is what all the driver refactoring I've done has been about enabling. We gain new daemons for each driver, for the primary virt drivers: virtlibxld virtlxcd virtqemud virtvboxd virtvzd And again for the secondary drivers virtinterfaced virtnetworkd virtnodedevd virtnwfilterd virtsecretd virtstoraged Finally to support IP connectivity, and also the legacy lbivirtd UNIX domain socket (for the old libvirt remote driver SSH tunnelling): virtproxyd The the sake of facilitating upgrades, the existing libvirtd still exists and works the same way it always has. You either run libvirtd, or you run the per-driver daemons, never both. The remote driver will look to see whether libvirtd is running to figure out whether to connect to libvirtd or the new per-driver daemons. When auto-spawning daemons for nonroot users, we default to spawning the per-driver daemons. This can be controlled with a UR parameter "?mode=direct|legacy|auto", where 'direct' means per-driver and 'legacy' means libvirtd (or indirect via virtproxyd if that's running). Changed in v3: - Add identity forwarding between daemons for polkit auth - Make virtxend sockets conditional on Xen kernel - Other misc review fixes Changed in v2: - Added systemd unit files for service & sockets, ensuring conflicts with libvirtd - Fixed proxy to actually handle probing of URIs (still not quite perfect) - Renamed virtlibxld to virtxend as "xen" is the user facing name of the URI - Lazy loading of secondary drivers, so connecting to virtqemud does't auto-spawn all secondary driver daemons, until a relevant API is actually invoked - Actually generated config files / augeas files per daemon - Abort daemon startup if driver fails to load - Many other fixes A few nice to have things, but not merge blocking - MAYBE make it possible to disable build of libvirtd, or of the per-driver daemons so downstream vendors can decide which to ship. Alternatively they can just not include the binary in the package file list ? - Tuning of the daemon defaults for worker threads to better suit the fact that we have per-driver daemons - More work on RPM packaging to allow install of per-driver daemosn without pulling in libvirtd too Daniel P. Berrangé (48): build: make augeas-gentest.pl write to stdout build: collapse rules adding augeas tests to CLEANFILES build: create all augeas test files in same dir as their source build: use a common rule for checking augeas test data files build: centralize rule for handling generated config files remote: stop trying to print help as giant blocks of text remote: conditionalize socket names in libvirtd daemon remote: conditionalize daemon name in libvirtd daemon remote: conditionalize driver loading in libvirtd daemon remote: conditionalize IP socket usage in libvirtd daemon build: use @CONFIG@ instead of ::CONFIG:: in augeas tests remote: conditionalize IP socket config in libvirtd.conf remote: conditionalize IP socket config in augeas definitions remote: refactor & rename variables for building libvirtd build: don't hardcode /etc in the config related files remote: reduce duplication in systemd unit file make rules into one remote: conditionalize systemd socket unit files remote: refactor how list of systemd unit files is built remote: in per-driver daemons ensure that state initialize succeeds remote: introduce virtproxyd daemon to handle IP connectivity secret: introduce virtsecretd daemon network: introduce virtnetworkd daemon interface: introduce virtinterfaced daemon storage: introduce virtstoraged daemon nodedev: introduce virtnodedevd daemon nwfilter: introduce virtnwfilterd daemon libxl: introduce virtxend daemon qemu: introduce virtqemud daemon lxc: introduce virtlxcd daemon vbox: introduce virtvboxd daemon bhyve: introduce virtbhyved daemon vz: introduce virtvzd daemon admin: add ability to connect to the per-driver daemon sockets remote: get rid of bogus ATTRIBUTE_UNUSED annotation client param remote: change generated methods to not directly access connection remote: fix lock ordering mistake in event registration remote: change hand written methods to not directly access connection remote: open secondary drivers via remote driver if needed remote: handle autoprobing of driver within virtproxyd remote: use enum helpers for parsing remote driver transport remote: refactor the code for choosing the UNIX socket path remote: enable connecting to the per-driver daemons api: introduce virConnectSetIdentity for pasing uid, gid, selinux info util: change identity class attribute names util: make generic identity accessors private util: storage identity attrs as virTypedParameter internally util: allow identity to be imported/exported as typed parameters remote: pass identity across to newly opened daemons .gitignore | 62 +- build-aux/augeas-gentest.pl | 22 +- docs/remote.html.in | 18 + include/libvirt/libvirt-host.h | 75 + libvirt.spec.in | 91 ++ m4/virt-driver-remote.m4 | 15 + src/Makefile.am | 30 +- src/access/viraccessdriverpolkit.c | 12 +- src/admin/admin_server.c | 10 +- src/admin/admin_server_dispatch.c | 9 + src/bhyve/Makefile.inc.am | 55 +- src/bhyve/bhyve_driver.c | 10 +- src/bhyve/test_libvirtd_bhyve.aug.in | 2 +- src/driver-hypervisor.h | 7 + src/driver-state.h | 8 +- src/driver.h | 2 + src/interface/Makefile.inc.am | 62 + src/interface/interface_backend_netcf.c | 8 +- src/interface/interface_backend_udev.c | 4 +- src/interface/virtinterfaced.service.in | 24 + src/libvirt-admin.c | 32 +- src/libvirt-host.c | 51 + src/libvirt.c | 42 +- src/libvirt_internal.h | 1 + src/libvirt_private.syms | 29 +- src/libvirt_public.syms | 1 + src/libvirt_remote.syms | 1 + src/libxl/Makefile.inc.am | 80 +- src/libxl/libxl_driver.c | 10 +- src/libxl/test_libvirtd_libxl.aug.in | 2 +- src/libxl/virtxend.service.in | 26 + src/locking/Makefile.inc.am | 77 +- src/locking/test_libvirt_lockd.aug.in | 2 +- src/locking/test_libvirt_sanlock.aug.in | 2 +- src/locking/test_virtlockd.aug.in | 2 +- src/locking/virtlockd.service.in | 2 +- src/logging/Makefile.inc.am | 23 +- src/logging/test_virtlogd.aug.in | 2 +- src/logging/virtlogd.service.in | 2 +- src/lxc/Makefile.inc.am | 77 +- src/lxc/lxc_driver.c | 12 +- src/lxc/test_libvirtd_lxc.aug.in | 2 +- src/lxc/virtlxcd.service.in | 40 + src/network/Makefile.inc.am | 61 + src/network/bridge_driver.c | 4 +- src/network/virtnetworkd.service.in | 25 + src/node_device/Makefile.inc.am | 62 + src/node_device/node_device_hal.c | 12 +- src/node_device/node_device_udev.c | 8 +- src/node_device/virtnodedevd.service.in | 24 + src/nwfilter/Makefile.inc.am | 62 + src/nwfilter/nwfilter_driver.c | 12 +- src/nwfilter/virtnwfilterd.service.in | 24 + src/qemu/Makefile.inc.am | 76 +- src/qemu/qemu_driver.c | 8 +- src/qemu/test_libvirtd_qemu.aug.in | 2 +- src/qemu/virtqemud.service.in | 40 + src/remote/Makefile.inc.am | 285 ++-- src/remote/libvirtd-admin.socket.in | 15 +- src/remote/libvirtd-ro.socket.in | 15 +- src/remote/libvirtd-tcp.socket.in | 13 +- src/remote/libvirtd-tls.socket.in | 13 +- src/remote/{libvirtd.aug => libvirtd.aug.in} | 26 +- .../{libvirtd.conf => libvirtd.conf.in} | 60 +- src/remote/libvirtd.service.in | 2 +- src/remote/libvirtd.socket.in | 11 +- src/remote/remote_daemon.c | 302 ++-- src/remote/remote_daemon.h | 13 + src/remote/remote_daemon_config.c | 47 +- src/remote/remote_daemon_config.h | 10 +- src/remote/remote_daemon_dispatch.c | 1354 ++++++++++------- src/remote/remote_driver.c | 424 ++++-- src/remote/remote_driver.h | 4 - src/remote/remote_protocol.x | 18 +- src/remote/test_libvirtd.aug.in | 24 +- src/remote/virtproxyd.service.in | 24 + src/remote_protocol-structs | 8 + src/rpc/gendispatch.pl | 96 +- src/rpc/virnetserverclient.c | 24 +- src/rpc/virnetserverclient.h | 2 + src/secret/Makefile.inc.am | 62 + src/secret/secret_driver.c | 8 +- src/secret/virtsecretd.service.in | 24 + src/storage/Makefile.inc.am | 61 + src/storage/storage_driver.c | 8 +- src/storage/virtstoraged.service.in | 26 + src/util/viridentity.c | 483 +++--- src/util/viridentity.h | 87 +- src/vbox/Makefile.inc.am | 62 + src/vbox/virtvboxd.service.in | 25 + src/vz/Makefile.inc.am | 62 + src/vz/virtvzd.service.in | 25 + src/vz/vz_driver.c | 14 +- tests/viridentitytest.c | 97 +- tests/virnetserverclienttest.c | 45 +- tools/libvirt-guests.service.in | 2 +- 96 files changed, 3642 insertions(+), 1703 deletions(-) create mode 100644 src/interface/virtinterfaced.service.in create mode 100644 src/libxl/virtxend.service.in create mode 100644 src/lxc/virtlxcd.service.in create mode 100644 src/network/virtnetworkd.service.in create mode 100644 src/node_device/virtnodedevd.service.in create mode 100644 src/nwfilter/virtnwfilterd.service.in create mode 100644 src/qemu/virtqemud.service.in rename src/remote/{libvirtd.aug => libvirtd.aug.in} (89%) rename src/remote/{libvirtd.conf => libvirtd.conf.in} (92%) create mode 100644 src/remote/virtproxyd.service.in create mode 100644 src/secret/virtsecretd.service.in create mode 100644 src/storage/virtstoraged.service.in create mode 100644 src/vbox/virtvboxd.service.in create mode 100644 src/vz/virtvzd.service.in -- 2.21.0

5 118