[libvirt] [PATCH 00/41] Split the libvirtd daemon into per-driver daemons

This is what all the driver refactoring I've done has been about enabling. We gain new daemons for each driver, for the primary virt drivers: virtlibxld virtlxcd virtqemud virtvboxd virtvzd And again for the secondary drivers virtinterfaced virtnetworkd virtnodedevd virtnwfilterd virtsecretd virtstoraged Finally to support IP connectivity, and also the legacy lbivirtd UNIX domain socket (for the old libvirt remote driver SSH tunnelling): virtproxyd The the sake of facilitating upgrades, the existing libvirtd still exists and works the same way it always has. You either run libvirtd, or you run the per-driver daemons, never both. The remote driver will look to see whether libvirtd is running to figure out whether to connect to libvirtd or the new per-driver daemons. When auto-spawning daemons for nonroot users, we default to spawning the per-driver daemons. This can be controlled with a UR parameter "?mode=direct|legacy|auto", where 'direct' means per-driver and 'legacy' means libvirtd (or indirect via virtproxyd if that's running). Changed in v2: - Added systemd unit files for service & sockets, ensuring conflicts with libvirtd - Fixed proxy to actually handle probing of URIs (still not quite perfect) - Renamed virtlibxld to virtxend as "xen" is the user facing name of the URI - Lazy loading of secondary drivers, so connecting to virtqemud does't auto-spawn all secondary driver daemons, until a relevant API is actually invoked - Actually generated config files / augeas files per daemon - Abort daemon startup if driver fails to load - Many other fixes Still a few things to do.... Important: - Tweak virtxend to not start unless on a xen host - Identity forwarding so that when connecting to virtproxyd the apps identity is sent onto virtqemud. Without this, virtqemud would see the client as "root" and so polkit would allow all access. Opps Nice to have - MAYBE make it possible to disable build of libvirtd, or of the per-driver daemons so downstream vendors can decide which to ship. Alternatively they can just not include the binary in the package file list ? - Tuning of the daemon defaults for worker threads to better suit the fact that we have per-driver daemons - More work on RPM packaging to allow install of per-driver daemosn without pulling in libvirtd too - Hopefully not much else. Daniel P. Berrangé (41): build: make augeas-gentest.pl write to stdout build: create all augeas test files in same dir as their source build: collapse rules adding augeas tests to CLEANFILES build: use a common rule for checking augeas test data files build: centralize rule for handling generated config files remote: stop trying to print help as giant blocks of text remote: conditionalize socket names in libvirtd daemon remote: conditionalize daemon name in libvirtd daemon remote: conditionalize driver loading in libvirtd daemon remote: conditionalize IP socket usage in libvirtd daemon remote: conditionalize IP socket config in libvirtd.conf remote: conditionalize IP socket config in augeas definitions remote: refactor & rename variables for building libvirtd remote: don't hardcode /etc in the systemd units remote: reduce duplication in systemd unit file make rules into one remote: conditionalize systemd socket unit files remote: refactor how list of systemd unit files is built remote: in per-driver daemons ensure that state initialize succeeds remote: introduce virtproxyd daemon to handle IP connectivity secret: introduce virtsecretd daemon network: introduce virtnetworkd daemon interface: introduce virtinterfaced daemon storage: introduce virtstoraged daemon nodedev: introduce virtnodedevd daemon nwfilter: introduce virtnwfilterd daemon libxl: introduce virtxend daemon qemu: introduce virtqemud daemon lxc: introduce virtlxcd daemon vbox: introduce virtvboxd daemon bhyve: introduce virtbhyved daemon vz: introduce virtvzd daemon admin: add ability to connect to the per-driver daemon sockets remote: get rid of bogus ATTRIBUTE_UNUSED annotation client param remote: change generated methods to not directly access connection remote: change hand written methods to not directly access connection remote: open secondary drivers via remote driver if needed remote: handle autoprobing of driver within virtproxyd remote: use enum helpers for parsing remote driver transport remote: refactor the code for choosing the UNIX socket path remote: switch to connect to per-driver daemons by default libvirt: correctly print out URI returned from probing .gitignore | 62 +- build-aux/augeas-gentest.pl | 22 +- libvirt.spec.in | 91 ++ m4/virt-driver-remote.m4 | 15 + src/Makefile.am | 26 +- src/admin/admin_server_dispatch.c | 9 + src/bhyve/Makefile.inc.am | 55 +- src/bhyve/bhyve_driver.c | 10 +- src/driver-state.h | 8 +- src/driver.h | 2 + src/interface/Makefile.inc.am | 62 + src/interface/interface_backend_netcf.c | 8 +- src/interface/interface_backend_udev.c | 4 +- src/interface/virtinterfaced.service.in | 24 + src/libvirt-admin.c | 32 +- src/libvirt.c | 42 +- src/libvirt_internal.h | 1 + src/libxl/Makefile.inc.am | 79 +- src/libxl/libxl_driver.c | 10 +- src/libxl/virtxend.service.in | 24 + src/locking/Makefile.inc.am | 76 +- src/logging/Makefile.inc.am | 22 +- src/lxc/Makefile.inc.am | 77 +- src/lxc/lxc_driver.c | 12 +- src/lxc/virtlxcd.service.in | 24 + src/network/Makefile.inc.am | 61 + src/network/bridge_driver.c | 4 +- src/network/virtnetworkd.service.in | 24 + src/node_device/Makefile.inc.am | 62 + src/node_device/node_device_hal.c | 12 +- src/node_device/node_device_udev.c | 8 +- src/node_device/virtnodedevd.service.in | 24 + src/nwfilter/Makefile.inc.am | 62 + src/nwfilter/nwfilter_driver.c | 12 +- src/nwfilter/virtnwfilterd.service.in | 24 + src/qemu/Makefile.inc.am | 76 +- src/qemu/qemu_driver.c | 8 +- src/qemu/virtqemud.service.in | 24 + src/remote/Makefile.inc.am | 281 ++-- src/remote/libvirtd-admin.socket.in | 15 +- src/remote/libvirtd-ro.socket.in | 15 +- src/remote/libvirtd-tcp.socket.in | 13 +- src/remote/libvirtd-tls.socket.in | 13 +- src/remote/{libvirtd.aug => libvirtd.aug.in} | 26 +- .../{libvirtd.conf => libvirtd.conf.in} | 42 +- src/remote/libvirtd.service.in | 2 +- src/remote/libvirtd.socket.in | 11 +- src/remote/remote_daemon.c | 291 ++-- src/remote/remote_daemon.h | 13 + src/remote/remote_daemon_config.c | 47 +- src/remote/remote_daemon_config.h | 9 +- src/remote/remote_daemon_dispatch.c | 1205 +++++++++-------- src/remote/remote_driver.c | 396 ++++-- src/remote/remote_driver.h | 4 - src/remote/test_libvirtd.aug.in | 16 +- src/remote/virtproxyd.service.in | 24 + src/rpc/gendispatch.pl | 96 +- src/secret/Makefile.inc.am | 62 + src/secret/secret_driver.c | 8 +- src/secret/virtsecretd.service.in | 24 + src/storage/Makefile.inc.am | 61 + src/storage/storage_driver.c | 8 +- src/storage/virtstoraged.service.in | 24 + src/vbox/Makefile.inc.am | 62 + src/vbox/virtvboxd.service.in | 24 + src/vz/Makefile.inc.am | 62 + src/vz/virtvzd.service.in | 24 + src/vz/vz_driver.c | 14 +- 68 files changed, 2832 insertions(+), 1258 deletions(-) create mode 100644 src/interface/virtinterfaced.service.in create mode 100644 src/libxl/virtxend.service.in create mode 100644 src/lxc/virtlxcd.service.in create mode 100644 src/network/virtnetworkd.service.in create mode 100644 src/node_device/virtnodedevd.service.in create mode 100644 src/nwfilter/virtnwfilterd.service.in create mode 100644 src/qemu/virtqemud.service.in rename src/remote/{libvirtd.aug => libvirtd.aug.in} (89%) rename src/remote/{libvirtd.conf => libvirtd.conf.in} (95%) create mode 100644 src/remote/virtproxyd.service.in create mode 100644 src/secret/virtsecretd.service.in create mode 100644 src/storage/virtstoraged.service.in create mode 100644 src/vbox/virtvboxd.service.in create mode 100644 src/vz/virtvzd.service.in -- 2.21.0