[libvirt] [PATCH 0/4] Don't destroy domains on daemon restart and autostarted pools

The reverts were not clean. A lot has changed since v1.1.1. Michal Prívozník (4): daemon: Register secret driver before storage driver bhyve: Move autostarting of domains into bhyveStateInitialize Revert "virStateDriver - Separate AutoStart from Initialize" Revert "Separate out StateAutoStart from StateInitialize" src/bhyve/bhyve_driver.c | 12 ++---------- src/driver-state.h | 4 ---- src/libvirt.c | 14 +------------- src/libxl/libxl_driver.c | 14 +++----------- src/lxc/lxc_driver.c | 28 ++++++++-------------------- src/network/bridge_driver.c | 22 ++++------------------ src/qemu/qemu_driver.c | 17 ++--------------- src/remote/remote_daemon.c | 8 ++++---- src/storage/storage_driver.c | 19 ++----------------- 9 files changed, 26 insertions(+), 112 deletions(-) -- 2.19.2

The order in which drivers are registered is important because their stateInitialize and stateAutoStart callback are called in that order. Well, stateAutoStart is going away and therefore if there is some dependency between two drivers (e.g. when initializing storage driver expects secret driver to be available already), the registration of such drivers must happen in correct order. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/remote/remote_daemon.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/remote/remote_daemon.c b/src/remote/remote_daemon.c index 54ddfc42a0..61eabe03be 100644 --- a/src/remote/remote_daemon.c +++ b/src/remote/remote_daemon.c @@ -312,6 +312,10 @@ static int daemonInitialize(void) if (virDriverLoadModule("interface", "interfaceRegister", false) < 0) return -1; #endif +#ifdef WITH_SECRETS + if (virDriverLoadModule("secret", "secretRegister", false) < 0) + return -1; +#endif #ifdef WITH_STORAGE if (virDriverLoadModule("storage", "storageRegister", false) < 0) return -1; @@ -320,10 +324,6 @@ static int daemonInitialize(void) if (virDriverLoadModule("nodedev", "nodedevRegister", false) < 0) return -1; #endif -#ifdef WITH_SECRETS - if (virDriverLoadModule("secret", "secretRegister", false) < 0) - return -1; -#endif #ifdef WITH_NWFILTER if (virDriverLoadModule("nwfilter", "nwfilterRegister", false) < 0) return -1; -- 2.19.2

On Mon, Mar 04, 2019 at 03:02:10PM +0100, Michal Privoznik wrote:
The order in which drivers are registered is important because their stateInitialize and stateAutoStart callback are called in that order. Well, stateAutoStart is going away and therefore if there is some dependency between two drivers (e.g. when initializing storage driver expects secret driver to be available already), the registration of such drivers must happen in correct order.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/remote/remote_daemon.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano

The stateAutoStart callback will go away shortly. Therefore, move the autostart call into state initialize callback. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/bhyve/bhyve_driver.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/src/bhyve/bhyve_driver.c b/src/bhyve/bhyve_driver.c index 3e192284cc..061888ab0b 100644 --- a/src/bhyve/bhyve_driver.c +++ b/src/bhyve/bhyve_driver.c @@ -1269,6 +1269,8 @@ bhyveStateInitialize(bool privileged, virBhyveProcessReconnectAll(bhyve_driver); + bhyveAutostartDomains(bhyve_driver); + return 0; cleanup: @@ -1296,15 +1298,6 @@ bhyveDriverGetGrubCaps(virConnectPtr conn) return 0; } -static void -bhyveStateAutoStart(void) -{ - if (!bhyve_driver) - return; - - bhyveAutostartDomains(bhyve_driver); -} - static int bhyveConnectGetMaxVcpus(virConnectPtr conn, const char *type) @@ -1712,7 +1705,6 @@ static virConnectDriver bhyveConnectDriver = { static virStateDriver bhyveStateDriver = { .name = "bhyve", .stateInitialize = bhyveStateInitialize, - .stateAutoStart = bhyveStateAutoStart, .stateCleanup = bhyveStateCleanup, }; -- 2.19.2

On Mon, Mar 04, 2019 at 03:02:11PM +0100, Michal Privoznik wrote:
The stateAutoStart callback will go away shortly. Therefore, move the autostart call into state initialize callback.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/bhyve/bhyve_driver.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano

https://bugzilla.redhat.com/show_bug.cgi?id=1685151 This reverts commit cefb97fb815c81fc882da752f45effd23bcb9b4b. The stateAutoStart callback will be removed in the next commit. Therefore move autostarting of domains, networks and storage pools back into stateInitialize callbacks. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/libxl/libxl_driver.c | 14 +++----------- src/lxc/lxc_driver.c | 28 ++++++++-------------------- src/network/bridge_driver.c | 22 ++++------------------ src/qemu/qemu_driver.c | 17 ++--------------- src/storage/storage_driver.c | 19 ++----------------- 5 files changed, 19 insertions(+), 81 deletions(-) diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 31b842aeee..a9edc8211d 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -779,6 +779,9 @@ libxlStateInitialize(bool privileged, NULL, NULL) < 0) goto error; + virDomainObjListForEach(libxl_driver->domains, libxlAutostartDomain, + libxl_driver); + virDomainObjListForEach(libxl_driver->domains, libxlDomainManagedSaveLoad, libxl_driver); @@ -790,16 +793,6 @@ libxlStateInitialize(bool privileged, return -1; } -static void -libxlStateAutoStart(void) -{ - if (!libxl_driver) - return; - - virDomainObjListForEach(libxl_driver->domains, libxlAutostartDomain, - libxl_driver); -} - static int libxlStateReload(void) { @@ -6611,7 +6604,6 @@ static virConnectDriver libxlConnectDriver = { static virStateDriver libxlStateDriver = { .name = "LIBXL", .stateInitialize = libxlStateInitialize, - .stateAutoStart = libxlStateAutoStart, .stateCleanup = libxlStateCleanup, .stateReload = libxlStateReload, }; diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 516a6b4de3..b1ef221c5c 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -1632,26 +1632,15 @@ static int lxcStateInitialize(bool privileged, NULL, NULL) < 0) goto cleanup; - virObjectUnref(caps); - return 0; - - cleanup: - virObjectUnref(caps); - lxcStateCleanup(); - return -1; -} - -/** - * lxcStateAutoStart: - * - * Function to autostart the LXC daemons - */ -static void lxcStateAutoStart(void) -{ - if (!lxc_driver) - return; - virLXCProcessAutostartAll(lxc_driver); + + virObjectUnref(caps); + return 0; + + cleanup: + virObjectUnref(caps); + lxcStateCleanup(); + return -1; } static void lxcNotifyLoadDomain(virDomainObjPtr vm, int newVM, void *opaque) @@ -5502,7 +5491,6 @@ static virConnectDriver lxcConnectDriver = { static virStateDriver lxcStateDriver = { .name = LXC_DRIVER_NAME, .stateInitialize = lxcStateInitialize, - .stateAutoStart = lxcStateAutoStart, .stateCleanup = lxcStateCleanup, .stateReload = lxcStateReload, }; diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index b3ca5b8a15..c3e1381124 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -660,6 +660,10 @@ networkStateInitialize(bool privileged, networkReloadFirewallRules(network_driver, true); networkRefreshDaemons(network_driver); + virNetworkObjListForEach(network_driver->networks, + networkAutostartConfig, + network_driver); + network_driver->networkEventState = virObjectEventStateNew(); #ifdef WITH_FIREWALLD @@ -699,23 +703,6 @@ networkStateInitialize(bool privileged, } -/** - * networkStateAutoStart: - * - * Function to AutoStart the bridge configs - */ -static void -networkStateAutoStart(void) -{ - if (!network_driver) - return; - - virNetworkObjListForEach(network_driver->networks, - networkAutostartConfig, - network_driver); -} - - /** * networkStateReload: * @@ -5652,7 +5639,6 @@ static virConnectDriver networkConnectDriver = { static virStateDriver networkStateDriver = { .name = "bridge", .stateInitialize = networkStateInitialize, - .stateAutoStart = networkStateAutoStart, .stateCleanup = networkStateCleanup, .stateReload = networkStateReload, }; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 36426cd65a..043cb817d2 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -914,6 +914,8 @@ qemuStateInitialize(bool privileged, qemuProcessReconnectAll(qemu_driver); + qemuAutostartDomains(qemu_driver); + return 0; error: @@ -924,20 +926,6 @@ qemuStateInitialize(bool privileged, return -1; } -/** - * qemuStateAutoStart: - * - * Function to auto start the QEMU daemons - */ -static void -qemuStateAutoStart(void) -{ - if (!qemu_driver) - return; - - qemuAutostartDomains(qemu_driver); -} - static void qemuNotifyLoadDomain(virDomainObjPtr vm, int newVM, void *opaque) { virQEMUDriverPtr driver = opaque; @@ -22672,7 +22660,6 @@ static virConnectDriver qemuConnectDriver = { static virStateDriver qemuStateDriver = { .name = QEMU_DRIVER_NAME, .stateInitialize = qemuStateInitialize, - .stateAutoStart = qemuStateAutoStart, .stateCleanup = qemuStateCleanup, .stateReload = qemuStateReload, .stateStop = qemuStateStop, diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index 34634e97d9..98be434005 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -294,6 +294,8 @@ storageStateInitialize(bool privileged, storagePoolUpdateAllState(); + storageDriverAutostart(); + driver->storageEventState = virObjectEventStateNew(); storageDriverUnlock(); @@ -306,22 +308,6 @@ storageStateInitialize(bool privileged, return -1; } -/** - * storageStateAutoStart: - * - * Function to auto start the storage driver - */ -static void -storageStateAutoStart(void) -{ - if (!driver) - return; - - storageDriverLock(); - storageDriverAutostart(); - storageDriverUnlock(); -} - /** * storageStateReload: * @@ -2832,7 +2818,6 @@ static virConnectDriver storageConnectDriver = { static virStateDriver stateDriver = { .name = "storage", .stateInitialize = storageStateInitialize, - .stateAutoStart = storageStateAutoStart, .stateCleanup = storageStateCleanup, .stateReload = storageStateReload, }; -- 2.19.2

On Mon, Mar 04, 2019 at 03:02:12PM +0100, Michal Privoznik wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1685151
This reverts commit cefb97fb815c81fc882da752f45effd23bcb9b4b.
The stateAutoStart callback will be removed in the next commit. Therefore move autostarting of domains, networks and storage pools back into stateInitialize callbacks.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/libxl/libxl_driver.c | 14 +++----------- src/lxc/lxc_driver.c | 28 ++++++++-------------------- src/network/bridge_driver.c | 22 ++++------------------ src/qemu/qemu_driver.c | 17 ++--------------- src/storage/storage_driver.c | 19 ++----------------- 5 files changed, 19 insertions(+), 81 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano

https://bugzilla.redhat.com/show_bug.cgi?id=1685151 This reverts commit e4a969092bda5b3b952963fdf6658895165040b7. Now that drivers may call virConnectOpen() on secondary drivers, it doesn't make much sense to have autostart separated from driver initialization callback. In fact, it creates a problem because one driver during its initialization might try to fetch an object from another driver but since the object is yet to be autostarted the fetch fails. This has been observed in reality: qemu driver performs qemuProcessReconnect() during qemu's stateInitialize phase which may call virDomainDiskTranslateSourcePool() which connects to the storage driver to look up the volume. But the storage driver did not autostarted its pools yet therefore volume lookup fails and the domain is killed. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/driver-state.h | 4 ---- src/libvirt.c | 14 +------------- 2 files changed, 1 insertion(+), 17 deletions(-) diff --git a/src/driver-state.h b/src/driver-state.h index 54ca17b260..a8595662af 100644 --- a/src/driver-state.h +++ b/src/driver-state.h @@ -30,9 +30,6 @@ typedef int virStateInhibitCallback callback, void *opaque); -typedef void -(*virDrvStateAutoStart)(void); - typedef int (*virDrvStateCleanup)(void); @@ -48,7 +45,6 @@ typedef virStateDriver *virStateDriverPtr; struct _virStateDriver { const char *name; virDrvStateInitialize stateInitialize; - virDrvStateAutoStart stateAutoStart; virDrvStateCleanup stateCleanup; virDrvStateReload stateReload; virDrvStateStop stateStop; diff --git a/src/libvirt.c b/src/libvirt.c index 854d70a2bf..7e665b6cba 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -632,11 +632,7 @@ virRegisterStateDriver(virStateDriverPtr driver) * @callback: callback to invoke to inhibit shutdown of the daemon * @opaque: data to pass to @callback * - * Initialize all virtualization drivers. Accomplished in two phases, - * the first being state and structure initialization followed by any - * auto start supported by the driver. This is done to ensure dependencies - * that some drivers may have on another driver having been initialized - * will exist, such as the storage driver's need to use the secret driver. + * Initialize all virtualization drivers. * * Returns 0 if all succeed, -1 upon any failure. */ @@ -664,14 +660,6 @@ virStateInitialize(bool privileged, } } } - - for (i = 0; i < virStateDriverTabCount; i++) { - if (virStateDriverTab[i]->stateAutoStart) { - VIR_DEBUG("Running global auto start for %s state driver", - virStateDriverTab[i]->name); - virStateDriverTab[i]->stateAutoStart(); - } - } return 0; } -- 2.19.2

On Mon, Mar 04, 2019 at 03:02:13PM +0100, Michal Privoznik wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1685151
This reverts commit e4a969092bda5b3b952963fdf6658895165040b7.
Now that drivers may call virConnectOpen() on secondary drivers, it doesn't make much sense to have autostart separated from driver initialization callback. In fact, it creates a problem because one driver during its initialization might try to fetch an object from another driver but since the object is yet to be autostarted the fetch fails. This has been observed in reality: qemu driver performs qemuProcessReconnect() during qemu's stateInitialize phase which may call virDomainDiskTranslateSourcePool() which connects to the storage driver to look up the volume. But the storage driver did not autostarted its pools yet therefore volume lookup fails and
s/autostarted/autostart/
the domain is killed.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/driver-state.h | 4 ---- src/libvirt.c | 14 +------------- 2 files changed, 1 insertion(+), 17 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano
participants (2)
-
Ján Tomko
-
Michal Privoznik