On 03/14/2018 06:53 AM, John Ferlan wrote:

On 03/13/2018 01:26 PM, Jim Fehlig wrote:

...

The libxlDomainMigrateConfirm3Params API locks and ref counts the associated virDomainObj but relies on the helper function libxlDomainMigrationConfirm to unlock the object. Unref'ing the object is not done in either function. libxlDomainMigrationConfirm is also used by libxlDomainMigratePerform3Params for p2p migration, but in that case the lock/ref and unref/unlock are properly handled in the API entry point.

Remove the unlock from libxlDomainMigrationConfirm and adjust libxlDomainMigrateConfirm3Params to properly unref/unlock the virDomainObj on success and error paths.

Signed-off-by: Jim Fehlig <jfehlig@suse.com> --- src/libxl/libxl_driver.c | 13 ++++++++----- src/libxl/libxl_migration.c | 2 -- 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index eff13e5aa..67a638da0 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -6161,6 +6161,7 @@ libxlDomainMigrateConfirm3Params(virDomainPtr domain, { libxlDriverPrivatePtr driver = domain->conn->privateData; virDomainObjPtr vm = NULL; + int ret = -1;

#ifdef LIBXL_HAVE_NO_SUSPEND_RESUME virReportUnsupportedError(); @@ -6174,12 +6175,14 @@ libxlDomainMigrateConfirm3Params(virDomainPtr domain, if (!(vm = libxlDomObjFromDomain(domain))) return -1;

- if (virDomainMigrateConfirm3ParamsEnsureACL(domain->conn, vm->def) < 0) { - virObjectUnlock(vm); - return -1; - } + if (virDomainMigrateConfirm3ParamsEnsureACL(domain->conn, vm->def) < 0) + goto cleanup;

- return libxlDomainMigrationConfirm(driver, vm, flags, cancelled); + ret = libxlDomainMigrationConfirm(driver, vm, flags, cancelled); + + cleanup: + virDomainObjEndAPI(&vm); + return ret; }

static int libxlNodeGetSecurityModel(virConnectPtr conn, diff --git a/src/libxl/libxl_migration.c b/src/libxl/libxl_migration.c index 4b848c920..fef1c998b 100644 --- a/src/libxl/libxl_migration.c +++ b/src/libxl/libxl_migration.c @@ -1400,8 +1400,6 @@ libxlDomainMigrationConfirm(libxlDriverPrivatePtr driver,

Above here there is a possible call to virDomainObjListRemove which would return @vm w/ at least 1 ref and unlocked. The reason it's "at least 1 ref" is because some call paths in libxl will call virDomainObjListAdd *and* then perform a virObjectRef(vm) while others don't do that ref. Return from ListAdd will have 2 refs and 1 lock on @vm (although it should be 3, hence my other series). It's not necessarily crystal clear which ListAdd was used in this path...

In any case, "for now" I think we're mostly OK here because at least @vm was obtained via libxlDomObjFromDomain which will return @vm w/ 1 extra ref and locked (e.g. 3 refs and locked). So calling virDomainObjListRemove will remove 2 refs and the lock.

All it means is the EndAPI call done will call Unlock even though it doesn't have the lock. Of course there's no error propagation, so it doesn't hurt. On the upside, the changes will at least allow @vm to be free'd once the EndAPI is called (when @vm refcnt == 0) as opposed to the changes before here where refcnt was never lowered and @vm was probably leaked once/if it was removed from the list.

I think what could be done is - after calling ListRemove, you could call virObjectLock(vm) and remove the vm = NULL. That way you know you "leave" this function with at least 1 ref and @vm being locked regardless of what happens. That way the caller using EndAPI will do the right thing too.

Thanks for the details. I agree with your suggestion and made the changes before pushing, along with adding a comment. E.g. virDomainObjListRemove(driver->domains, vm); /* Caller passed a locked vm and expects the same on return */ virObjectLock(vm);

Hopefully this makes sense, I have various stages of this ListAdd problem percolating inside my head...

Yes it does, and has helped me get my head around locking and ref counting the virDomainObj. It's also encouraged me to audit the libxl code for other locking and ref counting bugs. Regards, Jim

On 03/13/2018 01:26 PM, Jim Fehlig wrote:

libxlDomainMigrationPrepare adds the incoming domain def to the list of domains via virDomainObjListAdd, which returns a locked and ref counted virDomainObj. On exit the object is unlocked but not unref'ed. The same is true for libxlDomainMigrationPrepareTunnel3. Convert both to use the virDomainObjEndAPI function for cleanup.

Signed-off-by: Jim Fehlig <jfehlig@suse.com> --- src/libxl/libxl_migration.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)

These two leave me concerned - mainly because as I described in my review of 2/4 the ListAdd function will return 2 refs and 1 lock on the object unlike the libxlDomObjFromDomain where there are at least 3 refs and 1 lock. Since neither of these code paths adds a Ref(vm) after the ListAdd call (like CreateXML and RestoreFlags do) that means calling EndAPI will remove 1 ref and the lock leaving only 1 ref on the object. Later on when ListRemove is called it would enter with 2 refs and 1 lock and leave with @vm being destroyed. Not having a clear picture of all the paths a @vm could have in libxl makes me concerned because there are a few places where ListRemove is called *and* @vm is referenced afterwards such as libxlDomainDestroyFlags I think once ListAdd returns with the right number of refs, then yes, this is the proper adjustment, but for now unless there's an extra Ref done after the ListAdd, then we should leave things as is. Thoughts? And does this make sense? John

diff --git a/src/libxl/libxl_migration.c b/src/libxl/libxl_migration.c index fef1c998b..6b53f9587 100644 --- a/src/libxl/libxl_migration.c +++ b/src/libxl/libxl_migration.c @@ -639,9 +639,7 @@ libxlDomainMigrationPrepareTunnel3(virConnectPtr dconn, }

done: - if (vm) - virObjectUnlock(vm); - + virDomainObjEndAPI(&vm); return ret; }

@@ -820,8 +818,7 @@ libxlDomainMigrationPrepare(virConnectPtr dconn, VIR_FREE(hostname); else virURIFree(uri); - if (vm) - virObjectUnlock(vm); + virDomainObjEndAPI(&vm); virObjectUnref(cfg); return ret; }