Refactor the code to handle the NETFS pool separately from other pool types. When the original code was developed (commit id 'e1f27784') the virCommandSetUmask did not exist (commit id '0e1a1a8c4'), so there was no way to set the permissions for the creation. Now that it exists, we can use it. The code currently fails to create a volume in a NETFS pool from some other volume within the pool because the chmod done after file creation fails. So adjust the code to jump to cleanup once the file is successfully created in the NETFS pool. If it fails or for non NETFS files, just perform the create, chown, and chmod in order Signed-off-by: John Ferlan <jferlan(a)redhat.com&gt; --- src/storage/storage_backend.c | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/src/storage/storage_backend.c b/src/storage/storage_backend.c index a375fe0..037d6d7 100644 --- a/src/storage/storage_backend.c +++ b/src/storage/storage_backend.c @@ -678,7 +678,6 @@ virStorageBackendCreateExecCommand(virStoragePoolObjPtr pool, gid_t gid; uid_t uid; mode_t mode; - bool filecreated = false; int ret = -1; if ((pool->def->type == VIR_STORAGE_POOL_NETFS) @@ -690,26 +689,29 @@ virStorageBackendCreateExecCommand(virStoragePoolObjPtr pool, virCommandSetUID(cmd, vol->target.perms->uid); virCommandSetGID(cmd, vol->target.perms->gid); + mode = (vol->target.perms->mode == (mode_t) -1 ? + VIR_STORAGE_DEFAULT_VOL_PERM_MODE : vol->target.perms->mode); + virCommandSetUmask(cmd, 0777 - mode); - if (virCommandRun(cmd, NULL) == 0) { - /* command was successfully run, check if the file was created */ - if (stat(vol->target.path, &st) >= 0) - filecreated = true; + if ((ret = virCommandRun(cmd, NULL)) == 0) { + /* command was successfully run, if the file was created + * then we are done */ + if (virFileExists(vol->target.path)) + goto cleanup; } } - /* don't change uid/gid if we retry */ + /* don't change uid/gid/mode if we retry */ virCommandSetUID(cmd, -1); virCommandSetGID(cmd, -1); + virCommandSetUmask(cmd, 0); - if (!filecreated) { - if (virCommandRun(cmd, NULL) < 0) - goto cleanup; - if (stat(vol->target.path, &st) < 0) { - virReportSystemError(errno, - _("failed to create %s"), vol->target.path); - goto cleanup; - } + if (virCommandRun(cmd, NULL) < 0) + goto cleanup; + if (stat(vol->target.path, &st) < 0) { + virReportSystemError(errno, + _("failed to create %s"), vol->target.path); + goto cleanup; } uid = (vol->target.perms->uid != st.st_uid) ? vol->target.perms->uid -- 2.1.0

On 11/03/2015 04:47 AM, Ján Tomko wrote: The NETFS root-squash create file code will try both - which is what this follows. Look at the code prior to my changes - if the virCommandRun and the stat() inside the NETFS succeeds, then we skip the virCommandRun in the other path (even though it needlessly sets UID and GID in cmd). Next, if we have a successful NETFS creation, then the chown never happens since each will go to the ":" of the ternary as the virCommandRun would have been run under the provided UID/GID. The chmod is run in both cases, which is where the problem is if we have the pesky root squash environment. By not providing one at creation, we default to whatever nfs 'chooses' for that and we cannot change it using chmod() since nfs blocks that. So the "change" is essentially that the NETFS code will set the mode bits (via umask) during the virCommandRun which is done under the uid, gid, & umask provided just like would be done in our other root squash cases. If any of that fails, then we fallback to the non root squash mechanism where we try to set uid, gid, and mode. We document (in formatstorage) that if mode is not provided, then we default to 0600. So not attempting a change if mode == -1 isn't right. When I was first going through this - I considered checking for matching mode, but cannot recall why I chose not to. I can modify the check to be "if (mode != st.st_mode && chmod...)" in order to avoid the unnecessary chmod to the same mode bits. Hmm. right, so I'll move it to the top, e.g. mode_t mode = (vol->target.perms->mode == (mode_t) -1 ? VIR_STORAGE_DEFAULT_VOL_PERM_MODE : vol->target.perms->mode); OK - xor isn't one of those unary operations that I use all that often. I will change this to: virCommandSetUmask(cmd, 0777 ^ mode); I also could use ACCESSPERMS instead of 0777; however, it seems the majority of code uses 0777. My fear in using ACCESSPERMS is some platform doesn't provide it and some build fails... Not sure I understand the comment. If virCommandRun fails, then we fallback to the other methodology. If for some reason virCommandRun succeeds, but the file doesn't exist, then we fallback to the other mechanism (the difference in this being use of 'access' instead of 'stat' in order to determine that) which will attempt the chown & chmod on the created file. Hmm. right - so I'll change it to look like: if (virCommandRun(cmd, NULL) == 0) { /* command was successfully run, if the file was created * then we are done */ if (virFileExists(vol->target.path)) { ret = 0; goto cleanup; } } I was trying to save a ret = 0; step, but sure if ret = 0 and !virFileExists, then a subsequent jump to cleanup would return the wrong value. Do you wish to see the adjusted patch? Just the diffs between this and the changes or a diffs between the changes and top of tree? John FWIW: I probably should have carried over the response/comment from the initial adjustment (w/r/t the new virCommandSetUmask): Just to be clear - this fixes a bug in the existing code which cannot create 'qcow2' file in an NFS pool that has root squash enabled. The failure "currently" is : virsh vol-create-from bz1253609 qemu-img.xml test.img --inputpool bz1253609 error: Failed to create vol from qemu-img.xml error: cannot set mode of '/tmp/netfs-pool/qemu-img-test.img' to 0644: Operation not permitted Assuming that 'test.img' was created using : virsh vol-create bz1253609 vol.xml The qemu-img.xml is: <volume> <name>qemu-img-test.img</name> <source> </source> <capacity>4048576000</capacity> <allocation>0</allocation> <target> <format type='qcow2'/> <permissions> <mode>0644</mode> <owner>107</owner> <group>107</group> <label>system_u:object_r:nfs_t:s0</label> </permissions> </target> </volume> it fails it other ways if <mode> and/or <owner>/<group> are not supplied. FWIW: The 'vol.xml' file differs from the above by only the "type='raw'"

On Wed, Oct 28, 2015 at 07:54:49AM -0400, John Ferlan wrote: This variable was removed in the previous patch, reordering them would remove the need for that. Jan

After a successful qemu-img/qcow-create of the backing file, if we fail to stat the file, change it owner/group, or mode, then the cleanup path should remove the file. Signed-off-by: John Ferlan <jferlan(a)redhat.com&gt; --- Changes over v2: - The 'filecreated' was kept from the previous patch, but we still need to set it once virCommandRun and stat() agree the file was created. - Use virFileRemove and not just unlink in order to delete the file. If this was a root squash created file, then the only way to remove it properly is via virFileRemove src/storage/storage_backend.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/storage/storage_backend.c b/src/storage/storage_backend.c index 2ba6e27..15470e8 100644 --- a/src/storage/storage_backend.c +++ b/src/storage/storage_backend.c @@ -714,6 +714,7 @@ virStorageBackendCreateExecCommand(virStoragePoolObjPtr pool, _("failed to create %s"), vol->target.path); goto cleanup; } + filecreated = true; } uid = (vol->target.perms->uid != st.st_uid) ? vol->target.perms->uid @@ -740,6 +741,9 @@ virStorageBackendCreateExecCommand(virStoragePoolObjPtr pool, ret = 0; cleanup: + if (ret < 0 && filecreated) + virFileRemove(vol->target.path, vol->target.perms->uid, + vol->target.perms->gid); return ret; } -- 2.1.0

After successfully returning from virFileOpenAs, if subsequent calls fail, then we need to remove the file since our caller expects that failures after creation will remove the created file. Signed-off-by: John Ferlan <jferlan(a)redhat.com&gt; --- src/storage/storage_backend.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/storage/storage_backend.c b/src/storage/storage_backend.c index 487c914..8dcb1dc 100644 --- a/src/storage/storage_backend.c +++ b/src/storage/storage_backend.c @@ -485,6 +485,7 @@ virStorageBackendCreateRaw(virConnectPtr conn ATTRIBUTE_UNUSED, int operation_flags; bool reflink_copy = false; mode_t open_mode = VIR_STORAGE_DEFAULT_VOL_PERM_MODE; + bool created = false; virCheckFlags(VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA | VIR_STORAGE_VOL_CREATE_REFLINK, @@ -531,6 +532,7 @@ virStorageBackendCreateRaw(virConnectPtr conn ATTRIBUTE_UNUSED, vol->target.path); goto cleanup; } + created = true; if (vol->target.nocow) { #ifdef __linux__ @@ -557,6 +559,10 @@ virStorageBackendCreateRaw(virConnectPtr conn ATTRIBUTE_UNUSED, ret = -1; cleanup: + if (ret < 0 && created) + ignore_value(virFileRemove(vol->target.path, + vol->target.perms->uid, + vol->target.perms->gid)); VIR_FORCE_CLOSE(fd); return ret; } -- 2.1.0

Create a helper function to remove volume from the pool. Signed-off-by: John Ferlan <jferlan(a)redhat.com&gt; --- src/storage/storage_driver.c | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index 0aa2d6e..292ed9e 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -1619,6 +1619,25 @@ storagePoolLookupByTargetPath(virConnectPtr conn, } +static void +storageVolRemoveFromPool(virStoragePoolObjPtr pool, + virStorageVolDefPtr vol) +{ + size_t i; + + for (i = 0; i < pool->volumes.count; i++) { + if (pool->volumes.objs[i] == vol) { + VIR_INFO("Deleting volume '%s' from storage pool '%s'", + vol->name, pool->def->name); + virStorageVolDefFree(vol); + + VIR_DELETE_ELEMENT(pool->volumes.objs, i, pool->volumes.count); + break; + } + } +} + + static int storageVolDeleteInternal(virStorageVolPtr obj, virStorageBackendPtr backend, @@ -1627,7 +1646,6 @@ storageVolDeleteInternal(virStorageVolPtr obj, unsigned int flags, bool updateMeta) { - size_t i; int ret = -1; if (!backend->deleteVol) { @@ -1651,16 +1669,7 @@ storageVolDeleteInternal(virStorageVolPtr obj, } } - for (i = 0; i < pool->volumes.count; i++) { - if (pool->volumes.objs[i] == vol) { - VIR_INFO("Deleting volume '%s' from storage pool '%s'", - vol->name, pool->def->name); - virStorageVolDefFree(vol); - - VIR_DELETE_ELEMENT(pool->volumes.objs, i, pool->volumes.count); - break; - } - } + storageVolRemoveFromPool(pool, vol); ret = 0; cleanup: -- 2.1.0

https://bugzilla.redhat.com/show_bug.cgi?id=1233003 Commit id 'fdda3760' only managed a symptom where it was possible to create a file in a pool without libvirt's knowledge, so it was reverted. The real fix is to have all the createVol API's which actually create a volume (disk, logical, zfs) and the buildVol API's which handle the real creation of some volume file (fs, rbd, sheepdog) manage deleting any volume which they create when there is some sort of error in processing the volume. This way the onus isn't left up to the storage_driver to determine whether the buildVol failure was due to some failure as a result of adjustments made to the volume after creation such as getting sizes, changing ownership, changing volume protections, etc. or simple a failure in creation. Without needing to consider that the volume has to be removed, the buildVol failure path only needs to remove the volume from the pool. This way if a creation failed due to duplicate name, libvirt wouldn't remove a volume that it didn't create in the pool target. Signed-off-by: John Ferlan <jferlan(a)redhat.com&gt; --- src/storage/storage_backend.h | 10 ++++++++++ src/storage/storage_driver.c | 4 ++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/src/storage/storage_backend.h b/src/storage/storage_backend.h index 39c00b1..96b5f39 100644 --- a/src/storage/storage_backend.h +++ b/src/storage/storage_backend.h @@ -48,6 +48,16 @@ typedef int (*virStorageBackendStopPool)(virConnectPtr conn, typedef int (*virStorageBackendDeletePool)(virConnectPtr conn, virStoragePoolObjPtr pool, unsigned int flags); + +/* A 'buildVol' backend must remove any volume created on error since + * the storage driver does not distinguish whether the failure is due + * to failure to create the volume, to reserve any space necessary for + * the volume, to get data about the volume, to change it's accessibility, + * etc. This avoids issues arising from a creation failure due to some + * external action which created a volume of the same name that libvirt + * was not aware of between checking the pool and the create attempt. It + * also avoids extra round trips to just delete a file. + */ typedef int (*virStorageBackendBuildVol)(virConnectPtr conn, virStoragePoolObjPtr pool, virStorageVolDefPtr vol, diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index 31b7095..ee8b263 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -1869,8 +1869,8 @@ storageVolCreateXML(virStoragePoolPtr obj, pool->asyncjobs--; if (buildret < 0) { - storageVolDeleteInternal(volobj, backend, pool, voldef, - 0, false); + /* buildVol handles deleting volume on failure */ + storageVolRemoveFromPool(pool, voldef); voldef = NULL; goto cleanup; } -- 2.1.0