- Devel - libvirt List Archives

[libvirt] What is the robust/recommended way to retrieve the PID of a VM's init process ?
by Thierry Parmentelat 23 Jan '14

23 Jan '14

Hello there I am trying to locate the namespaces in place for a given lxc container (specifically /proc/<pid>/ns/*) And to this end I was wondering what is the recommended way to probe for an lxc container's init pid (mostly I'm after the mnt and pid namespaces, and probably network ones, but the actual list probably should not matter) I've found about "virsh domid" but this gives me the pid for libvirt_lxc, which turns out to have unmodified namespaces (at least as far as the mnt ns) OTOH this process has exactly one child which is the container's init, which seems to have the right set of namespaces My angle right now is to look in /proc/<domid_pid>/task/children for a - hopefully single - pid and that seems to work for now, but I am concerned this code may be fragile so I would rather use a more robust approach; or maybe this is robust ? Any insight would be most welcome -- Thanks in advance --- PS. I am using a stock fedora20 box with [root@vnode11 ~]# rpm -q libvirt libvirt-1.1.3.2-1.fc20.x86_64

4 11

[libvirt] [PATCH] add nocow feature option to vol-create
by Chunyan Liu 23 Jan '14

23 Jan '14

Btrfs has terrible performance when hosting VM images, even more when the guest in those VM are also using btrfs as file system. One way to mitigate this bad performance is to turn off COW attributes on VM files (since having copy on write for this kind of data is not useful). According to 'chattr' manpage, NOCOW could be set to new or empty file only on btrfs, so this patch tries to add nocow feature option in volume xml and handle it in vol-create, so that users could have a chance to set NOCOW to a new volume if that happens on a btrfs like file system. Signed-off-by: Chunyan Liu <cyliu(a)suse.com> --- This is a revised version to: http://www.redhat.com/archives/libvir-list/2013-December/msg00303.html Changes: * fix Daniel's comments --- docs/formatstorage.html.in | 12 +++++--- docs/schemas/storagefilefeatures.rng | 3 ++ src/conf/storage_conf.c | 9 ++++-- src/storage/storage_backend.c | 4 +- src/storage/storage_backend_fs.c | 48 ++++++++++++++++++++++++++++++++++ src/util/virstoragefile.c | 1 + src/util/virstoragefile.h | 1 + 7 files changed, 69 insertions(+), 9 deletions(-) diff --git a/docs/formatstorage.html.in b/docs/formatstorage.html.in index a089a31..3de1a2b 100644 --- a/docs/formatstorage.html.in +++ b/docs/formatstorage.html.in @@ -385,6 +385,7 @@ <compat>1.1</compat> <features> <lazy_refcounts/> + <nocow/> </features> </target></pre> @@ -423,11 +424,14 @@ Since 1.1.0 </dd> <dt><code>features</code></dt> - <dd>Format-specific features. Only used for <code>qcow2</code> now. - Valid sub-elements are: - <ul> + <dd>Format-specific features. Valid sub-elements are: + <ul> <li><code><lazy_refcounts/></code> - allow delayed reference - counter updates. Since 1.1.0</li> + counter updates. Only used for <code>qcow2</code> now. + Since 1.1.0</li> + <li><code><nocow/></code> - turn off copy-on-write. Only valid + to volume on <code>btrfs</code>, can improve performance. + Since 1.2.2</li> </ul> </dd> </dl> diff --git a/docs/schemas/storagefilefeatures.rng b/docs/schemas/storagefilefeatures.rng index 424b4e2..0cf3513 100644 --- a/docs/schemas/storagefilefeatures.rng +++ b/docs/schemas/storagefilefeatures.rng @@ -17,6 +17,9 @@ <element name='lazy_refcounts'> <empty/> </element> + <element name='nocow'> + <empty/> + </element> </optional> </interleave> </element> diff --git a/src/conf/storage_conf.c b/src/conf/storage_conf.c index 22e38c1..b6409a6 100644 --- a/src/conf/storage_conf.c +++ b/src/conf/storage_conf.c @@ -1398,9 +1398,6 @@ virStorageVolDefParseXML(virStoragePoolDefPtr pool, if ((n = virXPathNodeSet("./target/features/*", ctxt, &nodes)) < 0) goto error; - if (!ret->target.compat && VIR_STRDUP(ret->target.compat, "1.1") < 0) - goto error; - if (!(ret->target.features = virBitmapNew(VIR_STORAGE_FILE_FEATURE_LAST))) goto error; @@ -1412,6 +1409,12 @@ virStorageVolDefParseXML(virStoragePoolDefPtr pool, (const char*)nodes[i]->name); goto error; } + + if (f == VIR_STORAGE_FILE_FEATURE_LAZY_REFCOUNTS) { + if (!ret->target.compat && VIR_STRDUP(ret->target.compat, "1.1") < 0) + goto error; + } + ignore_value(virBitmapSetBit(ret->target.features, f)); } VIR_FREE(nodes); diff --git a/src/storage/storage_backend.c b/src/storage/storage_backend.c index b08d646..b4ab866 100644 --- a/src/storage/storage_backend.c +++ b/src/storage/storage_backend.c @@ -423,7 +423,7 @@ virStorageBackendCreateRaw(virConnectPtr conn ATTRIBUTE_UNUSED, operation_flags |= VIR_FILE_OPEN_FORK; if ((fd = virFileOpenAs(vol->target.path, - O_RDWR | O_CREAT | O_EXCL, + O_RDWR | O_CREAT, vol->target.perms.mode, vol->target.perms.uid, vol->target.perms.gid, @@ -729,7 +729,7 @@ virStorageBackendCreateQemuImgOpts(char **opts, break; /* coverity[dead_error_begin] */ - case VIR_STORAGE_FILE_FEATURE_LAST: + default: ; } virBufferAsprintf(&buf, "%s,", diff --git a/src/storage/storage_backend_fs.c b/src/storage/storage_backend_fs.c index 95783be..dc26f6d 100644 --- a/src/storage/storage_backend_fs.c +++ b/src/storage/storage_backend_fs.c @@ -51,6 +51,13 @@ #include "virfile.h" #include "virlog.h" #include "virstring.h" +#ifdef __linux__ +# include <sys/ioctl.h> +# include <linux/fs.h> +#ifndef FS_NOCOW_FL +#define FS_NOCOW_FL 0x00800000 /* Do not cow file */ +#endif +#endif #define VIR_FROM_THIS VIR_FROM_STORAGE @@ -1061,6 +1068,7 @@ _virStorageBackendFileSystemVolBuild(virConnectPtr conn, { virStorageBackendBuildVolFrom create_func; int tool_type; + bool nocow; if (inputvol) { if (vol->target.encryption != NULL) { @@ -1090,6 +1098,46 @@ _virStorageBackendFileSystemVolBuild(virConnectPtr conn, return -1; } + if (vol->target.features) + ignore_value(virBitmapGetBit(vol->target.features, + VIR_STORAGE_FILE_FEATURE_NOCOW, &nocow)); + if (nocow) { +#ifdef __linux__ + /* create an empty file and set nocow flag. + * This could optimize performance on file system like btrfs. + */ + int attr, fd; + int operation_flags = VIR_FILE_OPEN_FORCE_MODE | VIR_FILE_OPEN_FORCE_OWNER; + if (pool->def->type == VIR_STORAGE_POOL_NETFS) + operation_flags |= VIR_FILE_OPEN_FORK; + + if ((fd = virFileOpenAs(vol->target.path, + O_RDWR | O_CREAT | O_EXCL | O_LARGEFILE, + vol->target.perms.mode, + vol->target.perms.uid, + vol->target.perms.gid, + operation_flags)) < 0) { + virReportSystemError(-fd, + _("Failed to create file '%s'"), + vol->target.path); + return -1; + } + + /* This is an optimisation. The FS_IOC_SETFLAGS ioctl return value will + * be ignored since any failure of this operation should not block the + * left work. + */ + if (ioctl(fd, FS_IOC_GETFLAGS, &attr) == 0) { + attr |= FS_NOCOW_FL; + ioctl(fd, FS_IOC_SETFLAGS, &attr); + } + + VIR_FORCE_CLOSE(fd); +#endif + if (virBitmapClearBit(vol->target.features, VIR_STORAGE_FILE_FEATURE_NOCOW) < 0) + return -1; + } + if (create_func(conn, pool, vol, inputvol, flags) < 0) return -1; return 0; diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c index e45236f..6e34e4c 100644 --- a/src/util/virstoragefile.c +++ b/src/util/virstoragefile.c @@ -62,6 +62,7 @@ VIR_ENUM_IMPL(virStorageFileFormat, VIR_ENUM_IMPL(virStorageFileFeature, VIR_STORAGE_FILE_FEATURE_LAST, "lazy_refcounts", + "nocow", ) enum lv_endian { diff --git a/src/util/virstoragefile.h b/src/util/virstoragefile.h index 7bd2fe0..f45743a 100644 --- a/src/util/virstoragefile.h +++ b/src/util/virstoragefile.h @@ -62,6 +62,7 @@ VIR_ENUM_DECL(virStorageFileFormat); enum virStorageFileFeature { VIR_STORAGE_FILE_FEATURE_LAZY_REFCOUNTS = 0, + VIR_STORAGE_FILE_FEATURE_NOCOW = 1, VIR_STORAGE_FILE_FEATURE_LAST }; -- 1.6.0.2

2 2

[libvirt] [PATCH 0/2] Storage: Fixes for the "fc_host" type pool
by Osier Yang 23 Jan '14

23 Jan '14

*** Enough details in the patch commits *** Osier Yang (2): storage: Fix autostart of pool with "fc_host" type adapter storage: Polling the sysfs for pool with "fc_host" type adapter src/storage/storage_backend_scsi.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) -- 1.8.1.4

2 17

[libvirt] [PATCH] storage: Fix the memory leak
by Osier Yang 23 Jan '14

23 Jan '14

The return value of virGetFCHostNameByWWN is a strdup'ed string. Also add comments to declare that the caller should take care of freeing it. --- src/storage/storage_backend_scsi.c | 15 ++++++++++----- src/util/virutil.c | 5 ++++- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/src/storage/storage_backend_scsi.c b/src/storage/storage_backend_scsi.c index fce2bae..b38e530 100644 --- a/src/storage/storage_backend_scsi.c +++ b/src/storage/storage_backend_scsi.c @@ -668,6 +668,8 @@ static int deleteVport(virStoragePoolSourceAdapter adapter) { unsigned int parent_host; + char *name = NULL; + int ret = -1; if (adapter.type != VIR_STORAGE_POOL_SOURCE_ADAPTER_TYPE_FC_HOST) return 0; @@ -680,18 +682,21 @@ deleteVport(virStoragePoolSourceAdapter adapter) if (!adapter.data.fchost.parent) return 0; - if (!(virGetFCHostNameByWWN(NULL, adapter.data.fchost.wwnn, - adapter.data.fchost.wwpn))) + if (!(name = virGetFCHostNameByWWN(NULL, adapter.data.fchost.wwnn, + adapter.data.fchost.wwpn))) return -1; if (getHostNumber(adapter.data.fchost.parent, &parent_host) < 0) - return -1; + goto cleanup; if (virManageVport(parent_host, adapter.data.fchost.wwpn, adapter.data.fchost.wwnn, VPORT_DELETE) < 0) - return -1; + goto cleanup; - return 0; + ret = 0; +cleanup: + VIR_FREE(name); + return ret; } diff --git a/src/util/virutil.c b/src/util/virutil.c index 87cc2e7..7a2fbb0 100644 --- a/src/util/virutil.c +++ b/src/util/virutil.c @@ -1820,7 +1820,10 @@ cleanup: /* virGetHostNameByWWN: * * Iterate over the sysfs tree to get FC host name (e.g. host5) - * by wwnn,wwpn pair. + * by the provided "wwnn,wwpn" pair. + * + * Returns the FC host name which must be freed by the caller, + * or NULL on failure. */ char * virGetFCHostNameByWWN(const char *sysfs_prefix, -- 1.8.1.4

2 2

[libvirt] [libvirt-java] state of affairs
by Claudio Bley 23 Jan '14

23 Jan '14

Hi. It seems the Java wrapper is nearly dead. It has fallen way behind libvirt development. [in my local tree, there're still 120 libvirt API functions missing from the Java interface which /probably/ are worth to be added] I have send a few patches to the list, but no one is willing / able to review them. Some of those patches date almost a year back. Slowly I'm getting a bit frustrated and maybe also a tad impatient... Currently, I'm having +60 commits in my local git tree. As you might imagine, I'd really like to get these off my back. That's why I'm asking myself whether the ACKing / NACKing of patches is the right model for libvirt-java, given that there is, apparently, very little interest and at the same time next to nobody with good Java expertise on the list. Additionally, there are a few glitches in the API which are a thorn in my side ever since I began using the libvirt Java wrapper. It's obvious that the wrapper was written without much thinking about the Java environment and API. Some functions have only been wrapped just because it was possible or perhaps to just have a full coverage of libvirt version x.y.z, without any real use for a Java programmer. Do we really have to live with the failures of the past? I'd really like to fix these even if that means *breaking* the API. IMO, this would not be that bad in the Java world. It's not like that you suddenly happen to have an updated dynamic library on the system that's missing some symbols or has it's ABI changed which makes your program crash. Java libraries come with the API bundled and you get an error at the earliest time possible - at compilation time. Even if you upgrade a jar without recompiling your program you'll get a nice RuntimeException instead of undefined behavior. So, I'd say just bump the major or minor version up to the next number and send out a big "SORRY, we messed up" to everyone and be done with it. Alternatively, change the package and the artifact name to libvirt2 effectively creating some sort of fork?! But, given that there's not much review on list, failures are likely to happen again and the same situation would arise anew. Thank you. Claudio -- AV-Test GmbH, Henricistraße 20, 04155 Leipzig, Germany Phone: +49 341 265 310 19 Web:<http://www.av-test.org> Eingetragen am / Registered at: Amtsgericht Stendal (HRB 114076) Geschaeftsfuehrer (CEO): Andreas Marx, Guido Habicht, Maik Morgenstern

2 2

[libvirt] Build failed in Jenkins: libvirt-syntax-check #1898
by Jenkins CI 23 Jan '14

23 Jan '14

See <http://honk.sigxcpu.org:8001/job/libvirt-syntax-check/1898/> ------------------------------------------ Started by upstream project "libvirt-build" build number 2132 Started by upstream project "libvirt-build" build number 2133 Building on master in workspace <http://honk.sigxcpu.org:8001/job/libvirt-syntax-check/ws/> [workspace] $ /bin/sh -xe /tmp/hudson9098701041827209295.sh + make syntax-check GEN bracket-spacing-check src/util/virscsi.c:206: dev->shareable= shareable; maint.mk: incorrect whitespace, see HACKING for rules make: *** [bracket-spacing-check] Error 1 Build step 'Execute shell' marked build as failure

2 2

[libvirt] [PATCH] qemu: remove memset params array to zero in qemuDomainGetPercpuStats
by Gao feng 23 Jan '14

23 Jan '14

the array params is allocated by VIR_ALLOC_N in remoteDispatchDomainGetCPUStats. it had been set to zero. No need to reset it to zero again, and this reset here is incorrect too, nparams * ncpus is the array length not the size of params array. Signed-off-by: Gao feng <gaofeng(a)cn.fujitsu.com> --- src/qemu/qemu_driver.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index df4f5b5..1e54164 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -15818,7 +15818,6 @@ qemuDomainGetPercpuStats(virDomainObjPtr vm, if (virCgroupGetCpuacctPercpuUsage(priv->cgroup, &buf)) goto cleanup; pos = buf; - memset(params, 0, nparams * ncpus); /* return percpu cputime in index 0 */ param_idx = 0; -- 1.8.4.2

2 1

[libvirt] [PATCH 0/2 v2] Fixes on shareable SCSI host device
by Osier Yang 23 Jan '14

23 Jan '14

*** BLURB HERE *** Osier Yang (2): util: Add "shareable" field for virSCSIDevice struct qemu: Don't fail if the SCSI host device is shareable between domains src/libvirt_private.syms | 3 +- src/qemu/qemu_cgroup.c | 3 +- src/qemu/qemu_hostdev.c | 84 ++++++++++++++++++++++------------------ src/security/security_apparmor.c | 3 +- src/security/security_dac.c | 6 ++- src/security/security_selinux.c | 6 ++- src/util/virscsi.c | 59 +++++++++++++++++++++++----- src/util/virscsi.h | 11 ++++-- 8 files changed, 116 insertions(+), 59 deletions(-) -- 1.8.1.4

2 12

[libvirt] [PATCH v3] virsh nodecpustats returns incorrect stats of cpu on Linux when the number of online cpu exceed 9.
by mars＠linux.vnet.ibm.com 23 Jan '14

23 Jan '14

From: Bing Bu Cao <mars(a)linux.vnet.ibm.com> To retrieve node cpu statistics on Linux system, the linuxNodeGetCPUstats function simply uses STRPREFIX() to match the cpuid with the cpuid read from /proc/stat, it will cause obvious error. For example: 'virsh nodecpustats 1' will display stats of cpu1* if the latter is online and cpu1 is offline. This patch fixes this bug. Signed-off-by: Bing Bu Cao <mars(a)linux.vnet.ibm.com> --- src/nodeinfo.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/nodeinfo.c b/src/nodeinfo.c index 05bc038..cf6d29b 100644 --- a/src/nodeinfo.c +++ b/src/nodeinfo.c @@ -691,7 +691,7 @@ linuxNodeGetCPUStats(FILE *procstat, char line[1024]; unsigned long long usr, ni, sys, idle, iowait; unsigned long long irq, softirq, steal, guest, guest_nice; - char cpu_header[3 + INT_BUFSIZE_BOUND(cpuNum)]; + char cpu_header[4 + INT_BUFSIZE_BOUND(cpuNum)]; if ((*nparams) == 0) { /* Current number of cpu stats supported by linux */ @@ -708,9 +708,9 @@ linuxNodeGetCPUStats(FILE *procstat, } if (cpuNum == VIR_NODE_CPU_STATS_ALL_CPUS) { - strcpy(cpu_header, "cpu"); + strcpy(cpu_header, "cpu "); } else { - snprintf(cpu_header, sizeof(cpu_header), "cpu%d", cpuNum); + snprintf(cpu_header, sizeof(cpu_header), "cpu%d ", cpuNum); } while (fgets(line, sizeof(line), procstat) != NULL) { -- 1.7.7.6

2 1

[libvirt] [PATCH] util: Correct the NUMA node range checking
by Osier Yang 23 Jan '14

23 Jan '14

There are 2 issues here: First we shouldn't add "1" to the return value of numa_max_node(), since the semanteme of the error message was changed, it's not saying about the number of total NUMA nodes anymore. Second, the value of "bit" is the position of the first bit which exceeds either numa_max_node() or NUMA_NUM_NODES, it can be any number in the range, so saying "bigger than $bit" is quite confused now. For example, assuming there is a NUMA machine which has 10 NUMA nodes, and one specifies the "nodeset" as "0,5,88", the error message will be like: Nodeset is out of range, host cannot support NUMA node bigger than 88 It sounds like all NUMA node number less than 88 is fine, but actually the maximum NUMA node number the machine supports is 9. This patch fixes the issues by removing the addition with "1" and simplifies the error message as "NUMA node $bit is out of range". --- src/util/virnuma.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/src/util/virnuma.c b/src/util/virnuma.c index ab46591..500dca7 100644 --- a/src/util/virnuma.c +++ b/src/util/virnuma.c @@ -99,7 +99,6 @@ virNumaSetupMemoryPolicy(virNumaTuneDef numatune, int ret = -1; int bit = 0; size_t i; - int maxnode = 0; virBitmapPtr tmp_nodemask = NULL; if (numatune.memory.placement_mode == @@ -122,16 +121,13 @@ virNumaSetupMemoryPolicy(virNumaTuneDef numatune, return -1; } - maxnode = numa_max_node() + 1; - /* Convert nodemask to NUMA bitmask. */ nodemask_zero(&mask); bit = -1; while ((bit = virBitmapNextSetBit(tmp_nodemask, bit)) >= 0) { - if (bit > maxnode || bit > NUMA_NUM_NODES) { + if (bit > numa_max_node() || bit > NUMA_NUM_NODES) { virReportError(VIR_ERR_INTERNAL_ERROR, - _("Nodeset is out of range, host cannot support " - "NUMA node bigger than %d"), bit); + _("NUMA node %d is out of range"), bit); return -1; } nodemask_set(&mask, bit); -- 1.8.1.4

2 2