- Devel - libvirt List Archives

[libvirt] [RFC PATCH v1 0/4] numa: describe sibling nodes distances
by Wim Ten Have 27 Jun '17

27 Jun '17

From: Wim ten Have <wim.ten.have(a)oracle.com> This patch extents guest domain administration adding support to advertise node sibling distances when configuring HVM numa guests. NUMA (non-uniform memory access), a method of configuring a cluster of nodes within a single multiprocessing system such that it shares processor local memory amongst others improving performance and the ability of the system to be expanded. A NUMA system could be illustrated as shown below. Within this 4-node system, every socket is equipped with its own distinct memory. The whole typically resembles a SMP (symmetric multiprocessing) system being a "tightly-coupled," "share everything" system in which multiple processors are working under a single operating system and can access each others' memory over multiple "Bus Interconnect" paths. +-----+-----+-----+ +-----+-----+-----+ | M | CPU | CPU | | CPU | CPU | M | | E | | | | | | E | | M +- Socket0 -+ +- Socket3 -+ M | | O | | | | | | O | | R | CPU | CPU <---------> CPU | CPU | R | | Y | | | | | | Y | +-----+--^--+-----+ +-----+--^--+-----+ | | | Bus Interconnect | | | +-----+--v--+-----+ +-----+--v--+-----+ | M | | | | | | M | | E | CPU | CPU <---------> CPU | CPU | E | | M | | | | | | M | | O +- Socket1 -+ +- Socket2 -+ O | | R | | | | | | R | | Y | CPU | CPU | | CPU | CPU | Y | +-----+-----+-----+ +-----+-----+-----+ In contrast there is the limitation of a flat SMP system, not illustrated. Here, as sockets are added, the bus (data and address path), under high activity, gets overloaded and easily becomes a performance bottleneck. NUMA adds an intermediate level of memory shared amongst a few cores per socket as illustrated above, so that data accesses do not have to travel over a single bus. Unfortunately the way NUMA does this adds its own limitations. This, as visualized in the illustration above, happens when data is stored in memory associated with Socket2 and is accessed by a CPU (core) in Socket0. The processors use the "Bus Interconnect" to create gateways between the sockets (nodes) enabling inter-socket access to memory. These "Bus Interconnect" hops add data access delays when a CPU (core) accesses memory associated with a remote socket (node). For terminology we refer to sockets as "nodes" where access to each others' distinct resources such as memory make them "siblings" with a designated "distance" between them. A specific design is described under the ACPI (Advanced Configuration and Power Interface Specification) within the chapter explaining the system's SLIT (System Locality Distance Information Table). These patches extend core libvirt's XML description of a virtual machine's hardware to include NUMA distance information for sibling nodes, which is then passed to Xen guests via libxl. Recently qemu landed support for constructing the SLIT since commit 0f203430dd ("numa: Allow setting NUMA distance for different NUMA nodes"), hence these core libvirt extensions can also help other drivers in supporting this feature. The XML changes made allow to describe the <cell> (or node/sockets) <distances> amongst <sibling> node identifiers and propagate these towards the numa domain functionality finally adding support to libxl. [below is an example illustrating a 4 node/socket <cell> setup] <cpu> <numa> <cell id='0' cpus='0,4-7' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='10'/> <sibling id='1' value='21'/> <sibling id='2' value='31'/> <sibling id='3' value='41'/> </distances> </cell> <cell id='1' cpus='1,8-10,12-15' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='21'/> <sibling id='1' value='10'/> <sibling id='2' value='21'/> <sibling id='3' value='31'/> </distances> </cell> <cell id='2' cpus='2,11' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='31'/> <sibling id='1' value='21'/> <sibling id='2' value='10'/> <sibling id='3' value='21'/> </distances> </cell> <cell id='3' cpus='3' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='41'/> <sibling id='1' value='31'/> <sibling id='2' value='21'/> <sibling id='3' value='10'/> </distances> </cell> </numa> </cpu> By default on libxl, if no <distances> are given to describe the SLIT data between different <cell>s, this patch will default to a scheme using 10 for local and 21 for any remote node/socket, which is the assumption of guest OS when no SLIT is specified. While SLIT is optional, libxl requires that distances are set nonetheless. On Linux systems the SLIT detail can be listed with help of the 'numactl -H' command. An above HVM guest as described would on such prompt with below output. [root@f25 ~]# numactl -H available: 4 nodes (0-3) node 0 cpus: 0 4 5 6 7 node 0 size: 1988 MB node 0 free: 1743 MB node 1 cpus: 1 8 9 10 12 13 14 15 node 1 size: 1946 MB node 1 free: 1885 MB node 2 cpus: 2 11 node 2 size: 2011 MB node 2 free: 1912 MB node 3 cpus: 3 node 3 size: 2010 MB node 3 free: 1980 MB node distances: node 0 1 2 3 0: 10 21 31 41 1: 21 10 21 31 2: 31 21 10 21 3: 41 31 21 10 Wim ten Have (4): numa: describe siblings distances within cells libxl: vnuma support xenconfig: add domxml conversions for xen-xl xlconfigtest: add tests for numa cell sibling distances docs/schemas/basictypes.rng | 8 + docs/schemas/cputypes.rng | 18 ++ src/conf/cpu_conf.c | 2 +- src/conf/numa_conf.c | 260 +++++++++++++++++- src/conf/numa_conf.h | 25 +- src/libvirt_private.syms | 6 + src/libxl/libxl_conf.c | 128 +++++++++ src/xenconfig/xen_xl.c | 300 +++++++++++++++++++++ .../test-fullvirt-vnuma-nodistances.cfg | 26 ++ .../test-fullvirt-vnuma-nodistances.xml | 54 ++++ tests/xlconfigdata/test-fullvirt-vnuma.cfg | 26 ++ tests/xlconfigdata/test-fullvirt-vnuma.xml | 81 ++++++ tests/xlconfigtest.c | 4 + 13 files changed, 932 insertions(+), 6 deletions(-) create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.xml create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.xml -- 2.9.4

5 11

[libvirt] [PATCH] conf: Copy loadparm in virDomainDeviceInfoCopy()
by Andrea Bolognani 27 Jun '17

27 Jun '17

Commit 54fa1b44afc8 added virDomainDeviceInfo::loadparm and updated virDomainDeviceInfoClear() accordingly, but omitted the necessary virDomainDeviceInfoCopy() changes. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- src/conf/domain_conf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e5595d7..8fabb89 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -3812,12 +3812,14 @@ virDomainDeviceInfoCopy(virDomainDeviceInfoPtr dst, /* first a shallow copy of *everything* */ *dst = *src; - /* then redo the two fields that are pointers */ + /* then copy whatever's left */ dst->alias = NULL; dst->romfile = NULL; + dst->loadparm = NULL; if (VIR_STRDUP(dst->alias, src->alias) < 0 || - VIR_STRDUP(dst->romfile, src->romfile) < 0) + VIR_STRDUP(dst->romfile, src->romfile) < 0 || + VIR_STRDUP(dst->loadparm, src->loadparm) < 0) return -1; return 0; } -- 2.7.5

3 3

[libvirt] [PATCH 0/4] Drop HACKING, improve README
by Andrea Bolognani 27 Jun '17

27 Jun '17

This has annoyed me for a very, very long time. Andrea Bolognani (4): HACKING: Drop from the git repository docs: Point to hacking.html from contribute.html README: Remove 'git send-email' mention README: Various minor style tweaks HACKING | 1112 ----------------------------------------------- Makefile.am | 11 - README-hacking | 3 +- README.md | 21 +- cfg.mk | 13 +- docs/Makefile.am | 2 +- docs/contribute.html.in | 4 +- docs/hacking1.xsl | 40 -- docs/hacking2.xsl | 140 ------ 9 files changed, 21 insertions(+), 1325 deletions(-) delete mode 100644 HACKING delete mode 100644 docs/hacking1.xsl delete mode 100644 docs/hacking2.xsl -- 2.7.5

4 17

[libvirt] [PATCH] qemu: Support chardevs with ARM virt machines
by Christoffer Dall 27 Jun '17

27 Jun '17

The function to check if -chardev is supported by QEMU was written a long time ago, where adding chardevs did not make sense on the fixed ARM platforms. Since then, we now have a general purpose virt platform, which should support plugging in any device over PCIe which is supported in a similar fashion on x86. Signed-off-by: Christoffer Dall <cdall(a)linaro.org> --- src/qemu/qemu_capabilities.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 7f22492..1348af7 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -5507,6 +5507,11 @@ virQEMUCapsSupportsChardev(const virDomainDef *def, if ((def->os.arch != VIR_ARCH_ARMV7L) && (def->os.arch != VIR_ARCH_AARCH64)) return true; + /* The virt machine has a PCIe bus and allows plugging in the same type of + * devices as x86 systems do on a PCIe bus. */ + if (qemuDomainIsVirt(def)) + return true; + /* This may not be true for all ARM machine types, but at least * the only supported non-virtio serial devices of vexpress and versatile * don't have the -chardev property wired up. */ -- 2.9.0

4 10

[libvirt] [PATCH 1/2] lxc: add support for docker-json Memory and VCPU conversion
by Venkat Datta N H 27 Jun '17

27 Jun '17

Docker Memory and VCPU configuration is converted to fit for LXC container XML configuration --- po/POTFILES.in | 1 + src/Makefile.am | 1 + src/lxc/lxc_driver.c | 13 ++- src/lxc/lxc_native.h | 1 + src/lxc/lxc_native_docker.c | 112 ++++++++++++++++++ src/lxc/lxc_native_docker.h | 30 +++++ tests/Makefile.am | 8 +- .../dockerjson2xmldata-simple.json | 36 ++++++ .../dockerjson2xmldata-simple.xml | 15 +++ tests/dockerjson2xmltest.c | 127 +++++++++++++++++++++ 10 files changed, 338 insertions(+), 6 deletions(-) create mode 100644 src/lxc/lxc_native_docker.c create mode 100644 src/lxc/lxc_native_docker.h create mode 100644 tests/dockerjson2xmldata/dockerjson2xmldata-simple.json create mode 100644 tests/dockerjson2xmldata/dockerjson2xmldata-simple.xml create mode 100644 tests/dockerjson2xmltest.c diff --git a/po/POTFILES.in b/po/POTFILES.in index 275df1f..098be9f 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -111,6 +111,7 @@ src/lxc/lxc_driver.c src/lxc/lxc_fuse.c src/lxc/lxc_hostdev.c src/lxc/lxc_native.c +src/lxc/lxc_native_docker.c src/lxc/lxc_process.c src/network/bridge_driver.c src/network/bridge_driver_linux.c diff --git a/src/Makefile.am b/src/Makefile.am index eae32dc..53d1bca 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -839,6 +839,7 @@ LXC_DRIVER_SOURCES = \ lxc/lxc_process.c lxc/lxc_process.h \ lxc/lxc_fuse.c lxc/lxc_fuse.h \ lxc/lxc_native.c lxc/lxc_native.h \ + lxc/lxc_native_docker.c lxc/lxc_native_docker.h \ lxc/lxc_driver.c lxc/lxc_driver.h LXC_CONTROLLER_SOURCES = \ diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 22c8b58..0f5a7870 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -53,6 +53,7 @@ #include "lxc_driver.h" #include "lxc_native.h" #include "lxc_process.h" +#include "lxc_native_docker.h" #include "viralloc.h" #include "virnetdevbridge.h" #include "virnetdevveth.h" @@ -1062,15 +1063,17 @@ static char *lxcConnectDomainXMLFromNative(virConnectPtr conn, if (virConnectDomainXMLFromNativeEnsureACL(conn) < 0) goto cleanup; - if (STRNEQ(nativeFormat, LXC_CONFIG_FORMAT)) { + if (STREQ(nativeFormat, DOCKER_CONFIG_FORMAT)) { + if (!(def = dockerParseJSONConfig(caps, driver->xmlopt, nativeConfig))) + goto cleanup; + } else if (STREQ(nativeFormat, LXC_CONFIG_FORMAT)) { + if (!(def = lxcParseConfigString(nativeConfig, caps, driver->xmlopt))) + goto cleanup; + } else { virReportError(VIR_ERR_INVALID_ARG, _("unsupported config type %s"), nativeFormat); goto cleanup; } - - if (!(def = lxcParseConfigString(nativeConfig, caps, driver->xmlopt))) - goto cleanup; - xml = virDomainDefFormat(def, caps, 0); cleanup: diff --git a/src/lxc/lxc_native.h b/src/lxc/lxc_native.h index 15fa0d5..88263ae 100644 --- a/src/lxc/lxc_native.h +++ b/src/lxc/lxc_native.h @@ -26,6 +26,7 @@ # include "domain_conf.h" # define LXC_CONFIG_FORMAT "lxc-tools" +# define DOCKER_CONFIG_FORMAT "docker" virDomainDefPtr lxcParseConfigString(const char *config, virCapsPtr caps, diff --git a/src/lxc/lxc_native_docker.c b/src/lxc/lxc_native_docker.c new file mode 100644 index 0000000..a278309 --- /dev/null +++ b/src/lxc/lxc_native_docker.c @@ -0,0 +1,112 @@ +/* + * lxc_native_docker.c: LXC native docker configuration import + * + * Copyright (C) 2017 Venkat Datta N H + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Author: Venkat Datta N H <nhvenkatdatta(a)gmail.com> + */ +#include <config.h> +#include "util/viralloc.h" +#include "util/virfile.h" +#include "util/virstring.h" +#include "util/virconf.h" +#include "util/virjson.h" +#include "util/virutil.h" +#include "virerror.h" +#include "virlog.h" +#include "conf/domain_conf.h" +#include "lxc_native_docker.h" +#include "secret_conf.h" +#define VIR_FROM_THIS VIR_FROM_LXC +VIR_LOG_INIT("lxc.lxc_native_docker"); + +static int dockerParseVCpus(virDomainDefPtr dom, + virDomainXMLOptionPtr xmlopt, + virJSONValuePtr prop) +{ + int vcpus; + + if (virJSONValueObjectGetNumberInt(prop, "CpuShares", &vcpus) != 0) + return -1; + + if (virDomainDefSetVcpusMax(dom, vcpus, xmlopt) < 0) + return -1; + + if (virDomainDefSetVcpus(dom, vcpus) < 0) + return -1; + + return 0; +} + +static int dockerParseMem(virDomainDefPtr dom, + virJSONValuePtr prop) +{ + unsigned long long mem; + + if (virJSONValueObjectGetNumberUlong(prop, "Memory", &mem) != 0) + return -1; + + virDomainDefSetMemoryTotal(dom, mem / 1024); + dom->mem.cur_balloon = mem / 1024; + + return 0; +} + +virDomainDefPtr dockerParseJSONConfig(virCapsPtr caps ATTRIBUTE_UNUSED, + virDomainXMLOptionPtr xmlopt, + const char *config) +{ + virJSONValuePtr json_obj; + virJSONValuePtr host_config; + + if (!(json_obj = virJSONValueFromString(config))) + return NULL; + + virDomainDefPtr def; + + if (!(def = virDomainDefNew())) + goto error; + + def->id = -1; + def->mem.cur_balloon = 64*1024; + virDomainDefSetMemoryTotal(def, def->mem.cur_balloon); + + if ((host_config = virJSONValueObjectGetObject(json_obj, "HostConfig")) != NULL) { + if (dockerParseVCpus(def, xmlopt, host_config) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to parse VCpu")); + goto error; + } + + if (dockerParseMem(def, host_config) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to parse Memory")); + goto error; + } + } + + def->clock.offset = VIR_DOMAIN_CLOCK_OFFSET_UTC; + def->onReboot = VIR_DOMAIN_LIFECYCLE_RESTART; + def->onCrash = VIR_DOMAIN_LIFECYCLE_CRASH_DESTROY; + def->onPoweroff = VIR_DOMAIN_LIFECYCLE_DESTROY; + def->virtType = VIR_DOMAIN_VIRT_LXC; + def->os.type = VIR_DOMAIN_OSTYPE_EXE; + + return def; + + error: + virDomainDefFree(def); + return NULL; +} diff --git a/src/lxc/lxc_native_docker.h b/src/lxc/lxc_native_docker.h new file mode 100644 index 0000000..40285f9 --- /dev/null +++ b/src/lxc/lxc_native_docker.h @@ -0,0 +1,30 @@ +/* + * lxc_native_docker.h: header file for LXC native docker configuration + * + * Copyright (C) 2017 Venkat Datta N H + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Author: Venkat Datta N H <nhvenkatdatta(a)gmail.com> + */ +#ifndef __LXC_NATIVE_DOCKER_H__ +# define __LXC_NATIVE_DOCKER_H__ +# include "domain_conf.h" + +virDomainDefPtr dockerParseJSONConfig(virCapsPtr caps, + virDomainXMLOptionPtr xmlopt, + const char *config); + +#endif diff --git a/tests/Makefile.am b/tests/Makefile.am index 19986dc..3beb634 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -93,6 +93,7 @@ EXTRA_DIST = \ capabilityschemadata \ commanddata \ cputestdata \ + dockerjson2xmldata \ domaincapsschemadata \ domainconfdata \ domainschemadata \ @@ -295,7 +296,7 @@ test_libraries += libqemumonitortestutils.la \ endif WITH_QEMU if WITH_LXC -test_programs += lxcxml2xmltest lxcconf2xmltest +test_programs += lxcxml2xmltest lxcconf2xmltest dockerjson2xmltest endif WITH_LXC if WITH_OPENVZ @@ -693,6 +694,11 @@ lxcconf2xmltest_SOURCES = \ lxcconf2xmltest.c testutilslxc.c testutilslxc.h \ testutils.c testutils.h lxcconf2xmltest_LDADD = $(lxc_LDADDS) + +dockerjson2xmltest_SOURCES = \ + dockerjson2xmltest.c testutilslxc.c testutilslxc.h \ + testutils.c testutils.h +dockerjson2xmltest_LDADD = $(lxc_LDADDS) else ! WITH_LXC EXTRA_DIST += lxcxml2xmltest.c testutilslxc.c testutilslxc.h endif ! WITH_LXC diff --git a/tests/dockerjson2xmldata/dockerjson2xmldata-simple.json b/tests/dockerjson2xmldata/dockerjson2xmldata-simple.json new file mode 100644 index 0000000..63470be --- /dev/null +++ b/tests/dockerjson2xmldata/dockerjson2xmldata-simple.json @@ -0,0 +1,36 @@ +{ + "Id": "dbb1ae21dac15973d66e6c2b8516d270b32ca766e0cf7551d8b7973513e5f079", + "Created": "2017-05-25T18:55:17.922934825Z", + "Path": "/bin/bash", + "Args": [], + "HostConfig": { + "Binds": null, + "ContainerIDFile": "", + "LogConfig": { + "Type": "json-file", + "Config": {} + }, + "NetworkMode": "default", + "PortBindings": {}, + "ShmSize": 67108864, + "Runtime": "runc", + "Isolation": "", + "CpuShares": 2, + "Memory": 1073741824, + "CgroupParent": "", + "CpuPeriod": 0, + "CpuQuota": 0, + "CpusetCpus": "", + "CpusetMems": "", + "KernelMemory": 0, + "MemoryReservation": 0, + "MemorySwap": -1, + "MemorySwappiness": -1, + "PidsLimit": 0, + "Ulimits": null, + "CpuCount": 0, + "CpuPercent": 0, + "IOMaximumIOps": 0, + "IOMaximumBandwidth": 0 + } +} diff --git a/tests/dockerjson2xmldata/dockerjson2xmldata-simple.xml b/tests/dockerjson2xmldata/dockerjson2xmldata-simple.xml new file mode 100644 index 0000000..8be1ace --- /dev/null +++ b/tests/dockerjson2xmldata/dockerjson2xmldata-simple.xml @@ -0,0 +1,15 @@ +<domain type='lxc'> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>2</vcpu> + <os> + <type>exe</type> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + </devices> +</domain> diff --git a/tests/dockerjson2xmltest.c b/tests/dockerjson2xmltest.c new file mode 100644 index 0000000..41c46a1 --- /dev/null +++ b/tests/dockerjson2xmltest.c @@ -0,0 +1,127 @@ +#include <config.h> + +#include "testutils.h" + +#ifdef WITH_LXC + +# include "lxc/lxc_native_docker.h" +# include "lxc/lxc_conf.h" +# include "testutilslxc.h" + +# define VIR_FROM_THIS VIR_FROM_NONE + +static virCapsPtr caps; +static virDomainXMLOptionPtr xmlopt; + +static int testSanitizeDef(virDomainDefPtr vmdef) +{ + /* Remove UUID randomness */ + if (virUUIDParse("c7a5fdbd-edaf-9455-926a-d65c16db1809", vmdef->uuid) < 0) + return -1; + return 0; +} + +static int +testCompareXMLToConfigFiles(const char *xmlfile, + const char *configfile, + bool expectError) +{ + int ret = -1; + char *config = NULL; + char *actualxml = NULL; + virDomainDefPtr vmdef = NULL; + + if (virTestLoadFile(configfile, &config) < 0) + goto fail; + + vmdef = dockerParseJSONConfig(caps, xmlopt, config); + + if (vmdef && expectError) { + if (testSanitizeDef(vmdef) < 0) + goto fail; + + if (!(actualxml = virDomainDefFormat(vmdef, caps, 0))) + goto fail; + + if (virTestCompareToFile(actualxml, xmlfile) < 0) + goto fail; + } + + ret = 0; + + fail: + VIR_FREE(actualxml); + VIR_FREE(config); + virDomainDefFree(vmdef); + return ret; +} + +struct testInfo { + const char *name; + bool expectError; +}; + +static int +testCompareXMLToConfigHelper(const void *data) +{ + int result = -1; + const struct testInfo *info = data; + char *xml = NULL; + char *config = NULL; + + if (virAsprintf(&xml, "%s/dockerjson2xmldata/dockerjson2xmldata-%s.xml", + abs_srcdir, info->name) < 0 || + virAsprintf(&config, "%s/dockerjson2xmldata/dockerjson2xmldata-%s.json", + abs_srcdir, info->name) < 0) + goto cleanup; + + result = testCompareXMLToConfigFiles(xml, config, info->expectError); + + cleanup: + VIR_FREE(xml); + VIR_FREE(config); + return result; +} + +static int +mymain(void) +{ + int ret = EXIT_SUCCESS; + + if (!(caps = testLXCCapsInit())) + return EXIT_FAILURE; + + if (!(xmlopt = lxcDomainXMLConfInit())) { + virObjectUnref(caps); + return EXIT_FAILURE; + } + + +# define DO_TEST(name, expectError) \ + do { \ + const struct testInfo info = { name, expectError }; \ + if (virTestRun("DOCKER JSON-2-XML " name, \ + testCompareXMLToConfigHelper, \ + &info) < 0) \ + ret = EXIT_FAILURE; \ + } while (0) + + DO_TEST("simple", true); + + virObjectUnref(xmlopt); + virObjectUnref(caps); + + return ret; +} + +VIR_TEST_MAIN(mymain) + +#else + +int +main(void) +{ + return EXIT_AM_SKIP; +} + +#endif /* WITH_LXC */ -- 2.7.4

3 4

Re: [libvirt] [PATCH] qemu: Support chardevs with ARM virt machines
by Andrew Jones 26 Jun '17

26 Jun '17

On Sun, 2017-06-25 at 13:46 +0200, Christoffer Dall wrote: >>On Sun, Jun 25, 2017 at 4:07 AM, Andrea Bolognani <abologna redhat com> wrote: >> On Sat, 2017-06-24 at 16:07 +0200, Christoffer Dall wrote: >>> > The way I see it, the bug is about libvirt being unable to >>> > launch guests which use the <console><log> feature, and with >>> > that in mind your patch is correct but doesn't solve the >>> > issue, because even thought that specific error is gone you >>> > immediately run into a different one and your guest is still >>> > unable to start. >>> >>> I didn't experience this, it was actually working on my end. I wonder >>> if it's related to the QEMU version, where I seem to remember we >>> changed what some serial options turned into. But I for sure did not >>> see "-device isa-serial..." on the command line, so maybe not. >> >> That's very different from the behavior I'm seeing, and I >> can't figure out why that would be the case. That's why >> having your QEMU command line would be very useful. >> >> As for differences in QEMU binaries, there might be some >> capability that I haven't considered and influences the >> generated command line. I'll look into that. > >Cool, I'll have a look as well and will document my complete >environment, then hopefully we can diff with yours and see where this >ISA thing shows up. It's likely a pci-serial vs. isa-serial device getting created. Something like -device pcie-root-port,port=0xa,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \ -chardev stdio,logfile=logfile,id=chardev0,logappend=off \ -device pci-serial,chardev=chardev0,bus=pci.3,addr=0x0 works for me, but only with upstream (not RHEL) qemu, and when adding console=ttyS0 to the guest kernel command line. So far we haven't wanted the pci-serial device in AArch64 RHEL qemu, as we didn't see a need for it. Is there any reason it should be supported other than it allows the logfile feature to work? As Andrea pointed out, in [1], I wrote that I think the logfile feature should work even with the default PL011 serial console. Thanks, drew [1] https://bugzilla.redhat.com/show_bug.cgi?id=1456882#c4

2 1

[libvirt] [PATCH] nodedev: udev: Drop udev monitor fd check in udevEventHandleCallback
by Erik Skultety 26 Jun '17

26 Jun '17

We do perform that magical check every time an event is received, but unless we explicitly unref'd or disconnected the monitor, we should not get a spurious event from a different source. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- This is just a poke patch, since I also checked libudev's source and still fail to understand why we need that check. Unless entered by *_monitor_unref with the last reference or with *monitor_disconnect, libudev doesn't change the fd arbitrarily. So the only reason for us to check for that would be if we disconnected from our side, but didn't unregister the callback in which case it was our responsibility to clean up properly and we shouldn't get to the callback in the first place. Thanks for comments in advance. src/node_device/node_device_udev.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/src/node_device/node_device_udev.c b/src/node_device/node_device_udev.c index 174124a1b..7820c49c4 100644 --- a/src/node_device/node_device_udev.c +++ b/src/node_device/node_device_udev.c @@ -1602,24 +1602,15 @@ nodeStateCleanup(void) static void udevEventHandleCallback(int watch ATTRIBUTE_UNUSED, - int fd, + int fd ATTRIBUTE_UNUSED, int events ATTRIBUTE_UNUSED, void *data ATTRIBUTE_UNUSED) { struct udev_device *device = NULL; struct udev_monitor *udev_monitor = DRV_STATE_UDEV_MONITOR(driver); const char *action = NULL; - int udev_fd = -1; nodeDeviceLock(); - udev_fd = udev_monitor_get_fd(udev_monitor); - if (fd != udev_fd) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("File descriptor returned by udev %d does not " - "match node device file descriptor %d"), - fd, udev_fd); - goto cleanup; - } device = udev_monitor_receive_device(udev_monitor); if (device == NULL) { -- 2.13.2

2 2

[libvirt] [PATCH go-xml] support virtualport for interface and add test code
by ZhenweiPi 26 Jun '17

26 Jun '17

--- domain.go | 29 +++++++++++++++++------------ domain_test.go | 4 ++++ 2 files changed, 21 insertions(+), 12 deletions(-) diff --git a/domain.go b/domain.go index eb7ff9e..ecb8550 100644 --- a/domain.go +++ b/domain.go @@ -185,19 +185,24 @@ type DomainInterfaceDriver struct { Queues uint `xml:"queues,attr,omitempty"` } +type DomainInterfaceVirtualport struct { + Type string `xml:"type,attr"` +} + type DomainInterface struct { - XMLName xml.Name `xml:"interface"` - Type string `xml:"type,attr"` - MAC *DomainInterfaceMAC `xml:"mac"` - Model *DomainInterfaceModel `xml:"model"` - Source *DomainInterfaceSource `xml:"source"` - Target *DomainInterfaceTarget `xml:"target"` - Alias *DomainInterfaceAlias `xml:"alias"` - Link *DomainInterfaceLink `xml:"link"` - Boot *DomainDeviceBoot `xml:"boot"` - Script *DomainInterfaceScript `xml:"script"` - Driver *DomainInterfaceDriver `xml:"driver"` - Address *DomainAddress `xml:"address"` + XMLName xml.Name `xml:"interface"` + Type string `xml:"type,attr"` + MAC *DomainInterfaceMAC `xml:"mac"` + Model *DomainInterfaceModel `xml:"model"` + Source *DomainInterfaceSource `xml:"source"` + Target *DomainInterfaceTarget `xml:"target"` + Alias *DomainInterfaceAlias `xml:"alias"` + Link *DomainInterfaceLink `xml:"link"` + Boot *DomainDeviceBoot `xml:"boot"` + Script *DomainInterfaceScript `xml:"script"` + Driver *DomainInterfaceDriver `xml:"driver"` + Virtualport *DomainInterfaceVirtualport `xml:"virtualport"` + Address *DomainAddress `xml:"address"` } type DomainChardevSource struct { diff --git a/domain_test.go b/domain_test.go index 6b37719..9de725c 100644 --- a/domain_test.go +++ b/domain_test.go @@ -565,6 +565,9 @@ var domainTestData = []struct { Model: &DomainInterfaceModel{ Type: "virtio", }, + Virtualport: &DomainInterfaceVirtualport{ + Type: "openvswitch", + }, }, }, }, @@ -577,6 +580,7 @@ var domainTestData = []struct { ` <interface type="network">`, ` <mac address="00:11:22:33:44:55"></mac>`, ` <model type="virtio"></model>`, + ` <virtualport type="openvswitch"></virtualport>`, ` </interface>`, ` </devices>`, `</domain>`, -- 2.7.4

2 1

[libvirt] [PATCH go-xml] add append attr for chardev
by zhenwei.pi 26 Jun '17

26 Jun '17

--- domain.go | 5 +++-- domain_test.go | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/domain.go b/domain.go index 82bf728..00231b2 100644 --- a/domain.go +++ b/domain.go @@ -227,8 +227,9 @@ type DomainInterface struct { } type DomainChardevSource struct { - Mode string `xml:"mode,attr"` - Path string `xml:"path,attr"` + Mode string `xml:"mode,attr,omitempty"` + Path string `xml:"path,attr"` + Append string `xml:"append,attr,omitempty"` } type DomainChardevTarget struct { diff --git a/domain_test.go b/domain_test.go index 122aada..f50f1fd 100644 --- a/domain_test.go +++ b/domain_test.go @@ -297,6 +297,16 @@ var domainTestData = []struct { Port: &serialPort, }, }, + DomainSerial{ + Type: "file", + Source: &DomainChardevSource{ + Path: "/tmp/serial.log", + Append: "off", + }, + Target: &DomainSerialTarget{ + Port: &serialPort, + }, + }, }, Channels: []DomainChannel{ DomainChannel{ @@ -332,6 +342,10 @@ var domainTestData = []struct { ` <serial type="pty">`, ` <target type="isa" port="0"></target>`, ` </serial>`, + ` <serial type="file">`, + ` <source path="/tmp/serial.log" append="off"></source>`, + ` <target port="0"></target>`, + ` </serial>`, ` <console type="pty">`, ` <target type="virtio" port="0"></target>`, ` </console>`, -- 2.7.4

2 1

[libvirt] [PATCH] qemu: Avoid fd leak on incoming tunneled migration
by Jiri Denemark 26 Jun '17

26 Jun '17

While qemuProcessIncomingDefNew takes an fd argument and stores it in qemuProcessIncomingDef structure, the caller is still responsible for closing the file descriptor. Introduced by commit v1.2.21-140-ge7c6f4575. Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_migration.c | 1 - src/qemu/qemu_process.c | 3 +++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 8129dcd40..c23fffef2 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2690,7 +2690,6 @@ qemuMigrationPrepareAny(virQEMUDriverPtr driver, listenAddress, port, dataFD[0]))) goto stopjob; - dataFD[0] = -1; /* the FD is now owned by incoming */ if (qemuProcessPrepareDomain(dconn, driver, vm, VIR_QEMU_PROCESS_START_AUTODESTROY) < 0) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index fa9990e5d..443ec88fe 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4169,6 +4169,9 @@ qemuProcessIncomingDefFree(qemuProcessIncomingDefPtr inc) * This function does not copy @path, the caller is responsible for keeping * the @path pointer valid during the lifetime of the allocated * qemuProcessIncomingDef structure. + * + * The caller is responsible for closing @fd, calling + * qemuProcessIncomingDefFree will NOT close it. */ qemuProcessIncomingDefPtr qemuProcessIncomingDefNew(virQEMUCapsPtr qemuCaps, -- 2.13.1

2 2