- Devel - libvirt List Archives

[libvirt] nodeinfo test cases for x86_64 AMD CPUs
by Steffen Persvold 22 Jan '15

22 Jan '15

Hi, Lately we’ve been puzzled by the nodeinfo returned for AMD 63xx based platforms so I checked out libvirt from Git and checked out the testcases in test/nodeinfodata. Currently you have 3 AMD test cases there : linux-x86_64-test3 : AMD 6172 2.1 GHz (MCM, 12 cores per package/socket, 2 numa nodes per package/socket, 1 thread/core) 4 Sockets, 8 NUMA nodes, 48 CPU cores total. linux-x86_64-test7 : AMD 6174 2.2 GHz (MCM, 12 cores per package/socket, 2 numa nodes per package/socket, 1 thread/core) 2 Sockets, 4 NUMA nodes, 24 CPU cores total. linux-x86_64-test8 : AMD 6282 SE 2.6 GHz (MCM, 8 CU(core) per package/socket, 2 numa nodes per package/socket, 2 threads/CU(core)) 4 Sockets, 8 NUMA nodes, 64 CPU cores total. However, the “expected” output from each of these are wrong I believe : % ~/libvirt/tests/nodeinfodata$ cat linux-x86_64-test{3,7,8}.expected CPUs: 48/48, MHz: 2100, Nodes: 8, Sockets: 1, Cores: 6, Threads: 1 CPUs: 24/24, MHz: 2200, Nodes: 1, Sockets: 1, Cores: 24, Threads: 1 CPUs: 64/64, MHz: 2593, Nodes: 1, Sockets: 1, Cores: 64, Threads: 1 In my opinion it should have been : CPUs: 48/48, MHz: 2100, Nodes: 8, Sockets: 4, Cores: 12, Threads: 1 CPUs: 24/24, MHz: 2200, Nodes: 4, Sockets: 2, Cores: 12, Threads: 1 CPUs: 64/64, MHz: 2593, Nodes: 8, Sockets: 4, Cores: 8, Threads: 2 Atleast that’s more in line with what tools like “lscpu” and “lstopo” would report. For example, I have a dual-socket AMD 6376 based system. lscpu reports : $ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 32 On-line CPU(s) list: 0-31 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 2 NUMA node(s): 4 Vendor ID: AuthenticAMD CPU family: 21 Model: 2 Stepping: 0 CPU MHz: 2299.820 BogoMIPS: 4600.03 Virtualization: AMD-V L1d cache: 16K L1i cache: 64K L2 cache: 2048K L3 cache: 6144K NUMA node0 CPU(s): 0-7 NUMA node1 CPU(s): 8-15 NUMA node2 CPU(s): 16-23 NUMA node3 CPU(s): 24-31 virsh nodeinfo however : CPU model: x86_64 CPU(s): 32 CPU frequency: 2299 MHz CPU socket(s): 1 Core(s) per socket: 32 Thread(s) per core: 1 NUMA cell(s): 1 Memory size: 49448656 KiB Looking more closely at the code in src/nodeinfo.c:linuxNodeInfoCPUPopulate() I believe it makes some false assumptions. When iterating over the NUMA nodes, it expects that NUMA nodes can contain more than one socket, whereas on AMD systems atleast it’s the other way around (Sockets can contain more than 1 socket). Hence, the topology check at the end goes all wrong and libvirt uses just a flat topology : if ((nodeinfo->nodes * nodeinfo->sockets * nodeinfo->cores * nodeinfo->threads) != (nodeinfo->cpus + offline)) { nodeinfo->nodes = 1; nodeinfo->sockets = 1; nodeinfo->cores = nodeinfo->cpus + offline; nodeinfo->threads = 1; } If instead the “fallback” mechanism is used, which iterates over /sys/devices/system/cpu/cpu%d and finds sockets/cores/threads is used, *and* you leave nodeinfo->nodes out of the equation then it all makes sense. Iterating over entries in /sys/devices/system/node/node%d would then only be used to find nodeinfo->nodes. I have limited insight into how this would work on say an S390 system, so if my assumptions here are completely wrong please do tell :) Cheers, -- Steffen Persvold Chief Architect NumaChip, Numascale AS Tel: +47 23 16 71 88 Fax: +47 23 16 71 80 Skype: spersvold

1 1

[libvirt] (no subject)
by akrowiak＠linux.vnet.ibm.com 22 Jan '15

22 Jan '15

From: Tony Krowiak <akrowiak(a)linux.vnet.ibm.com> This patch set provides the code to synchonize some macvtap device modes when the values are changed on the guest's network device. The following modes will by synchronized: * PROMISC * MULTICAST * ALLMULTI I noticed something while testing this patch set that did not occur prior to installing more recent kernel and Qemu distributions. It seems that the VLAN table always has an entry for VLAN ID 0 when the rxfilter is queried during processing of the NIC_RX_FILTER_CHANGED event. That means that the PROMISC flag for macvtap0 will be set. I don't know if this will cause problems, but I thought I'd make note of it in case anybody wants to comment on that. Tony Krowiak (2): util: Functions for getting/setting device options qemu: change macvtap device options in response to NIC_RX_FILTER_CHANGED

1 4

[libvirt] [PATCH 0/2] fix two issues found by clang
by Pavel Hrdina 22 Jan '15

22 Jan '15

Pavel Hrdina (2): xenapi_driver: fix copy-paste typo esx_vi: fix possible segfault src/esx/esx_vi.c | 3 ++- src/xenapi/xenapi_driver.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) -- 2.0.5

2 6

[libvirt] [PATCH 0/8] Enable NIC reporting to systemd
by Daniel P. Berrange 22 Jan '15

22 Jan '15

This series enables the QEMU and LXC drivers to report the network interface backends they use to systemd. This gets then shown to the user in # machinectl status lxc-shell lxc-shell(95449419f969d649d9962566ec42af7d) Since: Fri 2015-01-16 16:53:37 GMT; 3s ago Leader: 28085 (sh) Service: libvirt-lxc; class container Iface: vnet0 Address: fe80::216:3eff:fe00:c317%124 OS: Fedora 21 (Twenty One) Unit: machine-lxc\x2dshell.scope └─28085 /bin/sh but most fun is that if you add nss-mymachines to the /etc/nsswitch.conf, the machine names can now be used as hostnames. eg if your guest is 'lxc-shell' this lets you just do 'ssh lxc-shell' and it'll use the detected link local address to connect. Daniel P. Berrange (8): qemu: report TAP device indexes to systemd systemd: don't report an error if the guest is already terminated lxc: don't build pidfile string multiple times lxc: re-arrange startup synchronization sequence with controller lxc: only write XML once for lxc controller lxc: delay setup of cgroup until we have the init pid lxc: more logging during startup paths lxc: report veth device indexes to systemd src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 5 ++ src/lxc/lxc_cgroup.c | 15 +++--- src/lxc/lxc_cgroup.h | 5 +- src/lxc/lxc_container.c | 8 +++ src/lxc/lxc_controller.c | 116 ++++++++++++++++++++++++++++++++++++---- src/lxc/lxc_process.c | 134 +++++++++++++++++++++++++++-------------------- src/qemu/qemu_cgroup.c | 13 +++-- src/qemu/qemu_cgroup.h | 4 +- src/qemu/qemu_command.c | 27 ++++++++-- src/qemu/qemu_command.h | 7 ++- src/qemu/qemu_driver.c | 7 ++- src/qemu/qemu_hotplug.c | 4 +- src/qemu/qemu_process.c | 9 +++- src/util/virsystemd.c | 25 +++++++-- tests/qemuxml2argvtest.c | 5 +- tests/qemuxmlnstest.c | 6 ++- 17 files changed, 288 insertions(+), 104 deletions(-) -- 2.1.0

2 11

[libvirt] [PATCH] network: verify proper address family in updates to <host> and <range>
by Laine Stump 22 Jan '15

22 Jan '15

By specifying parentIndex in a call to virNetworkUpdate(), it was possible to direct libvirt to add a dhcp range or static host of a non-matching address family to the <dhcp> element of an <ip>. For example, given: <ip address='192.168.122.1' netmask='255.255.255.0'/> <ip family='ipv6' address='2001:db6:ca3:45::1' prefix='64'/> you could provide a static host entry with an IPv4 address, and specify that it be added to the 2nd <ip> element (index 1): virsh net-update default add ip-dhcp-host --parent-index 1 \ '<host mac="52:54:00:00:00:01" ip="192.168.122.45"/>' This would be happily added with no error (and no concern of any possible future consequences). This patch checks that any dhcp range or host element being added to a network ip's <dhcp> subelement has addresses of the same family as the ip element they are being added to. This problem was noticed when looking at the reproduction case for https://bugzilla.redhat.com/show_bug.cgi?id=1182486 (but is not a solution to that bug). --- src/conf/network_conf.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/src/conf/network_conf.c b/src/conf/network_conf.c index 23ec369..15501ec 100644 --- a/src/conf/network_conf.c +++ b/src/conf/network_conf.c @@ -1,7 +1,7 @@ /* * network_conf.c: network XML handling * - * Copyright (C) 2006-2014 Red Hat, Inc. + * Copyright (C) 2006-2015 Red Hat, Inc. * Copyright (C) 2006-2008 Daniel P. Berrange * * This library is free software; you can redistribute it and/or @@ -3285,6 +3285,14 @@ virNetworkDefUpdateIPDHCPHost(virNetworkDefPtr def, &host, partialOkay) < 0) goto cleanup; + if (VIR_SOCKET_ADDR_FAMILY(&ipdef->address) + != VIR_SOCKET_ADDR_FAMILY(&host.ip)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("the address family of a host entry IP must match " + "the address family of the dhcp element's parent")); + goto cleanup; + } + if (command == VIR_NETWORK_UPDATE_COMMAND_MODIFY) { /* search for the entry with this (ip|mac|name), @@ -3422,6 +3430,14 @@ virNetworkDefUpdateIPDHCPRange(virNetworkDefPtr def, if (virSocketAddrRangeParseXML(def->name, ctxt->node, &range) < 0) goto cleanup; + if (VIR_SOCKET_ADDR_FAMILY(&ipdef->address) + != VIR_SOCKET_ADDR_FAMILY(&range.start)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("the address family of a dhcp range must match " + "the address family of the dhcp element's parent")); + goto cleanup; + } + /* check if an entry with same name/address/ip already exists */ for (i = 0; i < ipdef->nranges; i++) { if (virSocketAddrEqual(&range.start, &ipdef->ranges[i].start) && -- 1.9.3

2 2

[libvirt] [PATCH v3 1/2] qemu: output error when try to hotplug unsupport console
by Luyao Huang 22 Jan '15

22 Jan '15

https://bugzilla.redhat.com/show_bug.cgi?id=1164627 When using 'virsh attach-device' to hotplug an unsupported console type into a qemu guest, the attachment will erroneously allows the attachment. This patch will check to ensure that only the support target types are used for a running guest. NB, Although a guest can be defined with an improper target, startup will cause failure. Signed-off-by: Luyao Huang <lhuang(a)redhat.com> --- src/qemu/qemu_command.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index c041ee7..df87e1e 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -10067,13 +10067,17 @@ qemuBuildConsoleChrDeviceStr(char **deviceStr, break; case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_SERIAL: + break; case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_NONE: case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_XEN: case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_UML: case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_LXC: case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_OPENVZ: case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_LAST: - break; + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("unsupported console target type %s"), + NULLSTR(virDomainChrConsoleTargetTypeToString(chr->targetType))); + goto cleanup; } ret = 0; -- 1.8.3.1

3 7

[libvirt] [PATCH 0/2] CVE-2015-0236: Check ACLs for the VIR_DOMAIN_XML_SECURE flag for snapshots and save images
by Peter Krempa 22 Jan '15

22 Jan '15

Patches are pushed according to Eric's ACK on the security list. Peter Krempa (2): CVE-2015-0236: qemu: Check ACLs when dumping security info from save image CVE-2015-0236: qemu: Check ACLs when dumping security info from snapshots src/qemu/qemu_driver.c | 4 ++-- src/remote/remote_protocol.x | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) -- 2.2.1

1 2

[libvirt] [RFC PATCH 00/11] qemu: add support to hotplug memory device
by Zhu Guihua 22 Jan '15

22 Jan '15

Now qemu has already supported memory hotplug, so this patchset will make libvirt support hotplug memory device for qemu driver. First, add two parameters slots and maxmem for memory hotplug, and the xml format can be like this. <memory slot='10'>102400</memory> <maxMemory>1024000</maxMemory> Second, memory device's xml format can be like this. <dimm driver='pc-dimm' addr='0' node='0' slot='1'> <backend type='ram' size='131072'/> </dimm> Zhu Guihua (11): domain_conf: support slots and maxmem properties in memory xml domain_conf: introduce virDomainDimmGetFreeSlot domain_conf: add support for memory device configuration in XML domain_conf: introduce dimm def helpers domain_conf: introduce dimm device hotplug helpers qemu: implement dimm device hotplug on config level qemu_monitor: introduce qemuMonitorAddMemoryBackend qemu_command: introduce a func for memory device alias qemu: implement support for pc-dimm qemu: introduce qemuBuildDimmDeviceStr qemu: implement dimm device hotplug on live level src/conf/domain_conf.c | 306 ++++++++++++++++++++++++++++++++++++++++++- src/conf/domain_conf.h | 54 ++++++++ src/libvirt_private.syms | 6 + src/qemu/qemu_capabilities.c | 3 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 110 +++++++++++++++- src/qemu/qemu_command.h | 7 + src/qemu/qemu_driver.c | 26 ++++ src/qemu/qemu_hotplug.c | 71 ++++++++++ src/qemu/qemu_hotplug.h | 5 + src/qemu/qemu_monitor.c | 54 ++++++++ src/qemu/qemu_monitor.h | 4 + 12 files changed, 644 insertions(+), 3 deletions(-) -- 1.9.3

2 14

[libvirt] RFC: Building a virtlogd daemon
by Daniel P. Berrange 22 Jan '15

22 Jan '15

With QEMU there are a couple of areas where QEMU ends up logging data to a file on the host. - stdout/err - connected to /var/log/libvirt/qemu/$GUEST.log - serial/parallel/console when used with type=file The stdout/err is typically very small, typically only getting data when an error occurs in QEMU resulting in it existing. In the past there was the chance of it getting lots of data - eg spice used to write tonnes of data on stderr during normal operation, so a guest OS could cause large data to be saved to the host log file. This is harder todo now, but still theoretically possible. The serial/parallel/console log files can be arbitrarily sized and under complete control of the guest OS - it can write whatever it wants with no limits. In both these cases we would prefer to be able to limit the amount of data a guest can write to the host file, so we have a finite cap on disk usage. You might suggest using logrotate, but that only runs periodically so between invocations of logrotate the log files can still grow to arbitrary size. This is inherant problem in doing the log rotation out of band / asynchronously OpenStack has a further problem it would like to solve. It wants to record all guest serial port data to a log file, as it has an API for a client user to read historically captured data from the serial port. This is done using type=file chardev At the same time it wants to expose an interactive console for client users to interact with the guest. This is done using type=unix or type=pty chardevs. The obvious problem is that a single serial/parallel/console device can only be connected to one chardev backend at a time. A very long time ago (5-6 years?) I submitted patches to allow multiple chardev backends in QEMU but they were rejected. Downstream projects like OpenStack have considered setting up console log handling service themselves, but my view is that this is a problem that all users of libvirt face, so libvirt should provide a standard solution to it. This avoids each downstream app reinventing the wheel. So I'm intending to create a standalone virtlogd daemon to address this problem. Similarly to virtlockd, it will be able to re-exec itelf so that upgrades can be done with no interruption to logging, and libvirtd will talk to it over a simple RPC protocol. - For stdout/stderr - libvirtd will ask virtlogd will provide a pipe FD that can be connected to the guest stdout/err, in the sme way libvirtd provides a file FD today. - virtlogd will read from this pipe and write the data to /var/log/libvirt/qemu/$GUEST.log - virtlogd will either truncate or rotate the $GUEST.log file when it reaches a declared certain size limit. - For serial/parallel/console - libvirtd will ask virtlogd to setup a pair of UNIX domain sockets listening on /var/run/virtlogd/qemu/$GUEST.guest /var/run/virtlogd/qemu/$GUEST.client - libvirtd will ask virtlogd to (optionally) write the data to a file /var/log/virtlogd/qemu/$GUEST.log (Or /var/log/libvirt/qemu/$GUEST-$DEVICE.log perhaps, where $DEVICE is the <alias> string from the console, serial or parallel device ?) - QEMU will be told to connect to this $GUEST.guest socket with the type=unix chardev backend - The virDomainOpenConsole API will connect to the $GUEST.client socket - virtlogd will read data from $GUEST.guest socket and write it to $GUEST.client and optionally $GUEST.log too - virtlogd will read data from $GUEST.client socket and write it to $GUEST.guest socket - virtlogd will either truncate or rotate the $GUEST.log file when it reaches a declared certain size limit. So at the end of this we will have strictly size limited log files which can rotated at the size threshold, and the ability to have interactive console sessions and file logging at the same time. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

3 2

[libvirt] [libvirt-test-API][PATCH] Add connection_cpu_models test case
by jiahu 22 Jan '15

22 Jan '15

The connection_cpu_models.py uses getCPUModelNames() to validate new API virConnectGetCPUModelNames of libvirt. --- cases/test_connection.conf | 12 +++++ repos/virconn/connection_cpu_models.py | 82 ++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 repos/virconn/connection_cpu_models.py diff --git a/cases/test_connection.conf b/cases/test_connection.conf index ccde119..e916886 100644 --- a/cases/test_connection.conf +++ b/cases/test_connection.conf @@ -29,3 +29,15 @@ virconn:connection_nodeinfo virconn:connection_version conn lxc:/// + +virconn:connection_cpu_models + arch + x86_64 + +virconn:connection_cpu_models + arch + i686 + +virconn:connection_cpu_models + arch + ppc64 diff --git a/repos/virconn/connection_cpu_models.py b/repos/virconn/connection_cpu_models.py new file mode 100644 index 0000000..4588188 --- /dev/null +++ b/repos/virconn/connection_cpu_models.py @@ -0,0 +1,82 @@ +#!/usr/bin/env python +# test getCPUModelNames() API for libvirt + +import os +import libvirt + +from xml.dom import minidom +from libvirt import libvirtError +from src import sharedmod +from utils import utils + +required_params = ('arch',) +optional_params = {} + +CPU_MAP_FILE = "/usr/share/libvirt/cpu_map.xml" + +def get_cpu_archs_from_xml(logger): + """ + return supported cpu archs from cpu_map.xml + """ + cpu_archs_from_xml = [] + xml = minidom.parse(CPU_MAP_FILE) + for arch in xml.getElementsByTagName('arch'): + cpu_archs_from_xml.append(str(arch.getAttribute('name'))) + return cpu_archs_from_xml + +def get_cpu_models_from_xml(arch, logger): + """ + return supported cpu models from cpu_map.xml + """ + cpu_models_from_xml = [] + if arch == 'x86_64' or arch == 'i686': + real_arch = 'x86' + else: + real_arch = arch + + xml = minidom.parse(CPU_MAP_FILE) + for model in xml.getElementsByTagName('model'): + if model.parentNode.getAttribute('name') == real_arch: + cpu_models_from_xml.append(str(model.getAttribute('name'))) + return cpu_models_from_xml + +def connection_cpu_models(params): + """ + test API for getCPUModelNames in class virConnect + """ + logger = params['logger'] + arch_value = params['arch'] + try: + logger.info("get cpu archs from cpu_map.xml") + if not os.path.exists(CPU_MAP_FILE): + logger.error("%s is not exist" % CPU_MAP_FILE) + return 1 + cpu_archs_from_xml = get_cpu_archs_from_xml(logger) + logger.info("The supported cpu archs in xml are %s" \ + % cpu_archs_from_xml) + cpu_models_from_xml = get_cpu_models_from_xml(arch_value, logger) + logger.info("The supported cpu models in xml are %s" \ + % cpu_models_from_xml) + + conn = sharedmod.libvirtobj['conn'] + + cpu_models_from_libvirt = conn.getCPUModelNames(arch_value ,0) + logger.info("The specified architecture is %s" \ + % arch_value) + logger.info("The supported cpu models is %s" \ + % cpu_models_from_libvirt) + + #compare with cpu_map.xml + for cpu_model in cpu_models_from_libvirt: + if cpu_model in cpu_models_from_xml: + logger.debug("'%s' model: PASS" % cpu_model) + else: + logger.debug("'%s' model: FAIL, not in libvirt"\ + % cpu_model) + return 1 + logger.debug("check all cpu models: PASS") + except libvirtError, e: + logger.error("API error message: %s" % e.message) + return 1 + + return 0 -- 1.8.1.4

2 1