- Devel - libvirt List Archives

[libvirt] [ libvirt-python BUG ?] Memory leak in libvirt_virConnectDomainEventJobCompletedCallback
by Hailiang Zhang 20 Jan '17

20 Jan '17

Hi all, It seems that, there is a memory leak bug in libvirt_virConnectDomainEventJobCompletedCallback() function of libvirt-override.c. We didn't decrease the reference of 'pyobj_dict' in this function or any other places while pyt0bj_ret is not zero. libvirt_virConnectDomainEventJobCompletedCallback() { ... ... pyobj_ret = PyObject_CallMethod(pyobj_conn, (char*)"_dispatchDomainEventJobCompletedCallback", (char*)"OOO", pyobj_dom, pyobj_dict, pyobj_cbData); Py_DECREF(pyobj_cbData); Py_DECREF(pyobj_dom); cleanup: if (!pyobj_ret) { DEBUG("%s - ret:%p\n", __FUNCTION__, pyobj_ret); PyErr_Print(); Py_XDECREF(pyobj_dict); } else { Py_DECREF(pyobj_ret); ret = 0; } LIBVIRT_RELEASE_THREAD_STATE; return ret; } I don't think the reference of pyobj_dict will be stolen by PyObject_CallMethod when it is passed to the Python function _dispatchDomainEventJobCompletedCallback. In fact, we have a test which triggered lots of events and found that the memory of the test process will keep raising slowly and could not be reclaimed even we called gc.collect() explicitly in our codes. So, is it a bug ? Any comments ? Thanks, hailiang

2 1

[libvirt] [V4] RFC for support cache tune(CAT) in libvirt
by 乔立勇(Eli Qiao) 20 Jan '17

20 Jan '17

hi all Sorry for resend this again, I forget to add subject :( ... Thanks all for the comment for previous RFC version, I summary all input here also with Opens which I am not sure, can you please review again? #Propose Changes ## Expose cache information into capabilities Extend capabilities to expose all level cache resource information and should tell the topology of caches also tell which kinds of resources can be tuned/allocation There information comes from /sys/devices/system/cpu/cpu0/cache/index${level}/size also /sys/fs/resctrl/ (this is CAT sys interface in linux kernel) virsh capabilities <cache> <bank id='0, 'type="l3" size="56320" units="KiB" cpus="0,1,2,6,7,8"/> <--------------------- level 3 cache is per socket, so group them by socket id <control unit="KiB" min="2816"/> <bank id='1', type="l3" size="56320" units="KiB" cpus="3,4,5,9,10,11"/> <bank id='2' type="l2" size="256" units="KiB" cpus="0"/> <bank id='3' type="l2" size="256" units="KiB" cpus="1"/> <bank id='4' type="l2" size="256" units="KiB" cpus="2"/> <bank id='5' type="l2" size="256" units="KiB" cpus="3"/> <bank id='6' type="l2" size="256" units="KiB" cpus="4"/> ... <cache> Opens 1. how about add socket id to bank for bank type = l3 ? 2. do we really want to expose l2/l3 cache for now , they are per core resource and linux kernel don't support l2 yet (depend no hardware)? 3. if enable CDP in resctrl, for bank type=l3 , it will be split to l3data l3code, should expose this ability. <bank type="l3" size="56320" units="KiB" cpus="0,1,2,6,7,8"/> <--------------------- level 3 cache is per socket, so group them by socket id <control unit="KiB" min="2816" cdp="enabled"/> ## Provide a new API to get the avail cache on each bank, such as the output are: id=0 type=l3 avail=56320 total = ?? <--------- do we need this? id=1 type=l3 avail=56320 id=3 type=l2 avail=256 Opens: · Don't expose the avail cache information if the host can not do the allocation of that type cache(eg, for l2 currently) ? · We can not make all of the cache , the reservation amount is the min_cbm_len (=1) * min_unit . · do we need to expose total? ## enable CAT for a domain 1 Domain XML changes <cputune> <cache id="1" host_id="0" type="l3" size="5632" unit="KiB"/> <cache id="2" host_id="1" type="l3" size="5632" unit="KiB"/> <cpu_cache vcpus="0-3" id="1"/> <cpu_cache vcpus="4-7" id="2"/> <iothread_cache iothreads="0-1" id="1"/> <emulator_cache id="2"/> </cputune> 2. Extend cputune command ? Opens: 1. Do we accept to extend existed API ? or using new API/virsh? 2. How to calculate cache size -> CBM bit? eg: 5632/ 2816 = 2 bits 5733/ 2816 = 2 bits or 3 bits? ## Restriction for using cache tune on multiple sockets' host. The l3 cache is per socket resource, kernel need to know about what's affinity looks like, so for a VM which running on a multiple socket's host, it should have NUMA setting or vcpuset pin setting. Or cache tune will fail. [1] kernel support https://git.kernel.org/cgit/linux/kernel/git/tip/tip .git/tree/arch/x86/kernel/cpu/intel_rdt.c?h=x86/cache [2] libvirt PoC(not finished yet) https://github.com/taget/ libvirt/commits/cat_new -- Best regards - Eli 天涯无处不重逢 a leaf duckweed belongs to the sea , where not to meet in life

3 3

[libvirt] Mirror on Github failed to sync from libvirt.org
by Chen Hanxiao 20 Jan '17

20 Jan '17

Dear list, Mirror on Github: https://github.com/libvirt/libvirt did not sync from libvirt.org for almost a month. Regards, - Chen

2 1

[libvirt] [PATCH] Interface: return appropriate status for bridge device
by Lin Ma 20 Jan '17

20 Jan '17

In function udevInterfaceIsActive, The current design relies on the sys attribute 'operstate' to determine whether the interface is active. For the device node of virtual network(say virbr0), There is already a dummy tap device to maintain a fixed mac on it, So it's available and its status should be considered as active. But if no anyelse tap device connects to it, The value of operstate of virbr0 in sysfs is 'down', So udevInterfaceIsActive returns 0, It causes 'virsh iface-list --all' to report the status of virbr0 as 'inactive'. This patch fixes the issue by counting slave devices for bridge device. Signed-off-by: Lin Ma <lma(a)suse.com> --- src/interface/interface_backend_udev.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/interface/interface_backend_udev.c b/src/interface/interface_backend_udev.c index 5d0fc64..9ac4674 100644 --- a/src/interface/interface_backend_udev.c +++ b/src/interface/interface_backend_udev.c @@ -1127,6 +1127,11 @@ udevInterfaceIsActive(virInterfacePtr ifinfo) struct udev_device *dev; virInterfaceDefPtr def = NULL; int status = -1; + struct dirent **member_list = NULL; + const char *devtype; + int member_count = 0; + char *member_path; + size_t i; dev = udev_device_new_from_subsystem_sysname(udev, "net", ifinfo->name); @@ -1146,6 +1151,23 @@ udevInterfaceIsActive(virInterfacePtr ifinfo) /* Check if it's active or not */ status = STREQ(udev_device_get_sysattr_value(dev, "operstate"), "up"); + if (!status) { + devtype = udev_device_get_devtype(dev); + if (STREQ_NULLABLE(devtype, "bridge")) { + if (virAsprintf(&member_path, "%s/%s", + udev_device_get_syspath(dev), "brif") < 0) + goto cleanup; + member_count = scandir(member_path, &member_list, + udevBridgeScanDirFilter, alphasort); + if (member_count > 0) + status = 1; + for (i = 0; i < member_count; i++) + VIR_FREE(member_list[i]); + VIR_FREE(member_list); + VIR_FREE(member_path); + } + } + udev_device_unref(dev); cleanup: -- 2.9.2

1 0

[libvirt] [PATCH] bhyve: fix interface type handling for argv2xml
by Roman Bogorodskiy 20 Jan '17

20 Jan '17

When generating a domain XML from native command (i.e. via the connectDomainXMLFromNative call), we should use interface type 'bridge' rather than 'ethernet' because we only support bridges at this point. As we don't have bridge name explicitly specified on the command line, just use 'virbr0' as a default. --- src/bhyve/bhyve_parse_command.c | 9 ++++++--- tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net.xml | 6 ++++-- tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net4.xml | 3 ++- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/src/bhyve/bhyve_parse_command.c b/src/bhyve/bhyve_parse_command.c index 619004298..5b8712388 100644 --- a/src/bhyve/bhyve_parse_command.c +++ b/src/bhyve/bhyve_parse_command.c @@ -507,9 +507,12 @@ bhyveParsePCINet(virDomainDefPtr def, if (VIR_ALLOC(net) < 0) goto cleanup; - /* Let's just assume it is VIR_DOMAIN_NET_TYPE_ETHERNET, it could also be - * a bridge, but this is the most generic option. */ - net->type = VIR_DOMAIN_NET_TYPE_ETHERNET; + /* As we only support interface type='bridge' and cannot + * guess the actual bridge name from the command line, + * try to come up with some reasonable defaults */ + net->type = VIR_DOMAIN_NET_TYPE_BRIDGE; + if (VIR_STRDUP(net->data.bridge.brname, "virbr0") < 0) + goto error; net->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; net->info.addr.pci.slot = pcislot; diff --git a/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net.xml b/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net.xml index 09cc79b92..5895c8c53 100644 --- a/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net.xml +++ b/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net.xml @@ -12,13 +12,15 @@ <on_reboot>destroy</on_reboot> <on_crash>destroy</on_crash> <devices> - <interface type='ethernet'> + <interface type='bridge'> <mac address='52:54:00:00:00:00'/> + <source bridge='virbr0'/> <target dev='tap0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </interface> - <interface type='ethernet'> + <interface type='bridge'> <mac address='fe:ed:ad:ea:df:15'/> + <source bridge='virbr0'/> <target dev='tap1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </interface> diff --git a/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net4.xml b/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net4.xml index e1bda46a1..5f1972080 100644 --- a/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net4.xml +++ b/tests/bhyveargv2xmldata/bhyveargv2xml-virtio-net4.xml @@ -12,8 +12,9 @@ <on_reboot>destroy</on_reboot> <on_crash>destroy</on_crash> <devices> - <interface type='ethernet'> + <interface type='bridge'> <mac address='00:00:00:00:00:00'/> + <source bridge='virbr0'/> <target dev='tap1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </interface> -- 2.11.0

2 2

[libvirt] The libvirt Release Notes Game, v3.0.0 edition
by Andrea Bolognani 19 Jan '17

19 Jan '17

Hello everyone, and welcome to the libvirt Release Notes Game, v3.0.0 edition! Here's the deal: libvirt v3.0.0 is right around the corner, and so it's about time we get the release notes into shape. Most of the work has thankfully already been done during the development cycle, by bundling release notes entries with the code that implements the actual changes; however, some changes have managed to fly under the radar so far. Not documenting them properly in the release notes would be a disservice to our users, and would undersell what is once again a terrific release - I'd venture as far as calling it the best one of 2017! I've gone through all the commits between v2.5.0 and now, and highlighted the changes that look like they should be part of the release notes. Look for your name in the list below, and spend a few minutes to craft a beautiful release notes entry for each of the highlighted changes, then send the patch to the list as usual. Feel free to CC: me for a guaranteed[1] swift review! Please note that, unlike you, I don't have intimate knowledge of each and every line of code you've ever contributed to libvirt, and as such might have misjudged the weight and scope of the commits listed below, but that's the whole point: the only person worthy of documenting a change in the release notes is the author of the change himself! That said, if you feel that any of the changes below should not end up in the release notes feel free to raise an objection :) Happy documenting! ^^ * Cédric Bosdonnat - a30b08b libxl: define a per-domain logger. - 340bb6b libxl: add QED disk format support * Collin L. Walling, Jason J. Herne - 79d7201 s390: Cpu driver support for update and compare ..d47db7b qemu: command: Support new cpu feature argument syntax * Daniel P. Berrange - c500701 Add domain event for metadata changes ..dc2bfdc Update remote_protocol-structs for new events - 44f79a0 lxc: ensure libvirt_lxc and qemu-nbd move into systemd machine slice * intrigeri - de79efd AppArmor policy: support merged-/usr. ..a73e703 AppArmor: allow QEMU to set_process_name. * John Ferlan - 2b13361 nodedev: Add the ability to create vHBA by parent wwnn/wwpn or fabric_wwn * Laine Stump - 9838cad conf: use struct instead of int for each slot in virDomainPCIAddressBus ..5949b53 conf: eliminate virDomainPCIAddressReleaseSlot() in favor of ...Addr() * Lin Ma - 2922cd9 cpu: Add support for more AVX512 Intel features ..c80e6b9 cpu: Add support for pku and ospke Intel features for Memory Protection Keys * Marc Hartmayer - c344d4b conf: simplify functions virDomainSCSIDriveAddressIsUsedBy*() ..36d9965 tests: add test cases for address conflicts * Maxim Nestratov - ef5c8bb qemu: Fix pit timer tick policy=delay ..245d9ba tests: Add "no-kvm-pit-device" testcase * Mehdi Abaakouk - 013df87 Gathering vhostuser interface stats with ovs * Michal Privoznik - f55afd8 qemu: Create hugepage path on per domain basis * Nikolay Shirokovskiy - 61a0026 qemu: Fix xml dump of autogenerated websocket * Pavel Glushchak - 5bafa1d vz: set PVMT_DONT_CREATE_DISK migration flag ..b1f916a vz: added VIR_MIGRATE_NON_SHARED_INC migration flag support * Peter Krempa - b469853 qemu: blockjob: Fix locking of block copy/active block commit - f61e406 qemu: snapshot: Properly handle image locking - a4ed5b4 qemu: Don't try to find compression program for "raw" memory images - 9e93055 qemu: block copy: Forbid block copy to relative paths * Pino Toscano - 408a1ce rpc: libssh: allow a NULL known_hosts file ..1a5de3f remote: do not check for an existing config dir [1] Or you money back! -- Andrea Bolognani / Red Hat / Virtualization

4 4

[libvirt] [PATCH for 3.0.x] Disable use of namespaces by default
by Daniel P. Berrange 19 Jan '17

19 Jan '17

When namespaces are enabled there is currently breakage when using disk hotplug and when using AppArmor Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- I'm suggesting this for 3.0.x branch - we'll leave them enabled in master on the basis that we'll actually fix the real bugs there. src/qemu/qemu_conf.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 6613d59..aa05b46 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -317,13 +317,6 @@ virQEMUDriverConfigPtr virQEMUDriverConfigNew(bool privileged) if (!(cfg->namespaces = virBitmapNew(QEMU_DOMAIN_NS_LAST))) goto error; -#if defined(__linux__) - if (privileged && - virProcessNamespaceAvailable(VIR_PROCESS_NAMESPACE_MNT) == 0 && - virBitmapSetBit(cfg->namespaces, QEMU_DOMAIN_NS_MOUNT) < 0) - goto error; -#endif /* defined(__linux__) */ - #ifdef DEFAULT_LOADER_NVRAM if (virFirmwareParseList(DEFAULT_LOADER_NVRAM, &cfg->firmwares, -- 2.9.3

3 3

[libvirt] [PATCH] storage: avoid use of undefined GLUSTER_CLI variable
by Daniel P. Berrange 19 Jan '17

19 Jan '17

Previous commit tried to change configure logic such that the GLUSTER_CLI parameter would always be set: commit 9e97c8c0f0f3921d06bac2b92cd094a41373f748 Author: Peter Krempa <pkrempa(a)redhat.com> Date: Mon Jan 9 15:56:12 2017 +0100 storage: gluster: Remove build-time dependency on the 'gluster' cli tool This missed the fact that the AC_PATH_PROG call was itself inside an 'if' conditional that would not be called in with_storage_gluster was false. As a result, GLUSTER_CLI was still conditionally defined. Just kill the GLUSTER_CLI parameter and AC_PATH_PROG call entirely and pass a bare "gluster" string to virFindFileInPath instead. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- Technically this is a build-break fix for FreeBSD, but I want to get a review to double-check before pushing this. configure.ac | 9 --------- src/storage/storage_util.c | 2 +- 2 files changed, 1 insertion(+), 10 deletions(-) diff --git a/configure.ac b/configure.ac index a217fc1..7efaddb 100644 --- a/configure.ac +++ b/configure.ac @@ -603,15 +603,6 @@ LIBVIRT_STORAGE_CHECK_SHEEPDOG LIBVIRT_STORAGE_CHECK_GLUSTER LIBVIRT_STORAGE_CHECK_ZFS -if test "$with_storage_fs" = "yes" || - test "$with_storage_gluster" = "yes"; then - AC_PATH_PROG([GLUSTER_CLI], [gluster], [gluster], [$LIBVIRT_SBIN_PATH]) - if test "x$GLUSTER_CLI" != "x"; then - AC_DEFINE_UNQUOTED([GLUSTER_CLI], ["$GLUSTER_CLI"], - [Location or name of the gluster command line tool]) - fi -fi - with_storage=no for backend in dir fs lvm iscsi scsi mpath rbd disk; do if eval test \$with_storage_$backend = yes; then diff --git a/src/storage/storage_util.c b/src/storage/storage_util.c index 895c623..e16c1a4 100644 --- a/src/storage/storage_util.c +++ b/src/storage/storage_util.c @@ -2437,7 +2437,7 @@ virStorageBackendFindGlusterPoolSources(const char *host, int ret = -1; - if (!(glusterpath = virFindFileInPath(GLUSTER_CLI))) { + if (!(glusterpath = virFindFileInPath("gluster"))) { if (report) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("'gluster' command line tool not found")); -- 2.9.3

2 1

[libvirt] [PATCH] Fix segfault in GetIOThreadInfo
by Leonid Podolny 19 Jan '17

19 Jan '17

The pointers calculation included a typo which caused segfaults. Signed-off-by: Leonid Podolny <leonid(a)podolny.net> --- domain.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/domain.go b/domain.go index 515138a..74061c2 100644 --- a/domain.go +++ b/domain.go @@ -3887,7 +3887,7 @@ func (d *Domain) GetIOThreadInfo(flags DomainModificationImpact) ([]DomainIOThre info := make([]DomainIOThreadInfo, int(ret)) for i := 0; i < int(ret); i++ { - cinfo := (*(**C.virDomainIOThreadInfo)(unsafe.Pointer(uintptr(unsafe.Pointer(cinfolist)) + (unsafe.Sizeof(*cinfolist) + uintptr(i))))) + cinfo := (*(**C.virDomainIOThreadInfo)(unsafe.Pointer(uintptr(unsafe.Pointer(cinfolist)) + (unsafe.Sizeof(*cinfolist) * uintptr(i))))) ncpus := int(cinfo.cpumaplen * 8) cpumap := make([]bool, ncpus) -- 2.11.0

2 1

[libvirt] [PATCH go] Fix segfault in GetIOThreadInfo
by Leonid Podolny 19 Jan '17

19 Jan '17

2 1