August 2017 - Devel - libvirt List Archives

[libvirt] Exposing mem-path in domain XML
by Michal Privoznik 23 Oct '17

23 Oct '17

Dear list, there is the following bug [1] which I'm not quite sure how to grasp. So there is this application/infrastructure called Kove [2] that allows you to have memory for your application stored on a distant host in network and basically fetch needed region on pagefault. Now imagine that somebody wants to use it for backing up domain memory. However, the way that the tool works is it has some kernel module and then some userland binary that is fed with the path of the mmaped file. I don't know all the details, but the point is, in order to let users use this we need to expose the paths for mem-path for the guest memory. I know we did not want to do this in the past, but now it looks like we don't have a way around it, do we? Michal 1: https://bugzilla.redhat.com/show_bug.cgi?id=1461214 2: http://kove.net

3 15

[libvirt] [PATCH v2 00/12] Continue altering storage pool for privatization
by John Ferlan 19 Sep '17

19 Sep '17

Consider this round 1 of 2.... The next series will be 18 patches, but the majority of those deal with change every {pool|obj}->def->X to use the accessor virStoragePoolObjGetDef. v1: https://www.redhat.com/archives/libvir-list/2017-May/msg00218.html Probably not even worth looking at the v1, but this picks up where v1 left off somewhere around patch 10, but adding smaller steps between patches. John Ferlan (12): storage: Create accessor API's for virStoragePoolObj storage: Introduce virStoragePoolObjNew storage: Fill in storage pool @active properly storage: Introduce storage volume add, delete, count APIs storage: Introduce APIs to search/scan storage pool volumes list storage: Use virStoragePoolObj{Get|Set}ConfigFile storage: Use virStoragePoolObjGetAutostartLink storage: Use virStoragePoolObj{Is|Set}Active storage: Use virStoragePoolObj{Is|Set}Autostart storage: Internally represent @autostart to bool storage: Use virStoragePoolObj{Get|Incr}Decr}Asyncjobs storage: Use virStoragePoolObjDefUseNewDef src/conf/virstorageobj.c | 211 +++++++++++++++++++++++++++++++-- src/conf/virstorageobj.h | 84 ++++++++++++- src/libvirt_private.syms | 20 ++++ src/storage/storage_backend_disk.c | 93 +++++++++------ src/storage/storage_backend_gluster.c | 5 +- src/storage/storage_backend_logical.c | 4 +- src/storage/storage_backend_mpath.c | 3 +- src/storage/storage_backend_rbd.c | 4 +- src/storage/storage_backend_scsi.c | 4 +- src/storage/storage_backend_sheepdog.c | 4 +- src/storage/storage_backend_zfs.c | 6 +- src/storage/storage_driver.c | 142 +++++++++------------- src/storage/storage_util.c | 8 +- src/test/test_driver.c | 54 ++++----- tests/storagevolxml2argvtest.c | 20 ++-- 15 files changed, 465 insertions(+), 197 deletions(-) -- 2.9.5

2 19

[libvirt] [PATCHv2] nodedev: add switchdev to NIC capabilities
by Edan David 19 Sep '17

19 Sep '17

Adding functionality to libvirt that will allow querying the interface for the availability of switchdev Offloading NIC capabilities. The switchdev mode was introduced in kernel 4.8, the iproute2-devlink command to retrieve the swtichdev NIC feature, Command example: devlink dev eswitch show pci/0000:03:00.0 This feature is needed for Openstack so we can do a scheduling decision if the NIC is in Hardware Offload (switchdev) or regular SR-IOV (legacy) mode. And select the appropriate hypervisors with the requested capability see [1]. [1] - https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/enable… --- configure.ac | 13 ++ docs/formatnode.html.in | 1 + src/util/virnetdev.c | 187 +++++++++++++++++++++- src/util/virnetdev.h | 1 + tests/nodedevschemadata/net_00_13_02_b9_f9_d3.xml | 1 + tests/nodedevschemadata/net_00_15_58_2f_e9_55.xml | 1 + 6 files changed, 203 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index b12b7fa..c089798 100644 --- a/configure.ac +++ b/configure.ac @@ -627,6 +627,19 @@ if test "$with_linux" = "yes"; then AC_CHECK_HEADERS([linux/btrfs.h]) fi +dnl +dnl check for kernel headers required by devlink +dnl +if test "$with_linux" = "yes"; then + AC_CHECK_HEADERS([linux/devlink.h]) + AC_CHECK_DECLS([DEVLINK_GENL_VERSION, DEVLINK_GENL_NAME, DEVLINK_ATTR_MAX, DEVLINK_CMD_ESWITCH_GET, DEVLINK_ATTR_BUS_NAME, DEVLINK_ATTR_DEV_NAME, DEVLINK_ATTR_ESWITCH_MODE, DEVLINK_ESWITCH_MODE_SWITCHDEV], + [AC_DEFINE([HAVE_DECL_DEVLINK], + [1], + [whether devlink declarations is available])], + [], + [[#include <linux/devlink.h>]]) +fi + dnl Allow perl/python overrides AC_PATH_PROGS([PYTHON], [python2 python]) if test -z "$PYTHON"; then diff --git a/docs/formatnode.html.in b/docs/formatnode.html.in index 4d935b5..29244a8 100644 --- a/docs/formatnode.html.in +++ b/docs/formatnode.html.in @@ -227,6 +227,7 @@ <dt><code>rxhash</code></dt><dd>receive-hashing</dd> <dt><code>rdma</code></dt><dd>remote-direct-memory-access</dd> <dt><code>txudptnl</code></dt><dd>tx-udp-tunnel-segmentation</dd> + <dt><code>switchdev</code></dt><dd>kernel-forward-plane-offload</dd> </dl> </dd> <dt><code>capability</code></dt> diff --git a/src/util/virnetdev.c b/src/util/virnetdev.c index 51a6e42..fc7c961 100644 --- a/src/util/virnetdev.c +++ b/src/util/virnetdev.c @@ -59,6 +59,10 @@ # include <net/if_dl.h> #endif +#if HAVE_DECL_DEVLINK +# include <linux/devlink.h> +#endif + #ifndef IFNAMSIZ # define IFNAMSIZ 16 #endif @@ -2481,7 +2485,8 @@ VIR_ENUM_IMPL(virNetDevFeature, "ntuple", "rxhash", "rdma", - "txudptnl") + "txudptnl", + "switchdev") #ifdef __linux__ int @@ -2936,6 +2941,7 @@ int virNetDevGetRxFilter(const char *ifname, return ret; } + #if defined(SIOCETHTOOL) && defined(HAVE_STRUCT_IFREQ) /** @@ -3115,6 +3121,182 @@ virNetDevGetEthtoolFeatures(virBitmapPtr bitmap, } +#if HAVE_DECL_DEVLINK +/** + * virNetDevPutExtraHeader + * reserve and prepare room for an extra header + * This function sets to zero the room that is required to put the extra + * header after the initial Netlink header. This function also increases + * the nlmsg_len field. + * + * @nlh: pointer to Netlink header + * @size: size of the extra header that we want to put + * + * Returns pointer to the start of the extended header + */ +static void * +virNetDevPutExtraHeader(struct nlmsghdr *nlh, + size_t size) +{ + char *ptr = (char *)nlh + nlh->nlmsg_len; + size_t len = NLMSG_ALIGN(size); + nlh->nlmsg_len += len; + memset(ptr, 0, len); + return ptr; +} + + +/** + * virNetDevGetFamilyId: + * This function supplies the devlink family id + * + * @family_name: the name of the family to query + * + * Returns family id or 0 on failure. + */ +static uint32_t +virNetDevGetFamilyId(const char *family_name) +{ + struct nl_msg *nl_msg = NULL; + struct nlmsghdr *resp = NULL; + struct genlmsghdr* gmsgh = NULL; + struct nlattr *tb[CTRL_ATTR_MAX + 1] = {NULL, }; + unsigned int recvbuflen; + uint32_t family_id = 0; + + if (!(nl_msg = nlmsg_alloc_simple(GENL_ID_CTRL, + NLM_F_REQUEST | NLM_F_ACK))) { + virReportOOMError(); + goto cleanup; + } + + if (!(gmsgh = virNetDevPutExtraHeader(nlmsg_hdr(nl_msg), sizeof(struct genlmsghdr)))) + goto cleanup; + + gmsgh->cmd = CTRL_CMD_GETFAMILY; + gmsgh->version = DEVLINK_GENL_VERSION; + + if (nla_put_string(nl_msg, CTRL_ATTR_FAMILY_NAME, family_name) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("allocated netlink buffer is too small")); + goto cleanup; + } + + if (virNetlinkCommand(nl_msg, &resp, &recvbuflen, 0, 0, NETLINK_GENERIC, 0) < 0) + goto cleanup; + + if (nlmsg_parse(resp, sizeof(struct nlmsghdr), tb, CTRL_CMD_MAX, NULL) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed netlink response message")); + goto cleanup; + } + + if (tb[CTRL_ATTR_FAMILY_ID] == NULL) + goto cleanup; + + family_id = *(uint32_t *)RTA_DATA(tb[CTRL_ATTR_FAMILY_ID]); + + cleanup: + nlmsg_free(nl_msg); + VIR_FREE(resp); + return family_id; +} + + +/** + * virNetDevSwitchdevFeature + * This function checks for the availability of Switchdev feature + * and add it to bitmap + * + * @ifname: name of the interface + * @out: add Switchdev feature if exist to bitmap + * + * Returns 0 on success, -1 on failure. + */ +static int +virNetDevSwitchdevFeature(const char *ifname, + virBitmapPtr *out) +{ + struct nl_msg *nl_msg = NULL; + struct nlmsghdr *resp = NULL; + unsigned int recvbuflen; + struct nlattr *tb[DEVLINK_ATTR_MAX + 1] = {NULL, }; + virPCIDevicePtr pci_device_ptr = NULL; + struct genlmsghdr* gmsgh = NULL; + const char *pci_name; + char *pfname = NULL; + int is_vf = -1; + int ret = -1; + uint32_t family_id; + + if ((family_id = virNetDevGetFamilyId(DEVLINK_GENL_NAME)) <= 0) + return ret; + + if ((is_vf = virNetDevIsVirtualFunction(ifname)) < 0) + return ret; + + if (is_vf == 1 && virNetDevGetPhysicalFunction(ifname, &pfname) < 0) + goto cleanup; + + if (!(nl_msg = nlmsg_alloc_simple(family_id, + NLM_F_REQUEST | NLM_F_ACK))) { + virReportOOMError(); + goto cleanup; + } + + if (!(gmsgh = virNetDevPutExtraHeader(nlmsg_hdr(nl_msg), sizeof(struct genlmsghdr)))) + goto cleanup; + + gmsgh->cmd = DEVLINK_CMD_ESWITCH_GET; + gmsgh->version = DEVLINK_GENL_VERSION; + + pci_device_ptr = pfname ? virNetDevGetPCIDevice(pfname) : + virNetDevGetPCIDevice(ifname); + if (pci_device_ptr == NULL) + goto cleanup; + + pci_name = virPCIDeviceGetName(pci_device_ptr); + + if (nla_put(nl_msg, DEVLINK_ATTR_BUS_NAME, strlen("pci")+1, "pci") < 0 || + nla_put(nl_msg, DEVLINK_ATTR_DEV_NAME, strlen(pci_name)+1, pci_name) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("allocated netlink buffer is too small")); + goto cleanup; + } + + if (virNetlinkCommand(nl_msg, &resp, &recvbuflen, 0, 0, NETLINK_GENERIC, 0) < 0) + goto cleanup; + + if (nlmsg_parse(resp, sizeof(struct genlmsghdr), tb, DEVLINK_ATTR_MAX, NULL) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed netlink response message")); + goto cleanup; + } + + if (tb[DEVLINK_ATTR_ESWITCH_MODE] && + *(int *)RTA_DATA(tb[DEVLINK_ATTR_ESWITCH_MODE]) == DEVLINK_ESWITCH_MODE_SWITCHDEV) { + ignore_value(virBitmapSetBit(*out, VIR_NET_DEV_FEAT_SWITCHDEV)); + } + + ret = 0; + + cleanup: + nlmsg_free(nl_msg); + virPCIDeviceFree(pci_device_ptr); + VIR_FREE(resp); + VIR_FREE(pfname); + return ret; +} +#else +static int +virNetDevSwitchdevFeature(const char *ifname ATTRIBUTE_UNUSED, + virBitmapPtr *out ATTRIBUTE_UNUSED) +{ + return 0; +} +#endif + + # if HAVE_DECL_ETHTOOL_GFEATURES /** * virNetDevGFeatureAvailable @@ -3315,6 +3497,9 @@ virNetDevGetFeatures(const char *ifname, if (virNetDevRDMAFeature(ifname, out) < 0) goto cleanup; + if (virNetDevSwitchdevFeature(ifname, out) < 0) + goto cleanup; + ret = 0; cleanup: VIR_FORCE_CLOSE(fd); diff --git a/src/util/virnetdev.h b/src/util/virnetdev.h index 9205c0e..71eaf45 100644 --- a/src/util/virnetdev.h +++ b/src/util/virnetdev.h @@ -112,6 +112,7 @@ typedef enum { VIR_NET_DEV_FEAT_RXHASH, VIR_NET_DEV_FEAT_RDMA, VIR_NET_DEV_FEAT_TXUDPTNL, + VIR_NET_DEV_FEAT_SWITCHDEV, VIR_NET_DEV_FEAT_LAST } virNetDevFeature; diff --git a/tests/nodedevschemadata/net_00_13_02_b9_f9_d3.xml b/tests/nodedevschemadata/net_00_13_02_b9_f9_d3.xml index d4c96e8..88252e6 100644 --- a/tests/nodedevschemadata/net_00_13_02_b9_f9_d3.xml +++ b/tests/nodedevschemadata/net_00_13_02_b9_f9_d3.xml @@ -15,6 +15,7 @@ <feature name='rxhash'/> <feature name='rdma'/> <feature name='txudptnl'/> + <feature name='switchdev'/> <capability type='80211'/> </capability> </device> diff --git a/tests/nodedevschemadata/net_00_15_58_2f_e9_55.xml b/tests/nodedevschemadata/net_00_15_58_2f_e9_55.xml index 71bf90e..f77dfcc 100644 --- a/tests/nodedevschemadata/net_00_15_58_2f_e9_55.xml +++ b/tests/nodedevschemadata/net_00_15_58_2f_e9_55.xml @@ -15,6 +15,7 @@ <feature name='rxhash'/> <feature name='rdma'/> <feature name='txudptnl'/> + <feature name='switchdev'/> <capability type='80203'/> </capability> </device> -- 2.1.4

4 11

[libvirt] [PATCH v3 0/6] Work around the kernel mdev uevent race in nodedev
by Erik Skultety 18 Sep '17

18 Sep '17

v2 here: https://www.redhat.com/archives/libvir-list/2017-July/msg01268.html Since v2: - added patch 4/6 that fixes the issue with the handler thread spamming logs with "udev_monitor_receive_device returned NULL" -> the event loop callback now disables polling on the udev monitor's fd every time there's a new event, leaving the responsibility for re-enabling it back to the handler thread once it had removed the corresponding data from the socket. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1463285 Erik Skultety (6): nodedev: Introduce udevCheckMonitorFD helper function udev: Split udevEventHandleCallback in two functions udev: Convert udevEventHandleThread to an actual thread routine nodedev: Disable/re-enable polling on the udev fd util: Introduce virFileWaitForAccess nodedev: Work around the uevent race by hooking up virFileWaitForAccess src/libvirt_private.syms | 1 + src/node_device/node_device_udev.c | 208 ++++++++++++++++++++++++++++++++----- src/util/virfile.c | 29 ++++++ src/util/virfile.h | 2 + 4 files changed, 214 insertions(+), 26 deletions(-) -- 2.13.3

2 22

[libvirt] [libvirt-sandbox PATCH 0/2] virt-sandbox-image: unbreak start from library
by Guido Günther 15 Sep '17

15 Sep '17

This is basically a V2 of "Drop library/ from template name and image path" with Dan's comment implemented. Guido Günther (2): Drop library/ from image path Sanitize domain name libvirt-sandbox/image/cli.py | 8 ++++++-- libvirt-sandbox/image/sources/docker.py | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) -- 2.11.0

2 7

[libvirt] [PATCH] spec: Own %{_libdir}/libvirt{, /connection-driver} dirs, mark licenses as %license
by Cole Robinson 15 Sep '17

15 Sep '17

From: Ville Skyttä <ville.skytta(a)iki.fi> https://bugzilla.redhat.com/show_bug.cgi?id=1483293 Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- libvirt.spec.in | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index 2b58bf3a9..2d21e60bc 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1794,6 +1794,8 @@ exit 0 %dir %attr(0711, root, root) %{_localstatedir}/cache/libvirt/ +%dir %attr(0755, root, root) %{_libdir}/libvirt/ +%dir %attr(0755, root, root) %{_libdir}/libvirt/connection-driver/ %dir %attr(0755, root, root) %{_libdir}/libvirt/lock-driver %attr(0755, root, root) %{_libdir}/libvirt/lock-driver/lockd.so @@ -2028,7 +2030,7 @@ exit 0 %attr(0755, root, root) %{_libexecdir}/libvirt-guests.sh %files libs -f %{name}.lang -%doc COPYING COPYING.LESSER +%license COPYING COPYING.LESSER %config(noreplace) %{_sysconfdir}/libvirt/libvirt.conf %config(noreplace) %{_sysconfdir}/libvirt/libvirt-admin.conf %{_libdir}/libvirt.so.* -- 2.13.5

2 2

[libvirt] [PATCH] qemu: Clean up qemuDomainSecretPrepare
by John Ferlan 13 Sep '17

13 Sep '17

No need to pass a @driver parameter since all that's done is deref the @cfg especially since the only caller can just pass an already referenced @cfg. Also, looks like commit id '0298531b' at one time had a different name for the API, so I took the liberty of fixing the comments too since I would already be updating them for the @cfg variable. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Saw this while doing some updates for the recently posted VxHS series. src/qemu/qemu_domain.c | 11 ++++------- src/qemu/qemu_domain.h | 2 +- src/qemu/qemu_process.c | 2 +- 3 files changed, 6 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index cbee151..4cd0087 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -7454,7 +7454,7 @@ qemuDomainPrepareChannel(virDomainChrDefPtr channel, } -/* qemuProcessPrepareDomainChardevSourceTLS: +/* qemuDomainPrepareChardevSourceTLS: * @source: pointer to host interface data for char devices * @cfg: driver configuration * @@ -7478,19 +7478,18 @@ qemuDomainPrepareChardevSourceTLS(virDomainChrSourceDefPtr source, } -/* qemuProcessPrepareDomainChardevSource: +/* qemuDomainPrepareChardevSource: * @def: live domain definition - * @driver: qemu driver + * @cfg: driver configuration * * Iterate through all devices that use virDomainChrSourceDefPtr as host * interface part. */ void qemuDomainPrepareChardevSource(virDomainDefPtr def, - virQEMUDriverPtr driver) + virQEMUDriverConfigPtr cfg) { size_t i; - virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); for (i = 0; i < def->nserials; i++) qemuDomainPrepareChardevSourceTLS(def->serials[i]->source, cfg); @@ -7515,8 +7514,6 @@ qemuDomainPrepareChardevSource(virDomainDefPtr def, for (i = 0; i < def->nredirdevs; i++) qemuDomainPrepareChardevSourceTLS(def->redirdevs[i]->source, cfg); - - virObjectUnref(cfg); } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index f93b09b..4b030ae 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -842,7 +842,7 @@ void qemuDomainPrepareChardevSourceTLS(virDomainChrSourceDefPtr source, ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); void qemuDomainPrepareChardevSource(virDomainDefPtr def, - virQEMUDriverPtr driver) + virQEMUDriverConfigPtr cfg) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); int qemuDomainPrepareShmemChardev(virDomainShmemDefPtr shmem) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 364c359..44df858 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5327,7 +5327,7 @@ qemuProcessPrepareDomain(virConnectPtr conn, goto cleanup; VIR_DEBUG("Prepare chardev source backends for TLS"); - qemuDomainPrepareChardevSource(vm->def, driver); + qemuDomainPrepareChardevSource(vm->def, cfg); VIR_DEBUG("Add secrets to disks, hostdevs, and chardevs"); if (qemuDomainSecretPrepare(conn, driver, vm) < 0) -- 2.9.5

2 2

[libvirt] [PATCH v6 00/13] Add support for Veritas HyperScale (VxHS) block device protocol
by John Ferlan 11 Sep '17

11 Sep '17

Here's the reworked v5 series I promised: https://www.redhat.com/archives/libvir-list/2017-August/thread.html Each of the patches lists changes that I recall making in the area. I may have missed a few... and I may have missed something from my own review - so hopefully Ashish you can keep me honest and of course since you have the environment, please check/test that things actually work. I've done quite a bit of reformatting the order and splitting things up so that XML changes are in one patch and qemu changes are in a subsequent patch. Not too little change, but not too excessive. I think we do need to think about the default TLS environment and whether we really care to fail in the event that cfg->vxhsTLS = 0 and src->haveTLS = yes. Ashish Mittal (10): storage: Introduce VIR_STORAGE_NET_PROTOCOL_VXHS docs: Add schema and docs for Veritas HyperScale (VxHS) util: storage: Add JSON backing volume parse for VxHS qemu: Add qemu command line generation for a VxHS block device conf: Introduce TLS options for VxHS block device clients util: Add haveTLS to virStorageSource util: Add virstoragetest to parse/format a tls='yes' qemu: Add TLS support for Veritas HyperScale (VxHS) tests: Add test for failure when vxhs_tls=0 tests: Add a test case for multiple VxHS disk configuration John Ferlan (3): qemu: Add QEMU 2.10 x86_64 the generated capabilities qemu: Detect support for vxhs qemu: Introduce qemuDomainPrepareDiskSource docs/formatdomain.html.in | 46 +- docs/schemas/domaincommon.rng | 18 + src/conf/domain_conf.c | 19 + src/libxl/libxl_conf.c | 1 + src/qemu/libvirtd_qemu.aug | 4 + src/qemu/qemu.conf | 33 + src/qemu/qemu_block.c | 70 +- src/qemu/qemu_block.h | 4 +- src/qemu/qemu_capabilities.c | 4 + src/qemu/qemu_capabilities.h | 3 + src/qemu/qemu_command.c | 41 +- src/qemu/qemu_conf.c | 16 + src/qemu/qemu_conf.h | 3 + src/qemu/qemu_domain.c | 58 + src/qemu/qemu_domain.h | 5 + src/qemu/qemu_driver.c | 3 + src/qemu/qemu_parse_command.c | 15 + src/qemu/qemu_process.c | 4 + src/qemu/test_libvirtd_qemu.aug.in | 2 + src/util/virstoragefile.c | 54 +- src/util/virstoragefile.h | 4 + src/xenconfig/xen_xl.c | 1 + .../caps_2.10.0.x86_64.replies | 17994 +++++++++++++++++++ tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml | 792 + tests/qemucapabilitiestest.c | 1 + ...ml2argv-disk-drive-network-tlsx509-err-vxhs.xml | 34 + ...-disk-drive-network-tlsx509-multidisk-vxhs.args | 43 + ...v-disk-drive-network-tlsx509-multidisk-vxhs.xml | 50 + ...muxml2argv-disk-drive-network-tlsx509-vxhs.args | 30 + ...emuxml2argv-disk-drive-network-tlsx509-vxhs.xml | 32 + .../qemuxml2argv-disk-drive-network-vxhs.args | 27 + .../qemuxml2argv-disk-drive-network-vxhs.xml | 32 + tests/qemuxml2argvtest.c | 10 + ...uxml2xmlout-disk-drive-network-tlsx509-vxhs.xml | 34 + .../qemuxml2xmlout-disk-drive-network-vxhs.xml | 34 + tests/qemuxml2xmltest.c | 2 + tests/virstoragetest.c | 23 + 37 files changed, 19534 insertions(+), 12 deletions(-) create mode 100644 tests/qemucapabilitiesdata/caps_2.10.0.x86_64.replies create mode 100644 tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-err-vxhs.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-multidisk-vxhs.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-multidisk-vxhs.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-vxhs.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-vxhs.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-network-tlsx509-vxhs.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-network-vxhs.xml -- 2.9.5

4 39

[libvirt] [PATCH v3 0/4] numa: describe sibling nodes distances
by Wim Ten Have 11 Sep '17

11 Sep '17

From: Wim ten Have <wim.ten.have(a)oracle.com> This patch extents guest domain administration adding support to advertise node sibling distances when configuring HVM numa guests. NUMA (non-uniform memory access), a method of configuring a cluster of nodes within a single multiprocessing system such that it shares processor local memory amongst others improving performance and the ability of the system to be expanded. A NUMA system could be illustrated as shown below. Within this 4-node system, every socket is equipped with its own distinct memory. The whole typically resembles a SMP (symmetric multiprocessing) system being a "tightly-coupled," "share everything" system in which multiple processors are working under a single operating system and can access each others' memory over multiple "Bus Interconnect" paths. +-----+-----+-----+ +-----+-----+-----+ | M | CPU | CPU | | CPU | CPU | M | | E | | | | | | E | | M +- Socket0 -+ +- Socket3 -+ M | | O | | | | | | O | | R | CPU | CPU <---------> CPU | CPU | R | | Y | | | | | | Y | +-----+--^--+-----+ +-----+--^--+-----+ | | | Bus Interconnect | | | +-----+--v--+-----+ +-----+--v--+-----+ | M | | | | | | M | | E | CPU | CPU <---------> CPU | CPU | E | | M | | | | | | M | | O +- Socket1 -+ +- Socket2 -+ O | | R | | | | | | R | | Y | CPU | CPU | | CPU | CPU | Y | +-----+-----+-----+ +-----+-----+-----+ In contrast there is the limitation of a flat SMP system, not illustrated. Here, as sockets are added, the bus (data and address path), under high activity, gets overloaded and easily becomes a performance bottleneck. NUMA adds an intermediate level of memory shared amongst a few cores per socket as illustrated above, so that data accesses do not have to travel over a single bus. Unfortunately the way NUMA does this adds its own limitations. This, as visualized in the illustration above, happens when data is stored in memory associated with Socket2 and is accessed by a CPU (core) in Socket0. The processors use the "Bus Interconnect" to create gateways between the sockets (nodes) enabling inter-socket access to memory. These "Bus Interconnect" hops add data access delays when a CPU (core) accesses memory associated with a remote socket (node). For terminology we refer to sockets as "nodes" where access to each others' distinct resources such as memory make them "siblings" with a designated "distance" between them. A specific design is described under the ACPI (Advanced Configuration and Power Interface Specification) within the chapter explaining the system's SLIT (System Locality Distance Information Table). These patches extend core libvirt's XML description of a virtual machine's hardware to include NUMA distance information for sibling nodes, which is then passed to Xen guests via libxl. Recently qemu landed support for constructing the SLIT since commit 0f203430dd ("numa: Allow setting NUMA distance for different NUMA nodes"), hence these core libvirt extensions can also help other drivers in supporting this feature. The XML changes made allow to describe the <cell> (or node/sockets) <distances> amongst <sibling> node identifiers and propagate these towards the numa domain functionality finally adding support to libxl. [below is an example illustrating a 4 node/socket <cell> setup] <cpu> <numa> <cell id='0' cpus='0,4-7' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='10'/> <sibling id='1' value='21'/> <sibling id='2' value='31'/> <sibling id='3' value='41'/> </distances> </cell> <cell id='1' cpus='1,8-10,12-15' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='21'/> <sibling id='1' value='10'/> <sibling id='2' value='21'/> <sibling id='3' value='31'/> </distances> </cell> <cell id='2' cpus='2,11' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='31'/> <sibling id='1' value='21'/> <sibling id='2' value='10'/> <sibling id='3' value='21'/> </distances> </cell> <cell id='3' cpus='3' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='41'/> <sibling id='1' value='31'/> <sibling id='2' value='21'/> <sibling id='3' value='10'/> </distances> </cell> </numa> </cpu> By default on libxl, if no <distances> are given to describe the SLIT data between different <cell>s, this patch will default to a scheme using 10 for local and 21 for any remote node/socket, which is the assumption of guest OS when no SLIT is specified. While SLIT is optional, libxl requires that distances are set nonetheless. On Linux systems the SLIT detail can be listed with help of the 'numactl -H' command. An above HVM guest as described would on such prompt with below output. [root@f25 ~]# numactl -H available: 4 nodes (0-3) node 0 cpus: 0 4 5 6 7 node 0 size: 1988 MB node 0 free: 1743 MB node 1 cpus: 1 8 9 10 12 13 14 15 node 1 size: 1946 MB node 1 free: 1885 MB node 2 cpus: 2 11 node 2 size: 2011 MB node 2 free: 1912 MB node 3 cpus: 3 node 3 size: 2010 MB node 3 free: 1980 MB node distances: node 0 1 2 3 0: 10 21 31 41 1: 21 10 21 31 2: 31 21 10 21 3: 41 31 21 10 Wim ten Have (4): numa: describe siblings distances within cells libxl: vnuma support xenconfig: add domxml conversions for xen-xl xlconfigtest: add tests for numa cell sibling distances docs/formatdomain.html.in | 70 ++++- docs/schemas/basictypes.rng | 9 + docs/schemas/cputypes.rng | 18 ++ src/conf/cpu_conf.c | 2 +- src/conf/numa_conf.c | 323 +++++++++++++++++++- src/conf/numa_conf.h | 25 +- src/libvirt_private.syms | 6 + src/libxl/libxl_conf.c | 120 ++++++++ src/libxl/libxl_driver.c | 3 +- src/xenconfig/xen_xl.c | 333 +++++++++++++++++++++ .../test-fullvirt-vnuma-nodistances.cfg | 26 ++ .../test-fullvirt-vnuma-nodistances.xml | 53 ++++ tests/xlconfigdata/test-fullvirt-vnuma.cfg | 26 ++ tests/xlconfigdata/test-fullvirt-vnuma.xml | 81 +++++ tests/xlconfigtest.c | 4 + 15 files changed, 1089 insertions(+), 10 deletions(-) create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.xml create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.xml -- 2.9.5

4 12

[libvirt] [PATCH] cpu: Add new EPYC CPU model
by Brijesh Singh 08 Sep '17

08 Sep '17

Add a new CPU model called 'EPYC' to model processors from AMD EPYC family (which includes EPYC 76xx,75xx,74xx, 73xx and 72xx). The following features bits have been added/removed compare to Opteron_G5 Added: monitor, movbe, rdrand, mmxext, ffxsr, rdtscp, cr8legacy, osvw, fsgsbase, bmi1, avx2, smep, bmi2, rdseed, adx, smap, clfshopt, sha xsaveopt, xsavec, xgetbv1, arat Removed: xop, fma4, tbm The patch is depend on EPYC CPU model supported introduced in qemu [1] [1] https://patchwork.kernel.org/patch/9902205/ Cc: Tom Lendacky <Thomas.Lendacky(a)amd.com> Signed-off-by: Brijesh Singh <brijesh.singh(a)amd.com> --- src/cpu/cpu_map.xml | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml index 8e7ac49..522d66b 100644 --- a/src/cpu/cpu_map.xml +++ b/src/cpu/cpu_map.xml @@ -251,6 +251,9 @@ <feature name='clflushopt'> <cpuid eax_in='0x07' ecx_in='0x00' ebx='0x00800000'/> </feature> + <feature name='sha_ni'> + <cpuid eax_in='0x07' ecx_in='0x00' ebx='0x20000000'/> + </feature> <feature name='avx512pf'> <cpuid eax_in='0x07' ecx_in='0x00' ebx='0x04000000'/> </feature> @@ -1545,6 +1548,77 @@ <feature name='xop'/> <feature name='xsave'/> </model> + + <model name='EPYC'> + <signature family='23' model='1'/> + <vendor name='AMD'/> + <feature name='sse2'/> + <feature name='sse'/> + <feature name='fxsr'/> + <feature name='mmx'/> + <feature name='clflush'/> + <feature name='pse36'/> + <feature name='pat'/> + <feature name='cmov'/> + <feature name='mca'/> + <feature name='pge'/> + <feature name='mtrr'/> + <feature name='sep'/> + <feature name='apic'/> + <feature name='cx8'/> + <feature name='mce'/> + <feature name='pae'/> + <feature name='msr'/> + <feature name='tsc'/> + <feature name='pse'/> + <feature name='de'/> + <feature name='vme'/> + <feature name='fpu'/> + <feature name='rdrand'/> + <feature name='f16c'/> + <feature name='avx'/> + <feature name='xsave'/> + <feature name='aes'/> + <feature name='popcnt'/> + <feature name='movbe'/> + <feature name='sse4.2'/> + <feature name='sse4.1'/> + <feature name='cx16'/> + <feature name='fma'/> + <feature name='ssse3'/> + <feature name='monitor'/> + <feature name='pclmuldq'/> + <feature name='pni'/> + <feature name='lm'/> + <feature name='rdtscp'/> + <feature name='pdpe1gb'/> + <feature name='fxsr_opt'/> + <feature name='mmxext'/> + <feature name='nx'/> + <feature name='syscall'/> + <feature name='osvw'/> + <feature name='3dnowprefetch'/> + <feature name='misalignsse'/> + <feature name='sse4a'/> + <feature name='abm'/> + <feature name='cr8legacy'/> + <feature name='svm'/> + <feature name='lahf_lm'/> + <feature name='fsgsbase'/> + <feature name='bmi1'/> + <feature name='avx2'/> + <feature name='smep'/> + <feature name='bmi2'/> + <feature name='rdseed'/> + <feature name='adx'/> + <feature name='smap'/> + <feature name='clflushopt'/> + <feature name='sha_ni'/> + <feature name='xsaveopt'/> + <feature name='xsavec'/> + <feature name='xgetbv1'/> + <feature name='arat'/> + </model> </arch> <arch name='ppc64'> -- 2.9.4

3 2