November 2014 - Devel - libvirt List Archives

[libvirt] [PATCH 0/8] Add XML validation to the APIs
by Daniel P. Berrange 24 Nov '14

24 Nov '14

This proof of concept patch extends the virDomainDefineXML and virDomainCreateXML APIs so that they can validate the user supplied XML document against the RNG schemas. The virsh command will enable validation by default, it must be turned off with --skip-validation if desired. This series is not complete - The network, interface, storage pool, etc APIs are not wired up to support validation. - Only the QEMU virt driver is wired up to validate - The virsh edit command is not wired up to validate It is enough to demonstrate it working with 'virsh define' and the QEMU driver though. The biggest problem I see is the really awful error messages we get back from libxml2 when validation fails :-( They are essentially useless :-( Daniel P. Berrange (8): Add new virDomainDefineXMLFlags public API Add stub virDomainDefineXMLFlags impls Add virXMLValidateAgainstSchema helper method Fix use of flags when parsing/formatting snapshot domain XML Don't pass VIR_DOMAIN_XML_SECURE to virDomainDefParseString in phyp Fix flags passed to virDomainDefParseString by XenAPI driver Given virDomainDef parser & formatter their own flags Add support for schema validation when passing in XML include/libvirt/libvirt-domain.h | 9 ++ include/libvirt/virterror.h | 1 + src/bhyve/bhyve_driver.c | 21 ++- src/conf/domain_conf.c | 315 +++++++++++++++++++------------------- src/conf/domain_conf.h | 71 ++++++--- src/conf/snapshot_conf.c | 8 +- src/driver-hypervisor.h | 5 + src/esx/esx_driver.c | 23 ++- src/hyperv/hyperv_driver.c | 3 +- src/internal.h | 4 + src/libvirt-domain.c | 48 ++++++ src/libvirt_private.syms | 2 + src/libvirt_public.syms | 5 + src/libxl/libxl_domain.c | 2 +- src/libxl/libxl_driver.c | 35 +++-- src/libxl/libxl_migration.c | 6 +- src/lxc/lxc_driver.c | 25 ++- src/openvz/openvz_driver.c | 23 ++- src/parallels/parallels_driver.c | 11 +- src/phyp/phyp_driver.c | 7 +- src/qemu/qemu_domain.c | 10 +- src/qemu/qemu_driver.c | 48 ++++-- src/qemu/qemu_migration.c | 8 +- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 19 ++- src/remote_protocol-structs | 8 + src/test/test_driver.c | 31 ++-- src/uml/uml_driver.c | 22 ++- src/util/virerror.c | 6 + src/util/virxml.c | 74 +++++++++ src/util/virxml.h | 5 + src/vbox/vbox_common.c | 25 ++- src/vmware/vmware_driver.c | 19 ++- src/xen/xen_driver.c | 21 ++- src/xen/xend_internal.c | 6 +- src/xen/xm_internal.c | 4 +- src/xenapi/xenapi_driver.c | 15 +- tests/domainsnapshotxml2xmltest.c | 2 +- tests/lxcxml2xmltest.c | 4 +- tests/openvzutilstest.c | 2 +- tests/qemuhotplugtest.c | 6 +- tests/qemuxml2argvtest.c | 2 +- tests/qemuxml2xmltest.c | 9 +- tests/qemuxmlnstest.c | 2 +- tests/vmx2xmltest.c | 2 +- tests/xmconfigtest.c | 4 +- tests/xml2sexprtest.c | 2 +- tests/xml2vmxtest.c | 2 +- tools/virsh-domain.c | 19 ++- 49 files changed, 684 insertions(+), 318 deletions(-) -- 2.1.0

2 17

[libvirt] [PATCH v6] network: Add network bandwidth support to ethernet interfaces
by Anirban Chakraborty 24 Nov '14

24 Nov '14

Ethernet interfaces in libvirt currently do not support bandwidth setting. For example, following xml file for an interface will not apply these settings to corresponding qdiscs. <interface type="ethernet"> <mac address="02:36:1d:18:2a:e4"/> <model type="virtio"/> <script path=""/> <target dev="tap361d182a-e4"/> <bandwidth> <inbound average="984" peak="1024" burst="64"/> <outbound average="2000" peak="2048" burst="128"/> </bandwidth> </interface> Signed-off-by: Anirban Chakraborty <abchak(a)juniper.net> --- src/conf/netdev_bandwidth_conf.c | 14 ++++++++++- src/conf/netdev_bandwidth_conf.h | 25 +++++++++++++++++++ src/libvirt_private.syms | 1 + src/lxc/lxc_driver.c | 8 +++++++ src/lxc/lxc_process.c | 52 +++++++++++++++++++++++----------------- src/qemu/qemu_command.c | 16 ++++++++----- src/qemu/qemu_hotplug.c | 12 ++++++++++ src/qemu/qemu_process.c | 4 ++++ src/util/virnetdevmacvlan.c | 10 -------- src/util/virnetdevmacvlan.h | 1 - 10 files changed, 103 insertions(+), 40 deletions(-) diff --git a/src/conf/netdev_bandwidth_conf.c b/src/conf/netdev_bandwidth_conf.c index 2e5b564..63a39e3 100644 --- a/src/conf/netdev_bandwidth_conf.c +++ b/src/conf/netdev_bandwidth_conf.c @@ -25,7 +25,6 @@ #include "netdev_bandwidth_conf.h" #include "virerror.h" #include "viralloc.h" -#include "domain_conf.h" #include "virstring.h" #define VIR_FROM_THIS VIR_FROM_NONE @@ -269,3 +268,16 @@ virNetDevBandwidthFormat(virNetDevBandwidthPtr def, virBufferPtr buf) cleanup: return ret; } + +void +virDomainClearNetBandwidth(virDomainObjPtr vm) +{ + size_t i; + virDomainNetType type; + + for (i = 0; i < vm->def->nnets; i++) { + type = virDomainNetGetActualType(vm->def->nets[i]); + if (virNetDevSupportBandwidth(type)) + virNetDevBandwidthClear(vm->def->nets[i]->ifname); + } +} diff --git a/src/conf/netdev_bandwidth_conf.h b/src/conf/netdev_bandwidth_conf.h index 23aaaf3..60dacc6 100644 --- a/src/conf/netdev_bandwidth_conf.h +++ b/src/conf/netdev_bandwidth_conf.h @@ -27,6 +27,7 @@ # include "virnetdevbandwidth.h" # include "virbuffer.h" # include "virxml.h" +# include "domain_conf.h" virNetDevBandwidthPtr virNetDevBandwidthParse(xmlNodePtr node, int net_type) @@ -35,4 +36,28 @@ int virNetDevBandwidthFormat(virNetDevBandwidthPtr def, virBufferPtr buf) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); +void virDomainClearNetBandwidth(virDomainObjPtr vm) + ATTRIBUTE_NONNULL(1); + +static inline bool virNetDevSupportBandwidth(virDomainNetType type) +{ + switch (type) { + case VIR_DOMAIN_NET_TYPE_BRIDGE: + case VIR_DOMAIN_NET_TYPE_NETWORK: + case VIR_DOMAIN_NET_TYPE_DIRECT: + case VIR_DOMAIN_NET_TYPE_ETHERNET: + return true; + case VIR_DOMAIN_NET_TYPE_USER: + case VIR_DOMAIN_NET_TYPE_VHOSTUSER: + case VIR_DOMAIN_NET_TYPE_SERVER: + case VIR_DOMAIN_NET_TYPE_CLIENT: + case VIR_DOMAIN_NET_TYPE_MCAST: + case VIR_DOMAIN_NET_TYPE_INTERNAL: + case VIR_DOMAIN_NET_TYPE_HOSTDEV: + case VIR_DOMAIN_NET_TYPE_LAST: + break; + } + return false; +} + #endif /* __VIR_NETDEV_BANDWIDTH_CONF_H__ */ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0864618..1c0cfb1 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -509,6 +509,7 @@ virInterfaceRemove; # conf/netdev_bandwidth_conf.h virNetDevBandwidthFormat; virNetDevBandwidthParse; +virDomainClearNetBandwidth; # conf/netdev_vlan_conf.h diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index cf2a3c8..e9e0da3 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -72,6 +72,7 @@ #include "viraccessapicheck.h" #include "viraccessapichecklxc.h" #include "virhostdev.h" +#include "netdev_bandwidth_conf.h" #define VIR_FROM_THIS VIR_FROM_LXC @@ -4213,6 +4214,11 @@ lxcDomainAttachDeviceNetLive(virConnectPtr conn, _("Network device type is not supported")); goto cleanup; } + /* set network bandwidth */ + if (virNetDevSupportBandwidth(actualType) && + virNetDevBandwidthSet(net->ifname, + virDomainNetGetActualBandwidth(net), false) < 0) + goto cleanup; if (virNetDevSetNamespace(veth, priv->initpid) < 0) { virDomainAuditNet(vm, NULL, net, "attach", false); @@ -4644,6 +4650,8 @@ lxcDomainDetachDeviceNetLive(virDomainObjPtr vm, detach = vm->def->nets[detachidx]; + virDomainClearNetBandwidth(vm); + switch (virDomainNetGetActualType(detach)) { case VIR_DOMAIN_NET_TYPE_BRIDGE: case VIR_DOMAIN_NET_TYPE_NETWORK: diff --git a/src/lxc/lxc_process.c b/src/lxc/lxc_process.c index 37ddbbc..5bbd696 100644 --- a/src/lxc/lxc_process.c +++ b/src/lxc/lxc_process.c @@ -51,6 +51,7 @@ #include "viratomic.h" #include "virprocess.h" #include "virsystemd.h" +#include "netdev_bandwidth_conf.h" #define VIR_FROM_THIS VIR_FROM_LXC @@ -274,11 +275,6 @@ char *virLXCProcessSetupInterfaceBridged(virConnectPtr conn, if (virNetDevSetOnline(parentVeth, true) < 0) goto cleanup; - if (virNetDevBandwidthSet(net->ifname, - virDomainNetGetActualBandwidth(net), - false) < 0) - goto cleanup; - if (net->filter && virDomainConfNWFilterInstantiate(conn, vm->uuid, net) < 0) goto cleanup; @@ -300,6 +296,7 @@ char *virLXCProcessSetupInterfaceDirect(virConnectPtr conn, virNetDevBandwidthPtr bw; virNetDevVPortProfilePtr prof; virLXCDriverConfigPtr cfg = virLXCDriverGetConfig(driver); + const char *linkdev = virDomainNetGetActualDirectDev(net); /* XXX how todo bandwidth controls ? * Since the 'net-ifname' is about to be moved to a different @@ -329,14 +326,13 @@ char *virLXCProcessSetupInterfaceDirect(virConnectPtr conn, if (virNetDevMacVLanCreateWithVPortProfile( net->ifname, &net->mac, - virDomainNetGetActualDirectDev(net), + linkdev, virDomainNetGetActualDirectMode(net), false, def->uuid, - virDomainNetGetActualVirtPortProfile(net), + prof, &res_ifname, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, - cfg->stateDir, - virDomainNetGetActualBandwidth(net), 0) < 0) + cfg->stateDir, 0) < 0) goto cleanup; ret = res_ifname; @@ -368,6 +364,8 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, int ret = -1; size_t i; size_t niface = 0; + virDomainNetDefPtr net; + virDomainNetType type; for (i = 0; i < def->nnets; i++) { char *veth = NULL; @@ -375,13 +373,15 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, * network's pool of devices, or resolve bridge device name * to the one defined in the network definition. */ - if (networkAllocateActualDevice(def, def->nets[i]) < 0) + net = def->nets[i]; + if (networkAllocateActualDevice(def, net) < 0) goto cleanup; if (VIR_EXPAND_N(*veths, *nveths, 1) < 0) goto cleanup; - switch (virDomainNetGetActualType(def->nets[i])) { + type = virDomainNetGetActualType(net); + switch (type) { case VIR_DOMAIN_NET_TYPE_NETWORK: { virNetworkPtr network; char *brname = NULL; @@ -389,7 +389,7 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, virErrorPtr errobj; if (!(network = virNetworkLookupByName(conn, - def->nets[i]->data.network.name))) + net->data.network.name))) goto cleanup; if (!(brname = virNetworkGetBridgeName(network))) fail = true; @@ -405,7 +405,7 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, if (!(veth = virLXCProcessSetupInterfaceBridged(conn, def, - def->nets[i], + net, brname))) { VIR_FREE(brname); goto cleanup; @@ -414,7 +414,7 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, break; } case VIR_DOMAIN_NET_TYPE_BRIDGE: { - const char *brname = virDomainNetGetActualBridgeName(def->nets[i]); + const char *brname = virDomainNetGetActualBridgeName(net); if (!brname) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("No bridge name specified")); @@ -422,7 +422,7 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, } if (!(veth = virLXCProcessSetupInterfaceBridged(conn, def, - def->nets[i], + net, brname))) goto cleanup; } break; @@ -430,31 +430,39 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, case VIR_DOMAIN_NET_TYPE_DIRECT: if (!(veth = virLXCProcessSetupInterfaceDirect(conn, def, - def->nets[i]))) + net))) goto cleanup; break; - case VIR_DOMAIN_NET_TYPE_USER: case VIR_DOMAIN_NET_TYPE_ETHERNET: + break; + + case VIR_DOMAIN_NET_TYPE_USER: case VIR_DOMAIN_NET_TYPE_VHOSTUSER: case VIR_DOMAIN_NET_TYPE_SERVER: case VIR_DOMAIN_NET_TYPE_CLIENT: case VIR_DOMAIN_NET_TYPE_MCAST: case VIR_DOMAIN_NET_TYPE_INTERNAL: case VIR_DOMAIN_NET_TYPE_LAST: + case VIR_DOMAIN_NET_TYPE_HOSTDEV: virReportError(VIR_ERR_INTERNAL_ERROR, _("Unsupported network type %s"), - virDomainNetTypeToString( - virDomainNetGetActualType(def->nets[i]) - )); + virDomainNetTypeToString(type)); goto cleanup; + } + /* set network bandwidth */ + if (virNetDevSupportBandwidth(type) && + virNetDevBandwidthSet(net->ifname, + virDomainNetGetActualBandwidth(net), false) < 0) + goto cleanup; + (*veths)[(*nveths)-1] = veth; /* Make sure all net definitions will have a name in the container */ - if (!def->nets[i]->ifname_guest) { - if (virAsprintf(&def->nets[i]->ifname_guest, "eth%zu", niface) < 0) + if (!net->ifname_guest) { + if (virAsprintf(&net->ifname_guest, "eth%zu", niface) < 0) return -1; niface++; } diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 1399ce4..8025ff6 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -43,6 +43,7 @@ #include "domain_addr.h" #include "domain_audit.h" #include "domain_conf.h" +#include "netdev_bandwidth_conf.h" #include "snapshot_conf.h" #include "storage_conf.h" #include "secret_conf.h" @@ -192,7 +193,6 @@ qemuPhysIfaceConnect(virDomainDefPtr def, virDomainNetGetActualVirtPortProfile(net), &res_ifname, vmop, cfg->stateDir, - virDomainNetGetActualBandwidth(net), macvlan_create_flags); if (rc >= 0) { virDomainAuditNetDevice(def, net, res_ifname, true); @@ -372,11 +372,6 @@ qemuNetworkIfaceConnect(virDomainDefPtr def, &net->mac) < 0) goto cleanup; - if (virNetDevBandwidthSet(net->ifname, - virDomainNetGetActualBandwidth(net), - false) < 0) - goto cleanup; - if (net->filter && virDomainConfNWFilterInstantiate(conn, def->uuid, net) < 0) { goto cleanup; @@ -7501,6 +7496,15 @@ qemuBuildInterfaceCommandLine(virCommandPtr cmd, goto cleanup; } + /* Set Bandwidth */ + if (net->ifname && strlen(net->ifname)) { + if (virNetDevSupportBandwidth(actualType) && + virNetDevBandwidthSet(net->ifname, + virDomainNetGetActualBandwidth(net), + false) < 0) + goto cleanup; + } + if ((actualType == VIR_DOMAIN_NET_TYPE_NETWORK || actualType == VIR_DOMAIN_NET_TYPE_BRIDGE || actualType == VIR_DOMAIN_NET_TYPE_ETHERNET || diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index b9a0cee..3e7b7da 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -31,6 +31,7 @@ #include "qemu_command.h" #include "qemu_hostdev.h" #include "domain_audit.h" +#include "netdev_bandwidth_conf.h" #include "domain_nwfilter.h" #include "virlog.h" #include "datatypes.h" @@ -947,6 +948,12 @@ int qemuDomainAttachNetDevice(virConnectPtr conn, goto cleanup; } + /* Set Bandwidth */ + if (virNetDevSupportBandwidth(actualType) && + virNetDevBandwidthSet(net->ifname, + virDomainNetGetActualBandwidth(net), false) < 0) + goto cleanup; + for (i = 0; i < tapfdSize; i++) { if (virSecurityManagerSetTapFDLabel(driver->securityManager, vm->def, tapfd[i]) < 0) @@ -3545,6 +3552,11 @@ qemuDomainDetachNetDevice(virQEMUDriverPtr driver, goto cleanup; } + if (virNetDevSupportBandwidth(virDomainNetGetActualType(detach)) && + virNetDevBandwidthClear(detach->ifname) < 0) + VIR_WARN("cannot clear bandwidth setting for device : %s", + detach->ifname); + qemuDomainMarkDeviceForRemoval(vm, &detach->info); qemuDomainObjEnterMonitor(driver, vm); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 7518138..b782984 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -69,6 +69,7 @@ #include "storage/storage_driver.h" #include "configmake.h" #include "nwfilter_conf.h" +#include "netdev_bandwidth_conf.h" #define VIR_FROM_THIS VIR_FROM_QEMU @@ -4843,6 +4844,9 @@ void qemuProcessStop(virQEMUDriverPtr driver, virStrerror(errno, ebuf, sizeof(ebuf))); } + /* Clear network bandwidth */ + virDomainClearNetBandwidth(vm); + virDomainConfVMNWFilterTeardown(vm); if (cfg->macFilter) { diff --git a/src/util/virnetdevmacvlan.c b/src/util/virnetdevmacvlan.c index 89e221d..2ff24b1 100644 --- a/src/util/virnetdevmacvlan.c +++ b/src/util/virnetdevmacvlan.c @@ -810,7 +810,6 @@ int virNetDevMacVLanCreateWithVPortProfile(const char *tgifname, char **res_ifname, virNetDevVPortProfileOp vmOp, char *stateDir, - virNetDevBandwidthPtr bandwidth, unsigned int flags) { const char *type = (flags & VIR_NETDEV_MACVLAN_CREATE_WITH_TAP) ? @@ -923,14 +922,6 @@ int virNetDevMacVLanCreateWithVPortProfile(const char *tgifname, rc = 0; } - if (virNetDevBandwidthSet(cr_ifname, bandwidth, false) < 0) { - if (flags & VIR_NETDEV_MACVLAN_CREATE_WITH_TAP) - VIR_FORCE_CLOSE(rc); /* sets rc to -1 */ - else - rc = -1; - goto disassociate_exit; - } - if (vmOp == VIR_NETDEV_VPORT_PROFILE_OP_CREATE || vmOp == VIR_NETDEV_VPORT_PROFILE_OP_RESTORE) { /* Only directly register upon a create or restore (restarting @@ -1073,7 +1064,6 @@ int virNetDevMacVLanCreateWithVPortProfile(const char *ifname ATTRIBUTE_UNUSED, char **res_ifname ATTRIBUTE_UNUSED, virNetDevVPortProfileOp vmop ATTRIBUTE_UNUSED, char *stateDir ATTRIBUTE_UNUSED, - virNetDevBandwidthPtr bandwidth ATTRIBUTE_UNUSED, unsigned int flags) { virCheckFlags(0, -1); diff --git a/src/util/virnetdevmacvlan.h b/src/util/virnetdevmacvlan.h index 41aa4e2..f08d32b 100644 --- a/src/util/virnetdevmacvlan.h +++ b/src/util/virnetdevmacvlan.h @@ -68,7 +68,6 @@ int virNetDevMacVLanCreateWithVPortProfile(const char *ifname, char **res_ifname, virNetDevVPortProfileOp vmop, char *stateDir, - virNetDevBandwidthPtr bandwidth, unsigned int flags) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(6) ATTRIBUTE_NONNULL(8) ATTRIBUTE_NONNULL(10) ATTRIBUTE_RETURN_CHECK; -- 1.9.1

4 12

[libvirt] ideas on virDomainListBlockStats for allocation numbers
by Eric Blake 24 Nov '14

24 Nov '14

Based on Dan's recommendation [1], I'm looking at enhancing virDomainListBlockStats to report the allocation numbers of a backing file during a virDomainBlockCommit operation. Getting the information from qemu is not difficult, but the question is how should it be represented to the end user. See below for my ideas, and I'm open to feedback. [1] https://www.redhat.com/archives/libvir-list/2014-November/msg00604.html Some background - highest allocation is mainly applicable to using a qcow2 format on top of a raw block device (that's the only time that libvirt reports a different number based on querying qemu, other file formats just go off of stat information), so even setting this up to test can be interesting. Also, while qemu reports wr_highest_offset during 'query-blockstats', a disk has to actually have write activity performed during that given qemu process before the offset will be accurate. It took me a while to figure this out; when I set up a dummy guest with no OS (and therefore no writes), the offset being reported was 0 even when I had used qemu-io to poke data into the file prior to starting qemu. I finally figured out that metadata writes also count as part of the highest offset visited; so using 'blockdev-snapshot-internal-sync' followed by 'blockdev-snapshot-delete-internal-sync' is sufficient to cause qemu to write metadata and therefore reveal the highest offset. So I set up a playground to test things, where I first created two 1G partitions, then use this script to re-create a guest each time I want to reset things: ====== #!/bin/sh cd /tmp rm -f wrapper.qcow2 virsh destroy testvm2 2>/dev/null qemu-img create -f qcow2 /dev/sda6 $((750*1024*1024)) qemu-img create -f qcow2 /dev/sda6 $((1250*1024*1024)) virsh create /dev/stdin <<EOF <domain type='kvm'> <name>testvm2</name> <memory unit='MiB'>256</memory> <vcpu>1</vcpu> <os> <type arch='x86_64'>hvm</type> </os> <devices> <disk type='block' device='disk'> <driver name='qemu' type='qcow2'/> <source dev='/dev/sda6'/> <target dev='vda' bus='virtio'/> </disk> <disk type='block' device='disk'> <driver name='qemu' type='qcow2'/> <source dev='/dev/sda7'/> <target dev='vdb' bus='virtio'/> </disk> <graphics type='vnc'/> </devices> </domain> EOF virsh qemu-monitor-command testvm2 \ '{"execute":"blockdev-snapshot-internal-sync",' \ '"arguments":{"device":"drive-virtio-disk0", "name":"snap1"}}' virsh qemu-monitor-command testvm2 \ '{"execute":"blockdev-snapshot-delete-internal-sync",' \ '"arguments":{"device":"drive-virtio-disk0", "name":"snap1"}}' virsh domblkinfo testvm2 vda virsh domblkinfo testvm2 vdb ======== which shows this for my starting point: # virsh domblkinfo testvm2 vda Capacity: 786432000 Allocation: 458752 Physical: 1073741824 # virsh domblkinfo testvm2 vdb Capacity: 1310720000 Allocation: 0 Physical: 1073741824 After that, I can create external snapshots: # virsh snapshot-create-as testvm2 --disk-only --no-metadata \ --diskspec vda,file=/tmp/wrapper.qcow2 --diskspec vdb,snapshot=no at which point dumpxml shows this subset: <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/tmp/wrapper.qcow2'/> <backingStore type='block' index='1'> <format type='qcow2'/> <source dev='/dev/sda6'/> <backingStore/> </backingStore> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> Next, I can play with blockcommit, where a qemu-monitor-command on 'query-blockstats' will show me the growing allocation when the backing file is being written during the commit. The other useful output is the new virDomainListBlockStats: # virsh domstats --block testvm2Domain: 'testvm2' block.count=2 block.0.name=vda block.0.rd.reqs=1 block.0.rd.bytes=512 block.0.rd.times=31635 block.0.wr.reqs=0 block.0.wr.bytes=0 block.0.wr.times=0 block.0.fl.reqs=0 block.0.fl.times=0 block.0.allocation=458752 block.0.capacity=786432000 block.1.name=vdb block.1.rd.reqs=0 block.1.rd.bytes=0 block.1.rd.times=0 block.1.wr.reqs=0 block.1.wr.bytes=0 block.1.wr.times=0 block.1.fl.reqs=0 block.1.fl.times=0 block.1.allocation=0 block.1.capacity=1310720000 The problem is that once we have a domain with more than one <disk>, and where one or all disks have more than one <backingStore>, then how should virDomainListBlockStats represent that? One idea I have is to just expose a block.count equal to the total number of devices I'm about to report on, where the array can be larger than the number of disks, and using the name field to correlate back to dumpxml layout: block.count=3 block.0.name=vda # information on wrapper.qcow2 ... block.1.name=vda[1] # information on backingStore index 1 of vda block.1.rd.reqs=0 #+ that is, on /dev/sda6 ... block.2.name=vdb # information on /dev/sda7 ... It may make things easier if I also add a block.n.path that lists the file name of the block being described (might get tricky with NBD/gluster/sheepdog network disks). Also, I'm thinking of adding block.n.physical to match the older virDomainGetBlockInfo() information. Another possible layout is to mirror the nesting of XML backingChain. Something like: block.count=2 block.0.name=vda block.0.backing=1 block.0.allocation=... # information on /tmp/wrapper.qcow2 ... block.0.0.allocation=... # information on /dev/sda6 ... block.1.name=vdb block.1.backing=0 block.1.allocation=... # information on /dev/sda7 But there, we run into a possible problem: virTypedParameter has a finite length for field names, so we can only cover a finite depth of backing chain before we run out of space and can't report on the full chain. Any other ideas for the best way to lay this out, and for how to make it as easy as possible for client applications to correlate information on allocation back to the appropriate block device in the chain? I also wonder if adding more information to the existing --block flag for stats is okay, or whether it is better to add yet another statistics flag grouping that must be requested to turn on information about backing files. Technically, it's still block-related statistics, but as we have already got released libvirt that still has a 1:1 block.n mapping to <disk> elements, using a new flag would make it easier to learn if libvirt is new enough to support information on backing chains, all without confusing existing clients that aren't expecting backing chain stats. This question needs answering regardless of which layout we choose above for representing backing chain stats. Thoughts welcome. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

3 3

[libvirt] strange behavior when using iotune
by Vasiliy Tolstov 24 Nov '14

24 Nov '14

Hi. I'm try to shape disk via total_iops_sec in libvirt libvirt 1.2.10 qemu 2.0.0 Firstly when i'm run vm with predefined <total_iops_sec>5000</total_iops_sec> i have around 11000 iops (dd if=/dev/sda bs=512K of=/dev/null) After that i'm try to set via virsh --total_iops_sec 10 to want to minimize io, but nothing changed. After that i'm reboot vm with <total_iops_sec>10</total_iops_sec> and get very slow io, but this expected. But libvirt says that i have is around 600 iops. My questions is - why i can't change total_iops_sec in run-time, and why entered values does not equal values getting from libvirt ? Thanks for any suggestions and any help. -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru jabber: vase(a)selfip.ru

2 4

[libvirt] [PATCH v3 0/5] Guest filesystem information API
by Tomoki Sekiyama 24 Nov '14

24 Nov '14

Hi, This is v3 of patchset to add virDomainGetFSInfo API. * changes in v2->v3: -[all] squashed fixup patches by John Ferlan (thanks!) -[all] added 'ndevAlias' filed instead of NULL-termination of devAlias -[all] renamed 'type' field to 'fstype' to avoid wireshark genxdrstub.pl error -[2/5] use @acl: domain:fs_freeze (v2: http://www.redhat.com/archives/libvir-list/2014-November/msg00559.html ) * changes in v1->v2: -[all] removed redundant NULL element at the last of returned info array -[3/5] make error messages in qemu_agent.c consistent with other commands -[4/5] added a test case for 2 items in info->devAliases -[5/5] added a pod document for virsh domfsinfo command (v1: http://www.redhat.com/archives/libvir-list/2014-October/msg00001.html ) * summary This series implements a new virDomainGetFSInfo API, that returns a list of mounted filesystems information in the guest, collected via the guest agent. The returned info contains mountpoints and disk device alias named in libvirt, so we can know which mountpoints should be frozen by virDomainFSFreeze to take snapshots of a part of disks. --- Tomoki Sekiyama (5): Implement public API for virDomainGetFSInfo remote: Implement the remote protocol for virDomainGetFSInfo qemu: Implement the qemu driver for virDomainGetFSInfo qemu: add test for qemuAgentGetFSInfo virsh: expose virDomainGetFSInfo daemon/remote.c | 113 +++++++++++++++++++ include/libvirt/libvirt-domain.h | 22 ++++ src/conf/domain_conf.c | 71 ++++++++++++ src/conf/domain_conf.h | 6 + src/driver-hypervisor.h | 6 + src/libvirt.c | 68 ++++++++++++ src/libvirt_private.syms | 1 src/libvirt_public.syms | 6 + src/qemu/qemu_agent.c | 176 ++++++++++++++++++++++++++++++ src/qemu/qemu_agent.h | 2 src/qemu/qemu_driver.c | 48 ++++++++ src/remote/remote_driver.c | 93 ++++++++++++++++ src/remote/remote_protocol.x | 32 +++++ src/remote_protocol-structs | 21 ++++ src/rpc/gendispatch.pl | 1 tests/Makefile.am | 1 tests/qemuagentdata/qemuagent-fsinfo.xml | 39 +++++++ tests/qemuagenttest.c | 144 +++++++++++++++++++++++++ tools/virsh-domain.c | 70 ++++++++++++ tools/virsh.pod | 9 ++ 20 files changed, 928 insertions(+), 1 deletion(-) create mode 100644 tests/qemuagentdata/qemuagent-fsinfo.xml -- Tomoki Sekiyama

2 6

[libvirt] [PATCHv2 0/5] Add agent lifecycle event
by Peter Krempa 24 Nov '14

24 Nov '14

I'll post the promised channel state event later in a separate series. Peter Krempa (5): qemu: chardev: Extract more information about character devices qemu: process: Refresh virtio channel guest state when connecting to mon event: Add guest agent lifecycle event examples: Add support for the guest agent lifecycle event qemu: Emit the guest agent lifecycle event daemon/remote.c | 36 +++++++++++++++ examples/object-events/event-test.c | 67 ++++++++++++++++++++++++++- include/libvirt/libvirt-domain.h | 41 +++++++++++++++++ src/conf/domain_event.c | 78 ++++++++++++++++++++++++++++++++ src/conf/domain_event.h | 9 ++++ src/libvirt_private.syms | 2 + src/qemu/qemu_driver.c | 6 +++ src/qemu/qemu_monitor.c | 13 +++++- src/qemu/qemu_monitor.h | 6 +++ src/qemu/qemu_monitor_json.c | 51 ++++++++++++++------- src/qemu/qemu_monitor_text.c | 17 ++++--- src/qemu/qemu_process.c | 90 ++++++++++++++++++++++++++++++++++--- src/remote/remote_driver.c | 31 +++++++++++++ src/remote/remote_protocol.x | 16 ++++++- src/remote_protocol-structs | 7 +++ tests/qemumonitorjsontest.c | 41 ++++++++++++++--- tools/virsh-domain.c | 39 ++++++++++++++++ 17 files changed, 513 insertions(+), 37 deletions(-) -- 2.1.0

2 13

[libvirt] Plan for next releases
by Daniel Veillard 24 Nov '14

24 Nov '14

We are getting close to the end of the month and in theory we should enter freeze around next week if we want to release 1.2.11 early Dec. However: - we have 'only' 150 commits since 1.2.10 - December is usually heavilly truncated due to Xmas/etc... vacations - I'm actually on vacations next week and while I could try to push the release candidates while on the road it's not ideal What about pushing the 1.2.11 release to around 15th December (maybe a bit earlier) then push the following release to the last week of January, just before FOSDEM so it's available then, and also because Feb is really short. Opinions ? I'm tempted to do that '2 release in 3 months' trick, we did that in the past for end of year, Any objections ? thanks, Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 2

[libvirt] Error starting domain: internal error: missing IFLA_VF_INFO in netlink response
by Hong-Hua.Yin＠freescale.com 24 Nov '14

24 Nov '14

Hi, We try PCI Passthrough of host network devices on PPC platform. http://wiki.libvirt.org/page/Networking#Assignment_with_.3Cinterface_type.3… But we got a similar issue as below that reported on RedHat before: https://bugzilla.redhat.com/show_bug.cgi?id=1040626 With <hostdev>, it could start VM successfully. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x4'/> </source> </hostdev> Logged into VM and checked for VF using lspci root@model : qemu ppce500:~# lspci 00:00.0 PCI bridge: Freescale Semiconductor Inc MPC8533E 00:02.0 Power PC: Freescale Semiconductor Inc Device 0000 (rev 20) 00:03.0 Unclassified device [00ff]: Red Hat, Inc Virtio memory balloon But it will fail if using <interface type="hostdev">: <interface type="hostdev" managed="yes"> <mac address="00:e0:0c:00:20:01"/> <source> <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x4"/> </source> </interface> # virsh create vf.xml 2014-11-24 11:08:31.390+0000: 3556: info : libvirt version: 1.2.2 2014-11-24 11:08:31.390+0000: 3556: debug : virLogParseOutputs:1378 : outputs=1:file:virsh.log error: Failed to create domain from vf.xml error: internal error: missing IFLA_VF_INFO in netlink response Exactly We're using max_vfs=2 and libnl-3.2.22 # ls -l /usr/lib64/libnl-3.so.200.17.0 -rwxr-xr-x 1 root root 154440 Sep 17 07:15 /usr/lib64/libnl-3.so.200.17.0 Why this issue happened on PPC? Is there anything architecture-specific support needed to add? Thanks, Olivia

1 0

[libvirt] [PATCH] spec: Automatically apply all patches with git
by Jiri Denemark 24 Nov '14

24 Nov '14

With this change, any patch declared in libvirt.spec with Patch[0-9]* is automatically applied in %prep. Unlike with the standard %patch[0-9]*, patches are applied with "git am" to avoid some unexpected results. However, as a result of this, all patches must be in the right format for "git am" to be able to apply them; they should ideally be generated from git using "git format-patch". Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- libvirt.spec.in | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index 6fcaa3e..0959483 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -425,6 +425,7 @@ BuildRequires: gettext-devel BuildRequires: libtool BuildRequires: /usr/bin/pod2man %endif +BuildRequires: git BuildRequires: perl BuildRequires: python %if %{with_systemd} @@ -1198,6 +1199,41 @@ driver %prep %setup -q +# Patches have to be stored in a temporary file because RPM has +# a limit on the length of the result of any macro expansion; +# if the string is longer, it's silently cropped +%{lua: + tmp = os.tmpname(); + f = io.open(tmp, "w+"); + count = 0; + for i, p in ipairs(patches) do + f:write(p.."\n"); + count = count + 1; + end; + f:close(); + print("PATCHCOUNT="..count.."\n") + print("PATCHLIST="..tmp.."\n") +} + +git init -q +git config user.name rpm-build +git config user.email rpm-build +git config gc.auto 0 +git add . +git commit -q -a --author 'rpm-build <rpm-build>' \ + -m '%{name}-%{version} base' + +COUNT=$(grep '\.patch$' $PATCHLIST | wc -l) +if [ $COUNT -ne $PATCHCOUNT ]; then + echo "Found $COUNT patches in $PATCHLIST, expected $PATCHCOUNT" + exit 1 +fi +if [ $COUNT -gt 0 ]; then + xargs git am <$PATCHLIST || exit 1 +fi +echo "Applied $COUNT patches" +rm -f $PATCHLIST + %build %if ! %{with_xen} %define _without_xen --without-xen -- 2.1.3

3 3

[libvirt] lxcContainerResolveSymlinks failing on cascading FS
by Cedric Bosdonnat 24 Nov '14

24 Nov '14

Hi all, As an example, the following command virt-sandbox-service create --network dhcp,source=default --unitfile sshd.service mysshd -i 512 generates a container definition containing these filesystems (in the same order): <filesystem type='file' accessmode='passthrough'> <source file='/var/lib/libvirt/images/mysshd.raw'/> <target dir='/var/lib/libvirt/filesystems/mysshd'/> </filesystem> <filesystem type='bind' accessmode='passthrough'> <source dir='/var/lib/libvirt/filesystems/mysshd/var'/> <target dir='/var'/> </filesystem> Since /var/lib/libvirt/filesystems/mysshd contains nothing, the second FS needs the first one to be mounted for the source directory to exist. The problem comes with lxcContainerResolveSymlinks() being run before any file system is actually mounted. So the container can't be started and we get the following error: Failed to access '/var/lib/libvirt/filesystems/mysshd/var': No such file or directory This would work if the symlinks were resolved right before mounting the FS instead of before mounting any of them. Any strong opinion against it? -- Cedric

2 1