[libvirt] [PATCH 0/7] Fix reversed stats/QoS for macvtap
This is initiated by: https://bugzilla.redhat.com/show_bug.cgi?id=1497410 Long story short, in some cases host and domain are on the same side of RX/TX stats/QoS. Michal Privoznik (7): lxc: Drop useless ifdef __linux__ qemuDomainInterfaceStats: Check for the actual type of interface conf: Introduce virDomainNetFindByName src: Use virDomainNetFindByName virNetDevTapInterfaceStats: Allow caller to not swap the statistics QoS: Set classes and filters in proper direction libvirt-domain: Document interface stats POV src/conf/domain_conf.c | 31 ++++++++++--- src/conf/domain_conf.h | 38 +++++++++++++++- src/libvirt-domain.c | 4 +- src/libvirt_private.syms | 1 + src/libxl/libxl_driver.c | 23 +++++----- src/lxc/lxc_driver.c | 38 ++++++---------- src/lxc/lxc_process.c | 3 +- src/network/bridge_driver.c | 2 +- src/openvz/openvz_driver.c | 23 +++++----- src/qemu/qemu_command.c | 3 +- src/qemu/qemu_driver.c | 41 +++++++++-------- src/qemu/qemu_hotplug.c | 6 ++- src/test/test_driver.c | 28 +++--------- src/util/virnetdevbandwidth.c | 43 ++++++++++++------ src/util/virnetdevbandwidth.h | 3 +- src/util/virnetdevtap.c | 99 ++++++++++++++++++++++++++++-------------- src/util/virnetdevtap.h | 3 +- src/xen/xen_hypervisor.c | 2 +- tests/virnetdevbandwidthtest.c | 2 +- 19 files changed, 236 insertions(+), 157 deletions(-) -- 2.13.5
This code compiles only on Linux. Therefore the condition we check is always true. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/lxc/lxc_driver.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 5cc392b0e..4d24d2870 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -2847,7 +2847,6 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, } -#ifdef __linux__ static int lxcDomainInterfaceStats(virDomainPtr dom, const char *path, @@ -2895,16 +2894,7 @@ lxcDomainInterfaceStats(virDomainPtr dom, virDomainObjEndAPI(&vm); return ret; } -#else -static int -lxcDomainInterfaceStats(virDomainPtr dom, - const char *path ATTRIBUTE_UNUSED, - virDomainInterfaceStatsPtr stats ATTRIBUTE_UNUSED) -{ - virReportUnsupportedError(); - return -1; -} -#endif + static int lxcDomainGetAutostart(virDomainPtr dom, int *autostart) -- 2.13.5
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
This code compiles only on Linux. Therefore the condition we check is always true.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/lxc/lxc_driver.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-)
Hahaha - since 'e51cf5c19' (Nov 12, 2009)... Can't we last like another couple of years ;-) Reviewed-by: John Ferlan <jferlan@redhat.com> John
Users might have configured interface so that it's type of network, but the corresponding network plugs interfaces into an OVS bridge. Therefore, we have to check for the actual type of the interface instead of the configured one. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/qemu/qemu_driver.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 4855c9047..2c8ea19e3 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -11045,7 +11045,7 @@ qemuDomainInterfaceStats(virDomainPtr dom, goto cleanup; } - if (net->type == VIR_DOMAIN_NET_TYPE_VHOSTUSER) { + if (virDomainNetGetActualType(net) == VIR_DOMAIN_NET_TYPE_VHOSTUSER) { if (virNetDevOpenvswitchInterfaceStats(path, stats) < 0) goto cleanup; } else { @@ -19568,15 +19568,19 @@ qemuDomainGetStatsInterface(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, /* Check the path is one of the domain's network interfaces. */ for (i = 0; i < dom->def->nnets; i++) { + virDomainNetType actualType; + if (!dom->def->nets[i]->ifname) continue; memset(&tmp, 0, sizeof(tmp)); + actualType = virDomainNetGetActualType(dom->def->nets[i]); + QEMU_ADD_NAME_PARAM(record, maxparams, "net", "name", i, dom->def->nets[i]->ifname); - if (dom->def->nets[i]->type == VIR_DOMAIN_NET_TYPE_VHOSTUSER) { + if (actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER) { if (virNetDevOpenvswitchInterfaceStats(dom->def->nets[i]->ifname, &tmp) < 0) { virResetLastError(); -- 2.13.5
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
Users might have configured interface so that it's type of network, but the corresponding network plugs interfaces into an OVS bridge. Therefore, we have to check for the actual type of the interface instead of the configured one.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/qemu/qemu_driver.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
Reviewed-by: John Ferlan <jferlan@redhat.com> John
Small wrapper to lookup interface in domain definition by its name. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/conf/domain_conf.c | 29 +++++++++++++++++++++++------ src/conf/domain_conf.h | 1 + src/libvirt_private.syms | 1 + 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 87192eb2d..2289399cd 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -26952,17 +26952,34 @@ virDomainNetFind(virDomainDefPtr def, const char *device) } } } else { /* ifname */ - for (i = 0; i < def->nnets; i++) { - if (STREQ_NULLABLE(device, def->nets[i]->ifname)) { - net = def->nets[i]; - break; - } - } + net = virDomainNetFindByName(def, device); } return net; } +/** + * virDomainNetFindByName: + * @def: domain's def + * @ifname: interface name + * + * Finds a domain's net def given the interface name. + * + * Returns a pointer to the net def or NULL if not found. + */ +virDomainNetDefPtr +virDomainNetFindByName(virDomainDefPtr def, const char *ifname) +{ + size_t i; + + for (i = 0; i < def->nnets; i++) { + if (STREQ_NULLABLE(ifname, def->nets[i]->ifname)) + return def->nets[i]; + } + + return NULL; +} + /** * virDomainDeviceDefCopy: * @caps: Capabilities diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 05a035a16..9ba84a94d 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3000,6 +3000,7 @@ int virDomainDiskSourceParse(xmlNodePtr node, int virDomainNetFindIdx(virDomainDefPtr def, virDomainNetDefPtr net); virDomainNetDefPtr virDomainNetFind(virDomainDefPtr def, const char *device); +virDomainNetDefPtr virDomainNetFindByName(virDomainDefPtr def, const char *ifname); bool virDomainHasNet(virDomainDefPtr def, virDomainNetDefPtr net); int virDomainNetInsert(virDomainDefPtr def, virDomainNetDefPtr net); virDomainNetDefPtr virDomainNetRemove(virDomainDefPtr def, size_t i); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 5b1bc5e4f..7a12d6a14 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -422,6 +422,7 @@ virDomainNetDefClear; virDomainNetDefFormat; virDomainNetDefFree; virDomainNetFind; +virDomainNetFindByName; virDomainNetFindIdx; virDomainNetGenerateMAC; virDomainNetGetActualBandwidth; -- 2.13.5
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
Small wrapper to lookup interface in domain definition by its name.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/conf/domain_conf.c | 29 +++++++++++++++++++++++------ src/conf/domain_conf.h | 1 + src/libvirt_private.syms | 1 + 3 files changed, 25 insertions(+), 6 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 87192eb2d..2289399cd 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -26952,17 +26952,34 @@ virDomainNetFind(virDomainDefPtr def, const char *device) } } } else { /* ifname */ - for (i = 0; i < def->nnets; i++) { - if (STREQ_NULLABLE(device, def->nets[i]->ifname)) { - net = def->nets[i]; - break; - } - } + net = virDomainNetFindByName(def, device); }
return net; }
Two blank lines between functions
+/** + * virDomainNetFindByName: + * @def: domain's def + * @ifname: interface name + * + * Finds a domain's net def given the interface name. + * + * Returns a pointer to the net def or NULL if not found. + */ +virDomainNetDefPtr +virDomainNetFindByName(virDomainDefPtr def, const char *ifname)
Multiple lines per parameter
+{ + size_t i; + + for (i = 0; i < def->nnets; i++) { + if (STREQ_NULLABLE(ifname, def->nets[i]->ifname)) + return def->nets[i]; + } + + return NULL; +} +
Two blank lines Reviewed-by: John Ferlan <jferlan@redhat.com> John
/** * virDomainDeviceDefCopy: * @caps: Capabilities diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 05a035a16..9ba84a94d 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3000,6 +3000,7 @@ int virDomainDiskSourceParse(xmlNodePtr node,
int virDomainNetFindIdx(virDomainDefPtr def, virDomainNetDefPtr net); virDomainNetDefPtr virDomainNetFind(virDomainDefPtr def, const char *device); +virDomainNetDefPtr virDomainNetFindByName(virDomainDefPtr def, const char *ifname); bool virDomainHasNet(virDomainDefPtr def, virDomainNetDefPtr net); int virDomainNetInsert(virDomainDefPtr def, virDomainNetDefPtr net); virDomainNetDefPtr virDomainNetRemove(virDomainDefPtr def, size_t i); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 5b1bc5e4f..7a12d6a14 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -422,6 +422,7 @@ virDomainNetDefClear; virDomainNetDefFormat; virDomainNetDefFree; virDomainNetFind; +virDomainNetFindByName; virDomainNetFindIdx; virDomainNetGenerateMAC; virDomainNetGetActualBandwidth;
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/libxl/libxl_driver.c | 22 +++++++++------------- src/lxc/lxc_driver.c | 22 +++++++++------------- src/openvz/openvz_driver.c | 22 +++++++++------------- src/qemu/qemu_driver.c | 11 +---------- src/test/test_driver.c | 28 ++++++---------------------- 5 files changed, 34 insertions(+), 71 deletions(-) diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index bf3625e34..a174d892e 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -4961,7 +4961,7 @@ libxlDomainInterfaceStats(virDomainPtr dom, { libxlDriverPrivatePtr driver = dom->conn->privateData; virDomainObjPtr vm; - size_t i; + virDomainNetDefPtr net = NULL; int ret = -1; if (!(vm = libxlDomObjFromDomain(dom))) @@ -4979,20 +4979,16 @@ libxlDomainInterfaceStats(virDomainPtr dom, goto endjob; } - /* Check the path is one of the domain's network interfaces. */ - for (i = 0; i < vm->def->nnets; i++) { - if (vm->def->nets[i]->ifname && - STREQ(vm->def->nets[i]->ifname, path)) { - ret = 0; - break; - } - } - - if (ret == 0) - ret = virNetDevTapInterfaceStats(path, stats); - else + if (!(net = virDomainNetFindByName(vm->def, path))) { virReportError(VIR_ERR_INVALID_ARG, _("'%s' is not a known interface"), path); + goto endjob; + } + + if (virNetDevTapInterfaceStats(path, stats) < 0) + goto endjob; + + ret = 0; endjob: libxlDomainObjEndJob(driver, vm); diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 4d24d2870..c0ef0c210 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -2853,9 +2853,9 @@ lxcDomainInterfaceStats(virDomainPtr dom, virDomainInterfaceStatsPtr stats) { virDomainObjPtr vm; - size_t i; int ret = -1; virLXCDriverPtr driver = dom->conn->privateData; + virDomainNetDefPtr net = NULL; if (!(vm = lxcDomObjFromDomain(dom))) goto cleanup; @@ -2872,20 +2872,16 @@ lxcDomainInterfaceStats(virDomainPtr dom, goto endjob; } - /* Check the path is one of the domain's network interfaces. */ - for (i = 0; i < vm->def->nnets; i++) { - if (vm->def->nets[i]->ifname && - STREQ(vm->def->nets[i]->ifname, path)) { - ret = 0; - break; - } - } - - if (ret == 0) - ret = virNetDevTapInterfaceStats(path, stats); - else + if (!(net = virDomainNetFindByName(vm->def, path))) { virReportError(VIR_ERR_INVALID_ARG, _("Invalid path, '%s' is not a known interface"), path); + goto endjob; + } + + if (virNetDevTapInterfaceStats(path, stats) < 0) + goto endjob; + + ret = 0; endjob: virLXCDomainObjEndJob(driver, vm); diff --git a/src/openvz/openvz_driver.c b/src/openvz/openvz_driver.c index a1485fc88..3c24020ce 100644 --- a/src/openvz/openvz_driver.c +++ b/src/openvz/openvz_driver.c @@ -1985,7 +1985,7 @@ openvzDomainInterfaceStats(virDomainPtr dom, { struct openvz_driver *driver = dom->conn->privateData; virDomainObjPtr vm; - size_t i; + virDomainNetDefPtr net = NULL; int ret = -1; openvzDriverLock(driver); @@ -2006,20 +2006,16 @@ openvzDomainInterfaceStats(virDomainPtr dom, goto cleanup; } - /* Check the path is one of the domain's network interfaces. */ - for (i = 0; i < vm->def->nnets; i++) { - if (vm->def->nets[i]->ifname && - STREQ(vm->def->nets[i]->ifname, path)) { - ret = 0; - break; - } - } - - if (ret == 0) - ret = virNetDevTapInterfaceStats(path, stats); - else + if (!(net = virDomainNetFindByName(vm->def, path))) { virReportError(VIR_ERR_INVALID_ARG, _("invalid path, '%s' is not a known interface"), path); + goto cleanup; + } + + if (virNetDevTapInterfaceStats(path, stats) < 0) + goto cleanup; + + ret = 0; cleanup: if (vm) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 2c8ea19e3..1ab16e57c 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -11016,7 +11016,6 @@ qemuDomainInterfaceStats(virDomainPtr dom, { virDomainObjPtr vm; virDomainNetDefPtr net = NULL; - size_t i; int ret = -1; if (!(vm = qemuDomObjFromDomain(dom))) @@ -11031,15 +11030,7 @@ qemuDomainInterfaceStats(virDomainPtr dom, goto cleanup; } - /* Check the path is one of the domain's network interfaces. */ - for (i = 0; i < vm->def->nnets; i++) { - if (STREQ_NULLABLE(vm->def->nets[i]->ifname, path)) { - net = vm->def->nets[i]; - break; - } - } - - if (!net) { + if (!(net = virDomainNetFindByName(vm->def, path))) { virReportError(VIR_ERR_INVALID_ARG, _("invalid path, '%s' is not a known interface"), path); goto cleanup; diff --git a/src/test/test_driver.c b/src/test/test_driver.c index 9b434e9a0..e92768a97 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -593,25 +593,16 @@ testDomainGenerateIfname(virDomainDefPtr domdef) { int maxif = 1024; int ifctr; - size_t i; for (ifctr = 0; ifctr < maxif; ++ifctr) { + virDomainNetDefPtr net = NULL; char *ifname; - int found = 0; if (virAsprintf(&ifname, "testnet%d", ifctr) < 0) return NULL; /* Generate network interface names */ - for (i = 0; i < domdef->nnets; i++) { - if (domdef->nets[i]->ifname && - STREQ(domdef->nets[i]->ifname, ifname)) { - found = 1; - break; - } - } - - if (!found) + if (!(net = virDomainNetFindByName(domdef, ifname))) return ifname; VIR_FREE(ifname); } @@ -3176,8 +3167,9 @@ static int testDomainInterfaceStats(virDomainPtr domain, virDomainObjPtr privdom; struct timeval tv; unsigned long long statbase; - size_t i; - int found = 0, ret = -1; + virDomainNetDefPtr net = NULL; + int ret = -1; + if (!(privdom = testDomObjFromDomain(domain))) return -1; @@ -3188,15 +3180,7 @@ static int testDomainInterfaceStats(virDomainPtr domain, goto error; } - for (i = 0; i < privdom->def->nnets; i++) { - if (privdom->def->nets[i]->ifname && - STREQ(privdom->def->nets[i]->ifname, path)) { - found = 1; - break; - } - } - - if (!found) { + if (!(net = virDomainNetFindByName(privdom->def, path))) { virReportError(VIR_ERR_INVALID_ARG, _("invalid path, '%s' is not a known interface"), path); goto error; -- 2.13.5
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/libxl/libxl_driver.c | 22 +++++++++------------- src/lxc/lxc_driver.c | 22 +++++++++------------- src/openvz/openvz_driver.c | 22 +++++++++------------- src/qemu/qemu_driver.c | 11 +---------- src/test/test_driver.c | 28 ++++++---------------------- 5 files changed, 34 insertions(+), 71 deletions(-)
Reviewed-by: John Ferlan <jferlan@redhat.com> John
https://bugzilla.redhat.com/show_bug.cgi?id=1497410 The comment in virNetDevTapInterfaceStats() implementation for Linux states that packets transmitted by domain are received by the host and vice versa. Well, this is true but not for all types of interfaces. For instance, for macvtaps when TAP device is hooked right onto a physical device any packet that domain sends looks also like a packet sent to the host. Therefore, we should have caller chose if the stats returned should be straight copy or swapped. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 37 ++++++++++++++++- src/libxl/libxl_driver.c | 3 +- src/lxc/lxc_driver.c | 3 +- src/openvz/openvz_driver.c | 3 +- src/qemu/qemu_driver.c | 18 +++++---- src/util/virnetdevtap.c | 99 +++++++++++++++++++++++++++++++--------------- src/util/virnetdevtap.h | 3 +- src/xen/xen_hypervisor.c | 2 +- 9 files changed, 123 insertions(+), 47 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 2289399cd..6569b94fa 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -26729,7 +26729,7 @@ virDomainStateReasonFromString(virDomainState state, const char *reason) */ virDomainNetType -virDomainNetGetActualType(virDomainNetDefPtr iface) +virDomainNetGetActualType(const virDomainNetDef *iface) { if (iface->type != VIR_DOMAIN_NET_TYPE_NETWORK) return iface->type; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 9ba84a94d..6eda21ef7 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3021,7 +3021,7 @@ int virDomainGraphicsListenAppendSocket(virDomainGraphicsDefPtr def, const char *socket) ATTRIBUTE_NONNULL(1); -virDomainNetType virDomainNetGetActualType(virDomainNetDefPtr iface); +virDomainNetType virDomainNetGetActualType(const virDomainNetDef *iface); const char *virDomainNetGetActualBridgeName(virDomainNetDefPtr iface); int virDomainNetGetActualBridgeMACTableManager(virDomainNetDefPtr iface); const char *virDomainNetGetActualDirectDev(virDomainNetDefPtr iface); @@ -3393,4 +3393,39 @@ virDomainGenerateMachineName(const char *drivername, int id, const char *name, bool privileged); +/** + * virDomainNetTypeSharesHostView: + * @net: interface + * + * Some types of interfaces "share" the host view. For instance, + * for macvtap interface, every domain RX is the host RX too. And + * every domain TX is host TX too. IOW, for some types of + * interfaces guest and host are on the same side of RX/TX + * barrier. This is important so that we set up QoS correctly and + * report proper stats. + */ +static inline bool +virDomainNetTypeSharesHostView(const virDomainNetDef *net) +{ + virDomainNetType actualType = virDomainNetGetActualType(net); + switch (actualType) { + case VIR_DOMAIN_NET_TYPE_DIRECT: + case VIR_DOMAIN_NET_TYPE_ETHERNET: + return true; + case VIR_DOMAIN_NET_TYPE_USER: + case VIR_DOMAIN_NET_TYPE_VHOSTUSER: + case VIR_DOMAIN_NET_TYPE_SERVER: + case VIR_DOMAIN_NET_TYPE_CLIENT: + case VIR_DOMAIN_NET_TYPE_MCAST: + case VIR_DOMAIN_NET_TYPE_NETWORK: + case VIR_DOMAIN_NET_TYPE_BRIDGE: + case VIR_DOMAIN_NET_TYPE_INTERNAL: + case VIR_DOMAIN_NET_TYPE_HOSTDEV: + case VIR_DOMAIN_NET_TYPE_UDP: + case VIR_DOMAIN_NET_TYPE_LAST: + break; + } + return false; +} + #endif /* __DOMAIN_CONF_H */ diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index a174d892e..8483d6ecf 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -4985,7 +4985,8 @@ libxlDomainInterfaceStats(virDomainPtr dom, goto endjob; } - if (virNetDevTapInterfaceStats(path, stats) < 0) + if (virNetDevTapInterfaceStats(path, stats, + !virDomainNetTypeSharesHostView(net)) < 0) goto endjob; ret = 0; diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index c0ef0c210..f0a22ce3e 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -2878,7 +2878,8 @@ lxcDomainInterfaceStats(virDomainPtr dom, goto endjob; } - if (virNetDevTapInterfaceStats(path, stats) < 0) + if (virNetDevTapInterfaceStats(path, stats, + !virDomainNetTypeSharesHostView(net)) < 0) goto endjob; ret = 0; diff --git a/src/openvz/openvz_driver.c b/src/openvz/openvz_driver.c index 3c24020ce..11173898d 100644 --- a/src/openvz/openvz_driver.c +++ b/src/openvz/openvz_driver.c @@ -2012,7 +2012,8 @@ openvzDomainInterfaceStats(virDomainPtr dom, goto cleanup; } - if (virNetDevTapInterfaceStats(path, stats) < 0) + if (virNetDevTapInterfaceStats(path, stats, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; ret = 0; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 1ab16e57c..a46409d70 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -11040,7 +11040,8 @@ qemuDomainInterfaceStats(virDomainPtr dom, if (virNetDevOpenvswitchInterfaceStats(path, stats) < 0) goto cleanup; } else { - if (virNetDevTapInterfaceStats(path, stats) < 0) + if (virNetDevTapInterfaceStats(path, stats, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; } @@ -19559,29 +19560,30 @@ qemuDomainGetStatsInterface(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, /* Check the path is one of the domain's network interfaces. */ for (i = 0; i < dom->def->nnets; i++) { + virDomainNetDefPtr net = dom->def->nets[i]; virDomainNetType actualType; - if (!dom->def->nets[i]->ifname) + if (!net->ifname) continue; memset(&tmp, 0, sizeof(tmp)); - actualType = virDomainNetGetActualType(dom->def->nets[i]); + actualType = virDomainNetGetActualType(net); QEMU_ADD_NAME_PARAM(record, maxparams, - "net", "name", i, dom->def->nets[i]->ifname); + "net", "name", i, net->ifname); if (actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER) { - if (virNetDevOpenvswitchInterfaceStats(dom->def->nets[i]->ifname, + if (virNetDevOpenvswitchInterfaceStats(net->ifname, &tmp) < 0) { virResetLastError(); continue; } } else { - if (virNetDevTapInterfaceStats(dom->def->nets[i]->ifname, &tmp) < 0) { + if (virNetDevTapInterfaceStats(net->ifname, &tmp, + !virDomainNetTypeSharesHostView(net)) < 0) virResetLastError(); - continue; - } + continue; } QEMU_ADD_NET_PARAM(record, maxparams, i, diff --git a/src/util/virnetdevtap.c b/src/util/virnetdevtap.c index 175dc2bfa..a3ed59da8 100644 --- a/src/util/virnetdevtap.c +++ b/src/util/virnetdevtap.c @@ -676,14 +676,27 @@ int virNetDevTapCreateInBridgePort(const char *brname, } /*-------------------- interface stats --------------------*/ -/* Just reads the named interface, so not Xen or QEMU-specific. - * NB. Caller must check that libvirt user is trying to query - * the interface of a domain they own. We do no such checking. + +/** + * virNetDevTapInterfaceStats: + * @ifname: interface + * @stats: where to store statistics + * @swapped: whether to swap RX/TX fields + * + * Fetch RX/TX statistics for given named interface (@ifname) and + * store them at @stats. The returned statistics are always from + * domain POV. Because in some cases this means swapping RX/TX in + * the stats and in others this means no swapping (consider TAP + * vs macvtap) caller might choose if the returned stats should + * be @swapped or not. + * + * Returns 0 on success, -1 otherwise (with error reported). */ #ifdef __linux__ int virNetDevTapInterfaceStats(const char *ifname, - virDomainInterfaceStatsPtr stats) + virDomainInterfaceStatsPtr stats, + bool swapped) { int ifname_len; FILE *fp; @@ -718,30 +731,35 @@ virNetDevTapInterfaceStats(const char *ifname, *colon = '\0'; if (colon-ifname_len >= line && STREQ(colon-ifname_len, ifname)) { - /* IMPORTANT NOTE! - * /proc/net/dev vif<domid>.nn sees the network from the point - * of view of dom0 / hypervisor. So bytes TRANSMITTED by dom0 - * are bytes RECEIVED by the domain. That's why the TX/RX fields - * appear to be swapped here. - */ if (sscanf(colon+1, "%lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld", - &tx_bytes, &tx_packets, &tx_errs, &tx_drop, - &dummy, &dummy, &dummy, &dummy, &rx_bytes, &rx_packets, &rx_errs, &rx_drop, + &dummy, &dummy, &dummy, &dummy, + &tx_bytes, &tx_packets, &tx_errs, &tx_drop, &dummy, &dummy, &dummy, &dummy) != 16) continue; - stats->rx_bytes = rx_bytes; - stats->rx_packets = rx_packets; - stats->rx_errs = rx_errs; - stats->rx_drop = rx_drop; - stats->tx_bytes = tx_bytes; - stats->tx_packets = tx_packets; - stats->tx_errs = tx_errs; - stats->tx_drop = tx_drop; + if (swapped) { + stats->rx_bytes = tx_bytes; + stats->rx_packets = tx_packets; + stats->rx_errs = tx_errs; + stats->rx_drop = tx_drop; + stats->tx_bytes = rx_bytes; + stats->tx_packets = rx_packets; + stats->tx_errs = rx_errs; + stats->tx_drop = rx_drop; + } else { + stats->rx_bytes = rx_bytes; + stats->rx_packets = rx_packets; + stats->rx_errs = rx_errs; + stats->rx_drop = rx_drop; + stats->tx_bytes = tx_bytes; + stats->tx_packets = tx_packets; + stats->tx_errs = tx_errs; + stats->tx_drop = tx_drop; + } + VIR_FORCE_FCLOSE(fp); - return 0; } } @@ -754,7 +772,8 @@ virNetDevTapInterfaceStats(const char *ifname, #elif defined(HAVE_GETIFADDRS) && defined(AF_LINK) int virNetDevTapInterfaceStats(const char *ifname, - virDomainInterfaceStatsPtr stats) + virDomainInterfaceStatsPtr stats, + bool swapped) { struct ifaddrs *ifap, *ifa; struct if_data *ifd; @@ -775,18 +794,33 @@ virNetDevTapInterfaceStats(const char *ifname, if (STREQ(ifa->ifa_name, ifname)) { ifd = (struct if_data *)ifa->ifa_data; - stats->tx_bytes = ifd->ifi_ibytes; - stats->tx_packets = ifd->ifi_ipackets; - stats->tx_errs = ifd->ifi_ierrors; - stats->tx_drop = ifd->ifi_iqdrops; - stats->rx_bytes = ifd->ifi_obytes; - stats->rx_packets = ifd->ifi_opackets; - stats->rx_errs = ifd->ifi_oerrors; + if (swapped) { + stats->tx_bytes = ifd->ifi_ibytes; + stats->tx_packets = ifd->ifi_ipackets; + stats->tx_errs = ifd->ifi_ierrors; + stats->tx_drop = ifd->ifi_iqdrops; + stats->rx_bytes = ifd->ifi_obytes; + stats->rx_packets = ifd->ifi_opackets; + stats->rx_errs = ifd->ifi_oerrors; # ifdef HAVE_STRUCT_IF_DATA_IFI_OQDROPS - stats->rx_drop = ifd->ifi_oqdrops; + stats->rx_drop = ifd->ifi_oqdrops; # else - stats->rx_drop = 0; + stats->rx_drop = 0; # endif + } else { + stats->tx_bytes = ifd->ifi_obytes; + stats->tx_packets = ifd->ifi_opackets; + stats->tx_errs = ifd->ifi_oerrors; +# ifdef HAVE_STRUCT_IF_DATA_IFI_OQDROPS + stats->tx_drop = ifd->ifi_oqdrops; +# else + stats->tx_drop = 0; +# endif + stats->rx_bytes = ifd->ifi_ibytes; + stats->rx_packets = ifd->ifi_ipackets; + stats->rx_errs = ifd->ifi_ierrors; + stats->rx_drop = ifd->ifi_iqdrops; + } ret = 0; break; @@ -803,7 +837,8 @@ virNetDevTapInterfaceStats(const char *ifname, #else int virNetDevTapInterfaceStats(const char *ifname ATTRIBUTE_UNUSED, - virDomainInterfaceStatsPtr stats ATTRIBUTE_UNUSED) + virDomainInterfaceStatsPtr stats ATTRIBUTE_UNUSED, + bool swapped ATTRIBUTE_UNUSED) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("interface stats not implemented on this platform")); diff --git a/src/util/virnetdevtap.h b/src/util/virnetdevtap.h index 0b17feb8d..727368fe9 100644 --- a/src/util/virnetdevtap.h +++ b/src/util/virnetdevtap.h @@ -92,7 +92,8 @@ int virNetDevTapCreateInBridgePort(const char *brname, ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; int virNetDevTapInterfaceStats(const char *ifname, - virDomainInterfaceStatsPtr stats) + virDomainInterfaceStatsPtr stats, + bool swapped) ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; #endif /* __VIR_NETDEV_TAP_H__ */ diff --git a/src/xen/xen_hypervisor.c b/src/xen/xen_hypervisor.c index e18a4729d..70fdcbe55 100644 --- a/src/xen/xen_hypervisor.c +++ b/src/xen/xen_hypervisor.c @@ -1468,7 +1468,7 @@ xenHypervisorDomainInterfaceStats(virDomainDefPtr def, return -1; } - return virNetDevTapInterfaceStats(path, stats); + return virNetDevTapInterfaceStats(path, stats, true); #else virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("/proc/net/dev: Interface not found")); -- 2.13.5
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1497410
The comment in virNetDevTapInterfaceStats() implementation for Linux states that packets transmitted by domain are received by the host and vice versa. Well, this is true but not for all types of interfaces. For instance, for macvtaps when TAP device is hooked right onto a physical device any packet that domain sends looks also like a packet sent to the host. Therefore, we should have caller chose if the stats returned should be straight copy
s/have/allow/ s/chose/to choose BTW: My first read of this thought "caller" is user configurable :-)
or swapped.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 37 ++++++++++++++++- src/libxl/libxl_driver.c | 3 +- src/lxc/lxc_driver.c | 3 +- src/openvz/openvz_driver.c | 3 +- src/qemu/qemu_driver.c | 18 +++++---- src/util/virnetdevtap.c | 99 +++++++++++++++++++++++++++++++--------------- src/util/virnetdevtap.h | 3 +- src/xen/xen_hypervisor.c | 2 +- 9 files changed, 123 insertions(+), 47 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 2289399cd..6569b94fa 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -26729,7 +26729,7 @@ virDomainStateReasonFromString(virDomainState state, const char *reason) */
virDomainNetType -virDomainNetGetActualType(virDomainNetDefPtr iface) +virDomainNetGetActualType(const virDomainNetDef *iface)
Sigh, separable along w/ domain_conf.h change...
{ if (iface->type != VIR_DOMAIN_NET_TYPE_NETWORK) return iface->type; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 9ba84a94d..6eda21ef7 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3021,7 +3021,7 @@ int virDomainGraphicsListenAppendSocket(virDomainGraphicsDefPtr def, const char *socket) ATTRIBUTE_NONNULL(1);
-virDomainNetType virDomainNetGetActualType(virDomainNetDefPtr iface); +virDomainNetType virDomainNetGetActualType(const virDomainNetDef *iface);
e.g. this is separate and unrelated needs it's own already R-B patch ;-) The rest is "this" patch specific...
const char *virDomainNetGetActualBridgeName(virDomainNetDefPtr iface); int virDomainNetGetActualBridgeMACTableManager(virDomainNetDefPtr iface); const char *virDomainNetGetActualDirectDev(virDomainNetDefPtr iface); @@ -3393,4 +3393,39 @@ virDomainGenerateMachineName(const char *drivername, int id, const char *name, bool privileged); +/** + * virDomainNetTypeSharesHostView: + * @net: interface + * + * Some types of interfaces "share" the host view. For instance, + * for macvtap interface, every domain RX is the host RX too. And + * every domain TX is host TX too. IOW, for some types of + * interfaces guest and host are on the same side of RX/TX + * barrier. This is important so that we set up QoS correctly and + * report proper stats. + */ +static inline bool +virDomainNetTypeSharesHostView(const virDomainNetDef *net) +{ + virDomainNetType actualType = virDomainNetGetActualType(net); + switch (actualType) { + case VIR_DOMAIN_NET_TYPE_DIRECT: + case VIR_DOMAIN_NET_TYPE_ETHERNET: + return true; + case VIR_DOMAIN_NET_TYPE_USER: + case VIR_DOMAIN_NET_TYPE_VHOSTUSER: + case VIR_DOMAIN_NET_TYPE_SERVER: + case VIR_DOMAIN_NET_TYPE_CLIENT: + case VIR_DOMAIN_NET_TYPE_MCAST: + case VIR_DOMAIN_NET_TYPE_NETWORK: + case VIR_DOMAIN_NET_TYPE_BRIDGE: + case VIR_DOMAIN_NET_TYPE_INTERNAL: + case VIR_DOMAIN_NET_TYPE_HOSTDEV: + case VIR_DOMAIN_NET_TYPE_UDP: + case VIR_DOMAIN_NET_TYPE_LAST: + break; + } + return false; +} + #endif /* __DOMAIN_CONF_H */
[...]
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 1ab16e57c..a46409d70 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -11040,7 +11040,8 @@ qemuDomainInterfaceStats(virDomainPtr dom, if (virNetDevOpenvswitchInterfaceStats(path, stats) < 0) goto cleanup; } else { - if (virNetDevTapInterfaceStats(path, stats) < 0) + if (virNetDevTapInterfaceStats(path, stats, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; }
@@ -19559,29 +19560,30 @@ qemuDomainGetStatsInterface(virQEMUDriverPtr driver ATTRIBUTE_UNUSED,
/* Check the path is one of the domain's network interfaces. */ for (i = 0; i < dom->def->nnets; i++) { + virDomainNetDefPtr net = dom->def->nets[i]; virDomainNetType actualType;
- if (!dom->def->nets[i]->ifname) + if (!net->ifname) continue;
memset(&tmp, 0, sizeof(tmp));
- actualType = virDomainNetGetActualType(dom->def->nets[i]); + actualType = virDomainNetGetActualType(net);
QEMU_ADD_NAME_PARAM(record, maxparams, - "net", "name", i, dom->def->nets[i]->ifname); + "net", "name", i, net->ifname);
if (actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER) { - if (virNetDevOpenvswitchInterfaceStats(dom->def->nets[i]->ifname, + if (virNetDevOpenvswitchInterfaceStats(net->ifname, &tmp) < 0) {
This line may as well move to previous line as well for just one line.
virResetLastError(); continue; } } else { - if (virNetDevTapInterfaceStats(dom->def->nets[i]->ifname, &tmp) < 0) { + if (virNetDevTapInterfaceStats(net->ifname, &tmp, + !virDomainNetTypeSharesHostView(net)) < 0) virResetLastError(); - continue; - } + continue;
Is this one right? We won't QEMU_ADD_NET_PARAM when actual_type != VIR_DOMAIN_NET_TYPE_VHOSTUSER *and* virNetDevTapInterfaceStats succeeds? Looks like a bad transposition of extracted patches.
}
QEMU_ADD_NET_PARAM(record, maxparams, i, diff --git a/src/util/virnetdevtap.c b/src/util/virnetdevtap.c index 175dc2bfa..a3ed59da8 100644 --- a/src/util/virnetdevtap.c +++ b/src/util/virnetdevtap.c @@ -676,14 +676,27 @@ int virNetDevTapCreateInBridgePort(const char *brname, }
/*-------------------- interface stats --------------------*/ -/* Just reads the named interface, so not Xen or QEMU-specific. - * NB. Caller must check that libvirt user is trying to query - * the interface of a domain they own. We do no such checking. + +/** + * virNetDevTapInterfaceStats: + * @ifname: interface + * @stats: where to store statistics + * @swapped: whether to swap RX/TX fields + * + * Fetch RX/TX statistics for given named interface (@ifname) and + * store them at @stats. The returned statistics are always from + * domain POV. Because in some cases this means swapping RX/TX in + * the stats and in others this means no swapping (consider TAP + * vs macvtap) caller might choose if the returned stats should + * be @swapped or not. + * + * Returns 0 on success, -1 otherwise (with error reported). */ #ifdef __linux__ int virNetDevTapInterfaceStats(const char *ifname, - virDomainInterfaceStatsPtr stats) + virDomainInterfaceStatsPtr stats, + bool swapped) { int ifname_len; FILE *fp; @@ -718,30 +731,35 @@ virNetDevTapInterfaceStats(const char *ifname, *colon = '\0'; if (colon-ifname_len >= line && STREQ(colon-ifname_len, ifname)) { - /* IMPORTANT NOTE! - * /proc/net/dev vif<domid>.nn sees the network from the point - * of view of dom0 / hypervisor. So bytes TRANSMITTED by dom0 - * are bytes RECEIVED by the domain. That's why the TX/RX fields - * appear to be swapped here.
Maybe it's just a wording thing, but "dom0 / hypervisor" from the above note don't directly correspond to "are always from domain POV" described above - at least as how domain gets generally described... The command is run on the host right? So let's just be clear. Otherwise, looks fine to me with some adjustments as described above, Reviewed-by: John Ferlan <jferlan@redhat.com> John [...]
On 10/05/2017 12:38 AM, John Ferlan wrote:
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1497410
The comment in virNetDevTapInterfaceStats() implementation for Linux states that packets transmitted by domain are received by the host and vice versa. Well, this is true but not for all types of interfaces. For instance, for macvtaps when TAP device is hooked right onto a physical device any packet that domain sends looks also like a packet sent to the host. Therefore, we should have caller chose if the stats returned should be straight copy
s/have/allow/ s/chose/to choose
BTW: My first read of this thought "caller" is user configurable :-)
or swapped.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 37 ++++++++++++++++- src/libxl/libxl_driver.c | 3 +- src/lxc/lxc_driver.c | 3 +- src/openvz/openvz_driver.c | 3 +- src/qemu/qemu_driver.c | 18 +++++---- src/util/virnetdevtap.c | 99 +++++++++++++++++++++++++++++++--------------- src/util/virnetdevtap.h | 3 +- src/xen/xen_hypervisor.c | 2 +- 9 files changed, 123 insertions(+), 47 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 2289399cd..6569b94fa 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -26729,7 +26729,7 @@ virDomainStateReasonFromString(virDomainState state, const char *reason) */
virDomainNetType -virDomainNetGetActualType(virDomainNetDefPtr iface) +virDomainNetGetActualType(const virDomainNetDef *iface)
Sigh, separable along w/ domain_conf.h change...
{ if (iface->type != VIR_DOMAIN_NET_TYPE_NETWORK) return iface->type; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 9ba84a94d..6eda21ef7 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3021,7 +3021,7 @@ int virDomainGraphicsListenAppendSocket(virDomainGraphicsDefPtr def, const char *socket) ATTRIBUTE_NONNULL(1);
-virDomainNetType virDomainNetGetActualType(virDomainNetDefPtr iface); +virDomainNetType virDomainNetGetActualType(const virDomainNetDef *iface);
e.g. this is separate and unrelated needs it's own already R-B patch ;-)
Yeah, but I was too lazy to come up with yet another commit message. And one can argue that without the rest of the patch these two changes are not that much needed ;-)
The rest is "this" patch specific...
const char *virDomainNetGetActualBridgeName(virDomainNetDefPtr iface); int virDomainNetGetActualBridgeMACTableManager(virDomainNetDefPtr iface); const char *virDomainNetGetActualDirectDev(virDomainNetDefPtr iface); @@ -3393,4 +3393,39 @@ virDomainGenerateMachineName(const char *drivername, int id, const char *name, bool privileged); +/** + * virDomainNetTypeSharesHostView: + * @net: interface + * + * Some types of interfaces "share" the host view. For instance, + * for macvtap interface, every domain RX is the host RX too. And + * every domain TX is host TX too. IOW, for some types of + * interfaces guest and host are on the same side of RX/TX + * barrier. This is important so that we set up QoS correctly and + * report proper stats. + */ +static inline bool +virDomainNetTypeSharesHostView(const virDomainNetDef *net) +{ + virDomainNetType actualType = virDomainNetGetActualType(net); + switch (actualType) { + case VIR_DOMAIN_NET_TYPE_DIRECT: + case VIR_DOMAIN_NET_TYPE_ETHERNET: + return true; + case VIR_DOMAIN_NET_TYPE_USER: + case VIR_DOMAIN_NET_TYPE_VHOSTUSER: + case VIR_DOMAIN_NET_TYPE_SERVER: + case VIR_DOMAIN_NET_TYPE_CLIENT: + case VIR_DOMAIN_NET_TYPE_MCAST: + case VIR_DOMAIN_NET_TYPE_NETWORK: + case VIR_DOMAIN_NET_TYPE_BRIDGE: + case VIR_DOMAIN_NET_TYPE_INTERNAL: + case VIR_DOMAIN_NET_TYPE_HOSTDEV: + case VIR_DOMAIN_NET_TYPE_UDP: + case VIR_DOMAIN_NET_TYPE_LAST: + break; + } + return false; +} + #endif /* __DOMAIN_CONF_H */
[...]
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 1ab16e57c..a46409d70 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -11040,7 +11040,8 @@ qemuDomainInterfaceStats(virDomainPtr dom, if (virNetDevOpenvswitchInterfaceStats(path, stats) < 0) goto cleanup; } else { - if (virNetDevTapInterfaceStats(path, stats) < 0) + if (virNetDevTapInterfaceStats(path, stats, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; }
@@ -19559,29 +19560,30 @@ qemuDomainGetStatsInterface(virQEMUDriverPtr driver ATTRIBUTE_UNUSED,
/* Check the path is one of the domain's network interfaces. */ for (i = 0; i < dom->def->nnets; i++) { + virDomainNetDefPtr net = dom->def->nets[i]; virDomainNetType actualType;
- if (!dom->def->nets[i]->ifname) + if (!net->ifname) continue;
memset(&tmp, 0, sizeof(tmp));
- actualType = virDomainNetGetActualType(dom->def->nets[i]); + actualType = virDomainNetGetActualType(net);
QEMU_ADD_NAME_PARAM(record, maxparams, - "net", "name", i, dom->def->nets[i]->ifname); + "net", "name", i, net->ifname);
if (actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER) { - if (virNetDevOpenvswitchInterfaceStats(dom->def->nets[i]->ifname, + if (virNetDevOpenvswitchInterfaceStats(net->ifname, &tmp) < 0) {
This line may as well move to previous line as well for just one line.
virResetLastError(); continue; } } else { - if (virNetDevTapInterfaceStats(dom->def->nets[i]->ifname, &tmp) < 0) { + if (virNetDevTapInterfaceStats(net->ifname, &tmp, + !virDomainNetTypeSharesHostView(net)) < 0) virResetLastError(); - continue; - } + continue;
Is this one right? We won't QEMU_ADD_NET_PARAM when actual_type != VIR_DOMAIN_NET_TYPE_VHOSTUSER *and* virNetDevTapInterfaceStats succeeds? Looks like a bad transposition of extracted patches.
Ah right. Good catch.
}
QEMU_ADD_NET_PARAM(record, maxparams, i, diff --git a/src/util/virnetdevtap.c b/src/util/virnetdevtap.c index 175dc2bfa..a3ed59da8 100644 --- a/src/util/virnetdevtap.c +++ b/src/util/virnetdevtap.c @@ -676,14 +676,27 @@ int virNetDevTapCreateInBridgePort(const char *brname, }
/*-------------------- interface stats --------------------*/ -/* Just reads the named interface, so not Xen or QEMU-specific. - * NB. Caller must check that libvirt user is trying to query - * the interface of a domain they own. We do no such checking. + +/** + * virNetDevTapInterfaceStats: + * @ifname: interface + * @stats: where to store statistics + * @swapped: whether to swap RX/TX fields + * + * Fetch RX/TX statistics for given named interface (@ifname) and + * store them at @stats. The returned statistics are always from + * domain POV. Because in some cases this means swapping RX/TX in + * the stats and in others this means no swapping (consider TAP + * vs macvtap) caller might choose if the returned stats should + * be @swapped or not. + * + * Returns 0 on success, -1 otherwise (with error reported). */ #ifdef __linux__ int virNetDevTapInterfaceStats(const char *ifname, - virDomainInterfaceStatsPtr stats) + virDomainInterfaceStatsPtr stats, + bool swapped) { int ifname_len; FILE *fp; @@ -718,30 +731,35 @@ virNetDevTapInterfaceStats(const char *ifname, *colon = '\0'; if (colon-ifname_len >= line && STREQ(colon-ifname_len, ifname)) { - /* IMPORTANT NOTE! - * /proc/net/dev vif<domid>.nn sees the network from the point - * of view of dom0 / hypervisor. So bytes TRANSMITTED by dom0 - * are bytes RECEIVED by the domain. That's why the TX/RX fields - * appear to be swapped here.
Maybe it's just a wording thing, but "dom0 / hypervisor" from the above note don't directly correspond to "are always from domain POV" described above - at least as how domain gets generally described... The command is run on the host right? So let's just be clear.
Yes. The command is run on the host. And yes, it doesn't always correspond to "always from domain POV". Hence the bool @swapped. Michal
Similarly to previous patch, for some types of interface domain and host are on the same side of RX/TX barrier. In that case, we need to set up the QoS differently. Well, swapped. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/lxc/lxc_driver.c | 3 ++- src/lxc/lxc_process.c | 3 ++- src/network/bridge_driver.c | 2 +- src/qemu/qemu_command.c | 3 ++- src/qemu/qemu_driver.c | 6 ++++-- src/qemu/qemu_hotplug.c | 6 ++++-- src/util/virnetdevbandwidth.c | 43 +++++++++++++++++++++++++++++------------- src/util/virnetdevbandwidth.h | 3 ++- tests/virnetdevbandwidthtest.c | 2 +- 9 files changed, 48 insertions(+), 23 deletions(-) diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index f0a22ce3e..6ad61bdb7 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -3981,7 +3981,8 @@ lxcDomainAttachDeviceNetLive(virConnectPtr conn, actualBandwidth = virDomainNetGetActualBandwidth(net); if (actualBandwidth) { if (virNetDevSupportBandwidth(actualType)) { - if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false) < 0) + if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; } else { VIR_WARN("setting bandwidth on interfaces of " diff --git a/src/lxc/lxc_process.c b/src/lxc/lxc_process.c index 05f1dec4c..efd8a6900 100644 --- a/src/lxc/lxc_process.c +++ b/src/lxc/lxc_process.c @@ -601,7 +601,8 @@ static int virLXCProcessSetupInterfaces(virConnectPtr conn, actualBandwidth = virDomainNetGetActualBandwidth(net); if (actualBandwidth) { if (virNetDevSupportBandwidth(type)) { - if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false) < 0) + if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; } else { VIR_WARN("setting bandwidth on interfaces of " diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index e8d093a31..530d00ff3 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -2445,7 +2445,7 @@ networkStartNetworkVirtual(virNetworkDriverStatePtr driver, VIR_FORCE_CLOSE(tapfd); } - if (virNetDevBandwidthSet(def->bridge, def->bandwidth, true) < 0) + if (virNetDevBandwidthSet(def->bridge, def->bandwidth, true, true) < 0) goto err5; VIR_FREE(macTapIfName); diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 4f141e0ac..4f89235e3 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -8567,7 +8567,8 @@ qemuBuildInterfaceCommandLine(virQEMUDriverPtr driver, actualBandwidth = virDomainNetGetActualBandwidth(net); if (actualBandwidth) { if (virNetDevSupportBandwidth(actualType)) { - if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false) < 0) + if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; } else { VIR_WARN("setting bandwidth on interfaces of " diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index a46409d70..898079dbc 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -11182,11 +11182,13 @@ qemuDomainSetInterfaceParameters(virDomainPtr dom, if (!networkBandwidthChangeAllowed(net, newBandwidth)) goto endjob; - if (virNetDevBandwidthSet(net->ifname, newBandwidth, false) < 0 || + if (virNetDevBandwidthSet(net->ifname, newBandwidth, false, + !virDomainNetTypeSharesHostView(net)) < 0 || networkBandwidthUpdate(net, newBandwidth) < 0) { ignore_value(virNetDevBandwidthSet(net->ifname, net->bandwidth, - false)); + false, + !virDomainNetTypeSharesHostView(net))); goto endjob; } diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index b77731df0..2b9427df1 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1209,7 +1209,8 @@ qemuDomainAttachNetDevice(virQEMUDriverPtr driver, actualBandwidth = virDomainNetGetActualBandwidth(net); if (actualBandwidth) { if (virNetDevSupportBandwidth(actualType)) { - if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false) < 0) + if (virNetDevBandwidthSet(net->ifname, actualBandwidth, false, + !virDomainNetTypeSharesHostView(net)) < 0) goto cleanup; } else { VIR_WARN("setting bandwidth on interfaces of " @@ -3362,7 +3363,8 @@ qemuDomainChangeNet(virQEMUDriverPtr driver, if (needBandwidthSet) { if (virNetDevBandwidthSet(newdev->ifname, virDomainNetGetActualBandwidth(newdev), - false) < 0) + false, + !virDomainNetTypeSharesHostView(newdev)) < 0) goto cleanup; needReplaceDevDef = true; } diff --git a/src/util/virnetdevbandwidth.c b/src/util/virnetdevbandwidth.c index 4e4ac617b..9aebe56f4 100644 --- a/src/util/virnetdevbandwidth.c +++ b/src/util/virnetdevbandwidth.c @@ -174,6 +174,7 @@ virNetDevBandwidthManipulateFilter(const char *ifname, * @ifname: on which interface * @bandwidth: rates to set (may be NULL) * @hierarchical_class: whether to create hierarchical class + * @swapped: true if IN/OUT should be set contrariwise * * This function enables QoS on specified interface * and set given traffic limits for both, incoming @@ -182,14 +183,23 @@ virNetDevBandwidthManipulateFilter(const char *ifname, * hierarchical class. It is used to guarantee minimal * throughput ('floor' attribute in NIC). * + * If @swapped is set, the IN part of @bandwidth is set on + * @ifname's TX, and vice versa. If it is not set, IN is set on + * RX and OUT on TX. This is because for some types of interfaces + * domain and the host live on the same side of the interface (so + * domain's RX/TX is host's RX/TX), and for some it's swapped + * (domain's RX/TX is hosts's TX/RX). + * * Return 0 on success, -1 otherwise. */ int virNetDevBandwidthSet(const char *ifname, virNetDevBandwidthPtr bandwidth, - bool hierarchical_class) + bool hierarchical_class, + bool swapped) { int ret = -1; + virNetDevBandwidthRatePtr rx = NULL, tx = NULL; /* From domain POV */ virCommandPtr cmd = NULL; char *average = NULL; char *peak = NULL; @@ -215,16 +225,24 @@ virNetDevBandwidthSet(const char *ifname, return -1; } + if (swapped) { + rx = bandwidth->out; + tx = bandwidth->in; + } else { + rx = bandwidth->in; + tx = bandwidth->out; + } + virNetDevBandwidthClear(ifname); - if (bandwidth->in && bandwidth->in->average) { - if (virAsprintf(&average, "%llukbps", bandwidth->in->average) < 0) + if (tx && tx->average) { + if (virAsprintf(&average, "%llukbps", tx->average) < 0) goto cleanup; - if (bandwidth->in->peak && - (virAsprintf(&peak, "%llukbps", bandwidth->in->peak) < 0)) + if (tx->peak && + (virAsprintf(&peak, "%llukbps", tx->peak) < 0)) goto cleanup; - if (bandwidth->in->burst && - (virAsprintf(&burst, "%llukb", bandwidth->in->burst) < 0)) + if (tx->burst && + (virAsprintf(&burst, "%llukb", tx->burst) < 0)) goto cleanup; cmd = virCommandNew(TC); @@ -303,7 +321,7 @@ virNetDevBandwidthSet(const char *ifname, virCommandAddArgList(cmd, "class", "add", "dev", ifname, "parent", "1:", "classid", "1:1", "htb", "rate", average, "ceil", peak ? peak : average, NULL); - virNetDevBandwidthCmdAddOptimalQuantum(cmd, bandwidth->in); + virNetDevBandwidthCmdAddOptimalQuantum(cmd, tx); if (virCommandRun(cmd, NULL) < 0) goto cleanup; } @@ -319,7 +337,7 @@ virNetDevBandwidthSet(const char *ifname, if (burst) virCommandAddArgList(cmd, "burst", burst, NULL); - virNetDevBandwidthCmdAddOptimalQuantum(cmd, bandwidth->in); + virNetDevBandwidthCmdAddOptimalQuantum(cmd, tx); if (virCommandRun(cmd, NULL) < 0) goto cleanup; @@ -347,11 +365,10 @@ virNetDevBandwidthSet(const char *ifname, VIR_FREE(burst); } - if (bandwidth->out) { - if (virAsprintf(&average, "%llukbps", bandwidth->out->average) < 0) + if (rx) { + if (virAsprintf(&average, "%llukbps", rx->average) < 0) goto cleanup; - if (virAsprintf(&burst, "%llukb", bandwidth->out->burst ? - bandwidth->out->burst : bandwidth->out->average) < 0) + if (virAsprintf(&burst, "%llukb", rx->burst ? rx->burst : rx->average) < 0) goto cleanup; virCommandFree(cmd); diff --git a/src/util/virnetdevbandwidth.h b/src/util/virnetdevbandwidth.h index 64f35372e..63ac60897 100644 --- a/src/util/virnetdevbandwidth.h +++ b/src/util/virnetdevbandwidth.h @@ -45,7 +45,8 @@ void virNetDevBandwidthFree(virNetDevBandwidthPtr def); int virNetDevBandwidthSet(const char *ifname, virNetDevBandwidthPtr bandwidth, - bool hierarchical_class) + bool hierarchical_class, + bool swapped) ATTRIBUTE_RETURN_CHECK; int virNetDevBandwidthClear(const char *ifname); int virNetDevBandwidthCopy(virNetDevBandwidthPtr *dest, diff --git a/tests/virnetdevbandwidthtest.c b/tests/virnetdevbandwidthtest.c index a8f4ab196..2b031d55e 100644 --- a/tests/virnetdevbandwidthtest.c +++ b/tests/virnetdevbandwidthtest.c @@ -81,7 +81,7 @@ testVirNetDevBandwidthSet(const void *data) virCommandSetDryRun(&buf, NULL, NULL); - if (virNetDevBandwidthSet(iface, band, info->hierarchical_class) < 0) + if (virNetDevBandwidthSet(iface, band, info->hierarchical_class, true) < 0) goto cleanup; if (!(actual_cmd = virBufferContentAndReset(&buf))) { -- 2.13.5
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
Similarly to previous patch, for some types of interface domain and host are on the same side of RX/TX barrier. In that case, we need to set up the QoS differently. Well, swapped.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/lxc/lxc_driver.c | 3 ++- src/lxc/lxc_process.c | 3 ++- src/network/bridge_driver.c | 2 +- src/qemu/qemu_command.c | 3 ++- src/qemu/qemu_driver.c | 6 ++++-- src/qemu/qemu_hotplug.c | 6 ++++-- src/util/virnetdevbandwidth.c | 43 +++++++++++++++++++++++++++++------------- src/util/virnetdevbandwidth.h | 3 ++- tests/virnetdevbandwidthtest.c | 2 +- 9 files changed, 48 insertions(+), 23 deletions(-)
Reviewed-by: John Ferlan <jferlan@redhat.com> John (this reminds me, I need an eye exam ;-))
Interestingly enough, we don't document the point of view of the interface statistics. Therefore it's unknown to users if for instance rx_packets is the number of packets received by domain or received by host (from domain). Document this explicitly. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/libvirt-domain.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index d70027163..d2d022a66 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -5523,6 +5523,8 @@ virDomainBlockStatsFlags(virDomainPtr dom, * as -1, which indicates that the hypervisor does not support * that particular statistic. * + * The returned stats are from domain's point of view. + * * Returns: 0 in case of success or -1 in case of failure. */ int @@ -11301,7 +11303,7 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * as unsigned long long. * * VIR_DOMAIN_STATS_INTERFACE: - * Return network interface statistics. + * Return network interface statistics (from domain point of view). * The typed parameter keys are in this format: * * "net.count" - number of network interfaces on this domain -- 2.13.5
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
Interestingly enough, we don't document the point of view of the interface statistics. Therefore it's unknown to users if for instance rx_packets is the number of packets received by domain or received by host (from domain). Document this explicitly.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/libvirt-domain.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Maybe we should be even more explicit vis-a-vis when certain types of interface stats are swapped - direct & ethernet keep the same/current view, but every other type is changing. I know/understand this is a "tricky" and "slippery" slope in order to keep up with in the future, but I think some wording could be generated that would suffice Also a followup patch should modify docs/news.xml! Reviewed-by: John Ferlan <jferlan@redhat.com> John
On 10/05/2017 12:49 AM, John Ferlan wrote:
On 10/02/2017 11:05 AM, Michal Privoznik wrote:
Interestingly enough, we don't document the point of view of the interface statistics. Therefore it's unknown to users if for instance rx_packets is the number of packets received by domain or received by host (from domain). Document this explicitly.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/libvirt-domain.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Maybe we should be even more explicit vis-a-vis when certain types of interface stats are swapped - direct & ethernet keep the same/current view, but every other type is changing. I know/understand this is a "tricky" and "slippery" slope in order to keep up with in the future, but I think some wording could be generated that would suffice
I'm not fully convinced about this. I mean, it's just our implementation detail that we read stats from the host and therefore have to swap fields sometimes. The whole point of this series is that users reading stats do not need to fix/swap the fields themselves depending on the interface type.
Also a followup patch should modify docs/news.xml!
Ah right. I'll send it right away. Michal
On 10/02/2017 05:05 PM, Michal Privoznik wrote:
This is initiated by:
https://bugzilla.redhat.com/show_bug.cgi?id=1497410
Long story short, in some cases host and domain are on the same side of RX/TX stats/QoS.
Michal Privoznik (7): lxc: Drop useless ifdef __linux__ qemuDomainInterfaceStats: Check for the actual type of interface conf: Introduce virDomainNetFindByName src: Use virDomainNetFindByName virNetDevTapInterfaceStats: Allow caller to not swap the statistics QoS: Set classes and filters in proper direction libvirt-domain: Document interface stats POV
src/conf/domain_conf.c | 31 ++++++++++--- src/conf/domain_conf.h | 38 +++++++++++++++- src/libvirt-domain.c | 4 +- src/libvirt_private.syms | 1 + src/libxl/libxl_driver.c | 23 +++++----- src/lxc/lxc_driver.c | 38 ++++++---------- src/lxc/lxc_process.c | 3 +- src/network/bridge_driver.c | 2 +- src/openvz/openvz_driver.c | 23 +++++----- src/qemu/qemu_command.c | 3 +- src/qemu/qemu_driver.c | 41 +++++++++-------- src/qemu/qemu_hotplug.c | 6 ++- src/test/test_driver.c | 28 +++--------- src/util/virnetdevbandwidth.c | 43 ++++++++++++------ src/util/virnetdevbandwidth.h | 3 +- src/util/virnetdevtap.c | 99 ++++++++++++++++++++++++++++-------------- src/util/virnetdevtap.h | 3 +- src/xen/xen_hypervisor.c | 2 +- tests/virnetdevbandwidthtest.c | 2 +- 19 files changed, 236 insertions(+), 157 deletions(-)
Thanks John for all the fish^Wreviews. I've pushed this and I'm sending the follow up patch for news.xml. Michal
participants (2)
-
John Ferlan -
Michal Privoznik