February 2018 - Devel - libvirt List Archives

[libvirt] ANNOUNCE: virt-manager 1.5.0 released
by Cole Robinson 07 Feb '18

07 Feb '18

I'm happy to announce the release of virt-manager 1.5.0! virt-manager is a desktop application for managing KVM, Xen, and LXC virtualization via libvirt. The release can be downloaded from: http://virt-manager.org/download/ This release includes: - python3 prep work (Radostin Stoyanov, Cole Robinson, Cédric Bosdonnat) - Switch --location ISO to use isoinfo (Andrew Wong) - virt-install: add --cpu numa distance handling (Menno Lageman) - virt-install: fix --disk for rbd volumes with auth (Rauno Väli) - virt-install: add --cputune vcpupin handling (Wim ten Have) - details ui: Showing attached scsi devices per controller (Lin Ma) - network ui: Show details about SR-IOV VF pool (Lin Ma) - Greatly expand UI test suite coverage Note, the next major release will be within the next 2-3 months and will be a port to python3. There may be additional bugfix releases in the 1.5 series if needed. Thanks to everyone who has contributed to this release through testing, bug reporting, submitting patches, and otherwise sending in feedback! Thanks, Cole

1 0

[libvirt] [PATCH perl v2] Remove use of Data::Dumper from example programs
by Daniel P. Berrangé 06 Feb '18

06 Feb '18

Using Data::Dumper in examples does not help devs understand the data structures that the Perl APIs are returning. Change to explicit field accesses to illustrate it better Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- examples/dhcp-leases.pl | 7 +++++-- examples/dom-fsinfo.pl | 8 +++----- examples/dom-ifinfo.pl | 14 +++++++++----- examples/dom-stats.pl | 9 +++------ examples/node-info.pl | 26 +++++++++++++++++++------- 5 files changed, 39 insertions(+), 25 deletions(-) diff --git a/examples/dhcp-leases.pl b/examples/dhcp-leases.pl index 84a81ef..a2202d9 100644 --- a/examples/dhcp-leases.pl +++ b/examples/dhcp-leases.pl @@ -1,7 +1,6 @@ #!/usr/bin/perl use Sys::Virt; -use Data::Dumper; my $c = Sys::Virt->new(uri => "qemu:///system", readonly => 1); @@ -9,5 +8,9 @@ my $c = Sys::Virt->new(uri => "qemu:///system", $n = $c->get_network_by_name("default"); foreach my $lease ($n->get_dhcp_leases()) { - print Dumper($lease); + print "Interface ", $lease->{iface}, "\n"; + print " MAC: ", $lease->{mac}, "\n"; + print " IP: ", $lease->{ipaddr}, "\n"; + print " Host: ", $lease->{hostname}, "\n" if $lease->{hostname}; + print "\n"; } diff --git a/examples/dom-fsinfo.pl b/examples/dom-fsinfo.pl index 7763f78..46c64e9 100644 --- a/examples/dom-fsinfo.pl +++ b/examples/dom-fsinfo.pl @@ -1,6 +1,5 @@ #!/usr/bin/perl - use strict; use warnings; @@ -15,7 +14,6 @@ my $dom = $c->get_domain_by_name(shift @ARGV); my @fs = $dom->get_fs_info(); -use Data::Dumper; - -print Dumper($fs[0]); -print Dumper($fs[1]); +foreach my $fs (@fs) { + printf "%s (%s) at %s\n", $fs->{name}, $fs->{fstype}, $fs->{mountpoint}; +} diff --git a/examples/dom-ifinfo.pl b/examples/dom-ifinfo.pl index e10579b..66eb157 100644 --- a/examples/dom-ifinfo.pl +++ b/examples/dom-ifinfo.pl @@ -1,6 +1,5 @@ #!/usr/bin/perl - use strict; use warnings; @@ -13,9 +12,14 @@ my $c = Sys::Virt->new(uri => $uri); my $dom = $c->get_domain_by_name(shift @ARGV); -my @fs = $dom->get_interface_addresses( +my @nics = $dom->get_interface_addresses( Sys::Virt::Domain::INTERFACE_ADDRESSES_SRC_LEASE); -use Data::Dumper; - -print Dumper(@fs); +foreach my $nic (@nics) { + print "Interface ", $nic->{name}, "\n"; + print " MAC: ", $nic->{hwaddr}, "\n"; + foreach my $addr (@{$nic->{addrs}}) { + print " IP: ", $addr->{addr}, "\n"; + } + print "\n"; +} diff --git a/examples/dom-stats.pl b/examples/dom-stats.pl index 13d8fb7..1da0089 100644 --- a/examples/dom-stats.pl +++ b/examples/dom-stats.pl @@ -20,10 +20,7 @@ my @stats = $c->get_all_domain_stats(Sys::Virt::Domain::STATS_STATE, \@doms, Sys::Virt::Domain::GET_ALL_STATS_ENFORCE_STATS); -use Data::Dumper; - -print Dumper(\@stats); - -for (my $i = 0 ; $i <= $#stats ; $i++) { - print $stats[$i]->{'dom'}->get_name(), ": ", $stats[$i]->{'data'}->{'state.state'}, "\n"; +foreach my $stats (@stats) { + print "Guest ", $stats->{'dom'}->get_name(), "\n"; + print " State: ", $stats->{'data'}->{'state.state'}, "\n"; } diff --git a/examples/node-info.pl b/examples/node-info.pl index 89bc9ba..9655ab4 100644 --- a/examples/node-info.pl +++ b/examples/node-info.pl @@ -13,13 +13,25 @@ my $info = $hv->get_node_info(); my @models = $hv->get_cpu_model_names($info->{model}); -print join ("\n", sort{ lc $a cmp lc $b } @models), "\n"; - -my @info = $hv->get_node_free_pages([2048], 0, 0); - -use Data::Dumper; -print Dumper(\@info); - +print "Available CPU model names:\n"; +print join ("\n", map { " " . $_ } sort{ lc $a cmp lc $b } @models), "\n"; + +my @pagesizes = ( + 4, 2048, 1048576 + ); + +my @info = $hv->get_node_free_pages(\@pagesizes, 0, 0); + +print "Free pages per NUMA node:\n"; +foreach my $info (@info) { + print " Node: ", $info->{cell}, "\n"; + print " Free: "; + for (my $i = 0; $i <= $#pagesizes; $i++) { + my $pagesize = $pagesizes[$i]; + printf "%d @ %d KB, ", $info->{pages}->{$pagesize}, $pagesize; + } + print "\n"; +} my $xml = $hv->get_domain_capabilities(undef, "x86_64", undef, "kvm"); print $xml; -- 2.14.3

2 1

[libvirt] [PATCH perl] Remove use of Data::Dumper from example programs
by Daniel P. Berrangé 06 Feb '18

06 Feb '18

Using Data::Dumper in examples does not help devs understand the data structures that the Perl APIs are returning. Change to explicit field accesses to illustrate it better Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- Virt.xs | 1 - examples/dhcp-leases.pl | 2 -- examples/dom-fsinfo.pl | 8 +++----- examples/dom-ifinfo.pl | 14 +++++++++----- examples/dom-stats.pl | 9 +++------ examples/node-info.pl | 26 +++++++++++++++++++------- 6 files changed, 34 insertions(+), 26 deletions(-) diff --git a/Virt.xs b/Virt.xs index 7d2f1a7..1254df2 100644 --- a/Virt.xs +++ b/Virt.xs @@ -3358,7 +3358,6 @@ void get_all_domain_stats(con, stats, doms_sv=&PL_sv_undef, flags=0) if (SvOK(doms_sv)) { doms_av = (AV*)SvRV(doms_sv); ndoms = av_len(doms_av) + 1; - fprintf(stderr, "Len %d\n", ndoms); } else { ndoms = 0; } diff --git a/examples/dhcp-leases.pl b/examples/dhcp-leases.pl index af9bd41..a2202d9 100644 --- a/examples/dhcp-leases.pl +++ b/examples/dhcp-leases.pl @@ -1,8 +1,6 @@ #!/usr/bin/perl -use Acme::ChuckNorris; use Sys::Virt; -use Data::Dumper; my $c = Sys::Virt->new(uri => "qemu:///system", readonly => 1); diff --git a/examples/dom-fsinfo.pl b/examples/dom-fsinfo.pl index 7763f78..46c64e9 100644 --- a/examples/dom-fsinfo.pl +++ b/examples/dom-fsinfo.pl @@ -1,6 +1,5 @@ #!/usr/bin/perl - use strict; use warnings; @@ -15,7 +14,6 @@ my $dom = $c->get_domain_by_name(shift @ARGV); my @fs = $dom->get_fs_info(); -use Data::Dumper; - -print Dumper($fs[0]); -print Dumper($fs[1]); +foreach my $fs (@fs) { + printf "%s (%s) at %s\n", $fs->{name}, $fs->{fstype}, $fs->{mountpoint}; +} diff --git a/examples/dom-ifinfo.pl b/examples/dom-ifinfo.pl index e10579b..66eb157 100644 --- a/examples/dom-ifinfo.pl +++ b/examples/dom-ifinfo.pl @@ -1,6 +1,5 @@ #!/usr/bin/perl - use strict; use warnings; @@ -13,9 +12,14 @@ my $c = Sys::Virt->new(uri => $uri); my $dom = $c->get_domain_by_name(shift @ARGV); -my @fs = $dom->get_interface_addresses( +my @nics = $dom->get_interface_addresses( Sys::Virt::Domain::INTERFACE_ADDRESSES_SRC_LEASE); -use Data::Dumper; - -print Dumper(@fs); +foreach my $nic (@nics) { + print "Interface ", $nic->{name}, "\n"; + print " MAC: ", $nic->{hwaddr}, "\n"; + foreach my $addr (@{$nic->{addrs}}) { + print " IP: ", $addr->{addr}, "\n"; + } + print "\n"; +} diff --git a/examples/dom-stats.pl b/examples/dom-stats.pl index 13d8fb7..1da0089 100644 --- a/examples/dom-stats.pl +++ b/examples/dom-stats.pl @@ -20,10 +20,7 @@ my @stats = $c->get_all_domain_stats(Sys::Virt::Domain::STATS_STATE, \@doms, Sys::Virt::Domain::GET_ALL_STATS_ENFORCE_STATS); -use Data::Dumper; - -print Dumper(\@stats); - -for (my $i = 0 ; $i <= $#stats ; $i++) { - print $stats[$i]->{'dom'}->get_name(), ": ", $stats[$i]->{'data'}->{'state.state'}, "\n"; +foreach my $stats (@stats) { + print "Guest ", $stats->{'dom'}->get_name(), "\n"; + print " State: ", $stats->{'data'}->{'state.state'}, "\n"; } diff --git a/examples/node-info.pl b/examples/node-info.pl index 89bc9ba..9655ab4 100644 --- a/examples/node-info.pl +++ b/examples/node-info.pl @@ -13,13 +13,25 @@ my $info = $hv->get_node_info(); my @models = $hv->get_cpu_model_names($info->{model}); -print join ("\n", sort{ lc $a cmp lc $b } @models), "\n"; - -my @info = $hv->get_node_free_pages([2048], 0, 0); - -use Data::Dumper; -print Dumper(\@info); - +print "Available CPU model names:\n"; +print join ("\n", map { " " . $_ } sort{ lc $a cmp lc $b } @models), "\n"; + +my @pagesizes = ( + 4, 2048, 1048576 + ); + +my @info = $hv->get_node_free_pages(\@pagesizes, 0, 0); + +print "Free pages per NUMA node:\n"; +foreach my $info (@info) { + print " Node: ", $info->{cell}, "\n"; + print " Free: "; + for (my $i = 0; $i <= $#pagesizes; $i++) { + my $pagesize = $pagesizes[$i]; + printf "%d @ %d KB, ", $info->{pages}->{$pagesize}, $pagesize; + } + print "\n"; +} my $xml = $hv->get_domain_capabilities(undef, "x86_64", undef, "kvm"); print $xml; -- 2.14.3

3 4

[libvirt] [PATCH 0/2] Memory leak fix and some refactoring
by Marc Hartmayer 06 Feb '18

06 Feb '18

The second patch fixes the memory leak. Marc Hartmayer (2): qemu: Use the return value of virObjectRef directly qemu: Add and use qemuProcessEventFree for freeing qemuProcessEvents src/qemu/qemu_domain.c | 23 +++++++++++++++++++++++ src/qemu/qemu_domain.h | 2 ++ src/qemu/qemu_driver.c | 12 ++---------- src/qemu/qemu_process.c | 41 ++++++++++++++--------------------------- 4 files changed, 41 insertions(+), 37 deletions(-) -- 2.13.4

2 11

[libvirt] [PATCH] conf: Check for user aliases duplicates only
by Michal Privoznik 06 Feb '18

06 Feb '18

When validating a device XML config we check if user provided alias is unique. We do this by maintaining a hash table of device aliases as we iterated over all devices defined for the domain. However, it may happen that what appears as two devices in domain XML is in fact just one interface in hypervisor. For instance in qemu driver this is true for uhci/ehci controllers. In that case an error is reported even though it is not actually an error. At any rate, we can assume libvirt generated aliases to be unique and thus really check user provided ones only. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/conf/domain_conf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 34aae82f1..44724bd01 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -5569,7 +5569,7 @@ virDomainDeviceDefValidateAliasesIterator(virDomainDefPtr def, struct virDomainDefValidateAliasesData *data = opaque; const char *alias = info->alias; - if (!alias) + if (!alias || !STRPREFIX(alias, "ua-")) return 0; /* Some crazy backcompat for consoles. */ -- 2.13.6

2 2

[libvirt] [PATCH v4] virt-aa-helper: Set the supported features
by Shivaprasad G Bhat 06 Feb '18

06 Feb '18

The virt-aa-helper fails to parse the xmls with the memory/cpu hotplug features or user assigned aliases. Set the features in xmlopt->config for the parsing to succeed. Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com> --- src/conf/domain_conf.h | 2 ++ src/security/virt-aa-helper.c | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 21e004515..25d0b8187 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2532,6 +2532,8 @@ typedef bool (*virDomainObjListACLFilter)(virConnectPtr conn, virDomainDefPtr def); +/* NB: Any new flag to this list be considered to be set in + * virt-aa-helper code if the flag prevents parsing. */ typedef enum { VIR_DOMAIN_DEF_FEATURE_WIDE_SCSI = (1 << 0), VIR_DOMAIN_DEF_FEATURE_MEMORY_HOTPLUG = (1 << 1), diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index f7ccae0b0..29a459da2 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -654,6 +654,11 @@ caps_mockup(vahControl * ctl, const char *xmlStr) return rc; } +virDomainDefParserConfig virAAHelperDomainDefParserConfig = { + .features = VIR_DOMAIN_DEF_FEATURE_MEMORY_HOTPLUG | + VIR_DOMAIN_DEF_FEATURE_OFFLINE_VCPUPIN | + VIR_DOMAIN_DEF_FEATURE_INDIVIDUAL_VCPUS, +}; static int get_definition(vahControl * ctl, const char *xmlStr) @@ -673,7 +678,8 @@ get_definition(vahControl * ctl, const char *xmlStr) goto exit; } - if (!(ctl->xmlopt = virDomainXMLOptionNew(NULL, NULL, NULL, NULL, NULL))) { + if (!(ctl->xmlopt = virDomainXMLOptionNew(&virAAHelperDomainDefParserConfig, + NULL, NULL, NULL, NULL))) { vah_error(ctl, 0, _("Failed to create XML config object")); goto exit; }

3 4

[libvirt] [PATCH] qemu: Initialize @priv in qemuDomainCoreDumpWithFormat
by John Ferlan 06 Feb '18

06 Feb '18

Fix for a CI build failure Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Pushed under build breaker rule. src/qemu/qemu_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index dda8b4cbf..bff49e7be 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3968,7 +3968,7 @@ qemuDomainCoreDumpWithFormat(virDomainPtr dom, { virQEMUDriverPtr driver = dom->conn->privateData; virDomainObjPtr vm; - qemuDomainObjPrivatePtr priv; + qemuDomainObjPrivatePtr priv = NULL; bool resume = false, paused = false; int ret = -1; virObjectEventPtr event = NULL; -- 2.13.6

1 0

Re: [libvirt] [RFC] kvm: x86: export vCPU halted state to sysfs
by Eduardo Habkost 06 Feb '18

06 Feb '18

On Thu, Feb 01, 2018 at 09:15:15PM +0100, Radim Krčmář wrote: > 2018-02-01 12:54-0500, Luiz Capitulino: > > > > Libvirt needs to know when a vCPU is halted. To get this information, > > I don't see why upper level management should care about that, a single > bit about halted state that can be incorrect at the time it is processed > seems of very limited use. I don't see why, either. I'm CCing libvir-list and the people involved in the code that added halt state to libvirt domain statistics. > > (A much more sensible data point would be the fraction of time when VCPU > was running or runnable, which is roughly what you get by sampling the > halted state.) > > A halted vCPU it might even be halted in guest mode, so KVM doesn't know > about that state (unless you force a VM exit), which would complicate > the collection a bit more ... but really, what is the data being used > for? > > User might care about the state, for obscure reasons, but that isn't a > performance problem. > > > libvirt has started using the query-cpus command from QEMU. However, > > if in kernel irqchip is in use, query-cpus will force all vCPUs > > to user-space since they have to issue the KVM_GET_MP_STATE ioctl. > > Libvirt knows if KVM exits to userspace on halts, so it can just query > QEMU in that case and in the other case, there is a very dirty > "solution" that works on all architectures right now: > > grep kvm_vcpu_block /proc/$vcpu_task/stack > > If you get something, the vcpu is halted in KVM. Nice. > > > This has catastrophic implications to low-latency workloads like > > KVM-RT and zero packet loss with DPDK. To make matters worse, there's > > an OpenStack service called ceilometer that causes libvirt to > > issue query-cpus every few minutes. > > I'd expect that people running these workloads can setup the system. :( > > I bet that ceilometer just mindlessly collects everything, so we should > be able to configure libvirt to collect only some stats. Either libvirt > or upper layer would decide what is too expensive for its usefulness. Yes. Including expensive-to-collect halt state in VIR_DOMAIN_STATS_VCPU is a serious performance regression in libvirt. > > > The solution proposed in this patch is to export the vCPU > > halted state in the already existing vcpu directory in sysfs. > > This way, libvirt can read the vCPU halted state from sysfs and avoid > > using the query-cpus command. This solution seems to be sufficient > > for libvirt needs, but it has the following cons: > > > > * vcpu information in sysfs lives in a debug directory, so > > libvirt would be basing its API on debug info > > (It pains me to say there probably already are tools that depend on > kvm/debug.) > > It's slightly better than the stack hack, but needs more code in kernel > and the interface is in a gray compatibility zone, so I'd like to know > why does userspace do that in the first place. > > > * Currently, only x86 supports the vcpu dir in sysfs, so > > we'd have to expand this to other archs (should be doable) > > > > If we agree that this solution is feasible, I'll work on extending > > the vcpu debug information to other archs for my next posting. > > > > Signed-off-by: Luiz Capitulino <lcapitulino(a)redhat.com> > > --- > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > > @@ -6273,6 +6273,7 @@ void kvm_arch_exit(void) > > > > int kvm_vcpu_halt(struct kvm_vcpu *vcpu) > > { > > + kvm_vcpu_set_halted(vcpu); > > There is no point to care about !lapic_in_kernel(). I'd move the logic > into vcpu_block() to be shared among all architectures. > > > ++vcpu->stat.halt_exits; > > if (lapic_in_kernel(vcpu)) { > > vcpu->arch.mp_state = KVM_MP_STATE_HALTED; -- Eduardo

6 37

[libvirt] [PATCH] qemu: Limit refresh of CPU halted state to s390
by Viktor Mihajlovski 06 Feb '18

06 Feb '18

Refreshing the halted state can cause VM performance issues. Since s390 is currently the only architecture with a known interest in the halted state, we're avoiding to call QEMU on other platforms. Signed-off-by: Viktor Mihajlovski <mihajlov(a)linux.vnet.ibm.com> --- src/qemu/qemu_domain.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index df433c2..d2c833f 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -8634,6 +8634,10 @@ qemuDomainRefreshVcpuHalted(virQEMUDriverPtr driver, if (vm->def->virtType == VIR_DOMAIN_VIRT_QEMU) return 0; + /* Only supported on s390(x) */ + if (!ARCH_IS_S390(vm->def->os.arch)) + return 0; + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0) return -1; -- 1.9.1

3 4

[libvirt] [python PATCH] event-test.py: Remove extra ( in --help output
by Jiri Denemark 06 Feb '18

06 Feb '18

Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- Pushed as trivial. examples/event-test.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/event-test.py b/examples/event-test.py index 2587226..3de333c 100755 --- a/examples/event-test.py +++ b/examples/event-test.py @@ -666,8 +666,8 @@ def myConnectionCloseCallback(conn, reason, opaque): def usage(): print("usage: "+os.path.basename(sys.argv[0])+" [-hdl] [uri]") print(" uri will default to qemu:///system") - print(" --help, -h Print(this help message") - print(" --debug, -d Print(debug output") + print(" --help, -h Print this help message") + print(" --debug, -d Print debug output") print(" --loop=TYPE, -l Choose event-loop-implementation (native, poll, asyncio)") print(" --timeout=SECS Quit after SECS seconds running") -- 2.16.1

1 0