- Devel - libvirt List Archives

[libvirt] [PATCH v2 00/11] tests: qemu: Add cpu hotplug testing
by Peter Krempa 12 Jan '17

12 Jan '17

This is a repost of https://www.redhat.com/archives/libvir-list/2016-December/msg00420.html after a recent rebase conflict. Add some useful monitor infrastructure to simplify testing and add testing for the cpu hotplug code. The code has some complex logic which was not covered by current test suite. Peter Krempa (11): qemu: monitor: More strict checking of 'query-cpus' if hotplug is supported util: json: Add helper to reformat JSON strings tests: qemu: Document qemuMonitorTestNewFromFile qemu: Prepare for reuse of qemuDomainSetVcpusLive qemu: Move cpu hotplug code into qemu_hotplug.c tests: qemumonitor: Propagate better error messages tests: qemu: monitor: Add helpers to test full command syntax tests: qemu: Add helper to load full monitor conversation from file tests: hotplug: Add test infrastructure for testing qemu CPU hotplug code tests: hotplug: Add test data for legacy cpu hotplug tests: hotplug: Test CPU hotplug with ppc64 data src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 328 ------------ src/qemu/qemu_hotplug.c | 339 ++++++++++++ src/qemu/qemu_hotplug.h | 7 + src/qemu/qemu_monitor.c | 6 +- src/qemu/qemu_monitor_json.c | 6 +- src/qemu/qemu_monitor_json.h | 3 +- src/util/virjson.c | 29 + src/util/virjson.h | 2 + tests/qemuagenttest.c | 3 +- tests/qemuhotplugtest.c | 194 +++++++ tests/qemuhotplugtestcpus/ppc64-bulk-domain.xml | 20 + tests/qemuhotplugtestcpus/ppc64-bulk-monitor.json | 593 +++++++++++++++++++++ .../qemuhotplugtestcpus/ppc64-bulk-result-conf.xml | 64 +++ .../qemuhotplugtestcpus/ppc64-bulk-result-live.xml | 72 +++ .../qemuhotplugtestcpus/x86-modern-bulk-domain.xml | 21 + .../x86-modern-bulk-monitor.json | 471 ++++++++++++++++ .../x86-modern-bulk-result-conf.xml | 40 ++ .../x86-modern-bulk-result-live.xml | 48 ++ tests/qemuhotplugtestcpus/x86-old-bulk-domain.xml | 21 + .../qemuhotplugtestcpus/x86-old-bulk-monitor.json | 193 +++++++ .../x86-old-bulk-result-conf.xml | 30 ++ .../x86-old-bulk-result-live.xml | 38 ++ tests/qemumonitorjsontest.c | 2 +- tests/qemumonitortestutils.c | 322 ++++++++++- tests/qemumonitortestutils.h | 12 +- 26 files changed, 2516 insertions(+), 349 deletions(-) create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-domain.xml create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-monitor.json create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-result-conf.xml create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-result-live.xml create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-domain.xml create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-monitor.json create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-result-conf.xml create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-result-live.xml create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-domain.xml create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-monitor.json create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-result-conf.xml create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-result-live.xml -- 2.11.0

2 22

[libvirt] [PATCH] news: document libxl HVM network attach fix
by Joao Martins 12 Jan '17

12 Jan '17

Signed-off-by: Joao Martins <joao.m.martins(a)oracle.com> --- docs/news.xml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 92e39f9..c19bbbb 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -204,6 +204,16 @@ domain definition. </description> </change> + <change> + <summary> + libxl: reverse defaults on HVM net device attach + </summary> + <description> + Fixes network interface attach for HVM domains when no model is + specified. Emulated hotplug isn't yet supported and hence we should + default to the general working scenario. + </description> + </change> </section> </release> <release version="v2.5.0" date="2016-12-04"> -- 2.1.4

2 1

[libvirt] [PATCH 0/5] Use non-blacklisted family/model/stepping for Haswell CPU model
by Eduardo Habkost 12 Jan '17

12 Jan '17

A recent glibc commit[1] added a blacklist to ensure it won't use TSX on hosts that are known to have a broken TSX implementation. Our existing Haswell CPU model has a blacklisted family/model/stepping combination, so it has to be updated to make sure guests will really use TSX. This is done by patch 5/5. However, to do this safely we need to ensure the host CPU is not a blacklisted one, so we won't mislead guests by exposing known-to-be-good FMS values on a known-to-be-broken host. This is done by patch 3/5. [1] https://sourceware.org/git/?p=glibc.git;a=commit;h=2702856bf45c82cf8e69f206… --- Cc: dgilbert(a)redhat.com Cc: fweimer(a)redhat.com Cc: carlos(a)redhat.com Cc: triegel(a)redhat.com Cc: berrange(a)redhat.com Cc: jdenemar(a)redhat.com Cc: pbonzini(a)redhat.com Eduardo Habkost (5): i386: Add explicit array size to x86_cpu_vendor_words2str() i386: host_vendor_fms() helper function i386/kvm: Blacklist TSX on known broken hosts pc: Add 2.9 machine-types i386: Change stepping of Haswell to non-blacklisted value include/hw/i386/pc.h | 6 ++++++ target/i386/cpu.h | 1 + hw/i386/pc_piix.c | 15 ++++++++++++--- hw/i386/pc_q35.c | 13 +++++++++++-- target/i386/cpu.c | 32 ++++++++++++++++++++++---------- target/i386/kvm.c | 17 +++++++++++++++++ 6 files changed, 69 insertions(+), 15 deletions(-) -- 2.11.0.259.g40922b1

4 12

Re: [libvirt] "[V3] RFC for support cache tune in libvirt"
by Marcelo Tosatti 12 Jan '17

12 Jan '17

Hi, Comments/questions related to: https://www.redhat.com/archives/libvir-list/2017-January/msg00354.html 1) root s2600wt:~/linux# virsh cachetune kvm02 --l3.count 2 How does allocation of code/data look like? 2) 'nodecachestats' command: 3. Add new virsh command 'nodecachestats': This API is to expose vary cache resouce left on each hardware (cpu socket). It will be formated as: <resource_type>.<resource_id>: left size KiB Does this take into account that only contiguous regions of cbm masks can be used for allocations? Also, it should return the amount of free cache on each cacheid. 3) The interface should support different sizes for different cache-ids. See the KVM-RT use case at https://www.redhat.com/archives/libvir-list/2017-January/msg00415.html "WHAT THE USER NEEDS TO SPECIFY FOR VIRTUALIZATION (KVM-RT)". 4) Usefulness of exposing minimum unit size. Rather than specify unit sizes (which forces the user to convert every time the command is executed), why not specify in kbytes and round up? <resctrl name='L3' unit='KiB' cache_size='56320' cache_unit='2816'/> As noted in item 1 of https://www.redhat.com/archives/libvir-list/2017-January/msg00494.html, "1) Convertion of kbytes (user specification) --> number of CBM bits for host.", the format where the size is stored is kbytes, so its awkward to force users and OpenStack to perform the convertion themselves (and zero benefits... nothing changes if you know the unit size). Thanks!

3 7

[libvirt] [PATCH] news: document drive addresses fix
by Marc Hartmayer 12 Jan '17

12 Jan '17

Signed-off-by: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> --- docs/news.xml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 8175fd0..92e39f9 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -195,6 +195,15 @@ separate path which is also correctly labeled. </description> </change> + <change> + <summary> + conf: Reject domains with duplicate drive addresses + </summary> + <description> + Reject duplicate drive addresses for disks and hostdevs at + domain definition. + </description> + </change> </section> </release> <release version="v2.5.0" date="2016-12-04"> -- 2.5.5

2 2

[libvirt] [PATCH] news: document events changes and lxc fix
by Daniel P. Berrange 12 Jan '17

12 Jan '17

Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- docs/news.xml | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 300e553..23ff40f 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -14,6 +14,25 @@ <section title="New features"> <change> <summary> + Domain events for metadata content changes + </summary> + <description> + The domain events framework has a new event ID that can + be used to get notifications when domain metadata content + changes. + </description> + </change> + <change> + <summary> + Event notifications for the secret object + </summary> + <description> + The secret object now supports event notifications, covering + lifcycle changes and secret value changes. + </description> + </change> + <change> + <summary> New localPtr attribute for "ip" element in network XML </summary> </change> @@ -67,6 +86,17 @@ <section title="Improvements"> <change> <summary> + lxc: fix accidental killing of containers during libvirtd restart + </summary> + <description> + The libvirt_lxc process was previously not moved into the + container scope. As a result, if systemd reloads its config + after a container is started, when libvirtd is later restarted + it will accidentally kill the containers. + </description> + </change> + <change> + <summary> perf: Add more perf statistics </summary> <description> -- 2.9.3

2 1

[libvirt] [PATCH] docs: Add pit timer policy change description to news.xml
by Maxim Nestratov 12 Jan '17

12 Jan '17

Add bug fix description of appying correct pit timer policy. --- docs/news.xml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index e341ba2..d616ebd 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -184,6 +184,17 @@ libvirt can recognize. </description> </change> + <change> + <summary> + qemu: Fix pit timer tick policy + </summary> + <description> + By a mistake, when <code>policy=delay</code> pit timer tick policy + was specified, <code>policy=discard</code> was used instead. Now it + is possible to use both <code>discard</code> and <code>delay</code> + policies correctly. + </description> + </change> </section> </release> <release version="v2.5.0" date="2016-12-04"> -- 2.4.11

2 1

[libvirt] [V3] RFC for support cache tune in libvirt
by Qiao, Liyong 12 Jan '17

12 Jan '17

Add support for cache allocation. Thanks Martin for the previous version comments, this is the v3 version for RFC , I’v have some PoC code [2]. The follow changes are partly finished by the PoC. #Propose Changes ## virsh command line 1. Extend output of nodeinfo, to expose L3 cache size for Level 3 (last level cache size). This will expose how many cache on a host which can be used. root@s2600wt:~/linux# virsh nodeinfo | grep L3 L3 cache size: 56320 KiB 2. Extend capabilities outputs. virsh capabilities | grep resctrl <cpu> ... <resctrl name='L3' unit='KiB' cache_size='56320' cache_unit='2816'/> </cpu> This will tell that the host have enabled resctrl(which you can find it in /sys/fs/resctrl), And it supports to allocate 'L3' type cache, total 'L3' cache size is 56320 KiB, and the minimum unit size of 'L3' cache is 2816 KiB. P.S. L3 cache size unit is the minum l3 cache unit can be allocated. It's hardware related and can not be changed. 3. Add new virsh command 'nodecachestats': This API is to expose vary cache resouce left on each hardware (cpu socket). It will be formated as: <resource_type>.<resource_id>: left size KiB for example I have a 2 socket cpus host, and I'v enabled cat_l3 feature only root@s2600wt:~/linux# virsh nodecachestats L3.0 : 56320 KiB L3.1 : 56320 KiB P.S. resource_type can be L3, L3DATA, L3CODE, L2 for now. 4. Add new interface to manage how many cache can be allociated for a domain root@s2600wt:~/linux# virsh cachetune kvm02 --l3.count 2 root@s2600wt:~/linux# virsh cachetune kvm02 l3.count : 2 This will allocate 2 units(2816 * 2) l3 cache for domain kvm02 ## Domain XML changes Cache Tuneing <domain> ... <cachetune> <l3_cache_count>2</l3_cache_count> </cachetune> ... </domain> ## Restriction for using cache tune on multiple sockets' host. The l3 cache is per socket resource, kernel need to konw about what's affinity looks like, so for a VM which running on a mulitple socket's host, it should have NUMA setting or vcpuset pin setting. Or cache tune will fail. [1] kernel support https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/tree/arch/x86/kern… [2] libvirt PoC(not finished yet) https://github.com/taget/libvirt/commits/cat_new Best Regards Eli Qiao(乔立勇）OpenStack Core team OTC Intel. --

4 13

[libvirt] Query regarding sharing of vcpu's (cpu alloc ratio 1.0) between VM's
by BharaniKumar Gedela 12 Jan '17

12 Jan '17

HI, I have a Q about sharing a vcpu across Vm's (cpu alloc ratio 1.0 and HT not enabled)? I have a Use case where a VM needs vcpu's and one of the vcpu is dedicated to the VM for some traffic processing. We want the other vcpu which is used for control processing and can be shared with a similar VM's control processing. Is this possible and support in openstack/libvirt/KVM? If so could you please advice how to test it? Regards, Bharani..

2 1

[libvirt] Regarding Migration Statistics
by Anubhav Guleria 12 Jan '17

12 Jan '17

Greetings, I am writing a code using libvirt API to migrate VM between two physical hosts *(QEMU/KVM) , *say some *n *number of times. *1)* I am using right now* virDomainPtr virDomainMigrate (.......) *and to calculate the total migration time I am using something like this: *clock_gettime(CLOCK_MONOTONIC_RAW,&begin); * *migrate*(domainToMigrate,nodeToMigrate); *clock_gettime(CLOCK_MONOTONIC_RAW,&end);* *Total Migration Time = end.tv_sec-begin.tv_sec* Is this correct way to calculate total migration time. And is there some way to calculate the downtime (not how to set it)? *2) *I am interested in identifying in particular other statistics of migration like : *Number of iterations in Pre Copy*, *Memory transferred in each iteration* etc. I was going through the API and found* virDomainJobInfo <http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainJobInfo> and virDomainGetJobStats <http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetJobStats> *functions.But how to use them is not very clear. Can anyone point me to right place to achieve this objective? Thanks in advance. And sorry if that was too silly to ask. Anubhav

2 1

[libvirt] OpenStack/libvirt CAT interface
by Marcelo Tosatti 12 Jan '17

12 Jan '17

There have been queries about the OpenStack interface for CAT: http://bugzilla.redhat.com/show_bug.cgi?id=1299678 Comment 2 says: Sahid Ferdjaoui 2016-01-19 10:58:48 EST A spec will have to be addressed, after a first look this feature needs some work in several components of Nova to maintain/schedule/consume host's cache. I can work on that spec and implementation it when libvirt will provides information about cache and feature to use it for guests. I could add a comment about parameters to resctrltool, but since this depends on the libvirt interface, it would be good to know what the libvirt interface exposes first. I believe it should be essentially similar to OpenStack's "reserved_host_memory_mb": Set the reserved_host_memory_mb to reserve RAM for host processes. For the purposes of testing I am going to use the default of 512 MB: reserved_host_memory_mb=512 But rather use: rdt_cat_cache_reservation=type=code/data/both,size=10mb,cacheid=2; type=code/data/both,size=2mb,cacheid=1;... (per-vcpu). Where cache-id is optional. What is cache-id (from Documentation/x86/intel_rdt_ui.txt on recent kernel sources): Cache IDs --------- On current generation systems there is one L3 cache per socket and L2 caches are generally just shared by the hyperthreads on a core, but this isn't an architectural requirement. We could have multiple separate L3 caches on a socket, multiple cores could share an L2 cache. So instead of using "socket" or "core" to define the set of logical cpus sharing a resource we use a "Cache ID". At a given cache level this will be a unique number across the whole system (but it isn't guaranteed to be a contiguous sequence, there may be gaps). To find the ID for each logical CPU look in /sys/devices/system/cpu/cpu*/cache/index*/id WHAT THE USER NEEDS TO SPECIFY FOR VIRTUALIZATION (KVM-RT) ========================================================== For virtualization the following scenario is desired, on a given socket: * VM-A with VCPUs VM-A.vcpu-1, VM-A.vcpu-2. * VM-B with VCPUs VM-B.vcpu-1, VM-B.vcpu-2. With one realtime workload on each vcpu-2. Assume VM-A.vcpu-2 on pcpu 3. Assume VM-B.vcpu-2 on pcpu 5. Assume pcpus 0-5 on cacheid 0. We want VM-A.vcpu-2 to have a certain region of cache reserved, and VM-B.vcpu-2 as well. vcpu-1 for both VMs can use the default group (that is not have reserved L3 cache). This translates to the following resctrltool-style reservations: res.vm-a.vcpu-2 type=both,size=VM-A-RESSIZE,cache-id=0 res.vm-b.vcpu-2 type=both,size=VM-B-RESSIZE,cache-id=0 Which translate to the following in resctrlfs: res.vm-a.vcpu-2 type=both,size=VM-A-RESSIZE,cache-id=0 type=both,size=default-size,cache-id=1 ... res.vm-b.vcpu-2 type=both,size=VM-B-RESSIZE,cache-id=0 type=both,size=default-size,cache-id=1 ... Which is what we want, since the VCPUs are pinned. res.vm-a.vcpu-1 and res.vm-b.vcpu-1 don't need to be assigned to any reservation, which means they'll remain on the default group. RESTRICTIONS TO THE SYNTAX ABOVE ================================ Rules for the parameters: * type=code must be paired with type=data entry. ABOUT THE LIST INTERFACE ======================== About an interface for listing the reservations of the system to OpenStack. I think that what OpenStack needs is to check, before starting a guest on a given host, that there is sufficient space available for the reservation. To do that, it can: 1) resctrltool list (the end of the output mentions how much free space available there is), or via resctrlfs directly (have to lock the filesystem, read each directory, AND each schemata, and count number of zero bits). 2) Via libvirt Should fix resctrltool/API to list amount of contiguous free space BTW.

3 5

[libvirt] RFC: Use __attribute__ ((cleanup) in libvirt ?
by Daniel P. Berrange 12 Jan '17

12 Jan '17

For those who don't already know, GCC and CLang both implement a C language extension that enables automatic free'ing of resources when variables go out of scope. This is done by annotating the variable with the "cleanup" attribute, pointing to a function the compiler will wire up a call to when unwinding the stack. Since the annotation points to an arbitrary user defined function, you're not limited to simple free() like semantics. The cleanup function could unlock a mutex, or decrement a reference count, etc This annotation is used extensively by systemd, and libguestfs, amongst other projects. This obviously doesn't bring full garbage collection to C, but it does enable the code to be simplified. By removing the need to put in many free() (or equiv) calls to cleanup state, the "interesting" logic in the code stands out more, not being obscured by cleanup calls and goto jumps. I'm wondering what people think of making use of this in libvirt ? To my mind the only real reason to *not* use it, would be to maintain code portability to non-GCC/non-CLang compilers. OS-X, *BSD and *Linux all use GCC or CLang or both, so its a non-issue there. So the only place this could cause pain is people building libvirt on Win32, who are using the Microsoft compilers instead og GCC. IMHO, it is perfectly valid for us to declare that MSVC is unsupported with Libvirt and users must use GCC to build on Windows, either natively via cygwin, or cross-build from Linux hosts. As an example of what it would involve... This commit enables the basic helper macros in libguestfs: https://github.com/libguestfs/libguestfs/commit/98b64650c852ccc9a8eef8b9691… These commits make use of them https://github.com/libguestfs/libguestfs/commit/61162bdce1a00a921a47eb3e788… https://github.com/libguestfs/libguestfs/commit/5a3da366268825b26b470cde356… https://github.com/libguestfs/libguestfs/commit/791ad3e9e600ef528e3e5a8d50b… Finally, I'm absolutely *not* volunteering to actually implement this idea myself, as I don't have the free time. I just want to raise it as a discussion item, and if we agree its something we'd do, then we can make it a GSoC idea, or let any other interested person hack on it at will. There's no need for a "big bang" convert everything approach, we can do it incrementally. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|

7 14

[libvirt] Question about hypervisor <features> that are not tristate
by Jim Fehlig 11 Jan '17

11 Jan '17

Happy new year! Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg. The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before. To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use. Regards, Jim [1] https://www.redhat.com/archives/libvir-list/2016-February/msg00197.html [2] https://www.redhat.com/archives/libvir-list/2016-March/msg00001.html

4 6

[libvirt] [PATCH] news: Reflect hugepages patch
by Michal Privoznik 11 Jan '17

11 Jan '17

In f55afd8 I've made libvirt to construct hugepage path on per-domain basis. However, this change was not reflected in the NEWS file. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- I'd push this right away, but I rather wait for our NEWS police^Wvolunteer to check it. docs/news.xml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index e341ba248..f6f17d55f 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -184,6 +184,17 @@ libvirt can recognize. </description> </change> + <change> + <summary> + qemu: Create hugepage path on per domain basis + </summary> + <description> + Historically, all hugepage enabled domains shared the same path under + hugetlbfs. This left libvirt unable to correctly set security labels + on it. With this release, however, each domain is put into a + separate path which is also correctly labeled. + </description> + </change> </section> </release> <release version="v2.5.0" date="2016-12-04"> -- 2.11.0

2 1

[libvirt] [PATCH] qemuDomainSetupAllInputs: Update debug message
by Michal Privoznik 11 Jan '17

11 Jan '17

Due to a copy-paste error, the debug message reads: Setting up disks It should have been: Setting up inputs. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Pushed under trivial rule. src/qemu/qemu_domain.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index b26c02bda..473d0c1a2 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -7275,14 +7275,14 @@ qemuDomainSetupAllInputs(virQEMUDriverPtr driver, { size_t i; - VIR_DEBUG("Setting up disks"); + VIR_DEBUG("Setting up inputs"); for (i = 0; i < vm->def->ninputs; i++) { if (qemuDomainSetupInput(driver, vm->def->inputs[i], devPath) < 0) return -1; } - VIR_DEBUG("Setup all disks"); + VIR_DEBUG("Setup all inputs"); return 0; } -- 2.11.0

1 0

[libvirt] [PATCH for 3.0.0 0/2] Fix qemuMonitorJSONParseCPUModelProperty
by Jiri Denemark 11 Jan '17

11 Jan '17

qemuMonitorJSONParseCPUModelProperty was made a little bit to strict which could cause compatibility issues between libvirt 3.0.0 and QEMU 2.9.0 (or newer depending on when query-cpu-model-expansion is implemented for x86). Jiri Denemark (2): qemu: Don't check CPU model property key qemu: Ignore non-boolean CPU model properties src/qemu/qemu_monitor_json.c | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) -- 2.11.0

2 3

[libvirt] [PATCH 00/11] Adjust build/start format checking for fs, disk, and logical backends
by John Ferlan 11 Jan '17

11 Jan '17

Two of these patches (2 and 10) are followups to no-overwrite patches previously posted (I left links in the patches for reference). Essentially this series works to unify the probing code from the file system backend using blkid and the parted format reading code from the disk backend in one common set of functions. In doing so, the disk backend will now pick up the ability to use the blkid 'partitions' APIs in order to "probe" the target device for a valid/known partition format type. Additionally, the file system probe will pick up a fallback to the parted mechanism from the disk backend if the 'blkid' APIs are not built into the system. A few extra bells/whistles were added for fs pool startup to ensure that the format in the startup XML matches the format on the target disk, which is essentially the mechanism the disk backend used (although it was slightly broken). Then in order to really pile on, the overwrite logic was added to the logical pool which didn't have any such checking (both build and start). Testing was done locally using an iSCSI device and laying down different formats and seeing what would happen on various usages of the format (with --overwrite and --no-overwrite). John Ferlan (11): storage: Introduce virStorageBackendDeviceProbeEmpty storage: Fix implementation of no-overwrite for file system backend storage: Add partition type checks for BLKID probing storage: Add writelabel bool for virStorageBackendDeviceProbe storage: For FS pool check for properly formatted target volume storage: Move and rename disk backend label checking storage: Adjust disk label found to match labels storage: Clean up logical pool devices on build failure storage: Extract logical device initialize into a helper storage: Add overwrite flag checking for logical pool storage: Validate the device formats at logical startup src/libvirt_private.syms | 1 + src/storage/storage_backend.c | 396 ++++++++++++++++++++++++++++++++++ src/storage/storage_backend.h | 4 + src/storage/storage_backend_disk.c | 150 ++----------- src/storage/storage_backend_fs.c | 118 ++-------- src/storage/storage_backend_fs.h | 5 - src/storage/storage_backend_logical.c | 180 ++++++++++------ tools/virsh-pool.c | 2 +- tools/virsh.pod | 42 ++-- 9 files changed, 585 insertions(+), 313 deletions(-) -- 2.7.4

3 22

[libvirt] [PATCH v2] libxl: fix usb inputs loop error
by Cédric Bosdonnat 11 Jan '17

11 Jan '17

List indexes where mixed up in the code looping over the USB input devices. --- * v2: Increment nusbdevice if LIBXL_HAVE_BUILDINFO_USBDEVICE_LIST is not defined. src/libxl/libxl_conf.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index ac83b51c7..a24f9e052 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -479,7 +479,8 @@ libxlMakeDomBuildInfo(virDomainDefPtr def, if (VIR_EXPAND_N(b_info->u.hvm.usbdevice_list, nusbdevice, 1) < 0) return -1; #else - if (i > 1) { + nusbdevice++; + if (nusbdevice > 1) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("libxenlight supports only one input device")); return -1; @@ -487,7 +488,7 @@ libxlMakeDomBuildInfo(virDomainDefPtr def, #endif #ifdef LIBXL_HAVE_BUILDINFO_USBDEVICE_LIST - usbdevice = &b_info->u.hvm.usbdevice_list[i]; + usbdevice = &b_info->u.hvm.usbdevice_list[nusbdevice - 1]; #else usbdevice = &b_info->u.hvm.usbdevice; #endif -- 2.11.0

2 1

[libvirt] [PATCH] Extend l3 cache to nodeinfo
by Eli Qiao 11 Jan '17

11 Jan '17

This patch extends l3 cache infomation to nodeinfo output. Signed-off-by: Eli Qiao <liyong.qiao(a)intel.com> --- include/libvirt/libvirt-host.h | 1 + src/nodeinfo.c | 3 ++- src/remote/remote_protocol.x | 1 + src/test/test_driver.c | 1 + src/util/virhostcpu.c | 29 +++++++++++++++++++++++++---- src/util/virhostcpu.h | 3 ++- src/util/virhostcpupriv.h | 3 ++- tests/virhostcputest.c | 3 ++- tools/virsh-host.c | 1 + 9 files changed, 37 insertions(+), 8 deletions(-) diff --git a/include/libvirt/libvirt-host.h b/include/libvirt/libvirt-host.h index 07b5d15..ba926df 100644 --- a/include/libvirt/libvirt-host.h +++ b/include/libvirt/libvirt-host.h @@ -167,6 +167,7 @@ struct _virNodeInfo { processors in case of unusual NUMA topology*/ unsigned int threads; /* number of threads per core, 1 in case of unusual numa topology */ + unsigned int l3_cache; /* l3 cache in kilobytes */ }; /** diff --git a/src/nodeinfo.c b/src/nodeinfo.c index f2ded02..f54972b 100644 --- a/src/nodeinfo.c +++ b/src/nodeinfo.c @@ -152,7 +152,8 @@ nodeGetInfo(virNodeInfoPtr nodeinfo) if (virHostCPUGetInfo(hostarch, &nodeinfo->cpus, &nodeinfo->mhz, &nodeinfo->nodes, &nodeinfo->sockets, - &nodeinfo->cores, &nodeinfo->threads) < 0) + &nodeinfo->cores, &nodeinfo->threads, + &nodeinfo->l3_cache) < 0) return -1; return 0; diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index b846ef2..6a16b4e 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -489,6 +489,7 @@ struct remote_node_get_info_ret { /* insert@1 */ int sockets; int cores; int threads; + int l3_cache; }; struct remote_connect_get_capabilities_ret { diff --git a/src/test/test_driver.c b/src/test/test_driver.c index de92a01..b49c07b 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -138,6 +138,7 @@ static const virNodeInfo defaultNodeInfo = { 2, 2, 2, + 4096, }; static void diff --git a/src/util/virhostcpu.c b/src/util/virhostcpu.c index f29f312..698813b 100644 --- a/src/util/virhostcpu.c +++ b/src/util/virhostcpu.c @@ -530,7 +530,8 @@ virHostCPUGetInfoPopulateLinux(FILE *cpuinfo, unsigned int *nodes, unsigned int *sockets, unsigned int *cores, - unsigned int *threads) + unsigned int *threads, + unsigned int *l3_cache) { virBitmapPtr present_cpus_map = NULL; virBitmapPtr online_cpus_map = NULL; @@ -546,7 +547,7 @@ virHostCPUGetInfoPopulateLinux(FILE *cpuinfo, int direrr; *mhz = 0; - *cpus = *nodes = *sockets = *cores = *threads = 0; + *cpus = *nodes = *sockets = *cores = *threads = *l3_cache = 0; /* Start with parsing CPU clock speed from /proc/cpuinfo */ while (fgets(line, sizeof(line), cpuinfo) != NULL) { @@ -571,6 +572,24 @@ virHostCPUGetInfoPopulateLinux(FILE *cpuinfo, (*p == '\0' || *p == '.' || c_isspace(*p))) *mhz = ui; } + if (STRPREFIX(buf, "cache size")) { + char *p; + unsigned int ui; + buf += 10; + + while (*buf && c_isspace(*buf)) + buf++; + + if (*buf != ':' || !buf[1]) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("parsing cache size from cpuinfo")); + goto cleanup; + } + + if (virStrToLong_ui(buf+1, &p, 10, &ui) == 0 && + *(p+1)=='K' && *(p+2)=='B') + *l3_cache = ui; + } } else if (ARCH_IS_PPC(arch)) { char *buf = line; if (STRPREFIX(buf, "clock")) { @@ -960,7 +979,8 @@ virHostCPUGetInfo(virArch hostarch ATTRIBUTE_UNUSED, unsigned int *nodes ATTRIBUTE_UNUSED, unsigned int *sockets ATTRIBUTE_UNUSED, unsigned int *cores ATTRIBUTE_UNUSED, - unsigned int *threads ATTRIBUTE_UNUSED) + unsigned int *threads ATTRIBUTE_UNUSED, + unsigned int *l3_cache ATTRIBUTE_UNUSED) { #ifdef __linux__ int ret = -1; @@ -974,7 +994,8 @@ virHostCPUGetInfo(virArch hostarch ATTRIBUTE_UNUSED, ret = virHostCPUGetInfoPopulateLinux(cpuinfo, hostarch, cpus, mhz, nodes, - sockets, cores, threads); + sockets, cores, threads, + l3_cache); if (ret < 0) goto cleanup; diff --git a/src/util/virhostcpu.h b/src/util/virhostcpu.h index 39f7cf8..fc579fe 100644 --- a/src/util/virhostcpu.h +++ b/src/util/virhostcpu.h @@ -50,7 +50,8 @@ int virHostCPUGetInfo(virArch hostarch, unsigned int *nodes, unsigned int *sockets, unsigned int *cores, - unsigned int *threads); + unsigned int *threads, + unsigned int *l3_cache); int virHostCPUGetKVMMaxVCPUs(void); diff --git a/src/util/virhostcpupriv.h b/src/util/virhostcpupriv.h index de30983..69d5e34 100644 --- a/src/util/virhostcpupriv.h +++ b/src/util/virhostcpupriv.h @@ -34,7 +34,8 @@ int virHostCPUGetInfoPopulateLinux(FILE *cpuinfo, unsigned int *nodes, unsigned int *sockets, unsigned int *cores, - unsigned int *threads); + unsigned int *threads, + unsigned int *l3_cache); int virHostCPUGetStatsLinux(FILE *procstat, int cpuNum, diff --git a/tests/virhostcputest.c b/tests/virhostcputest.c index 8387956..2b2a680 100644 --- a/tests/virhostcputest.c +++ b/tests/virhostcputest.c @@ -44,7 +44,8 @@ linuxTestCompareFiles(const char *cpuinfofile, if (virHostCPUGetInfoPopulateLinux(cpuinfo, arch, &nodeinfo.cpus, &nodeinfo.mhz, &nodeinfo.nodes, &nodeinfo.sockets, - &nodeinfo.cores, &nodeinfo.threads) < 0) { + &nodeinfo.cores, &nodeinfo.threads, + &nodeinfo.l3_cache) < 0) { if (virTestGetDebug()) { if (virGetLastError()) VIR_TEST_DEBUG("\n%s\n", virGetLastErrorMessage()); diff --git a/tools/virsh-host.c b/tools/virsh-host.c index 24ebde2..2b85372 100644 --- a/tools/virsh-host.c +++ b/tools/virsh-host.c @@ -671,6 +671,7 @@ cmdNodeinfo(vshControl *ctl, const vshCmd *cmd ATTRIBUTE_UNUSED) vshPrint(ctl, "%-20s %d\n", _("Thread(s) per core:"), info.threads); vshPrint(ctl, "%-20s %d\n", _("NUMA cell(s):"), info.nodes); vshPrint(ctl, "%-20s %lu KiB\n", _("Memory size:"), info.memory); + vshPrint(ctl, "%-20s %d KiB\n", _("L3 cache size:"), info.l3_cache); return true; } -- 1.9.1

5 13

[libvirt] [PATCH] NEWS: Add trailing periods to all sentences
by Andrea Bolognani 11 Jan '17

11 Jan '17

Suggested-by: John Ferlan <jferlan(a)redhat.com> --- docs/news.xml | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/news.xml b/docs/news.xml index 909b4ca..672e988 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -24,7 +24,7 @@ <description> Add the capability to allow group I/O throttling via a new domain <disk> <iotune> subelement "group_name" - to allow sharing I/O throttling quota between multiple drives + to allow sharing I/O throttling quota between multiple drives. </description> </change> <change> @@ -33,7 +33,7 @@ </summary> <description> New <code>libvirt-guest</code> nss module that translates libvirt - guest names into IP addresses + guest names into IP addresses. </description> </change> <change> @@ -43,7 +43,7 @@ <description> Logging-related settings like log outputs and filters can now be adjusted during runtime using the admin interface without the - necessity of the daemon's restart + necessity of the daemon's restart. </description> </change> <change> @@ -55,7 +55,7 @@ flag in order to return the host physical size in bytes of the image container in the allocation field of the _virStorageVolInfo structure. The --physical flag has been - added to the virsh vol-info command to access the data + added to the virsh vol-info command to access the data. </description> </change> <change> @@ -73,7 +73,7 @@ Add support to get the count of branch instructions executed, branch misses, bus cycles, stalled frontend cpu cycles, stalled backend cpu cycles, and ref cpu - cycles by applications running on the platform + cycles by applications running on the platform. </description> </change> <change> @@ -82,7 +82,7 @@ </summary> <description> Add a display of the <physical> size of a disk - volume in the output of the volume XML + volume in the output of the volume XML. </description> </change> <change> @@ -93,7 +93,7 @@ virtio-pci provides several advantages over virtio-mmio, such as the ability to hotplug devices and improved performance. While opting in to virtio-pci has been possible for a while, - newly-defined guests will now use it automatically + newly-defined guests will now use it automatically. </description> </change> </section> @@ -106,7 +106,7 @@ For an active domain, correct the physical value provided for a raw sparse file backed storage and the allocation value provided for a qcow2 file backed storage that hasn't yet been opened on - the domain + the domain. </description> </change> <change> @@ -115,7 +115,7 @@ </summary> <description> The chardev detection code has been improved and can now handle this - configuration properly + configuration properly. </description> </change> <change> @@ -138,7 +138,7 @@ </summary> <description> The shmem device can now utilize QEMU's ivshmem-plain and - ivshmem-doorbell, more modern versions of ivshmem + ivshmem-doorbell, more modern versions of ivshmem. </description> </change> <change> @@ -153,7 +153,7 @@ <description> The new libssh transport allows one to connect to a running libvirtd via SSH, using the libssh library; for example: - <tt>qemu+libssh://<i>server</i>/system</tt> + <tt>qemu+libssh://<i>server</i>/system</tt>. </description> </change> <change> @@ -162,7 +162,7 @@ </summary> <description> Add the capability to pass through a scsi_host HBA and the - associated LUNs to the guest + associated LUNs to the guest. </description> </change> <change> @@ -190,7 +190,7 @@ e1000e, nec-xhci, vfio assigned devices) will be placed on an Express controller (i.e. a pcie-root-port) instead of a legacy PCI controller (i.e. pci-bridge) with the root ports - added as needed + added as needed. </description> </change> </section> @@ -222,7 +222,7 @@ </summary> <description> List user-visible changes instead of single commits for a better - high-level overview of differences between libvirt releases + high-level overview of differences between libvirt releases. </description> </change> <change> @@ -232,7 +232,7 @@ <description> The libvirt website looked very cluttered and outdated; it has now been completely overhauled, resulting in a design that's better - organized and more pleasant to look at + organized and more pleasant to look at. </description> </change> </section> -- 2.7.4

2 2

[libvirt] Paradox cpu topology in capabilities outputs
by Qiao, Liyong 11 Jan '17

11 Jan '17

Hi, I observe that virsh capabilities give wrong cpu topology on a multiple sockets host taget@jfz1r04h13:~/libvirt$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 72 On-line CPU(s) list: 0-71 Thread(s) per core: 2 Core(s) per socket: 18 Socket(s): 2 <------------------------ NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 63 Model name: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz Stepping: 2 CPU MHz: 1201.660 CPU max MHz: 3600.0000 CPU min MHz: 1200.0000 BogoMIPS: 4590.78 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 46080K NUMA node0 CPU(s): 0-17,36-53 NUMA node1 CPU(s): 18-35,54-71 But output of virsh capabilities only gives. <topology sockets='1' cores='18' threads='2'/> looking into code and got this: https://libvirt.org/git/?p=libvirt.git;a=blob;f=src/util/virhostcpu.c;h=f29… should we change it into 704<https://libvirt.org/git/?p=libvirt.git;a=blob;f=src/util/virhostcpu.c;h=f29…> *sockets += nodesockets; This also affect nodeinfo.sockets. Attached file is the full output of capabilities of the host Best Regards Eli Qiao(乔立勇）OpenStack Core team OTC Intel. --

3 3

[libvirt] [PATCH 00/11] tests: qemu: Add cpu hotplug testing
by Peter Krempa 11 Jan '17

11 Jan '17

Add some useful monitor infrastructure to simplify testing and add testing for the cpu hotplug code. The code has some complex logic which was not covered by current test suite. Peter Krempa (11): qemu: monitor: More strict checking of 'query-cpus' if hotplug is supported util: json: Add helper to reformat JSON strings tests: qemu: Document qemuMonitorTestNewFromFile qemu: Prepare for reuse of qemuDomainSetVcpusLive qemu: Move cpu hotplug code into qemu_hotplug.c tests: qemumonitor: Propagate better error messages tests: qemu: monitor: Add helpers to test full command syntax tests: qemu: Add helper to load full monitor conversation from file tests: hotplug: Add test infrastructure for testing qemu CPU hotplug code tests: hotplug: Add test data for legacy cpu hotplug tests: hotplug: Test CPU hotplug with ppc64 data src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 319 ----------- src/qemu/qemu_hotplug.c | 328 ++++++++++++ src/qemu/qemu_hotplug.h | 7 + src/qemu/qemu_monitor.c | 6 +- src/qemu/qemu_monitor_json.c | 6 +- src/qemu/qemu_monitor_json.h | 3 +- src/util/virjson.c | 29 + src/util/virjson.h | 2 + tests/qemuagenttest.c | 3 +- tests/qemuhotplugtest.c | 194 +++++++ tests/qemuhotplugtestcpus/ppc64-bulk-domain.xml | 20 + tests/qemuhotplugtestcpus/ppc64-bulk-monitor.json | 593 +++++++++++++++++++++ .../qemuhotplugtestcpus/ppc64-bulk-result-conf.xml | 64 +++ .../qemuhotplugtestcpus/ppc64-bulk-result-live.xml | 72 +++ .../qemuhotplugtestcpus/x86-modern-bulk-domain.xml | 21 + .../x86-modern-bulk-monitor.json | 471 ++++++++++++++++ .../x86-modern-bulk-result-conf.xml | 40 ++ .../x86-modern-bulk-result-live.xml | 48 ++ tests/qemuhotplugtestcpus/x86-old-bulk-domain.xml | 21 + .../qemuhotplugtestcpus/x86-old-bulk-monitor.json | 193 +++++++ .../x86-old-bulk-result-conf.xml | 30 ++ .../x86-old-bulk-result-live.xml | 38 ++ tests/qemumonitorjsontest.c | 2 +- tests/qemumonitortestutils.c | 322 ++++++++++- tests/qemumonitortestutils.h | 12 +- 26 files changed, 2505 insertions(+), 340 deletions(-) create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-domain.xml create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-monitor.json create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-result-conf.xml create mode 100644 tests/qemuhotplugtestcpus/ppc64-bulk-result-live.xml create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-domain.xml create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-monitor.json create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-result-conf.xml create mode 100644 tests/qemuhotplugtestcpus/x86-modern-bulk-result-live.xml create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-domain.xml create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-monitor.json create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-result-conf.xml create mode 100644 tests/qemuhotplugtestcpus/x86-old-bulk-result-live.xml -- 2.10.2

1 13

[libvirt] [PATCH v3 0/8] aggregate multiple pcie-root-ports onto a single slot
by Laine Stump 11 Jan '17

11 Jan '17

Another rebase of the series with no other changes Laine Stump (8): conf: use struct instead of int for each slot in virDomainPCIAddressBus conf: eliminate concept of "reserveEntireSlot" conf: eliminate repetitive code in virDomainPCIAddressGetNextSlot() conf: start search for next unused PCI address at same slot as previous find conf: new function virDomainPCIAddressIsMulti() qemu: use virDomainPCIAddressIsMulti() to determine multifunction setting conf: aggregate multiple devices on a slot when assigning PCI addresses conf: aggregate multiple pcie-root-ports onto a single slot src/conf/domain_addr.c | 290 ++++++++++++++------- src/conf/domain_addr.h | 48 +++- src/libvirt_private.syms | 1 + src/qemu/qemu_command.c | 16 +- src/qemu/qemu_domain_address.c | 35 ++- .../qemuxml2argv-pcie-root-port.args | 5 +- .../qemuxml2argv-pcie-switch-upstream-port.args | 5 +- .../qemuxml2argv-q35-default-devices-only.args | 7 +- .../qemuxml2argv-q35-multifunction.args | 43 +++ .../qemuxml2argv-q35-multifunction.xml | 51 ++++ .../qemuxml2argv-q35-pcie-autoadd.args | 30 ++- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args | 28 +- .../qemuxml2argv-q35-virt-manager-basic.args | 13 +- .../qemuxml2argv-q35-virtio-pci.args | 28 +- tests/qemuxml2argvtest.c | 25 ++ .../qemuxml2xmlout-pcie-root-port.xml | 2 +- .../qemuxml2xmlout-pcie-switch-upstream-port.xml | 4 +- .../qemuxml2xmlout-q35-default-devices-only.xml | 8 +- .../qemuxml2xmlout-q35-multifunction.xml | 120 +++++++++ .../qemuxml2xmlout-q35-pcie-autoadd.xml | 52 ++-- .../qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml | 48 ++-- .../qemuxml2xmlout-q35-virt-manager-basic.xml | 20 +- .../qemuxml2xmlout-q35-virtio-pci.xml | 48 ++-- tests/qemuxml2xmltest.c | 25 ++ 24 files changed, 676 insertions(+), 276 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-multifunction.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-multifunction.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-multifunction.xml -- 2.7.4

4 26

[libvirt] [PATCH v4] libxl: define a per-domain logger.
by Cédric Bosdonnat 11 Jan '17

11 Jan '17

libxl doesn't provide a way to write one log for each domain. Thus we need to demux the messages. If our logger doesn't know to which domain to attribute a message, then it will write it to the default log file. Starting with Xen 4.9 (commit f9858025 and following), libxl will write the domain ID in an easy to grab manner. The logger introduced by this commit will use it to demux the libxl log messages. Thanks to the default log file, this logger will also work with older versions of Xen. --- * v4: delay the log file opening to the first message write. * v3: add JSON-formatted libxl_domain_config to the newly opened file when creating a guest. * v2: addressed Jim's comments src/Makefile.am | 1 + src/libxl/libxl_conf.c | 38 +------ src/libxl/libxl_conf.h | 4 +- src/libxl/libxl_domain.c | 7 ++ src/libxl/libxl_driver.c | 2 + src/libxl/libxl_logger.c | 257 +++++++++++++++++++++++++++++++++++++++++++++++ src/libxl/libxl_logger.h | 40 ++++++++ 7 files changed, 312 insertions(+), 37 deletions(-) create mode 100644 src/libxl/libxl_logger.c create mode 100644 src/libxl/libxl_logger.h diff --git a/src/Makefile.am b/src/Makefile.am index 0dd2faaf6..58e3108a6 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -851,6 +851,7 @@ LIBXL_DRIVER_SOURCES = \ libxl/libxl_capabilities.c libxl/libxl_capabilities.h \ libxl/libxl_domain.c libxl/libxl_domain.h \ libxl/libxl_driver.c libxl/libxl_driver.h \ + libxl/libxl_logger.c libxl/libxl_logger.h \ libxl/libxl_migration.c libxl/libxl_migration.h UML_DRIVER_SOURCES = \ diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index b569ddad8..ac83b51c7 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -76,9 +76,7 @@ libxlDriverConfigDispose(void *obj) virObjectUnref(cfg->caps); libxl_ctx_free(cfg->ctx); - xtl_logger_destroy(cfg->logger); - if (cfg->logger_file) - VIR_FORCE_FCLOSE(cfg->logger_file); + libxlLoggerFree(cfg->logger); VIR_FREE(cfg->configDir); VIR_FREE(cfg->autostartDir); @@ -1356,8 +1354,6 @@ libxlDriverConfigPtr libxlDriverConfigNew(void) { libxlDriverConfigPtr cfg; - char *log_file = NULL; - xentoollog_level log_level = XTL_DEBUG; char ebuf[1024]; unsigned int free_mem; @@ -1386,9 +1382,6 @@ libxlDriverConfigNew(void) if (VIR_STRDUP(cfg->channelDir, LIBXL_CHANNEL_DIR) < 0) goto error; - if (virAsprintf(&log_file, "%s/libxl-driver.log", cfg->logDir) < 0) - goto error; - if (virFileMakePath(cfg->logDir) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("failed to create log dir '%s': %s"), @@ -1397,37 +1390,13 @@ libxlDriverConfigNew(void) goto error; } - if ((cfg->logger_file = fopen(log_file, "a")) == NULL) { - VIR_ERROR(_("Failed to create log file '%s': %s"), - log_file, virStrerror(errno, ebuf, sizeof(ebuf))); - goto error; - } - VIR_FREE(log_file); - - switch (virLogGetDefaultPriority()) { - case VIR_LOG_DEBUG: - log_level = XTL_DEBUG; - break; - case VIR_LOG_INFO: - log_level = XTL_INFO; - break; - case VIR_LOG_WARN: - log_level = XTL_WARN; - break; - case VIR_LOG_ERROR: - log_level = XTL_ERROR; - break; - } - - cfg->logger = - (xentoollog_logger *)xtl_createlogger_stdiostream(cfg->logger_file, - log_level, XTL_STDIOSTREAM_SHOW_DATE); + cfg->logger = libxlLoggerNew(cfg->logDir, virLogGetDefaultPriority()); if (!cfg->logger) { VIR_ERROR(_("cannot create logger for libxenlight, disabling driver")); goto error; } - if (libxl_ctx_alloc(&cfg->ctx, LIBXL_VERSION, 0, cfg->logger)) { + if (libxl_ctx_alloc(&cfg->ctx, LIBXL_VERSION, 0, (xentoollog_logger *)cfg->logger)) { VIR_ERROR(_("cannot initialize libxenlight context, probably not " "running in a Xen Dom0, disabling driver")); goto error; @@ -1478,7 +1447,6 @@ libxlDriverConfigNew(void) return cfg; error: - VIR_FREE(log_file); virObjectUnref(cfg); return NULL; } diff --git a/src/libxl/libxl_conf.h b/src/libxl/libxl_conf.h index 851f3afb4..69d78851a 100644 --- a/src/libxl/libxl_conf.h +++ b/src/libxl/libxl_conf.h @@ -41,6 +41,7 @@ # include "locking/lock_manager.h" # include "virfirmware.h" # include "libxl_capabilities.h" +# include "libxl_logger.h" # define LIBXL_DRIVER_NAME "xenlight" # define LIBXL_VNC_PORT_MIN 5900 @@ -74,8 +75,7 @@ struct _libxlDriverConfig { unsigned int version; /* log stream for driver-wide libxl ctx */ - FILE *logger_file; - xentoollog_logger *logger; + libxlLoggerPtr logger; /* libxl ctx for driver wide ops; getVersion, getNodeInfo, ... */ libxl_ctx *ctx; diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 5cde576ef..fbe7ee5ff 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -809,6 +809,8 @@ libxlDomainCleanup(libxlDriverPrivatePtr driver, VIR_FREE(xml); } + libxlLoggerCloseFile(cfg->logger, vm->def->id); + virDomainObjRemoveTransientDef(vm); virObjectUnref(cfg); } @@ -1127,6 +1129,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, libxl_asyncprogress_how aop_console_how; libxl_domain_restore_params params; unsigned int hostdev_flags = VIR_HOSTDEV_SP_PCI; + char *config_json = NULL; #ifdef LIBXL_HAVE_PVUSB hostdev_flags |= VIR_HOSTDEV_SP_USB; @@ -1290,6 +1293,9 @@ libxlDomainStart(libxlDriverPrivatePtr driver, * be cleaned up if there are any subsequent failures. */ vm->def->id = domid; + config_json = libxl_domain_config_to_json(cfg->ctx, &d_config); + + libxlLoggerOpenFile(cfg->logger, domid, vm->def->name, config_json); /* Always enable domain death events */ if (libxl_evenable_domain_death(cfg->ctx, vm->def->id, 0, &priv->deathW)) @@ -1366,6 +1372,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, cleanup: libxl_domain_config_dispose(&d_config); + VIR_FREE(config_json); VIR_FREE(dom_xml); VIR_FREE(managed_save_path); virDomainDefFree(def); diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 7e5d9b69e..325636c9a 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -406,6 +406,8 @@ libxlReconnectDomain(virDomainObjPtr vm, /* Update domid in case it changed (e.g. reboot) while we were gone? */ vm->def->id = d_info.domid; + libxlLoggerOpenFile(cfg->logger, vm->def->id, vm->def->name, NULL); + /* Update hostdev state */ if (virHostdevUpdateActiveDomainDevices(hostdev_mgr, LIBXL_DRIVER_NAME, vm->def, hostdev_flags) < 0) diff --git a/src/libxl/libxl_logger.c b/src/libxl/libxl_logger.c new file mode 100644 index 000000000..437dbb34b --- /dev/null +++ b/src/libxl/libxl_logger.c @@ -0,0 +1,257 @@ +/* + * libxl_logger.c: libxl logger implementation + * + * Copyright (c) 2016 SUSE LINUX Products GmbH, Nuernberg, Germany. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Cédric Bosdonnat <cbosdonnat(a)suse.com> + */ +#include <config.h> + +#include <string.h> +#include <libxl.h> + +#include "internal.h" +#include "libxl_logger.h" +#include "util/viralloc.h" +#include "util/virerror.h" +#include "util/virfile.h" +#include "util/virhash.h" +#include "util/virstring.h" +#include "util/virtime.h" + +#define VIR_FROM_THIS VIR_FROM_LIBXL + +VIR_LOG_INIT("libxl.libxl_logger"); + +typedef struct xentoollog_logger_libvirt xentoollog_logger_libvirt; + +typedef struct { + FILE *handle; + char *path; +} libxlLoggerFile; + +struct xentoollog_logger_libvirt { + xentoollog_logger vtable; + xentoollog_level minLevel; + const char *logDir; + + /* map storing the opened fds: "domid" -> FILE* */ + virHashTablePtr files; + FILE *defaultLogFile; +}; + +static void +libxlLoggerFileFree(void *payload, const void *key ATTRIBUTE_UNUSED) +{ + libxlLoggerFile *file = payload; + if (file->handle) { + VIR_FORCE_FCLOSE(file->handle); + file->handle = NULL; + } + VIR_FREE(file->path); +} + +static int +libxlLoggerFileOpen(libxlLoggerFile *file) { + int ret = 0; + char ebuf[1024]; + + if (!file->handle && !(file->handle = fopen(file->path, "a"))) { + VIR_WARN("Failed to open log file %s: %s", + file->path, virStrerror(errno, ebuf, sizeof(ebuf))); + ret = -1; + } + + return ret; +} + +ATTRIBUTE_FMT_PRINTF(5, 0) static void +libvirt_vmessage(xentoollog_logger *logger_in, + xentoollog_level level, + int errnoval, + const char *context, + const char *format, + va_list args) +{ + xentoollog_logger_libvirt *lg = (xentoollog_logger_libvirt *)logger_in; + FILE *logFile = lg->defaultLogFile; + char timestamp[VIR_TIME_STRING_BUFLEN]; + char *message = NULL; + char *start, *end; + char ebuf[1024]; + + VIR_DEBUG("libvirt_vmessage: context='%s' format='%s'", context, format); + + if (level < lg->minLevel) + return; + + if (virVasprintf(&message, format, args) < 0) + return; + + /* Should we print to a domain-specific log file? */ + if ((start = strstr(message, ": Domain ")) && + (end = strstr(start + 9, ":"))) { + libxlLoggerFile *file = NULL; + + start = start + 9; + *end = '\0'; + + file = virHashLookup(lg->files, start); + if (file) { + ignore_value(libxlLoggerFileOpen(file)); + logFile = file->handle; + } + + *end = ':'; + } + + /* Do the actual print to the log file */ + if (virTimeStringNowRaw(timestamp) < 0) + timestamp[0] = '\0'; + + fprintf(logFile, "%s: ", timestamp); + if (context) + fprintf(logFile, "%s: ", context); + + fprintf(logFile, "%s", message); + + if (errnoval >= 0) + fprintf(logFile, ": %s", virStrerror(errnoval, ebuf, sizeof(ebuf))); + + fputc('\n', logFile); + fflush(logFile); + + VIR_FREE(message); +} + +static void +libvirt_progress(xentoollog_logger *logger_in ATTRIBUTE_UNUSED, + const char *context ATTRIBUTE_UNUSED, + const char *doingwhat ATTRIBUTE_UNUSED, + int percent ATTRIBUTE_UNUSED, + unsigned long done ATTRIBUTE_UNUSED, + unsigned long total ATTRIBUTE_UNUSED) +{ + /* This function purposedly does nothing: it's no logging info */ +} + +static void +libvirt_destroy(xentoollog_logger *logger_in) +{ + xentoollog_logger_libvirt *lg = (xentoollog_logger_libvirt*)logger_in; + VIR_FREE(lg); +} + + +libxlLoggerPtr libxlLoggerNew(const char *logDir, + virLogPriority minLevel) +{ + xentoollog_logger_libvirt logger; + libxlLoggerPtr logger_out = NULL; + char *path = NULL; + + switch (minLevel) { + case VIR_LOG_DEBUG: + logger.minLevel = XTL_DEBUG; + break; + case VIR_LOG_INFO: + logger.minLevel = XTL_INFO; + break; + case VIR_LOG_WARN: + logger.minLevel = XTL_WARN; + break; + case VIR_LOG_ERROR: + logger.minLevel = XTL_ERROR; + break; + } + logger.logDir = logDir; + + if ((logger.files = virHashCreate(3, libxlLoggerFileFree)) == NULL) + return NULL; + + if (virAsprintf(&path, "%s/libxl-driver.log", logDir) < 0) + goto error; + + if ((logger.defaultLogFile = fopen(path, "a")) == NULL) + goto error; + + logger_out = XTL_NEW_LOGGER(libvirt, logger); + + cleanup: + VIR_FREE(path); + return logger_out; + + error: + virHashFree(logger.files); + goto cleanup; +} + +void libxlLoggerFree(libxlLoggerPtr logger) +{ + xentoollog_logger *xtl_logger = (xentoollog_logger*)logger; + if (logger->defaultLogFile) + VIR_FORCE_FCLOSE(logger->defaultLogFile); + virHashFree(logger->files); + xtl_logger_destroy(xtl_logger); +} + +void libxlLoggerOpenFile(libxlLoggerPtr logger, int id, + const char *name, + const char *domain_config) +{ + FILE *logFile = NULL; + char *domidstr = NULL; + libxlLoggerFile *file; + + if (VIR_ALLOC(file) < 0) + return; + + if (virAsprintf(&file->path, "%s/%s.log", logger->logDir, name) < 0 || + virAsprintf(&domidstr, "%d", id) < 0) + goto error; + + ignore_value(virHashAddEntry(logger->files, domidstr, file)); + + /* domain_config is non NULL only when starting a new domain */ + if (domain_config) { + if (libxlLoggerFileOpen(file) < 0) + goto cleanup; + + fprintf(logFile, "Domain start: %s\n", domain_config); + fflush(logFile); + } + goto cleanup; + + error: + VIR_FREE(file->path); + VIR_FREE(file); + + cleanup: + VIR_FREE(domidstr); +} + +void libxlLoggerCloseFile(libxlLoggerPtr logger, int id) +{ + char *domidstr = NULL; + if (virAsprintf(&domidstr, "%d", id) < 0) + return; + + ignore_value(virHashRemoveEntry(logger->files, domidstr)); + + VIR_FREE(domidstr); +} diff --git a/src/libxl/libxl_logger.h b/src/libxl/libxl_logger.h new file mode 100644 index 000000000..92b222578 --- /dev/null +++ b/src/libxl/libxl_logger.h @@ -0,0 +1,40 @@ +/* + * libxl_logger.h: libxl logger implementation + * + * Copyright (c) 2016 SUSE LINUX Products GmbH, Nuernberg, Germany. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Cédric Bosdonnat <cbosdonnat(a)suse.com> + */ + +#ifndef __LIBXL_LOGGER_H +# define __LIBXL_LOGGER_H + +# include "util/virlog.h" + +typedef struct xentoollog_logger_libvirt libxlLogger; +typedef libxlLogger *libxlLoggerPtr; + +libxlLoggerPtr libxlLoggerNew(const char *logDir, + virLogPriority minLevel); +void libxlLoggerFree(libxlLoggerPtr logger); + +void libxlLoggerOpenFile(libxlLoggerPtr logger, int id, const char *name, + const char *domain_config); +void libxlLoggerCloseFile(libxlLoggerPtr logger, int id); + +#endif /* __LIBXL_LOGGER_H */ -- 2.11.0

4 4

[libvirt] Do different Virgil3D users running have the same CANVAS fingerprint?
by bancfc＠openmailbox.org 11 Jan '17

11 Jan '17

Background: Canvas fingerprinting is a technique to track users based on their GPUs. https://en.wikipedia.org/wiki/Canvas_fingerprinting Someone did some testing and the good news is the fingerprint is common for all users of the same OS across all KVM users because it presents the same virtual GPU/drivers for everybody. What I need your help to find out is whether this holds for Virgil3D users that enable graphics acceleration. (I am running on Debian and the libvirt version in the repos is too old to have this included also I don't have multiple machines) To test this you need: To run plain FireFox on a virtualized distro (with GL acceleration enabled in libvirt) of your choice and visit https://browserleaks.com/canvas . Then repeat the same steps with that same distro (in a VM) on another physical machine and compare values. A good result is if you see the same numbers for both machines. That means the virtual device fingerprint is uniform for all virtual users.

2 3

[libvirt] [PATCH] libxl: fix usb inputs loop error
by Cédric Bosdonnat 10 Jan '17

10 Jan '17

List indexes where mixed up in the code looping over the USB input devices. --- src/libxl/libxl_conf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index ac83b51c7..1053e60a1 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -479,7 +479,7 @@ libxlMakeDomBuildInfo(virDomainDefPtr def, if (VIR_EXPAND_N(b_info->u.hvm.usbdevice_list, nusbdevice, 1) < 0) return -1; #else - if (i > 1) { + if (nusbdevice > 1) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("libxenlight supports only one input device")); return -1; @@ -487,7 +487,7 @@ libxlMakeDomBuildInfo(virDomainDefPtr def, #endif #ifdef LIBXL_HAVE_BUILDINFO_USBDEVICE_LIST - usbdevice = &b_info->u.hvm.usbdevice_list[i]; + usbdevice = &b_info->u.hvm.usbdevice_list[nusbdevice - 1]; #else usbdevice = &b_info->u.hvm.usbdevice; #endif -- 2.11.0

2 1

[libvirt] [PATCH] virsh: pool-info: introduce option --bytes
by Chen Hanxiao 10 Jan '17

10 Jan '17

From: Chen Hanxiao <chenhanxiao(a)gmail.com> By default, pool-info will convert sizes to human friendly units. This patch will introduce option [--bytes]. If specified, the raw sizes will be in the output. Signed-off-by: Chen Hanxiao <chenhanxiao(a)gmail.com> --- tools/virsh-pool.c | 25 +++++++++++++++++++------ tools/virsh.pod | 5 +++-- 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/tools/virsh-pool.c b/tools/virsh-pool.c index 8313be8..cbd7042 100644 --- a/tools/virsh-pool.c +++ b/tools/virsh-pool.c @@ -1589,6 +1589,10 @@ static const vshCmdInfo info_pool_info[] = { static const vshCmdOptDef opts_pool_info[] = { VIRSH_COMMON_OPT_POOL_FULL, + {.name = "bytes", + .type = VSH_OT_BOOL, + .help = N_("Reture pool info in bytes"), + }, {.name = NULL} }; @@ -1600,11 +1604,14 @@ cmdPoolInfo(vshControl *ctl, const vshCmd *cmd) int autostart = 0; int persistent = 0; bool ret = true; + bool bytes = false; char uuid[VIR_UUID_STRING_BUFLEN]; if (!(pool = virshCommandOptPool(ctl, cmd, "pool", NULL))) return false; + bytes = vshCommandOptBool(cmd, "bytes"); + vshPrint(ctl, "%-15s %s\n", _("Name:"), virStoragePoolGetName(pool)); if (virStoragePoolGetUUIDString(pool, &uuid[0]) == 0) @@ -1633,14 +1640,20 @@ cmdPoolInfo(vshControl *ctl, const vshCmd *cmd) if (info.state == VIR_STORAGE_POOL_RUNNING || info.state == VIR_STORAGE_POOL_DEGRADED) { - val = vshPrettyCapacity(info.capacity, &unit); - vshPrint(ctl, "%-15s %2.2lf %s\n", _("Capacity:"), val, unit); + if (bytes) { + vshPrint(ctl, "%-15s %llu\n", _("Capacity:"), info.capacity); + vshPrint(ctl, "%-15s %llu\n", _("Allocation:"), info.allocation); + vshPrint(ctl, "%-15s %llu\n", _("Available:"), info.available); + } else { + val = vshPrettyCapacity(info.capacity, &unit); + vshPrint(ctl, "%-15s %2.2lf %s\n", _("Capacity:"), val, unit); - val = vshPrettyCapacity(info.allocation, &unit); - vshPrint(ctl, "%-15s %2.2lf %s\n", _("Allocation:"), val, unit); + val = vshPrettyCapacity(info.allocation, &unit); + vshPrint(ctl, "%-15s %2.2lf %s\n", _("Allocation:"), val, unit); - val = vshPrettyCapacity(info.available, &unit); - vshPrint(ctl, "%-15s %2.2lf %s\n", _("Available:"), val, unit); + val = vshPrettyCapacity(info.available, &unit); + vshPrint(ctl, "%-15s %2.2lf %s\n", _("Available:"), val, unit); + } } } else { ret = false; diff --git a/tools/virsh.pod b/tools/virsh.pod index ef91223..7c418a5 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -3605,9 +3605,10 @@ except that it does some error checking. The editor used can be supplied by the C<$VISUAL> or C<$EDITOR> environment variables, and defaults to C<vi>. -=item B<pool-info> I<pool-or-uuid> +=item B<pool-info> [I<--bytes>] I<pool-or-uuid> -Returns basic information about the I<pool> object. +Returns basic information about the I<pool> object. If I<--bytes> is specified the sizes +of basic info are not converted to human friendly units. =item B<pool-list> [I<--inactive>] [I<--all>] [I<--persistent>] [I<--transient>] -- 2.7.4

2 1

[libvirt] [PATCH 00/12] vbox: remove support for vbox 3.x and older.
by Dawid Zamirski 10 Jan '17

10 Jan '17

Hello, This series removes support for legacy VirtualBox versions (3.x and older) that did not have upstream support for a while and won't even work on any relatively recent distro. The idea for doing this was first mentioned here [1] and this makes the driver code cleaner as the APIs for those old versions were quite different from v4.0+ and required abstractions to handle the differences. Please note that the first patch in the series is sent to ML as a compressed attachment because it deletes VirtualBox SDK header files for those old verions which are large and the patch would not pass ML message size limit. [1] https://www.redhat.com/archives/libvir-list/2016-November/msg01133.html Regards, Dawid Dawid Zamirski (12): vbox: remove SDK header files for vbox 3 and older. vbox: remove calls to *InstallUniformedAPI macros. vbox: remove code for old API versions. vbox: remove _vboxAttachDrivesOld vbox: do not use IHardDisk anymore. vbox: remove getMachineForSession flag. vbox: remove domain events support. vbox: remove code dealing with oldMediumInterface vbox: IVRDxServer to IVRDEServer. vbox: fix _displayTakeScreenShotPNGToArray vbox: consolidate vbox IID structures. docs: add news entry in improvements section. docs/news.html.in | 8 + src/Makefile.am | 4 - src/vbox/vbox_CAPI_v2_2.h | 4869 ----------------------------------- src/vbox/vbox_CAPI_v3_0.h | 5396 --------------------------------------- src/vbox/vbox_CAPI_v3_1.h | 5297 -------------------------------------- src/vbox/vbox_CAPI_v3_2.h | 5625 ----------------------------------------- src/vbox/vbox_V2_2.c | 37 - src/vbox/vbox_V3_0.c | 37 - src/vbox/vbox_V3_1.c | 37 - src/vbox/vbox_V3_2.c | 37 - src/vbox/vbox_XPCOMCGlue.h | 2 +- src/vbox/vbox_common.c | 402 +-- src/vbox/vbox_common.h | 13 +- src/vbox/vbox_network.c | 12 +- src/vbox/vbox_storage.c | 76 +- src/vbox/vbox_tmpl.c | 3006 ++-------------------- src/vbox/vbox_uniformed_api.h | 171 +- 17 files changed, 438 insertions(+), 24591 deletions(-) delete mode 100644 src/vbox/vbox_CAPI_v2_2.h delete mode 100644 src/vbox/vbox_CAPI_v3_0.h delete mode 100644 src/vbox/vbox_CAPI_v3_1.h delete mode 100644 src/vbox/vbox_CAPI_v3_2.h delete mode 100644 src/vbox/vbox_V2_2.c delete mode 100644 src/vbox/vbox_V3_0.c delete mode 100644 src/vbox/vbox_V3_1.c delete mode 100644 src/vbox/vbox_V3_2.c -- 2.9.3

2 14

[libvirt] [PATCH 1/3] rpc: libssh: allow a NULL known_hosts file
by Pino Toscano 10 Jan '17

10 Jan '17

Make sure that virNetLibsshSessionSetHostKeyVerification accepts a NULL value for the path to the known_hosts file: - call ssh_options_set(SSH_OPTIONS_KNOWNHOSTS) anyway, using /dev/null, otherwise libssh will use its default path - do not call ssh_write_knownhost when no known hosts file was set Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1406457 --- src/rpc/virnetlibsshsession.c | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/src/rpc/virnetlibsshsession.c b/src/rpc/virnetlibsshsession.c index 5de6629..25f93ce 100644 --- a/src/rpc/virnetlibsshsession.c +++ b/src/rpc/virnetlibsshsession.c @@ -382,14 +382,16 @@ virNetLibsshCheckHostKey(virNetLibsshSessionPtr sess) VIR_FREE(askKey.result); } - /* write the host key file */ - if (ssh_write_knownhost(sess->session) < 0) { - errmsg = ssh_get_error(sess->session); - virReportError(VIR_ERR_LIBSSH, - _("failed to write known_host file '%s': %s"), - sess->knownHostsFile, - errmsg); - return -1; + /* write the host key file, if specified */ + if (sess->knownHostsFile) { + if (ssh_write_knownhost(sess->session) < 0) { + errmsg = ssh_get_error(sess->session); + virReportError(VIR_ERR_LIBSSH, + _("failed to write known_host file '%s': %s"), + sess->knownHostsFile, + errmsg); + return -1; + } } /* key was accepted and added */ return 0; @@ -1172,13 +1174,20 @@ virNetLibsshSessionSetHostKeyVerification(virNetLibsshSessionPtr sess, goto error; } - /* set the known hosts file */ - if (ssh_options_set(sess->session, SSH_OPTIONS_KNOWNHOSTS, hostsfile) < 0) - goto error; + /* set the known hosts file, if specified */ + if (hostsfile) { + if (ssh_options_set(sess->session, SSH_OPTIONS_KNOWNHOSTS, hostsfile) < 0) + goto error; - VIR_FREE(sess->knownHostsFile); - if (VIR_STRDUP(sess->knownHostsFile, hostsfile) < 0) - goto error; + VIR_FREE(sess->knownHostsFile); + if (VIR_STRDUP(sess->knownHostsFile, hostsfile) < 0) + goto error; + } else { + /* libssh does not support trying no known_host file at all: + * hence use /dev/null here, without storing it as file */ + if (ssh_options_set(sess->session, SSH_OPTIONS_KNOWNHOSTS, "/dev/null") < 0) + goto error; + } virObjectUnlock(sess); return 0; -- 2.7.4

2 5

[libvirt] [PATCH] maint: update to latest gnulib
by Eric Blake 10 Jan '17

10 Jan '17

Among other recent changes, this includes a workaround to avoid Clang compiler bug https://llvm.org/bugs/show_bug.cgi?id=16404 having spurious link failures. * .gnulib: Update. * bootstrap: Synchronize to upstream. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- I'm debating about re-enabling test-lock (right now, we explicitly pass --avoid=lock-tests to gnulib-tool via our bootstrap.conf), now that upstream is finally starting to fix the bugs in the test that were causing extremely slow completion or outright hangs on multi-processor setups. But that's probably something better done right after a libvirt release, not right before a release candidate. Pushing under the gnulib maintenance rule; contains these gnulib patches: * .gnulib e210a3c...94386a1 (38): > maint.mk: enforce spelling of "timestamp" (i.e., no space) > dfa: minor simplification with emptyset > dfa: shrink constraints from 4 bits to 3 > dfa: omit unnecessary ptrdiff_t check > dfa: omit unnecessary allocation > dfa: omit unused local > maint: time stamp -> timestamp > maint: remove stray .texi files > getprogname: fix port to IRIX > dfa: melt down dfastate into build_state > dfa: simplify transition table allocation > dfa: fix reallocation bug when matching newlines > Avoid -Wundef warning about undefined WINDOWS_SOCKETS. > Avoid -Wundef warning about undefined __USE_FILE_OFFSET64. > stdioext: Port to Minix 3.2 and newer. > getprogname: port to IRIX > localename-tests: port to NetBSD 7 > glob, intprops, xalloc: work around Clang bug > dfa: fix 'return' typo > lock tests: Prefer semaphore over mutex. > parse-datetime: fix generated paths for coverage files > maint.mk: support parallel execution of coverage > lock: Provide guarantee to avoid writer starvation for rwlocks. > thread: Fix pth port. > parse-datetime: fix debug message on lone year number > parse-datetime: fix local timezone debug messages > parse-datetime: add debug warning about DST changes > parse-datetime: add debug warning about date arithmetic > parse-datetime: fix debug message of relative part after timezone > parse-datetime: fix incorrect debug message on lone number > exec > build-aux/mdate-sh > doc: fix typo in previous change > Revert copyright-year change to synced files > doc: modernize for C11 etc. > dfa: prefer functions to FETCH_WC macro > dfa: narrow more local var scopes > dfa: remove duplicate assignment .gnulib | 2 +- bootstrap | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.gnulib b/.gnulib index e210a3c..94386a1 160000 --- a/.gnulib +++ b/.gnulib @@ -1 +1 @@ -Subproject commit e210a3cbaec0ee82a67ff8fc427e21bdd64dba1b +Subproject commit 94386a13667c645fd42544a7fd302cdddd39fcdf diff --git a/bootstrap b/bootstrap index 26066b2..932ff85 100755 --- a/bootstrap +++ b/bootstrap @@ -1,6 +1,6 @@ #! /bin/sh # Print a version string. -scriptversion=2016-11-03.18; # UTC +scriptversion=2017-01-09.19; # UTC # Bootstrap this package from checked-out sources. @@ -790,7 +790,7 @@ symlink_to_dir() # Leave any existing symlink alone, if it already points to the source, # so that broken build tools that care about symlink times # aren't confused into doing unnecessary builds. Conversely, if the - # existing symlink's time stamp is older than the source, make it afresh, + # existing symlink's timestamp is older than the source, make it afresh, # so that broken tools aren't confused into skipping needed builds. See # <http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00326.html>. test -h "$dst" && -- 2.9.3

1 0

[libvirt] [PATCH v3 0/3] Improve release notes process
by Andrea Bolognani 10 Jan '17

10 Jan '17

Dan raised some concerns, but it doesn't look like they're critical enough for him to NACK the series; moreover, it's all stuff that we can easily address later on, so I'd like to move forward with this before the freeze. Details of changes can be found in the single patches. Andrea Bolognani (3): NEWS: Improve building pipeline NEWS: Reformat at generation time docs: Document the release notes process for contributors .gitignore | 1 + HACKING | 7 ++ Makefile.am | 26 ++++-- docs/Makefile.am | 28 +++++- docs/hacking.html.in | 10 ++ docs/news-ascii.xsl | 68 ++++++++++++++ docs/news-html.xsl | 96 +++++++++++++++++++ docs/news.html.in | 164 -------------------------------- docs/news.xml | 252 ++++++++++++++++++++++++++++++++++++++++++++++++++ docs/news.xsl | 50 ---------- docs/reformat-news.py | 104 +++++++++++++++++++++ 11 files changed, 583 insertions(+), 223 deletions(-) create mode 100644 docs/news-ascii.xsl create mode 100644 docs/news-html.xsl delete mode 100644 docs/news.html.in create mode 100644 docs/news.xml delete mode 100644 docs/news.xsl create mode 100755 docs/reformat-news.py -- 2.7.4

2 7

[libvirt] [PATCH 0/2] Couple of build fixes
by Michal Privoznik 10 Jan '17

10 Jan '17

Spotted by our CI. Michal Privoznik (2): qemu_domain: Move qemuDomainGetPreservedMounts virSecuritySELinuxSetFileconHelper: Fix build with broken selinux.h src/qemu/qemu_domain.c | 140 ++++++++++++++++++++-------------------- src/security/security_selinux.c | 2 +- 2 files changed, 71 insertions(+), 71 deletions(-) -- 2.11.0

2 3

[libvirt] [PATCH] libxl: implement virDomainGetMaxVcpus
by Jim Fehlig 10 Jan '17

10 Jan '17

The libxl driver already supports getting maximum vcpu count via libxlDomainGetVcpusFlags, allowing to trivially implement virDomainGetMaxVcpus. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- docs/news.html.in | 2 ++ src/libxl/libxl_driver.c | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/docs/news.html.in b/docs/news.html.in index c222075..2365c39 100644 --- a/docs/news.html.in +++ b/docs/news.html.in @@ -39,6 +39,8 @@ _virStorageVolInfo structure. The --physical flag has been added to the virsh vol-info command to access the data </li> + <li>libxl: Implement virDomainGetMaxVcpus API + </li> </ul> </li> <li><strong>Improvements</strong> diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 7e5d9b6..51fe3d8 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -2363,6 +2363,13 @@ libxlDomainGetVcpusFlags(virDomainPtr dom, unsigned int flags) } static int +libxlDomainGetMaxVcpus(virDomainPtr dom) +{ + return libxlDomainGetVcpusFlags(dom, (VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_VCPU_MAXIMUM)); +} + +static int libxlDomainPinVcpuFlags(virDomainPtr dom, unsigned int vcpu, unsigned char *cpumap, int maplen, unsigned int flags) @@ -6446,6 +6453,7 @@ static virHypervisorDriver libxlHypervisorDriver = { .domainSetVcpus = libxlDomainSetVcpus, /* 0.9.0 */ .domainSetVcpusFlags = libxlDomainSetVcpusFlags, /* 0.9.0 */ .domainGetVcpusFlags = libxlDomainGetVcpusFlags, /* 0.9.0 */ + .domainGetMaxVcpus = libxlDomainGetMaxVcpus, /* 3.0.0 */ .domainPinVcpu = libxlDomainPinVcpu, /* 0.9.0 */ .domainPinVcpuFlags = libxlDomainPinVcpuFlags, /* 1.2.1 */ .domainGetVcpus = libxlDomainGetVcpus, /* 0.9.0 */ -- 2.9.2

2 2

[libvirt] [PATCH 0/2] replacement for Patches 5/8 & 6/8 of "aggregate multiple pcie-root-ports onto a single slot"
by Laine Stump 10 Jan '17

10 Jan '17

During review of the aforementioned series, Andrea expressed misgivings about patch 6, which automatically set the multifunction=on option transiently at the time the qemu commandline was generated. Dan agreed. These two patches replace patches 5 and 6 of that series with two patches that will set multi='on' in the domain's config rather than just temporarily in the commandline. This way it will remain set even if all the devices on the non-0 function are later removed (unless it's explicitly changed by the user/management app, that is). The rest of the series is ACKed and ready to push, which I'd like to do before the freeze on Wednesday; these two patche are all that's holding it up. Laine Stump (2): conf: new function virDomainPCIAddressSetAllMulti() qemu: use virDomainPCIAddressSetAllMulti() to set multi when needed src/conf/domain_addr.c | 75 +++++++++++++ src/conf/domain_addr.h | 4 + src/libvirt_private.syms | 1 + src/qemu/qemu_domain_address.c | 5 + .../qemuxml2argv-q35-multifunction.args | 42 ++++++++ .../qemuxml2argv-q35-multifunction.xml | 51 +++++++++ tests/qemuxml2argvtest.c | 23 ++++ .../qemuxml2xmlout-q35-multifunction.xml | 120 +++++++++++++++++++++ tests/qemuxml2xmltest.c | 23 ++++ 9 files changed, 344 insertions(+) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-multifunction.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-multifunction.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-multifunction.xml -- 2.7.4

2 6

[libvirt] qemu-kvm blocked for more than 120 seconds when "libvirt-guests" is enabled
by Oscar Segarra 10 Jan '17

10 Jan '17

Hi, I just have two virtual machines in my environment, I want them to gracefully stop when host is powered off gracefully. Nevertheless, system hangs on shutdown: [root@vdicnode01 ~]# virsh list Id Name State ---------------------------------------------------- 1 vdicdb01 running 2 vdicone01 running The configuration: /usr/libexec/libvirt-guests.sh URIS=default ON_BOOT=ignore ON_SHUTDOWN=shutdown SHUTDOWN_TIMEOUT=60 PARALLEL_SHUTDOWN=5 START_DELAY=0 BYPASS_CACHE=0 CONNECT_RETRIES=10 RETRIES_SLEEP=1 SYNC_TIME=0 [root@vdicnode01 ~]# cat /etc/libvirt/qemu.conf user = "oneadmin" group = "oneadmin" dynamic_ownership = 0 spice_tls = 0 With libvirt-guests service started: [root@vdicnode01 ~]# service libvirt-guests status Redirecting to /bin/systemctl status libvirt-guests.service ● libvirt-guests.service - Suspend Active Libvirt Guests Loaded: loaded (/usr/lib/systemd/system/libvirt-guests.service; disabled; vendor preset: disabled) Active: active (exited) since Sat 2016-12-31 17:13:34 CET; 5s ago Docs: man:libvirtd(8) http://libvirt.org Process: 6619 ExecStart=/usr/libexec/libvirt-guests.sh start (code=exited, status=0/SUCCESS) Main PID: 6619 (code=exited, status=0/SUCCESS) Dec 31 17:13:34 vdicnode01.vdicube.com systemd[1]: Starting Suspend Active Libvirt Guests... Dec 31 17:13:34 vdicnode01.vdicube.com libvirt-guests.sh[6619]: libvirt-guests is configured not to start any guests on boot Dec 31 17:13:34 vdicnode01.vdicube.com systemd[1]: Started Suspend Active Libvirt Guests. [root@vdicnode01 ~]# If I stop locally libvirt-guests (as root) Looks work perfectly: [root@vdicnode01 ~]# virsh list Id Name State ---------------------------------------------------- [root@vdicnode01 ~]# --> Now I start again the virtual machines and the libvirt-guests service <-- [root@vdicnode01 ~]# service libvirtd restart Redirecting to /bin/systemctl restart libvirtd.service [root@vdicnode01 ~]# service libvirt-guests start Redirecting to /bin/systemctl start libvirt-guests.service [root@vdicnode01 ~]# virsh list Id Name State ---------------------------------------------------- 1 vdicone01 running 2 vdicdb01 running [root@vdicnode01 ~]# But If I shutdown the host it look not work properly (I attack screenshots of the shutdown process). Any help will be welcome. Thanks a lot.

3 11

[libvirt] [PATCH] storage: Fix storage_backend probing when PARTED not installed.
by John Ferlan 10 Jan '17

10 Jan '17

Commit id 'a48c674f' caused problems for systems without PARTED installed. So move the PARTED probing code back to storage_backend_disk.c and create a shim within storage_backend.c to call it if WITH_STORAGE_DISK is true; otherwise, just return -1 with the error. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Reported to me internally via Peter Krempa... Not pushing as a build breaker - I'll let Peter review and ACK. src/storage/storage_backend.c | 155 ++++--------------------------------- src/storage/storage_backend_disk.c | 152 ++++++++++++++++++++++++++++++++++++ src/storage/storage_backend_disk.h | 3 + 3 files changed, 169 insertions(+), 141 deletions(-) diff --git a/src/storage/storage_backend.c b/src/storage/storage_backend.c index 060b1c0..18433e9 100644 --- a/src/storage/storage_backend.c +++ b/src/storage/storage_backend.c @@ -2851,157 +2851,30 @@ virStorageBackendBLKIDFindEmpty(const char *device ATTRIBUTE_UNUSED, #endif /* #if WITH_BLKID */ -typedef enum { - VIR_STORAGE_PARTED_ERROR = -1, - VIR_STORAGE_PARTED_MATCH, /* Valid label found and matches format */ - VIR_STORAGE_PARTED_DIFFERENT, /* Valid label found but not match format */ - VIR_STORAGE_PARTED_UNKNOWN, /* No or unrecognized label */ - VIR_STORAGE_PARTED_NOPTTYPE, /* Did not find the Partition Table type */ - VIR_STORAGE_PARTED_PTTYPE_UNK, /* Partition Table type unknown*/ -} virStorageBackendPARTEDResult; - -/** - * Check for a valid disk label (partition table) on device using - * the PARTED command - * - * returns virStorageBackendPARTEDResult - */ -static virStorageBackendPARTEDResult -virStorageBackendPARTEDFindLabel(const char *device, - const char *format) -{ - const char *const args[] = { - device, "print", "--script", NULL, - }; - virCommandPtr cmd = virCommandNew(PARTED); - char *output = NULL; - char *error = NULL; - char *start, *end; - int ret = VIR_STORAGE_PARTED_ERROR; - - virCommandAddArgSet(cmd, args); - virCommandAddEnvString(cmd, "LC_ALL=C"); - virCommandSetOutputBuffer(cmd, &output); - virCommandSetErrorBuffer(cmd, &error); - - /* if parted succeeds we have a valid partition table */ - ret = virCommandRun(cmd, NULL); - if (ret < 0) { - if ((output && strstr(output, "unrecognised disk label")) || - (error && strstr(error, "unrecognised disk label"))) { - ret = VIR_STORAGE_PARTED_UNKNOWN; - } - goto cleanup; - } - - /* Search for "Partition Table:" in the output. If not present, - * then we cannot validate the partition table type. - */ - if (!(start = strstr(output, "Partition Table: ")) || - !(end = strstr(start, "\n"))) { - VIR_DEBUG("Unable to find tag in output: %s", output); - ret = VIR_STORAGE_PARTED_NOPTTYPE; - goto cleanup; - } - start += strlen("Partition Table: "); - *end = '\0'; - - /* on disk it's "msdos", but we document/use "dos" so deal with it here */ - if (STREQ(start, "msdos")) - start += 2; - - /* Make sure we know about this type */ - if (virStoragePoolFormatDiskTypeFromString(start) < 0) { - ret = VIR_STORAGE_PARTED_PTTYPE_UNK; - goto cleanup; - } - - /* Does the on disk match what the pool desired? */ - if (STREQ(start, format)) - ret = VIR_STORAGE_PARTED_MATCH; - - ret = VIR_STORAGE_PARTED_DIFFERENT; - - cleanup: - virCommandFree(cmd); - VIR_FREE(output); - VIR_FREE(error); - return ret; -} - +#if WITH_STORAGE_DISK -/** - * Determine whether the label on the disk is valid or in a known format - * for the purpose of rewriting the label during build or being able to - * start a pool on a device. - * - * When 'writelabel' is true, if we find a valid disk label on the device, - * then we shouldn't be attempting to write as the volume may contain - * data. Force the usage of the overwrite flag to the build command in - * order to be certain. When the disk label is unrecognized, then it - * should be safe to write. - * - * When 'writelabel' is false, only if we find a valid disk label on the - * device should we allow the start since for this path we won't be - * rewriting the label. - * - * Return: 0 if it's OK - * -1 if something's wrong - */ static int virStorageBackendPARTEDValidLabel(const char *device, const char *format, bool writelabel) { - int ret = -1; - virStorageBackendPARTEDResult check; - - check = virStorageBackendPARTEDFindLabel(device, format); - switch (check) { - case VIR_STORAGE_PARTED_ERROR: - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("Error checking for disk label, failed to get " - "disk partition information")); - break; - - case VIR_STORAGE_PARTED_MATCH: - if (writelabel) - virReportError(VIR_ERR_OPERATION_INVALID, - _("Disk label already formatted using '%s'"), - format); - else - ret = 0; - break; - - case VIR_STORAGE_PARTED_DIFFERENT: - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("Known, but different label format present, " - "requires build --overwrite")); - break; - - case VIR_STORAGE_PARTED_UNKNOWN: - if (writelabel) - ret = 0; - else - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("Unrecognized disk label found, requires build")); - break; - - case VIR_STORAGE_PARTED_NOPTTYPE: - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("Unable to determine Partition Type, " - "requires build --overwrite")); - break; + return virStorageBackendDiskValidLabel(device, format, writelabel); +} - case VIR_STORAGE_PARTED_PTTYPE_UNK: - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("Unknown Partition Type, requires build --overwrite")); - break; - } +#else - return ret; +static int +virStorageBackendPARTEDValidLabel(const char *device ATTRIBUTE_UNUSED, + const char *format ATTRIBUTE_UNUSED, + bool writelabel ATTRIBUTE_UNUSED) +{ + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("PARTED is unsupported by this build")); + return -1; } +#endif /* #if WITH_STORAGE_DISK */ + /* virStorageBackendDeviceIsEmpty: * @devpath: Path to the device to check diff --git a/src/storage/storage_backend_disk.c b/src/storage/storage_backend_disk.c index eae6c90..04ffc6c 100644 --- a/src/storage/storage_backend_disk.c +++ b/src/storage/storage_backend_disk.c @@ -418,6 +418,158 @@ virStorageBackendDiskRefreshPool(virConnectPtr conn ATTRIBUTE_UNUSED, } +typedef enum { + VIR_STORAGE_PARTED_ERROR = -1, + VIR_STORAGE_PARTED_MATCH, /* Valid label found and matches format */ + VIR_STORAGE_PARTED_DIFFERENT, /* Valid label found but not match format */ + VIR_STORAGE_PARTED_UNKNOWN, /* No or unrecognized label */ + VIR_STORAGE_PARTED_NOPTTYPE, /* Did not find the Partition Table type */ + VIR_STORAGE_PARTED_PTTYPE_UNK, /* Partition Table type unknown*/ +} virStorageBackendPARTEDResult; + +/** + * Check for a valid disk label (partition table) on device using + * the PARTED command + * + * returns virStorageBackendPARTEDResult + */ +static virStorageBackendPARTEDResult +virStorageBackendPARTEDFindLabel(const char *device, + const char *format) +{ + const char *const args[] = { + device, "print", "--script", NULL, + }; + virCommandPtr cmd = virCommandNew(PARTED); + char *output = NULL; + char *error = NULL; + char *start, *end; + int ret = VIR_STORAGE_PARTED_ERROR; + + virCommandAddArgSet(cmd, args); + virCommandAddEnvString(cmd, "LC_ALL=C"); + virCommandSetOutputBuffer(cmd, &output); + virCommandSetErrorBuffer(cmd, &error); + + /* if parted succeeds we have a valid partition table */ + ret = virCommandRun(cmd, NULL); + if (ret < 0) { + if ((output && strstr(output, "unrecognised disk label")) || + (error && strstr(error, "unrecognised disk label"))) { + ret = VIR_STORAGE_PARTED_UNKNOWN; + } + goto cleanup; + } + + /* Search for "Partition Table:" in the output. If not present, + * then we cannot validate the partition table type. + */ + if (!(start = strstr(output, "Partition Table: ")) || + !(end = strstr(start, "\n"))) { + VIR_DEBUG("Unable to find tag in output: %s", output); + ret = VIR_STORAGE_PARTED_NOPTTYPE; + goto cleanup; + } + start += strlen("Partition Table: "); + *end = '\0'; + + /* on disk it's "msdos", but we document/use "dos" so deal with it here */ + if (STREQ(start, "msdos")) + start += 2; + + /* Make sure we know about this type */ + if (virStoragePoolFormatDiskTypeFromString(start) < 0) { + ret = VIR_STORAGE_PARTED_PTTYPE_UNK; + goto cleanup; + } + + /* Does the on disk match what the pool desired? */ + if (STREQ(start, format)) + ret = VIR_STORAGE_PARTED_MATCH; + + ret = VIR_STORAGE_PARTED_DIFFERENT; + + cleanup: + virCommandFree(cmd); + VIR_FREE(output); + VIR_FREE(error); + return ret; +} + + +/** + * Determine whether the label on the disk is valid or in a known format + * for the purpose of rewriting the label during build or being able to + * start a pool on a device. + * + * When 'writelabel' is true, if we find a valid disk label on the device, + * then we shouldn't be attempting to write as the volume may contain + * data. Force the usage of the overwrite flag to the build command in + * order to be certain. When the disk label is unrecognized, then it + * should be safe to write. + * + * When 'writelabel' is false, only if we find a valid disk label on the + * device should we allow the start since for this path we won't be + * rewriting the label. + * + * Return: 0 if it's OK + * -1 if something's wrong + */ +int +virStorageBackendDiskValidLabel(const char *device, + const char *format, + bool writelabel) +{ + int ret = -1; + virStorageBackendPARTEDResult check; + + check = virStorageBackendPARTEDFindLabel(device, format); + switch (check) { + case VIR_STORAGE_PARTED_ERROR: + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("Error checking for disk label, failed to get " + "disk partition information")); + break; + + case VIR_STORAGE_PARTED_MATCH: + if (writelabel) + virReportError(VIR_ERR_OPERATION_INVALID, + _("Disk label already formatted using '%s'"), + format); + else + ret = 0; + break; + + case VIR_STORAGE_PARTED_DIFFERENT: + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("Known, but different label format present, " + "requires build --overwrite")); + break; + + case VIR_STORAGE_PARTED_UNKNOWN: + if (writelabel) + ret = 0; + else + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("Unrecognized disk label found, requires build")); + break; + + case VIR_STORAGE_PARTED_NOPTTYPE: + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("Unable to determine Partition Type, " + "requires build --overwrite")); + break; + + case VIR_STORAGE_PARTED_PTTYPE_UNK: + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("Unknown Partition Type, requires build --overwrite")); + break; + } + + return ret; +} + + static int virStorageBackendDiskStartPool(virConnectPtr conn ATTRIBUTE_UNUSED, virStoragePoolObjPtr pool) diff --git a/src/storage/storage_backend_disk.h b/src/storage/storage_backend_disk.h index aaabe62..79c0e6b 100644 --- a/src/storage/storage_backend_disk.h +++ b/src/storage/storage_backend_disk.h @@ -28,4 +28,7 @@ extern virStorageBackend virStorageBackendDisk; +int virStorageBackendDiskValidLabel(const char *device, + const char *format, + bool writelabel); #endif /* __VIR_STORAGE_BACKEND_DISK_H__ */ -- 2.7.4

2 1

[libvirt] [PATCH] rpc: libssh: don't crash when known_hosts file path is not provided
by Peter Krempa 10 Jan '17

10 Jan '17

When connecting as root, the "hostsfile" variable would be NULL due to the code leading to this point. This would result into a crash when attempting to set the known hosts file path. To avoid deviating from the approach taken in the libssh2 driver set the file to /dev/null so that all entries are discarded unless explicitly specified. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1406457 --- Note that it would be much simpler just to skip ssh_options_set if 'hostsfile' is NULL. This would result in using /root/.ssh/known_hosts (according to the config) which would be different to the approach taken in libssh2. With libssh2 this can't be done (at least the last time I checked) as it happened to corrupt the file in some cases. src/rpc/virnetlibsshsession.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/rpc/virnetlibsshsession.c b/src/rpc/virnetlibsshsession.c index 5de6629d7..5fc16ba8a 100644 --- a/src/rpc/virnetlibsshsession.c +++ b/src/rpc/virnetlibsshsession.c @@ -1172,6 +1172,9 @@ virNetLibsshSessionSetHostKeyVerification(virNetLibsshSessionPtr sess, goto error; } + if (!hostsfile) + hostsfile = "/dev/null"; + /* set the known hosts file */ if (ssh_options_set(sess->session, SSH_OPTIONS_KNOWNHOSTS, hostsfile) < 0) goto error; -- 2.11.0

2 2

[libvirt] [PATCH 0/6] qemu: blockjob: make automatic disk locking less useless
by Peter Krempa 10 Jan '17

10 Jan '17

If automatic disk locking would be enabled with either lock manager the operations would most probably fail. This was mostly as libvirt did not bother to release images that were no longer written to by qemu, or tried re-lock the same image. This fixes the locking code for creation of external snapshots (both disk-only and with memory), block copy and active block commit. This series also kills the horrible "rollback" code in the external disk snapshot code. Regular block commit work will be tracked by: https://bugzilla.redhat.com/show_bug.cgi?id=1405537 Peter Krempa (6): qemu: blockcopy: Save monitor error prior to calling into lock manager locking: Fix documentation on how automatic sanlock leases are stored qemu: snapshot: Don't redetect backing chain after snapshot qemu: snapshot: Refactor snapshot rollback on failure qemu: snapshot: Properly handle image locking qemu: blockjob: Fix locking of block copy/active block commit src/locking/sanlock.conf | 2 +- src/qemu/qemu_blockjob.c | 26 ++-- src/qemu/qemu_driver.c | 389 +++++++++++++++++++++++++++-------------------- 3 files changed, 242 insertions(+), 175 deletions(-) -- 2.11.0

3 14

[libvirt] [PATCH go-xml] Added Storage Pool and Storage Volume XML schemes.
by Alexey Slaykovsky 10 Jan '17

10 Jan '17

Signed-off-by: Alexey Slaykovsky <aslaikov(a)redhat.com> --- storage_encryption.go | 50 ++++++++++ storage_pool.go | 121 +++++++++++++++++++++++++ storage_pool_test.go | 240 ++++++++++++++++++++++++++++++++++++++++++++++++ storage_vol.go | 83 +++++++++++++++++ storage_vol_test.go | 246 ++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 740 insertions(+) create mode 100644 storage_encryption.go create mode 100644 storage_pool.go create mode 100644 storage_pool_test.go create mode 100644 storage_vol.go create mode 100644 storage_vol_test.go diff --git a/storage_encryption.go b/storage_encryption.go new file mode 100644 index 0000000..f66314c --- /dev/null +++ b/storage_encryption.go @@ -0,0 +1,50 @@ +/* + * This file is part of the libvirt-go-xml project + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * Copyright (C) 2017 Red Hat, Inc. + * + */ + +package libvirtxml + +type StorageEncryptionSecret struct { + Type string `xml:"type,attr"` + UUID string `xml:"uuid,attr"` +} + +type StorageEncryptionCipher struct { + Name string `xml:"name,attr"` + Size uint64 `xml:"size,attr"` + Mode string `xml:"mode,attr"` + Hash string `xml:"hash,attr"` +} + +type StorageEncryptionIvgen struct { + Name string `xml:"name,attr"` + Hash string `xml:"hash,attr"` +} + +type StorageEncryption struct { + Format string `xml:"format,attr"` + Secret *StorageEncryptionSecret `xml:"secret"` + Cipher *StorageEncryptionCipher `xml:"cipher"` + Ivgen *StorageEncryptionIvgen `xml:"ivgen"` +} diff --git a/storage_pool.go b/storage_pool.go new file mode 100644 index 0000000..3481361 --- /dev/null +++ b/storage_pool.go @@ -0,0 +1,121 @@ +/* + * This file is part of the libvirt-go-xml project + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * Copyright (C) 2017 Red Hat, Inc. + * + */ + +package libvirtxml + +import "encoding/xml" + +type StoragePoolTargetPermissions struct { + Owner string `xml:"owner,omitempty"` + Group string `xml:"group,omitempty"` + Mode string `xml:"mode,omitempty"` + Label string `xml:"label,omitempty"` +} + +type StoragePoolTargetTimestamps struct { + Atime string `xml:"atime"` + Mtime string `xml:"mtime"` + Ctime string `xml:"ctime"` +} + +type StoragePoolTarget struct { + Path string `xml:"path,omitempty"` + Permissions *StoragePoolTargetPermissions `xml:"permissions"` + Timestamps *StoragePoolTargetTimestamps `xml:"timestamps"` + Encryption *StorageEncryption `xml:"encryption"` +} + +type StoragePoolSourceFormat struct { + Type string `xml:"type,attr"` +} +type StoragePoolSourceHost struct { + Name string `xml:"name,attr"` +} + +type StoragePoolSourceDevice struct { + Path string `xml:"path,attr"` + PartSeparator string `xml:"part_separator,attr,omitempty"` +} + +type StoragePoolSourceAuthSecret struct { + Usage string `xml:"usage,attr"` +} + +type StoragePoolSourceAuth struct { + Type string `xml:"type,attr"` + Username string `xml:"username,attr"` + Secret *StoragePoolSourceAuthSecret `xml:"secret"` +} + +type StoragePoolSourceVendor struct { + Name string `xml:"name,attr"` +} + +type StoragePoolSourceProduct struct { + Name string `xml:"name,attr"` +} + +type StoragePoolSourceAdapterParentAddrAddress struct { + Domain string `xml:"domain,attr"` + Bus string `xml:"bus,attr"` + Slot string `xml:"slot,attr"` + Addr string `xml:"addr,attr"` +} + +type StoragePoolSourceAdapterParentAddr struct { + UniqueID uint64 `xml:"unique_id,attr"` + Address *StoragePoolSourceAdapterParentAddrAddress `xml:"address"` +} + +type StoragePoolSourceAdapter struct { + Type string `xml:"type,attr"` + Name string `xml:"name,attr,omitempty"` + Parent string `xml:"parent,attr,omitempty"` + WWNN string `xml:"wwnn,attr,omitempty"` + WWPN string `xml:"wwpn,attr,omitempty"` + ParentAddr *StoragePoolSourceAdapterParentAddr `xml:"parentaddr"` +} + +type StoragePoolSource struct { + Host *StoragePoolSourceHost `xml:"host"` + Device *StoragePoolSourceDevice `xml:"device"` + Auth *StoragePoolSourceAuth `xml:"auth"` + Vendor *StoragePoolSourceVendor `xml:"vendor"` + Product *StoragePoolSourceProduct `xml:"product"` + Format *StoragePoolSourceFormat `xml:"format"` + Adapter *StoragePoolSourceAdapter `xml:"adapter"` +} + +type StoragePool struct { + XMLName xml.Name `xml:"pool"` + Type string `xml:"type,attr"` + Name string `xml:"name"` + UUID string `xml:"uuid,omitempty"` + Allocation uint64 `xml:"allocation,omitempty"` + Capacity uint64 `xml:"capacity,omitempty"` + Available uint64 `xml:"available,omitempty"` + Target *StoragePoolTarget `xml:"target"` + Source *StoragePoolSource `xml:"source"` +} diff --git a/storage_pool_test.go b/storage_pool_test.go new file mode 100644 index 0000000..8c1b606 --- /dev/null +++ b/storage_pool_test.go @@ -0,0 +1,240 @@ +/* + * This file is part of the libvirt-go-xml project + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * Copyright (C) 2017 Red Hat, Inc. + * + */ + +package libvirtxml + +import ( + "encoding/xml" + "strings" + "testing" +) + +var storagePoolTestData = []struct { + Object *StoragePool + Expected []string +}{ + { + Object: &StoragePool{ + Type: "dir", + Name: "pool", + UUID: "3e3fce45-4f53-4fa7-bb32-11f34168b82b", + Allocation: 1000000, + Capacity: 5000000, + Available: 3000000, + }, + Expected: []string{ + `<pool type="dir">`, + ` <name>pool</name>`, + ` <uuid>3e3fce45-4f53-4fa7-bb32-11f34168b82b</uuid>`, + ` <allocation>1000000</allocation>`, + ` <capacity>5000000</capacity>`, + ` <available>3000000</available>`, + `</pool>`, + }, + }, + { + Object: &StoragePool{ + Type: "iscsi", + Name: "pool", + Source: &StoragePoolSource{ + Host: &StoragePoolSourceHost{ + Name: "host.example.com", + }, + Device: &StoragePoolSourceDevice{ + Path: "pool.example.com:iscsi-pool", + }, + Auth: &StoragePoolSourceAuth{ + Type: "chap", + Username: "username", + Secret: &StoragePoolSourceAuthSecret{ + Usage: "cluster", + }, + }, + Vendor: &StoragePoolSourceVendor{ + Name: "vendor", + }, + Product: &StoragePoolSourceProduct{ + Name: "product", + }, + }, + }, + Expected: []string{ + `<pool type="iscsi">`, + ` <name>pool</name>`, + ` <source>`, + ` <host name="host.example.com"></host>`, + ` <device path="pool.example.com:iscsi-pool"></device>`, + ` <auth type="chap" username="username">`, + ` <secret usage="cluster"></secret>`, + ` </auth>`, + ` <vendor name="vendor"></vendor>`, + ` <product name="product"></product>`, + ` </source>`, + `</pool>`, + }, + }, + { + Object: &StoragePool{ + Type: "disk", + Name: "pool", + Source: &StoragePoolSource{ + Device: &StoragePoolSourceDevice{ + Path: "/dev/mapper/pool", + PartSeparator: "no", + }, + Format: &StoragePoolSourceFormat{ + Type: "gpt", + }, + }, + }, + Expected: []string{ + `<pool type="disk">`, + ` <name>pool</name>`, + ` <source>`, + ` <device path="/dev/mapper/pool" part_separator="no"></device>`, + ` <format type="gpt"></format>`, + ` </source>`, + `</pool>`, + }, + }, + { + Object: &StoragePool{ + Type: "scsi", + Name: "pool", + Source: &StoragePoolSource{ + Adapter: &StoragePoolSourceAdapter{ + Type: "scsi_host", + Name: "scsi_host", + }, + }, + }, + Expected: []string{ + `<pool type="scsi">`, + ` <name>pool</name>`, + ` <source>`, + ` <adapter type="scsi_host" name="scsi_host"></adapter>`, + ` </source>`, + `</pool>`, + }, + }, + { + Object: &StoragePool{ + Type: "scsi", + Name: "pool", + Source: &StoragePoolSource{ + Adapter: &StoragePoolSourceAdapter{ + Type: "scsi_host", + ParentAddr: &StoragePoolSourceAdapterParentAddr{ + UniqueID: 1, + Address: &StoragePoolSourceAdapterParentAddrAddress{ + Domain: "0x0000", + Bus: "0x00", + Slot: "0x1f", + Addr: "0x2", + }, + }, + }, + }, + }, + Expected: []string{ + `<pool type="scsi">`, + ` <name>pool</name>`, + ` <source>`, + ` <adapter type="scsi_host">`, + ` <parentaddr unique_id="1">`, + ` <address domain="0x0000" bus="0x00" slot="0x1f" addr="0x2"></address>`, + ` </parentaddr>`, + ` </adapter>`, + ` </source>`, + `</pool>`, + }, + }, + { + Object: &StoragePool{ + Type: "fs", + Name: "pool", + Source: &StoragePoolSource{ + Adapter: &StoragePoolSourceAdapter{ + Type: "fc_host", + Parent: "scsi_parent", + WWNN: "20000000c9831b4b", + WWPN: "10000000c9831b4b", + }, + }, + }, + Expected: []string{ + `<pool type="fs">`, + ` <name>pool</name>`, + ` <source>`, + ` <adapter type="fc_host" parent="scsi_parent" wwnn="20000000c9831b4b" wwpn="10000000c9831b4b"></adapter>`, + ` </source>`, + `</pool>`, + }, + }, + { + Object: &StoragePool{ + Type: "dir", + Name: "pool", + Target: &StoragePoolTarget{ + Path: "/pool", + Permissions: &StoragePoolTargetPermissions{ + Owner: "1", + Group: "1", + Mode: "0744", + Label: "pool", + }, + }, + }, + Expected: []string{ + `<pool type="dir">`, + ` <name>pool</name>`, + ` <target>`, + ` <path>/pool</path>`, + ` <permissions>`, + ` <owner>1</owner>`, + ` <group>1</group>`, + ` <mode>0744</mode>`, + ` <label>pool</label>`, + ` </permissions>`, + ` </target>`, + `</pool>`, + }, + }, +} + +func TestStoragePool(t *testing.T) { + for _, test := range storagePoolTestData { + doc, err := xml.MarshalIndent(test.Object, "", " ") + if err != nil { + t.Fatal(err) + } + + expect := strings.Join(test.Expected, "\n") + + if string(doc) != expect { + t.Fatal("Bad xml:\n", string(doc), "\n does not match\n", expect, "\n") + } + } +} diff --git a/storage_vol.go b/storage_vol.go new file mode 100644 index 0000000..7d42b53 --- /dev/null +++ b/storage_vol.go @@ -0,0 +1,83 @@ +/* + * This file is part of the libvirt-go-xml project + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * Copyright (C) 2017 Red Hat, Inc. + * + */ + +package libvirtxml + +import "encoding/xml" + +type StorageVolumeMemory struct { + Unit string `xml:"unit,attr,omitempty"` + Value uint64 `xml:",chardata"` +} + +type StorageVolumeTargetPermissions struct { + Owner string `xml:"owner,omitempty"` + Group string `xml:"group,omitempty"` + Mode string `xml:"mode,omitempty"` + Label string `xml:"label,omitempty"` +} + +type StorageVolumeTargetFeature struct { + LazyRefcounts *struct{} `xml:"lazy_refcounts"` +} + +type StorageVolumeTargetFormat struct { + Type string `xml:"type,attr"` +} + +type StorageVolumeTargetTimestamps struct { + Atime string `xml:"atime"` + Mtime string `xml:"mtime"` + Ctime string `xml:"ctime"` +} + +type StorageVolumeTarget struct { + Path string `xml:"path,omitempty"` + Format *StorageVolumeTargetFormat `xml:"format"` + Permissions *StorageVolumeTargetPermissions `xml:"permissions"` + Timestamps *StorageVolumeTargetTimestamps `xml:"timestamps"` + Compat string `xml:"compat,omitempty"` + NoCOW *struct{} `xml:"nocow"` + Features []StorageVolumeTargetFeature `xml:"features"` + Encryption *StorageEncryption `xml:"encryption"` +} + +type StorageVolumeBackingStore struct { + Path string `xml:"path"` + Format *StorageVolumeTargetFormat `xml:"format"` + Permissions *StorageVolumeTargetPermissions `xml:"permissions"` +} + +type StorageVolume struct { + XMLName xml.Name `xml:"volume"` + Type string `xml:"type,attr,omitempty"` + Name string `xml:"name"` + Key string `xml:"key,omitempty"` + Allocation *StorageVolumeMemory `xml:"allocation"` + Capacity *StorageVolumeMemory `xml:"capacity"` + Physical *StorageVolumeMemory `xml:"physical"` + Target *StorageVolumeTarget `xml:"target"` + BackingStore *StorageVolumeBackingStore `xml:"backingStore"` +} diff --git a/storage_vol_test.go b/storage_vol_test.go new file mode 100644 index 0000000..e57e94d --- /dev/null +++ b/storage_vol_test.go @@ -0,0 +1,246 @@ +/* + * This file is part of the libvirt-go-xml project + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + * Copyright (C) 2017 Red Hat, Inc. + * + */ + +package libvirtxml + +import ( + "encoding/xml" + "strings" + "testing" +) + +var storageVolumeTestData = []struct { + Object *StorageVolume + Expected []string +}{ + { + Object: &StorageVolume{ + Type: "file", + Name: "file.img", + Key: "/file.img", + Allocation: &StorageVolumeMemory{ + Value: 0, + }, + + Capacity: &StorageVolumeMemory{ + Unit: "T", + Value: 1, + }, + }, + Expected: []string{ + `<volume type="file">`, + ` <name>file.img</name>`, + ` <key>/file.img</key>`, + ` <allocation>0</allocation>`, + ` <capacity unit="T">1</capacity>`, + `</volume>`, + }, + }, + { + Object: &StorageVolume{ + Type: "file", + Name: "file.img", + Target: &StorageVolumeTarget{ + Path: "/file.img", + Format: &StorageVolumeTargetFormat{ + Type: "qcow2", + }, + Permissions: &StorageVolumeTargetPermissions{ + Owner: "107", + Group: "107", + Mode: "0744", + Label: "image", + }, + Timestamps: &StorageVolumeTargetTimestamps{ + Atime: "1341933637.273190990", + Mtime: "1341930622.047245868", + Ctime: "1341930622.047245868", + }, + Compat: "1.1", + NoCOW: &struct{}{}, + Features: []StorageVolumeTargetFeature{ + StorageVolumeTargetFeature{ + LazyRefcounts: &struct{}{}, + }, + }, + }, + }, + Expected: []string{ + `<volume type="file">`, + ` <name>file.img</name>`, + ` <target>`, + ` <path>/file.img</path>`, + ` <format type="qcow2"></format>`, + ` <permissions>`, + ` <owner>107</owner>`, + ` <group>107</group>`, + ` <mode>0744</mode>`, + ` <label>image</label>`, + ` </permissions>`, + ` <timestamps>`, + ` <atime>1341933637.273190990</atime>`, + ` <mtime>1341930622.047245868</mtime>`, + ` <ctime>1341930622.047245868</ctime>`, + ` </timestamps>`, + ` <compat>1.1</compat>`, + ` <nocow></nocow>`, + ` <features>`, + ` <lazy_refcounts></lazy_refcounts>`, + ` </features>`, + ` </target>`, + `</volume>`, + }, + }, + { + Object: &StorageVolume{ + Type: "file", + Name: "file.img", + BackingStore: &StorageVolumeBackingStore{ + Path: "/master.img", + Format: &StorageVolumeTargetFormat{ + Type: "raw", + }, + Permissions: &StorageVolumeTargetPermissions{ + Owner: "107", + Group: "107", + Mode: "0744", + Label: "label", + }, + }, + }, + Expected: []string{ + `<volume type="file">`, + ` <name>file.img</name>`, + ` <backingStore>`, + ` <path>/master.img</path>`, + ` <format type="raw"></format>`, + ` <permissions>`, + ` <owner>107</owner>`, + ` <group>107</group>`, + ` <mode>0744</mode>`, + ` <label>label</label>`, + ` </permissions>`, + ` </backingStore>`, + `</volume>`, + }, + }, + { + Object: &StorageVolume{ + Name: "luks.img", + Capacity: &StorageVolumeMemory{ + Unit: "G", + Value: 5, + }, + Target: &StorageVolumeTarget{ + Path: "/luks.img", + Format: &StorageVolumeTargetFormat{ + Type: "raw", + }, + Encryption: &StorageEncryption{ + Format: "luks", + Secret: &StorageEncryptionSecret{ + Type: "passphrase", + UUID: "f52a81b2-424e-490c-823d-6bd4235bc572", + }, + }, + }, + }, + Expected: []string{ + `<volume>`, + ` <name>luks.img</name>`, + ` <capacity unit="G">5</capacity>`, + ` <target>`, + ` <path>/luks.img</path>`, + ` <format type="raw"></format>`, + ` <encryption format="luks">`, + ` <secret type="passphrase" uuid="f52a81b2-424e-490c-823d-6bd4235bc572"></secret>`, + ` </encryption>`, + ` </target>`, + `</volume>`, + }, + }, + { + Object: &StorageVolume{ + Name: "twofish", + Capacity: &StorageVolumeMemory{ + Unit: "G", + Value: 5, + }, + Target: &StorageVolumeTarget{ + Path: "/twofish.luks", + Format: &StorageVolumeTargetFormat{ + Type: "raw", + }, + Encryption: &StorageEncryption{ + Format: "luks", + Secret: &StorageEncryptionSecret{ + Type: "passphrase", + UUID: "f52a81b2-424e-490c-823d-6bd4235bc572", + }, + Cipher: &StorageEncryptionCipher{ + Name: "twofish", + Size: 256, + Mode: "cbc", + Hash: "sha256", + }, + Ivgen: &StorageEncryptionIvgen{ + Name: "plain64", + Hash: "sha256", + }, + }, + }, + }, + Expected: []string{ + `<volume>`, + ` <name>twofish</name>`, + ` <capacity unit="G">5</capacity>`, + ` <target>`, + ` <path>/twofish.luks</path>`, + ` <format type="raw"></format>`, + ` <encryption format="luks">`, + ` <secret type="passphrase" uuid="f52a81b2-424e-490c-823d-6bd4235bc572"></secret>`, + ` <cipher name="twofish" size="256" mode="cbc" hash="sha256"></cipher>`, + ` <ivgen name="plain64" hash="sha256"></ivgen>`, + ` </encryption>`, + ` </target>`, + `</volume>`, + }, + }, +} + +func TestStorageVolume(t *testing.T) { + for _, test := range storageVolumeTestData { + doc, err := xml.MarshalIndent(test.Object, "", " ") + if err != nil { + t.Fatal(err) + } + + expect := strings.Join(test.Expected, "\n") + + if string(doc) != expect { + t.Fatal("Bad xml:\n", string(doc), "\n does not match\n", expect, "\n") + } + } +} -- 2.11.0

2 1

[libvirt] [PATCH 0/2] Attempt, and fail, to fix build with Clang 3.9 / Linux
by Andrea Bolognani 10 Jan '17

10 Jan '17

Martin asked for this a while ago. I was surprised, at first, because according to my experience the only issue with Clang builds were a couple of test cases failing to run, but trying again on Clang 3.9 / Fedora rawhide caused me to run into at least three separate compilation issues. Two of them I take care with this series, even though I'm fairly unconvinced the bugs are actually in libvirt rather than Clang or glibc; more details in the relevant commit messages. The last one I was unable to work around in any sensible manner: when linking, clang fails with ../gnulib/lib/.libs/libgnu.a(mgetgroups.o): In function `realloc_groupbuf': .../libvirt/gnulib/lib/mgetgroups.c:45: undefined reference to `__muloti4' This seems to be a well-known issue[1] with no solution in sight; passing CFLAGS='-rtlib=compiler-rt' to configure allows the compilation to succeed but, yeah, way gross. So basically I've spent way too much time trying to get this to work and I'm stuck now, so I'm posting whatever I have so far in hope others will pitch in and help me out :) [1] https://llvm.org/bugs/show_bug.cgi?id=16404 Andrea Bolognani (2): util: Turn virFirewallAddRule() into a macro util: Cast to 'long double' when calling isnan() src/libvirt_private.syms | 1 - src/util/virfirewall.c | 23 ----------------------- src/util/virfirewall.h | 16 ++++++++++++---- src/util/virxml.c | 10 +++++----- 4 files changed, 17 insertions(+), 33 deletions(-) -- 2.11.0

4 11

[libvirt] [PATCH v2 0/5] Don't run whole sec driver in namespace
by Michal Privoznik 10 Jan '17

10 Jan '17

v2 of: https://www.redhat.com/archives/libvir-list/2016-December/msg00907.html diff to v1: - Pushed 1/6 from the original series as it was ACKed - Added more comments - Fixed return value of virSecurityDACTransactionAppend and virSecuritySELinuxTransactionAppend - Dropped ignore_value() around virThreadLocalSet() - Unset thread local just in transactionCommit APIs I have not implemented any rollback yet, but I've added comments where the implementation should go ;-) Michal Privoznik (5): security_dac: Resolve virSecurityDACSetOwnershipInternal const correctness security driver: Introduce transaction APIs security_dac: Implement transaction APIs security_selinux: Implement transaction APIs qemu: Use transactions from security driver src/libvirt_private.syms | 3 + src/qemu/qemu_driver.c | 25 ++-- src/qemu/qemu_security.c | 100 ++++--------- src/security/security_dac.c | 261 +++++++++++++++++++++++++++++++++- src/security/security_driver.h | 9 ++ src/security/security_manager.c | 68 +++++++++ src/security/security_manager.h | 7 +- src/security/security_selinux.c | 256 ++++++++++++++++++++++++++++++++- src/security/security_stack.c | 49 +++++++ src/storage/storage_backend.h | 2 +- src/storage/storage_backend_fs.c | 2 +- src/storage/storage_backend_gluster.c | 2 +- src/storage/storage_driver.c | 6 +- src/storage/storage_driver.h | 4 +- src/util/virstoragefile.c | 2 +- src/util/virstoragefile.h | 2 +- 16 files changed, 703 insertions(+), 95 deletions(-) -- 2.11.0

2 9

[libvirt] [PATCH v3 0/3] qemu: Use virtio-pci by default for mach-virt guests
by Andrea Bolognani 10 Jan '17

10 Jan '17

Changes from [v2]: * rename qemuDomainCountVirtioMMIODevices() to qemuDomainHasVirtioMMIODevices() and make it exit as soon as the first virtio-mmio device is encountered, as suggested by Laine * tweak test suite and note no new test cases are needed * add comments and user documentation * add release notes entry * can actually be merged now that all patches it builds on have been merged :) Changes from [v1]: * use virDomainDeviceInfoIterate(), as suggested by Martin and Laine, which results in cleaner and more robust code [v1] https://www.redhat.com/archives/libvir-list/2016-October/msg00988.html [v2] https://www.redhat.com/archives/libvir-list/2016-October/msg01042.html Andrea Bolognani (3): qemu: Use virtio-pci by default for mach-virt guests docs: Document virtio-mmio by default for mach-virt guests NEWS: Update for virtio-pci by default for mach-virt guests docs/formatdomain.html.in | 8 +++- docs/news.html.in | 6 +++ src/qemu/qemu_domain_address.c | 51 ++++++++++++++++++++-- ...l2argv-aarch64-virt-2.6-virtio-pci-default.args | 14 +++--- .../qemuxml2argv-aarch64-virtio-pci-default.args | 17 +++++--- .../qemuxml2argv-aarch64-virtio-pci-default.xml | 3 -- tests/qemuxml2argvtest.c | 1 + .../qemuxml2xmlout-aarch64-virtio-pci-default.xml | 40 ++++++++++++++--- tests/qemuxml2xmltest.c | 1 + 9 files changed, 119 insertions(+), 22 deletions(-) -- 2.7.4

2 8

[libvirt] [PATCH 0/4] qemu: Don't leave temp mount dirs around
by Michal Privoznik 10 Jan '17

10 Jan '17

After yesterday's patches [1] and fixing BZ [2] with them, the comment 4 got me thinking: why leave /var/run/libvirt/$domain.pts and similar temporary directories around? We need them just for a very short time. After that we might as well remove them. 1: https://www.redhat.com/archives/libvir-list/2017-January/msg00073.html 2: https://bugzilla.redhat.com/show_bug.cgi?id=1406837 Michal Privoznik (4): qemuDomainCreateNamespace: s/unlink/rmdir/ qemuDomainGetPreservedMounts: Do not special case /dev qemuDomainCreateNamespace: move mkdir to qemuDomainBuildNamespace qemu: Drop qemuDomainDeleteNamespace src/qemu/qemu_domain.c | 126 +++++++++++++----------------------------------- src/qemu/qemu_domain.h | 3 -- src/qemu/qemu_process.c | 2 - 3 files changed, 34 insertions(+), 97 deletions(-) -- 2.11.0

2 5

[libvirt] [PATCH go] Add support for perf cache_l1d event
by Daniel P. Berrange 10 Jan '17

10 Jan '17

Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- Pushed as a build fix connect.go | 6 ++++++ domain.go | 6 ++++++ domain_compat.h | 4 ++++ 3 files changed, 16 insertions(+) diff --git a/connect.go b/connect.go index 35eb7fd..41c54b1 100644 --- a/connect.go +++ b/connect.go @@ -2253,6 +2253,8 @@ type DomainStatsPerf struct { StalledCyclesBackend uint64 RefCpuCyclesSet bool RefCpuCycles uint64 + CacheL1DSet bool + CacheL1D uint64 } func getDomainStatsPerfFieldInfo(params *DomainStatsPerf) map[string]typedParamsFieldInfo { @@ -2309,6 +2311,10 @@ func getDomainStatsPerfFieldInfo(params *DomainStatsPerf) map[string]typedParams set: &params.RefCpuCyclesSet, ul: &params.RefCpuCycles, }, + "perf.cache_l1d": typedParamsFieldInfo{ + set: &params.CacheL1DSet, + ul: &params.CacheL1D, + }, } } diff --git a/domain.go b/domain.go index 515138a..3e66dab 100644 --- a/domain.go +++ b/domain.go @@ -3183,6 +3183,8 @@ type DomainPerfEvents struct { StalledCyclesBackend bool RefCpuCyclesSet bool RefCpuCycles bool + CacheL1DSet bool + CacheL1D bool } /* Remember to also update DomainStatsPerf in connect.go when adding to the stuct above */ @@ -3241,6 +3243,10 @@ func getDomainPerfEventsFieldInfo(params *DomainPerfEvents) map[string]typedPara set: &params.RefCpuCyclesSet, b: &params.RefCpuCycles, }, + C.VIR_PERF_PARAM_CACHE_L1D: typedParamsFieldInfo{ + set: &params.CacheL1DSet, + b: &params.CacheL1D, + }, } } diff --git a/domain_compat.h b/domain_compat.h index 14072d0..4da562b 100644 --- a/domain_compat.h +++ b/domain_compat.h @@ -45,6 +45,10 @@ #define VIR_PERF_PARAM_REF_CPU_CYCLES "ref_cpu_cycles" #endif +#ifndef VIR_PERF_PARAM_CACHE_L1D +#define VIR_PERF_PARAM_CACHE_L1D "cache_l1d" +#endif + #ifndef VIR_DOMAIN_EVENT_ID_METADATA_CHANGE #define VIR_DOMAIN_EVENT_ID_METADATA_CHANGE 23 #endif -- 2.9.3

1 0

[libvirt] [libvirt-python][PATCH] examples: Update event-test.py
by Michal Privoznik 10 Jan '17

10 Jan '17

With recent changes there are new events known to libvirt. Reflect those changes in our event-test.py example script. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- examples/event-test.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/examples/event-test.py b/examples/event-test.py index e43a2f8..d5af33c 100755 --- a/examples/event-test.py +++ b/examples/event-test.py @@ -538,6 +538,9 @@ def myDomainEventJobCompletedCallback(conn, dom, params, opaque): def myDomainEventDeviceRemovalFailedCallback(conn, dom, dev, opaque): print("myDomainEventDeviceRemovalFailedCallback: Domain %s(%s) failed to remove device: %s" % ( dom.name(), dom.ID(), dev)) +def myDomainEventMetadataChangeCallback(conn, dom, mtype, nsuri, opaque): + print("myDomainEventMetadataChangeCallback: Domain %s(%s) changed metadata mtype=%d nsuri=%s" % ( + dom.name(), dom.ID(), mtype, nsuri)) ########################################################################## # Network events @@ -600,6 +603,23 @@ def myNodeDeviceEventLifecycleCallback(conn, dev, event, detail, opaque): def myNodeDeviceEventUpdateCallback(conn, dev, opaque): print("myNodeDeviceEventUpdateCallback: Node device %s" % dev.name()) +########################################################################## +# Secret events +########################################################################## +def secretEventToString(event): + secretEventStrings = ( "Defined", + "Undefined", + ) + return secretEventStrings[event] + +def mySecretEventLifecycleCallback(conn, secret, event, detail, opaque): + print("mySecretEventLifecycleCallback: Secret %s %s %d" % (secret.UUIDString(), + secretEventToString(event), + detail)) + +def mySecretEventValueChanged(conn, secret, opaque): + print("mySecretEventValueChanged: Secret %s" % secret.UUIDString()) + ########################################################################## # Set up and run the program ########################################################################## @@ -689,6 +709,7 @@ def main(): vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_MIGRATION_ITERATION, myDomainEventMigrationIteration, None) vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_JOB_COMPLETED, myDomainEventJobCompletedCallback, None) vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_DEVICE_REMOVAL_FAILED, myDomainEventDeviceRemovalFailedCallback, None) + vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_METADATA_CHANGE, myDomainEventMetadataChangeCallback, None) vc.networkEventRegisterAny(None, libvirt.VIR_NETWORK_EVENT_ID_LIFECYCLE, myNetworkEventLifecycleCallback, None) @@ -698,6 +719,9 @@ def main(): vc.nodeDeviceEventRegisterAny(None, libvirt.VIR_NODE_DEVICE_EVENT_ID_LIFECYCLE, myNodeDeviceEventLifecycleCallback, None) vc.nodeDeviceEventRegisterAny(None, libvirt.VIR_NODE_DEVICE_EVENT_ID_UPDATE, myNodeDeviceEventUpdateCallback, None) + vc.secretEventRegisterAny(None, libvirt.VIR_SECRET_EVENT_ID_LIFECYCLE, mySecretEventLifecycleCallback, None) + vc.secretEventRegisterAny(None, libvirt.VIR_SECRET_EVENT_ID_VALUE_CHANGED, mySecretEventValueChanged, None) + vc.setKeepAlive(5, 3) # The rest of your app would go here normally, but for sake -- 2.11.0

2 1

[libvirt] [PATCH python 0/2] Add support for new events
by Daniel P. Berrange 10 Jan '17

10 Jan '17

Daniel P. Berrange (2): Add support for secret event APIs Add support for domain metadata change event generator.py | 2 + libvirt-override-virConnect.py | 52 +++++++++ libvirt-override.c | 260 +++++++++++++++++++++++++++++++++++++++++ sanitytest.py | 3 + 4 files changed, 317 insertions(+) -- 2.9.3

2 5

[libvirt] [libvirt-perl PATCH] Add perf cache_l1d event
by John Ferlan 10 Jan '17

10 Jan '17

Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Daniel - feel free to add to the pile of changes you posted if it's easier. Changes | 1 + Virt.xs | 1 + lib/Sys/Virt/Domain.pm | 6 ++++++ 3 files changed, 8 insertions(+) diff --git a/Changes b/Changes index 672b566..2a13f8f 100644 --- a/Changes +++ b/Changes @@ -10,6 +10,7 @@ Revision history for perl module Sys::Virt - Add PERF_PARAM_STALLED_CYCLES_BACKEND constant - Add PERF_PARAM_REF_CPU_CYCLES constant - Add virStorageVolGetInfoFlags & associated constants + - Add PERF_PARAM_CACHE_L1D constant 2.5.0 2016-12-05 diff --git a/Virt.xs b/Virt.xs index 1eb94a8..8fbbc4a 100644 --- a/Virt.xs +++ b/Virt.xs @@ -8313,6 +8313,7 @@ BOOT: REGISTER_CONSTANT_STR(VIR_PERF_PARAM_STALLED_CYCLES_FRONTEND, PERF_PARAM_STALLED_CYCLES_FRONTEND); REGISTER_CONSTANT_STR(VIR_PERF_PARAM_STALLED_CYCLES_BACKEND, PERF_PARAM_STALLED_CYCLES_BACKEND); REGISTER_CONSTANT_STR(VIR_PERF_PARAM_REF_CPU_CYCLES, PERF_PARAM_REF_CPU_CYCLES); + REGISTER_CONSTANT_STR(VIR_PERF_PARAM_CACHE_L1D, PERF_PARAM_CACHE_L1D); REGISTER_CONSTANT_STR(VIR_DOMAIN_BANDWIDTH_IN_AVERAGE, BANDWIDTH_IN_AVERAGE); REGISTER_CONSTANT_STR(VIR_DOMAIN_BANDWIDTH_IN_PEAK, BANDWIDTH_IN_PEAK); diff --git a/lib/Sys/Virt/Domain.pm b/lib/Sys/Virt/Domain.pm index bd59139..2133a30 100644 --- a/lib/Sys/Virt/Domain.pm +++ b/lib/Sys/Virt/Domain.pm @@ -2809,6 +2809,12 @@ frequency scaling by applications running on the platform. It corresponds to the "perf.ref_cpu_cycles" field in the *Stats APIs. +=item Sys::Virt::Domain::PERF_PARAM_CACHE_L1D +The cache_l1d perf event counter which can be used to +measure the count of total level 1 data cache by applications +running on the platform. It corresponds to the "perf.cache_l1d" +field in the *Stats APIs. + =back =head2 VCPU FLAGS -- 2.7.4

2 1

[libvirt] [V2]RFC for support cache tune in libvirt
by Qiao, Liyong 10 Jan '17

10 Jan '17

Hi folks I’v send the V1 version RFC early this week, but no reply yet. Thanks Qiaowei for the comment, I’v change the RFC much more libvirt specify, please help to comments on. ##Propose Changes #Libvirtd configure changes Add a new configure option cache_allocation_ratio to libvirtd, which let libvirt to allocate how many cache to domains. Default is 0.5 Eg. On a host which has 55M cache, libvirt can allocate 55M * cache_allocation_ratio cache to domains ## Virsh command line changes: NAME cachetune - control or query domain cache allocation SYNOPSIS cachetune <domain> [--enabled true/false] [--type <type>][--size <number>] [--config] [--live] [--current] DESCRIPTION Allocate cache usage for domain. OPTIONS [--domain] <string> domain name, id or uuid --enabled <true/false> enable cache allocation --type <string> cache allocations type, support l3/l2 etc --size <number> the cache size in KB --config affect next boot --live affect running domain --current affect current domain This will allow libvirt to allocate specify type l3 cache for a domain Domain xml changes: <cachetune> <enabled=’yes’, size=4096, actual_size=4680,type=’l3’> <enabled=’no’, size=256, actual_size=0, type=’l2’> </cachetune> For more information about the detail design, please refer https://www.redhat.com/archives/libvir-list/2016-December/msg01011.html CAT intro: https://software.intel.com/en-us/articles/software-enabling-for-cache-alloc… Best Regards Eli Qiao(乔立勇）OpenStack Core team OTC Intel. --

2 2

[libvirt] [PATCH] util: fix domain object leaks on closecallbacks
by Wang King 10 Jan '17

10 Jan '17

From: wangjing <king.wang(a)huawei.com> The virCloseCallbacksSet method increase object reference for VM, and decrease object reference in virCloseCallbacksUnset. But VM UUID will be deleted from closecallbacks list in virCloseCallbacksRun when connection disconnected, and then object reference cannot be decreased by virCloseCallbacksUnset in callback functions. Signed-off-by: Wang King <king.wang(a)huawei.com> --- src/util/virclosecallbacks.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/src/util/virclosecallbacks.c b/src/util/virclosecallbacks.c index 891a92b..26d5075 100644 --- a/src/util/virclosecallbacks.c +++ b/src/util/virclosecallbacks.c @@ -300,7 +300,9 @@ virCloseCallbacksGetForConn(virCloseCallbacksPtr closeCallbacks, data.list = list; data.oom = false; + virObjectLock(closeCallbacks); virHashForEach(closeCallbacks->list, virCloseCallbacksGetOne, &data); + virObjectUnlock(closeCallbacks); if (data.oom) { VIR_FREE(list->entries); @@ -329,22 +331,15 @@ virCloseCallbacksRun(virCloseCallbacksPtr closeCallbacks, * them all from the hash. At that point we can release * the lock and run the callbacks safely. */ - virObjectLock(closeCallbacks); list = virCloseCallbacksGetForConn(closeCallbacks, conn); if (!list) return; for (i = 0; i < list->nentries; i++) { - char uuidstr[VIR_UUID_STRING_BUFLEN]; - virUUIDFormat(list->entries[i].uuid, uuidstr); - virHashRemoveEntry(closeCallbacks->list, uuidstr); - } - virObjectUnlock(closeCallbacks); - - for (i = 0; i < list->nentries; i++) { virDomainObjPtr vm; + virDomainObjPtr dom; - if (!(vm = virDomainObjListFindByUUID(domains, + if (!(vm = virDomainObjListFindByUUIDRef(domains, list->entries[i].uuid))) { char uuidstr[VIR_UUID_STRING_BUFLEN]; virUUIDFormat(list->entries[i].uuid, uuidstr); @@ -352,10 +347,20 @@ virCloseCallbacksRun(virCloseCallbacksPtr closeCallbacks, continue; } - vm = list->entries[i].callback(vm, conn, opaque); - if (vm) - virObjectUnlock(vm); + dom = list->entries[i].callback(vm, conn, opaque); + if (dom) + virObjectUnlock(dom); + virObjectUnref(vm); } + + virObjectLock(closeCallbacks); + for (i = 0; i < list->nentries; i++) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(list->entries[i].uuid, uuidstr); + virHashRemoveEntry(closeCallbacks->list, uuidstr); + } + virObjectUnlock(closeCallbacks); + VIR_FREE(list->entries); VIR_FREE(list); } -- 2.8.3

2 2

[libvirt] [PATCH v3] libxl: define a per-domain logger.
by Cédric Bosdonnat 10 Jan '17

10 Jan '17

libxl doesn't provide a way to write one log for each domain. Thus we need to demux the messages. If our logger doesn't know to which domain to attribute a message, then it will write it to the default log file. Starting with Xen 4.9 (commit f9858025 and following), libxl will write the domain ID in an easy to grab manner. The logger introduced by this commit will use it to demux the libxl log messages. Thanks to the default log file, this logger will also work with older versions of Xen. --- * v3: add JSON-formatted libxl_domain_config to the newly opened file when creating a guest. * v2: addressed Jim's comments src/Makefile.am | 1 + src/libxl/libxl_conf.c | 38 +------- src/libxl/libxl_conf.h | 4 +- src/libxl/libxl_domain.c | 7 ++ src/libxl/libxl_driver.c | 2 + src/libxl/libxl_logger.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++ src/libxl/libxl_logger.h | 40 ++++++++ 7 files changed, 286 insertions(+), 37 deletions(-) create mode 100644 src/libxl/libxl_logger.c create mode 100644 src/libxl/libxl_logger.h diff --git a/src/Makefile.am b/src/Makefile.am index 0dd2faaf6..58e3108a6 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -851,6 +851,7 @@ LIBXL_DRIVER_SOURCES = \ libxl/libxl_capabilities.c libxl/libxl_capabilities.h \ libxl/libxl_domain.c libxl/libxl_domain.h \ libxl/libxl_driver.c libxl/libxl_driver.h \ + libxl/libxl_logger.c libxl/libxl_logger.h \ libxl/libxl_migration.c libxl/libxl_migration.h UML_DRIVER_SOURCES = \ diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index b569ddad8..ac83b51c7 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -76,9 +76,7 @@ libxlDriverConfigDispose(void *obj) virObjectUnref(cfg->caps); libxl_ctx_free(cfg->ctx); - xtl_logger_destroy(cfg->logger); - if (cfg->logger_file) - VIR_FORCE_FCLOSE(cfg->logger_file); + libxlLoggerFree(cfg->logger); VIR_FREE(cfg->configDir); VIR_FREE(cfg->autostartDir); @@ -1356,8 +1354,6 @@ libxlDriverConfigPtr libxlDriverConfigNew(void) { libxlDriverConfigPtr cfg; - char *log_file = NULL; - xentoollog_level log_level = XTL_DEBUG; char ebuf[1024]; unsigned int free_mem; @@ -1386,9 +1382,6 @@ libxlDriverConfigNew(void) if (VIR_STRDUP(cfg->channelDir, LIBXL_CHANNEL_DIR) < 0) goto error; - if (virAsprintf(&log_file, "%s/libxl-driver.log", cfg->logDir) < 0) - goto error; - if (virFileMakePath(cfg->logDir) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("failed to create log dir '%s': %s"), @@ -1397,37 +1390,13 @@ libxlDriverConfigNew(void) goto error; } - if ((cfg->logger_file = fopen(log_file, "a")) == NULL) { - VIR_ERROR(_("Failed to create log file '%s': %s"), - log_file, virStrerror(errno, ebuf, sizeof(ebuf))); - goto error; - } - VIR_FREE(log_file); - - switch (virLogGetDefaultPriority()) { - case VIR_LOG_DEBUG: - log_level = XTL_DEBUG; - break; - case VIR_LOG_INFO: - log_level = XTL_INFO; - break; - case VIR_LOG_WARN: - log_level = XTL_WARN; - break; - case VIR_LOG_ERROR: - log_level = XTL_ERROR; - break; - } - - cfg->logger = - (xentoollog_logger *)xtl_createlogger_stdiostream(cfg->logger_file, - log_level, XTL_STDIOSTREAM_SHOW_DATE); + cfg->logger = libxlLoggerNew(cfg->logDir, virLogGetDefaultPriority()); if (!cfg->logger) { VIR_ERROR(_("cannot create logger for libxenlight, disabling driver")); goto error; } - if (libxl_ctx_alloc(&cfg->ctx, LIBXL_VERSION, 0, cfg->logger)) { + if (libxl_ctx_alloc(&cfg->ctx, LIBXL_VERSION, 0, (xentoollog_logger *)cfg->logger)) { VIR_ERROR(_("cannot initialize libxenlight context, probably not " "running in a Xen Dom0, disabling driver")); goto error; @@ -1478,7 +1447,6 @@ libxlDriverConfigNew(void) return cfg; error: - VIR_FREE(log_file); virObjectUnref(cfg); return NULL; } diff --git a/src/libxl/libxl_conf.h b/src/libxl/libxl_conf.h index 851f3afb4..69d78851a 100644 --- a/src/libxl/libxl_conf.h +++ b/src/libxl/libxl_conf.h @@ -41,6 +41,7 @@ # include "locking/lock_manager.h" # include "virfirmware.h" # include "libxl_capabilities.h" +# include "libxl_logger.h" # define LIBXL_DRIVER_NAME "xenlight" # define LIBXL_VNC_PORT_MIN 5900 @@ -74,8 +75,7 @@ struct _libxlDriverConfig { unsigned int version; /* log stream for driver-wide libxl ctx */ - FILE *logger_file; - xentoollog_logger *logger; + libxlLoggerPtr logger; /* libxl ctx for driver wide ops; getVersion, getNodeInfo, ... */ libxl_ctx *ctx; diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 5cde576ef..fbe7ee5ff 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -809,6 +809,8 @@ libxlDomainCleanup(libxlDriverPrivatePtr driver, VIR_FREE(xml); } + libxlLoggerCloseFile(cfg->logger, vm->def->id); + virDomainObjRemoveTransientDef(vm); virObjectUnref(cfg); } @@ -1127,6 +1129,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, libxl_asyncprogress_how aop_console_how; libxl_domain_restore_params params; unsigned int hostdev_flags = VIR_HOSTDEV_SP_PCI; + char *config_json = NULL; #ifdef LIBXL_HAVE_PVUSB hostdev_flags |= VIR_HOSTDEV_SP_USB; @@ -1290,6 +1293,9 @@ libxlDomainStart(libxlDriverPrivatePtr driver, * be cleaned up if there are any subsequent failures. */ vm->def->id = domid; + config_json = libxl_domain_config_to_json(cfg->ctx, &d_config); + + libxlLoggerOpenFile(cfg->logger, domid, vm->def->name, config_json); /* Always enable domain death events */ if (libxl_evenable_domain_death(cfg->ctx, vm->def->id, 0, &priv->deathW)) @@ -1366,6 +1372,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, cleanup: libxl_domain_config_dispose(&d_config); + VIR_FREE(config_json); VIR_FREE(dom_xml); VIR_FREE(managed_save_path); virDomainDefFree(def); diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 7e5d9b69e..325636c9a 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -406,6 +406,8 @@ libxlReconnectDomain(virDomainObjPtr vm, /* Update domid in case it changed (e.g. reboot) while we were gone? */ vm->def->id = d_info.domid; + libxlLoggerOpenFile(cfg->logger, vm->def->id, vm->def->name, NULL); + /* Update hostdev state */ if (virHostdevUpdateActiveDomainDevices(hostdev_mgr, LIBXL_DRIVER_NAME, vm->def, hostdev_flags) < 0) diff --git a/src/libxl/libxl_logger.c b/src/libxl/libxl_logger.c new file mode 100644 index 000000000..9ca745a93 --- /dev/null +++ b/src/libxl/libxl_logger.c @@ -0,0 +1,231 @@ +/* + * libxl_logger.c: libxl logger implementation + * + * Copyright (c) 2016 SUSE LINUX Products GmbH, Nuernberg, Germany. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Cédric Bosdonnat <cbosdonnat(a)suse.com> + */ +#include <config.h> + +#include <string.h> +#include <libxl.h> + +#include "internal.h" +#include "libxl_logger.h" +#include "util/viralloc.h" +#include "util/virerror.h" +#include "util/virfile.h" +#include "util/virhash.h" +#include "util/virstring.h" +#include "util/virtime.h" + +#define VIR_FROM_THIS VIR_FROM_LIBXL + +VIR_LOG_INIT("libxl.libxl_logger"); + +typedef struct xentoollog_logger_libvirt xentoollog_logger_libvirt; + +struct xentoollog_logger_libvirt { + xentoollog_logger vtable; + xentoollog_level minLevel; + const char *logDir; + + /* map storing the opened fds: "domid" -> FILE* */ + virHashTablePtr files; + FILE *defaultLogFile; +}; + +static void +libxlLoggerFileFree(void *payload, const void *key ATTRIBUTE_UNUSED) +{ + FILE *file = payload; + VIR_FORCE_FCLOSE(file); + file = NULL; +} + +ATTRIBUTE_FMT_PRINTF(5, 0) static void +libvirt_vmessage(xentoollog_logger *logger_in, + xentoollog_level level, + int errnoval, + const char *context, + const char *format, + va_list args) +{ + xentoollog_logger_libvirt *lg = (xentoollog_logger_libvirt *)logger_in; + FILE *logFile = lg->defaultLogFile; + char timestamp[VIR_TIME_STRING_BUFLEN]; + char *message = NULL; + char *start, *end; + char ebuf[1024]; + + VIR_DEBUG("libvirt_vmessage: context='%s' format='%s'", context, format); + + if (level < lg->minLevel) + return; + + if (virVasprintf(&message, format, args) < 0) + return; + + /* Should we print to a domain-specific log file? */ + if ((start = strstr(message, ": Domain ")) && + (end = strstr(start + 9, ":"))) { + FILE *domainLogFile; + + VIR_DEBUG("Found domain log message"); + + start = start + 9; + *end = '\0'; + + domainLogFile = virHashLookup(lg->files, start); + if (domainLogFile) + logFile = domainLogFile; + + *end = ':'; + } + + /* Do the actual print to the log file */ + if (virTimeStringNowRaw(timestamp) < 0) + timestamp[0] = '\0'; + + fprintf(logFile, "%s: ", timestamp); + if (context) + fprintf(logFile, "%s: ", context); + + fprintf(logFile, "%s", message); + + if (errnoval >= 0) + fprintf(logFile, ": %s", virStrerror(errnoval, ebuf, sizeof(ebuf))); + + fputc('\n', logFile); + fflush(logFile); + + VIR_FREE(message); +} + +static void +libvirt_progress(xentoollog_logger *logger_in ATTRIBUTE_UNUSED, + const char *context ATTRIBUTE_UNUSED, + const char *doingwhat ATTRIBUTE_UNUSED, + int percent ATTRIBUTE_UNUSED, + unsigned long done ATTRIBUTE_UNUSED, + unsigned long total ATTRIBUTE_UNUSED) +{ + /* This function purposedly does nothing: it's no logging info */ +} + +static void +libvirt_destroy(xentoollog_logger *logger_in) +{ + xentoollog_logger_libvirt *lg = (xentoollog_logger_libvirt*)logger_in; + VIR_FREE(lg); +} + + +libxlLoggerPtr libxlLoggerNew(const char *logDir, + virLogPriority minLevel) +{ + xentoollog_logger_libvirt logger; + libxlLoggerPtr logger_out = NULL; + char *path = NULL; + + switch (minLevel) { + case VIR_LOG_DEBUG: + logger.minLevel = XTL_DEBUG; + break; + case VIR_LOG_INFO: + logger.minLevel = XTL_INFO; + break; + case VIR_LOG_WARN: + logger.minLevel = XTL_WARN; + break; + case VIR_LOG_ERROR: + logger.minLevel = XTL_ERROR; + break; + } + logger.logDir = logDir; + + if ((logger.files = virHashCreate(3, libxlLoggerFileFree)) == NULL) + return NULL; + + if (virAsprintf(&path, "%s/libxl-driver.log", logDir) < 0) + goto error; + + if ((logger.defaultLogFile = fopen(path, "a")) == NULL) + goto error; + + logger_out = XTL_NEW_LOGGER(libvirt, logger); + + cleanup: + VIR_FREE(path); + return logger_out; + + error: + virHashFree(logger.files); + goto cleanup; +} + +void libxlLoggerFree(libxlLoggerPtr logger) +{ + xentoollog_logger *xtl_logger = (xentoollog_logger*)logger; + if (logger->defaultLogFile) + VIR_FORCE_FCLOSE(logger->defaultLogFile); + virHashFree(logger->files); + xtl_logger_destroy(xtl_logger); +} + +void libxlLoggerOpenFile(libxlLoggerPtr logger, int id, + const char *name, + const char *domain_config) +{ + char *path = NULL; + FILE *logFile = NULL; + char *domidstr = NULL; + char ebuf[1024]; + + if (virAsprintf(&path, "%s/%s.log", logger->logDir, name) < 0 || + virAsprintf(&domidstr, "%d", id) < 0) + goto cleanup; + + if (!(logFile = fopen(path, "a"))) { + VIR_WARN("Failed to open log file %s: %s", + path, virStrerror(errno, ebuf, sizeof(ebuf))); + goto cleanup; + } + ignore_value(virHashAddEntry(logger->files, domidstr, logFile)); + + /* domain_config is non NULL only when starting a new domain */ + if (domain_config) { + fprintf(logFile, "Domain start: %s\n", domain_config); + fflush(logFile); + } + + cleanup: + VIR_FREE(path); + VIR_FREE(domidstr); +} + +void libxlLoggerCloseFile(libxlLoggerPtr logger, int id) +{ + char *domidstr = NULL; + if (virAsprintf(&domidstr, "%d", id) < 0) + return; + + ignore_value(virHashRemoveEntry(logger->files, domidstr)); + + VIR_FREE(domidstr); +} diff --git a/src/libxl/libxl_logger.h b/src/libxl/libxl_logger.h new file mode 100644 index 000000000..92b222578 --- /dev/null +++ b/src/libxl/libxl_logger.h @@ -0,0 +1,40 @@ +/* + * libxl_logger.h: libxl logger implementation + * + * Copyright (c) 2016 SUSE LINUX Products GmbH, Nuernberg, Germany. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Cédric Bosdonnat <cbosdonnat(a)suse.com> + */ + +#ifndef __LIBXL_LOGGER_H +# define __LIBXL_LOGGER_H + +# include "util/virlog.h" + +typedef struct xentoollog_logger_libvirt libxlLogger; +typedef libxlLogger *libxlLoggerPtr; + +libxlLoggerPtr libxlLoggerNew(const char *logDir, + virLogPriority minLevel); +void libxlLoggerFree(libxlLoggerPtr logger); + +void libxlLoggerOpenFile(libxlLoggerPtr logger, int id, const char *name, + const char *domain_config); +void libxlLoggerCloseFile(libxlLoggerPtr logger, int id); + +#endif /* __LIBXL_LOGGER_H */ -- 2.11.0

2 1