[libvirt] [PATCH v2 0/4] tests: json: Improve testing of parsing and formatting
by Peter Krempa
Now rebased to master after changes to the virjsontest.
Purpose of this series was to test json nested in json. While it won't be used,
it may be worth to test that our parser and formatter handles escaping/nesting
properly.
Peter Krempa (4):
tests: virjson: Modify logic in testJSONFromString
tests: virjson: Remove spaces from 'very-hard' parsing example
tests: virjson: Test formatting along with parsing of JSON objects
tests: virjson: Test string escaping
tests/virjsontest.c | 194 +++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 145 insertions(+), 49 deletions(-)
--
2.12.2
7 years, 5 months
[libvirt] [PATCH 00/11] introduce virFileCache and use it for qemu capabilities
by Pavel Hrdina
Pavel Hrdina (11):
util/virhash: add name parameter to virHashSearch
util: introduce virFileCache
tests: add virfilecachetest
qemu: move libvirt ctime and version into _virQEMUCaps struct
qemu: move virQEMUCapsIsValid before its usage and make it static
qemu: move libvirt ctime and version check into virQEMUCapsIsValid
qemu: don't pass qemuctime into virQEMUCapsIsValid
qemu: separate virQEMUCapsInitCached out of
virQEMUCapsNewForBinaryInternal
qemu: introduce struct _virQEMUCapsCachePriv
qemu: switch QEMU capabilities to use virFileCache
qemu: privatize _virQEMUCapsCachePriv struct
po/POTFILES.in | 1 +
src/Makefile.am | 1 +
src/conf/virdomainobjlist.c | 2 +-
src/conf/virnetworkobj.c | 4 +-
src/conf/virsecretobj.c | 2 +-
src/libvirt_private.syms | 9 +
src/qemu/qemu_capabilities.c | 517 +++++++--------------
src/qemu/qemu_capabilities.h | 22 +-
src/qemu/qemu_capspriv.h | 18 +-
src/qemu/qemu_conf.h | 3 +-
src/qemu/qemu_driver.c | 2 +-
src/util/virfilecache.c | 391 ++++++++++++++++
src/util/virfilecache.h | 136 ++++++
src/util/virhash.c | 11 +-
src/util/virhash.h | 2 +-
src/xen/xm_internal.c | 5 +-
tests/Makefile.am | 12 +
tests/qemucapabilitiestest.c | 4 +-
tests/qemucapsprobe.c | 2 +-
tests/testutilsqemu.c | 14 +-
tests/testutilsqemu.h | 3 +-
...a15b1658aa16923e46497dd8deeb6be287ddb0ca0.cache | 1 +
...ae4bc3a28e75bc3e262757001e8b953580f5e75ef.cache | 1 +
tests/virfilecachetest.c | 238 ++++++++++
tests/virhashtest.c | 2 +-
25 files changed, 1008 insertions(+), 395 deletions(-)
create mode 100644 src/util/virfilecache.c
create mode 100644 src/util/virfilecache.h
create mode 100644 tests/virfilecachedata/5f3154560c130108b282a2aa15b1658aa16923e46497dd8deeb6be287ddb0ca0.cache
create mode 120000 tests/virfilecachedata/9ca150bf3119b75dcac8e8bae4bc3a28e75bc3e262757001e8b953580f5e75ef.cache
create mode 100644 tests/virfilecachetest.c
--
2.13.2
7 years, 5 months
[libvirt] [PATCH] qemu: avoid deadlock on domain object enter monitor fail
by Wang King
Should be followed with qemuDomainObjExitMonitor only if
qemuDomainObjEnterMonitorAsync returns 0.
---
src/qemu/qemu_migration.c | 2 +-
src/qemu/qemu_process.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c
index c23fffe..dc56263 100644
--- a/src/qemu/qemu_migration.c
+++ b/src/qemu/qemu_migration.c
@@ -108,7 +108,7 @@ qemuMigrationCheckTLSCreds(virQEMUDriverPtr driver,
qemuMonitorMigrationParams migParams = { 0 };
if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0)
- goto cleanup;
+ return -1;
if (qemuMonitorGetMigrationParams(priv->mon, &migParams) < 0)
goto cleanup;
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index c2c3e8b..8d3cfe0 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -4349,7 +4349,7 @@ qemuProcessSetupBalloon(virQEMUDriverPtr driver,
return 0;
if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0)
- goto cleanup;
+ return -1;
if (vm->def->memballoon->period)
qemuMonitorSetMemoryStatsPeriod(priv->mon, vm->def->memballoon,
--
2.8.3
7 years, 5 months
[libvirt] [PATCH] security: apparmor: Properly link with storage driver in helper program
by Peter Krempa
The refactor to split up storage driver into modules broke the apparmor
helper program, since that did not initialize the storage driver
properly and thus detection of the backing chain could not work.
Register the storage driver backends explicitly. Unfortunately it's now
necessary to link with the full storage driver to satisfy dependencies
of the loadable modules.
---
src/Makefile.am | 2 +-
src/security/virt-aa-helper.c | 8 ++++++++
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/src/Makefile.am b/src/Makefile.am
index 399d031dd..e637dfd91 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -3281,7 +3281,7 @@ virt_aa_helper_LDADD = \
libvirt.la \
libvirt_conf.la \
libvirt_util.la \
- libvirt_driver_storage_impl.la \
+ libvirt_driver_storage.la \
../gnulib/lib/libgnu.la
if WITH_DTRACE_PROBES
virt_aa_helper_LDADD += libvirt_probes.lo
diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c
index 695272076..a751d6deb 100644
--- a/src/security/virt-aa-helper.c
+++ b/src/security/virt-aa-helper.c
@@ -56,6 +56,7 @@
#include "virgettext.h"
#include "storage/storage_source.h"
+#include "storage/storage_backend.h"
#define VIR_FROM_THIS VIR_FROM_SECURITY
@@ -926,6 +927,11 @@ get_files(vahControl * ctl)
goto cleanup;
}
+ if (virStorageBackendDriversRegister(false) < 0) {
+ vah_error(ctl, 0, _("failed to register storage driver backend"));
+ goto cleanup;
+ }
+
for (i = 0; i < ctl->def->ndisks; i++) {
virDomainDiskDefPtr disk = ctl->def->disks[i];
@@ -1283,6 +1289,8 @@ main(int argc, char **argv)
exit(EXIT_FAILURE);
}
+ virFileActivateDirOverride(argv[0]);
+
/* Initialize the log system */
virLogSetFromEnv();
--
2.13.2
7 years, 5 months
[libvirt] [PATCH] virFileInData: Report an error if unable to reposition file
by Michal Privoznik
The purpose of this function is to tell if the current position
in given FD is in data section or a hole and how much bytes there
is remaining until the end of the section. This is achieved by
couple of lseeks(). The most important part is that we reposition
the FD back, so that the position is unchanged from the caller
POV. And until now the final lseek() back to the original
position was done with no check for errors. And I was convinced
that that's okay since nothing can go wrong. However, John
persuaded me, that it's better to be safe than sorry. Therefore,
lets check if the final lseek() succeeded and if it doesn't
report an error.
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
src/util/virfile.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/src/util/virfile.c b/src/util/virfile.c
index d444b32f8..2f28e83f4 100644
--- a/src/util/virfile.c
+++ b/src/util/virfile.c
@@ -3900,8 +3900,12 @@ virFileInData(int fd,
ret = 0;
cleanup:
/* At any rate, reposition back to where we started. */
- if (cur != (off_t) -1)
- ignore_value(lseek(fd, cur, SEEK_SET));
+ if (cur != (off_t) -1 &&
+ lseek(fd, cur, SEEK_SET) == (off_t) -1) {
+ virReportSystemError(errno, "%s",
+ _("unable to restore position in file"));
+ ret = -1;
+ }
return ret;
}
--
2.13.0
7 years, 5 months
Re: [libvirt] [PATCH] spapr: make default PHB optionnal
by Andrea Bolognani
[libvir-list added to the loop]
On Tue, 2017-07-04 at 10:47 +0200, Greg Kurz wrote:
> On Tue, 4 Jul 2017 17:29:01 +1000 David Gibson <david(a)gibson.dropbear.id.au> wrote:
> > On Mon, Jul 03, 2017 at 06:48:25PM +0200, Greg Kurz wrote:
> > >
> > > The sPAPR machine always create a default PHB during initialization, even
> > > if -nodefaults was passed on the command line. This forces the user to
> > > rely on -global if she wants to set properties of the default PHB, such
> > > as numa_node.
> > >
> > > This patch introduces a new machine create-default-phb property to control
> > > whether the default PHB must be created or not. It defaults to on in order
> > > to preserve old setups (which is also the motivation to not alter the
> > > current behavior of -nodefaults).
> > >
> > > If create-default-phb is set to off, the default PHB isn't created, nor
> > > any other device usually created with it. It is mandatory to provide
> > > a PHB on the command line to be able to use PCI devices (otherwise QEMU
> > > won't start). For example, the following creates a PHB with the same
> > > mappings as the default PHB and also sets the NUMA affinity:
> > >
> > > -machine type=pseries,create-default-phb=off \
> > > -numa node,nodeid=0 -device spapr-pci-host-bridge,index=0,numa_node=0
> >
> > So, I agree that the distinction between default devices that are
> > disabled with -nodefaults and default devices that aren't is a big
> > mess in qemu configuration. But on the other hand this only addresses
> > one tiny aspect of that, and in the meantime means we will silently
> > ignore some other configuration options in some conditions.
> >
> > So, what's the immediate benefit / use case for this?
>
> With the current code base, the only way to set properties of the default
> PHB, is to pass -global spapr-pci-host-bridge.prop=value for each property.
> The immediate benefit of this patch is to unify the way libvirt passes
> PHB description to the command line:
>
> ie, do:
>
> -machine type=pseries,create-default-phb=off \
> -device spapr-pci-host-bridge,prop1=a,prop2=b,prop3=c \
> -device spapr-pci-host-bridge,prop1=d,prop2=e,prop3=f
>
> instead of:
>
> -machine type=pseries \
> -global spapr-pci-host-bridge.prop1=a \
> -global spapr-pci-host-bridge.prop2=b \
> -global spapr-pci-host-bridge.prop3=c \
> -device spapr-pci-host-bridge,prop1=d,prop2=e,prop3=f
So, I'm thinking about this mostly in terms of NUMA nodes
because that's the use case I'm aware of.
The problem with using -global is not that it requires using
a different syntax to set properties for the default PHB,
but rather that such properties are then inherited by all
other PHBs unless explicitly overridden. Not creating the
default PHB at all would solve the issue.
On the other hand, libvirt would then need to either
1) only allow setting NUMA nodes for PHBs if QEMU supports
the new option, leaving QEMU < 2.10 users behind; or
2) implement handling for both the new and old behavior.
I'm not sure we could get away with 1), and going for 2)
means more work both for QEMU and libvirt developers for
very little actual gain, so I'd be inclined to scrap this
and just build the libvirt glue on top of the existing
interface.
That is, of course, unless
1) having a random selection of PHBs not assigned to any
NUMA node is a sensible use case. This is something
we just can't do reliably with the current interface:
we can decide to set the NUMA node only for say, PHBs
1 and 3 leaving 0 and 2 alone, but once we set it for
the default PHB we *have* to set it for all remaining
ones as well. libvirt will by default assign emulated
devices to the default PHB, so I would rather expect
users to leave that one alone and set a NUMA node for
all other PHBs; or
2) there are other properties outside of numa_node we
might want to deal with; or
3) it turns out it's okay to require a recent QEMU :)
--
Andrea Bolognani / Red Hat / Virtualization
7 years, 5 months
[libvirt] [PATCH] docs: schema: make disk driver name attribute optional
by Jim Fehlig
/domain/devices/disk/driver/@name is not a required or mandatory
attribute according to formatdomain, and indeed it was agreed on
IRC that the attribute is "optional for input, recommended (but
not required) for output". Currently the schema requires the
attribute, causing virt-xml-validate to fail on disk config where
the driver name is not explicitly specified. E.g.
# cat test.xml | grep -A 5 cdrom
<disk type='file' device='cdrom'>
<driver type='raw'/>
<target dev='hdb' bus='ide'/>
<readonly/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
# virt-xml-validate test.xml
Relax-NG validity error : Extra element devices in interleave
test.xml:21: element devices: Relax-NG validity error : Element domain failed to validate content
test.xml fails to validate
Relaxing the name attribute to be optional fixes the validation
# virt-xml-validate test.xml
test.xml validates
Signed-off-by: Jim Fehlig <jfehlig(a)suse.com>
---
docs/schemas/domaincommon.rng | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index fc1a40f96..a49ce9303 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -1804,9 +1804,11 @@
</element>
</define>
<define name="driverFormat">
- <attribute name="name">
- <ref name="genericName"/>
- </attribute>
+ <optional>
+ <attribute name="name">
+ <ref name="genericName"/>
+ </attribute>
+ </optional>
<optional>
<attribute name='type'>
<choice>
--
2.13.1
7 years, 5 months
[libvirt] Xen device section defaults miss name='qemu'
by Christian Ehrhardt
Hi,
we have found an issue caused by 321a28c6 "libxl: set default disk format
in device post-parse" (and maybe more changes I couldn't identify yet).
TL;DR: The auto-added driver section in xen changed and now causes issues.
An example to to trigger:
1. Create a XEN xml with a disk device of type cdrom, but do not add a
driver section, like.
<disk type='file' device='cdrom'>
<target dev='hdb' bus='ide'/>
<readonly/>
</disk>
2. on virsh define if that XML
2b. You could also "virsh edit" any working xml file, remove the
<driver../> which will trigger the same
2c. You could also define via virt-manager as it does not explicitly
specify the device section
What happens is that before the changes this auto-added a driver section
like:
<driver name='qemu' type='raw'/>
But now it does only add
<driver type='raw'/>
Which fails to verify like:
Interestingly the same is not true for KVM, there the section is as it was
in the past which made it more likely the post-parse step might be involved.
Also rather expected, if one adds a full <driver name='qemu' type='raw'/>
in the XML libvirt doesn't have to provide (broken) defaults and things
work as they should - yet from a user with formerly working XMLs or using
virt-manager this is a regression.
I have beg your pardon I seem to not be experienced enough on this part of
libvirt to provide a valid fix - I tried but nothing good came out so far.
For now I reverted 321a28c6 which also affects a test later added by
4cd3f241. With those changes it will not add anything which lets us start
Xen VMs for now, but it clearly isn't a solution.
I wanted to ask you if you could take a look into this?
--
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd
7 years, 5 months
[libvirt] [PATCH] test/Makefile.am: drop WARN_CFLAGS from LDFLAGS
by Ján Tomko
Introduced by commit 0832c58, with the intention to link with
the stack protector library.
Another instance introduced by commit 4cbc15d which separated
commandhelper_LDADD from LDADDS.
Not needed because per commit 71b54636, automake should pass
all the CFLAGS to the linker.
---
tests/Makefile.am | 2 --
1 file changed, 2 deletions(-)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 19986dc99..3d3038e62 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -76,7 +76,6 @@ GNULIB_LIBS = \
../gnulib/lib/libgnu.la
LDADDS = \
- $(WARN_CFLAGS) \
$(NO_INDIRECT_LDFLAGS) \
$(PROBES_O) \
$(GNULIB_LIBS) \
@@ -933,7 +932,6 @@ commandtest_LDADD = $(LDADDS)
commandhelper_SOURCES = \
commandhelper.c
commandhelper_LDADD = \
- $(WARN_CFLAGS) \
$(NO_INDIRECT_LDFLAGS) \
$(PROBES_O) \
../src/libvirt_util.la \
--
2.13.0
7 years, 5 months
[libvirt] [RFC PATCH v2 0/4] numa: describe sibling nodes distances
by Wim Ten Have
From: Wim ten Have <wim.ten.have(a)oracle.com>
This patch extents guest domain administration adding support to advertise
node sibling distances when configuring HVM numa guests.
NUMA (non-uniform memory access), a method of configuring a cluster of nodes
within a single multiprocessing system such that it shares processor
local memory amongst others improving performance and the ability of the
system to be expanded.
A NUMA system could be illustrated as shown below. Within this 4-node
system, every socket is equipped with its own distinct memory. The whole
typically resembles a SMP (symmetric multiprocessing) system being a
"tightly-coupled," "share everything" system in which multiple processors
are working under a single operating system and can access each others'
memory over multiple "Bus Interconnect" paths.
+-----+-----+-----+ +-----+-----+-----+
| M | CPU | CPU | | CPU | CPU | M |
| E | | | | | | E |
| M +- Socket0 -+ +- Socket3 -+ M |
| O | | | | | | O |
| R | CPU | CPU <---------> CPU | CPU | R |
| Y | | | | | | Y |
+-----+--^--+-----+ +-----+--^--+-----+
| |
| Bus Interconnect |
| |
+-----+--v--+-----+ +-----+--v--+-----+
| M | | | | | | M |
| E | CPU | CPU <---------> CPU | CPU | E |
| M | | | | | | M |
| O +- Socket1 -+ +- Socket2 -+ O |
| R | | | | | | R |
| Y | CPU | CPU | | CPU | CPU | Y |
+-----+-----+-----+ +-----+-----+-----+
In contrast there is the limitation of a flat SMP system, not illustrated.
Here, as sockets are added, the bus (data and address path), under high
activity, gets overloaded and easily becomes a performance bottleneck.
NUMA adds an intermediate level of memory shared amongst a few cores per
socket as illustrated above, so that data accesses do not have to travel
over a single bus.
Unfortunately the way NUMA does this adds its own limitations. This,
as visualized in the illustration above, happens when data is stored in
memory associated with Socket2 and is accessed by a CPU (core) in Socket0.
The processors use the "Bus Interconnect" to create gateways between the
sockets (nodes) enabling inter-socket access to memory. These "Bus
Interconnect" hops add data access delays when a CPU (core) accesses
memory associated with a remote socket (node).
For terminology we refer to sockets as "nodes" where access to each
others' distinct resources such as memory make them "siblings" with a
designated "distance" between them. A specific design is described under
the ACPI (Advanced Configuration and Power Interface Specification)
within the chapter explaining the system's SLIT (System Locality Distance
Information Table).
These patches extend core libvirt's XML description of a virtual machine's
hardware to include NUMA distance information for sibling nodes, which
is then passed to Xen guests via libxl. Recently qemu landed support for
constructing the SLIT since commit 0f203430dd ("numa: Allow setting NUMA
distance for different NUMA nodes"), hence these core libvirt extensions
can also help other drivers in supporting this feature.
The XML changes made allow to describe the <cell> (or node/sockets) <distances>
amongst <sibling> node identifiers and propagate these towards the numa
domain functionality finally adding support to libxl.
[below is an example illustrating a 4 node/socket <cell> setup]
<cpu>
<numa>
<cell id='0' cpus='0,4-7' memory='2097152' unit='KiB'>
<distances>
<sibling id='0' value='10'/>
<sibling id='1' value='21'/>
<sibling id='2' value='31'/>
<sibling id='3' value='41'/>
</distances>
</cell>
<cell id='1' cpus='1,8-10,12-15' memory='2097152' unit='KiB'>
<distances>
<sibling id='0' value='21'/>
<sibling id='1' value='10'/>
<sibling id='2' value='21'/>
<sibling id='3' value='31'/>
</distances>
</cell>
<cell id='2' cpus='2,11' memory='2097152' unit='KiB'>
<distances>
<sibling id='0' value='31'/>
<sibling id='1' value='21'/>
<sibling id='2' value='10'/>
<sibling id='3' value='21'/>
</distances>
</cell>
<cell id='3' cpus='3' memory='2097152' unit='KiB'>
<distances>
<sibling id='0' value='41'/>
<sibling id='1' value='31'/>
<sibling id='2' value='21'/>
<sibling id='3' value='10'/>
</distances>
</cell>
</numa>
</cpu>
By default on libxl, if no <distances> are given to describe the SLIT data
between different <cell>s, this patch will default to a scheme using 10
for local and 21 for any remote node/socket, which is the assumption of
guest OS when no SLIT is specified. While SLIT is optional, libxl requires
that distances are set nonetheless.
On Linux systems the SLIT detail can be listed with help of the 'numactl -H'
command. An above HVM guest as described would on such prompt with below output.
[root@f25 ~]# numactl -H
available: 4 nodes (0-3)
node 0 cpus: 0 4 5 6 7
node 0 size: 1988 MB
node 0 free: 1743 MB
node 1 cpus: 1 8 9 10 12 13 14 15
node 1 size: 1946 MB
node 1 free: 1885 MB
node 2 cpus: 2 11
node 2 size: 2011 MB
node 2 free: 1912 MB
node 3 cpus: 3
node 3 size: 2010 MB
node 3 free: 1980 MB
node distances:
node 0 1 2 3
0: 10 21 31 41
1: 21 10 21 31
2: 31 21 10 21
3: 41 31 21 10
Wim ten Have (4):
numa: describe siblings distances within cells
libxl: vnuma support
xenconfig: add domxml conversions for xen-xl
xlconfigtest: add tests for numa cell sibling distances
docs/formatdomain.html.in | 64 ++++-
docs/schemas/basictypes.rng | 8 +
docs/schemas/cputypes.rng | 18 ++
src/conf/cpu_conf.c | 2 +-
src/conf/numa_conf.c | 260 +++++++++++++++++-
src/conf/numa_conf.h | 25 +-
src/libvirt_private.syms | 6 +
src/libxl/libxl_conf.c | 124 +++++++++
src/xenconfig/xen_xl.c | 303 +++++++++++++++++++++
.../test-fullvirt-vnuma-nodistances.cfg | 26 ++
.../test-fullvirt-vnuma-nodistances.xml | 53 ++++
tests/xlconfigdata/test-fullvirt-vnuma.cfg | 26 ++
tests/xlconfigdata/test-fullvirt-vnuma.xml | 81 ++++++
tests/xlconfigtest.c | 4 +
14 files changed, 993 insertions(+), 7 deletions(-)
create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.cfg
create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.xml
create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.cfg
create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.xml
--
2.9.4
7 years, 5 months
