[libvirt] [ocaml PATCH 0/4] Misc build improvements
by Pino Toscano
A round of few build system improvements:
- use the OCaml build macros from libguestfs
- remove generated stuff
- use pkg-config to find & use libvirt
Pino Toscano (4):
build: move OCaml macros to a m4 subdir
build: sync OCaml macros from libguestfs
build: remove config.h.in
build: use pkg-config to find libvirt
.gitignore | 2 +
aclocal.m4 | 170 ----------------------------------
config.h.in | 61 -------------
configure.ac | 26 +-----
libvirt/Makefile.in | 14 +--
m4/ocaml.m4 | 217 ++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 232 insertions(+), 258 deletions(-)
delete mode 100644 aclocal.m4
delete mode 100644 config.h.in
create mode 100644 m4/ocaml.m4
--
2.17.2
6 years, 1 month
[libvirt] [PULL 0/2] Ui2 20181012 patches
by Gerd Hoffmann
The following changes since commit 69ac8c4cb93f2685839ff7b857cef306b388ff3c:
Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20181012' into staging (2018-10-12 12:40:04 +0100)
are available in the git repository at:
git://git.kraxel.org/qemu tags/ui2-20181012-pull-request
for you to fetch changes up to 58296cb61866195297510e946a51acc5f0b9639e:
ui: increase min required GTK3 version to 3.14.0 (2018-10-12 15:22:18 +0200)
----------------------------------------------------------------
ui: drop gtk2 support.
----------------------------------------------------------------
Daniel P. Berrangé (2):
ui: remove support for GTK2 in favour of GTK3
ui: increase min required GTK3 version to 3.14.0
configure | 51 ++-----------
include/ui/gtk.h | 9 ---
ui/gtk-egl.c | 10 +--
ui/gtk.c | 202 ++++-----------------------------------------------
qemu-deprecated.texi | 7 --
5 files changed, 26 insertions(+), 253 deletions(-)
--
2.9.3
6 years, 1 month
[libvirt] [RFC 0/7] Warn at runtime when deprecated features are used
by Andrea Bolognani
Background
==========
We have plenty of features in libvirt, some of which were designed at
a time when the virtualization story was much more straightforward
than the multi-architecture, multi-hypervisor, multi-machine world we
currently live in and, while we have found ways to keep the APIs
chugging along, the result is sometimes somewhat confusing for users
and application developers, as well as requiring libvirt developers
themselves to spend quite a bit of collective time working around
decisions that, in hindsight, turn out to have been less than
fortunate.
Two concrete examples are considered here: one is the
virConnectNumOfDomains() API which, while known to be racy and having
non-racy alternatives, can still be used by developers without
getting any kind of warning in the process; the other one is the
ability to define a domain without specifying the machine type, which
is becoming increasingly problematic now with some x86_64 features
being limited to q35 and downstreams looking to push for its
adoption, as well as being a manifestation of the more general
problem of libvirt's default being sometimes too conservative and at
odds with the existence of slimmed-down QEMU binaries being built
with reducing the total attack surface in mind.
Having a proper deprecation story in libvirt would allow us to point
users and developers towards the recommended solution in each case,
be it using a different API or querying libosinfo for information;
after a generous grace period, we could then remove the problematic
functionality altogether. This would be a more conservative version
of the process we already have in place for dropping support for
older QEMU releases, which recently has allowed us to ax massive
chunks of effectively dead code and simplify parts of libvirt quite
significantly.
This series explores one possible approach to the problem and aims
to spark project-wide discussion around the topic.
Further work
============
* Fix the known issues listed below as well as all not-yet-known
issues that will undoubtably surface through discussion :)
* Introduce a mechanism to catch use of deprecated APIs at build
time, similar to GLib's G_DISABLE_DEPRECATED, to help application
developers proactively move off problematic APIs.
* Create a formal deprecation policy with well-defined rules and
time scales in the spirit of the existing one covering our
relationship with QEMU.
Know issues
===========
* For the more granular (and more interesting) type of deprecation
shown in patch 6/7, warnings are not being reported back to the
client as expected. I believe this is caused by the RPC code
looking for either a failure, in which case the virError is
transmitted, or a success, in which case the actual return value
is: we'll have to figure out a way for the error to travel across
the wire regardless of whether or not the API call was ultimately
successful if we want clients to actually receive warnings when
non-local drivers are involved.
Andrea Bolognani (7):
util: Add 'level' argument to virReportErrorHelper()
util: Introduce virReportWarning()
tools: Print warnings in virsh
util: Introduce VIR_ERR_DEPRECATED_FEATURE
Deprecate virConnectNumOfDomains()
Deprecate missing machine type in virDomainDefineXMLFlags()
tools: Force virsh to use deprecated features
include/libvirt/virterror.h | 1 +
src/access/viraccessdriverpolkit.c | 2 +-
src/access/viraccessmanager.c | 2 +-
src/conf/domain_conf.c | 11 ++++++++---
src/datatypes.h | 30 ++++++++++++++++++++++++++----
src/libvirt-domain.c | 6 ++++++
src/libvirt.c | 1 +
src/util/virbuffer.c | 4 ++--
src/util/virconf.c | 6 ++++--
src/util/virerror.c | 10 +++++++++-
src/util/virerror.h | 15 ++++++++++-----
src/util/virkeyfile.c | 6 ++++--
src/util/virxml.c | 2 +-
tools/virsh-domain-monitor.c | 2 ++
tools/vsh.c | 3 +++
15 files changed, 79 insertions(+), 22 deletions(-)
--
2.17.1
6 years, 1 month
[libvirt] [PATCH 0/2] util: storage: Fix parsing of URIs as disk sources
by Peter Krempa
see 2/2 for explanation
Peter Krempa (2):
util: storage: Rename '@path' argument of
virStorageSourceParseBackingURI
util: storage: Properly parse URIs with missing trailing slash
src/util/virstoragefile.c | 28 +++++++++++++++++++++-------
tests/qemublocktest.c | 3 +++
tests/virstoragetest.c | 20 ++++++++++++++++++++
3 files changed, 44 insertions(+), 7 deletions(-)
--
2.17.1
6 years, 1 month
[libvirt] [RFC PATCH 0/3] qemu: guest dedicated crypto adapters
by Boris Fiuczynski
This RFC patch series introduces initial libvirt support for guest
dedicated crypto adapters on S390.
It essentially allows to specify a vfio-ap mediated device in a domain.
Extensive documentation about AP is available in patch 6 of
the QEMU patch series.
KVM/kernel: guest dedicated crypto adapters
https://lkml.org/lkml/2018/9/26/25
QEMU: s390x: vfio-ap: guest dedicated crypto adapters
https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg03538.html
Boris Fiuczynski (3):
qemu: add vfio-ap capability
qemu: vfio-ap device support
news: Update news for vfio-ap support
docs/formatdomain.html.in | 3 ++-
docs/news.xml | 9 +++++++++
docs/schemas/domaincommon.rng | 1 +
src/qemu/qemu_capabilities.c | 2 ++
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_command.c | 8 ++++++++
src/qemu/qemu_domain_address.c | 4 ++++
src/util/virmdev.c | 3 ++-
src/util/virmdev.h | 1 +
9 files changed, 30 insertions(+), 2 deletions(-)
--
2.17.0
6 years, 1 month
[libvirt] [PATCH v1] rpc: reproducible genprotocol output
by Olaf Hering
If the same source gets built twice ('build same source on different
hosts at different times') the resulting files may differ.
Fix this by sorting the hash keys before usage.
Signed-off-by: Olaf Hering <olaf(a)aepfle.de>
---
src/rpc/genprotocol.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/rpc/genprotocol.pl b/src/rpc/genprotocol.pl
index 6ce268c024..6baa4f22d5 100755
--- a/src/rpc/genprotocol.pl
+++ b/src/rpc/genprotocol.pl
@@ -104,7 +104,7 @@ while (<RPCGEN>) {
if (keys %uses >= 1) {
my $i = 1;
- foreach (keys %uses) {
+ foreach (sort(keys %uses)) {
$i = $uses{$_};
unshift @function,
(" char **objp_cpp$i = (char **) (void *) &$_;\n");
6 years, 1 month
[libvirt] [PATCH] virsh: Fix regression with duplicated error messages
by Eric Blake
Commit 4f4c3b13 (v3.3) fixed an issue where cleaning libvirt objects
lost error messages, by adding code to copy the libvirt error into
last_error prior to cleanup paths. However, it caused a regression:
some errors are now printed twice, because libvirt still remembers in
its thread-local storage that an error was set. For example:
$ virsh -c test:///default snapshot-delete test blah
error: Domain snapshot not found: no domain snapshot with matching name 'blah'
error: Domain snapshot not found: no domain snapshot with matching name 'blah'
Fix things by telling libvirt to discard any thread-local errors at
the same time virsh prints an error message (whether or not the libvirt
error is the same as what is stored in last_error).
Update the virsh-undefine testsuite (partially reverting portions of
commit b620bdee, by removing -q, to more easily pinpoint which commands
are causing which messages), now that there is only one error message
instead of two.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
tests/virsh-undefine | 20 ++++++++++++++++----
tools/vsh.c | 1 +
2 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/tests/virsh-undefine b/tests/virsh-undefine
index 974b0c71f7..4a9f68dd39 100755
--- a/tests/virsh-undefine
+++ b/tests/virsh-undefine
@@ -30,34 +30,46 @@ fail=0
# connection is opened to the test driver, it starts life with a new
# persistent running domain named 'test' with a different uuid, so
# testing this command requires batch mode use of virsh.
-$abs_top_builddir/tools/virsh -q -c test:///default \
+$abs_top_builddir/tools/virsh -c test:///default \
'dominfo test; undefine test; dominfo test' > out1 2>&1
test $? = 0 || fail=1
sed '/^Persistent/n; /:/d' < out1 > out
cat <<\EOF > exp || fail=1
Persistent: yes
+
+Domain test has been undefined
+
Persistent: no
+
EOF
compare exp out || fail=1
# A similar diagnostic when specifying a domain ID
-$abs_top_builddir/tools/virsh -q -c test:///default \
+$abs_top_builddir/tools/virsh -c test:///default \
'dominfo 1; undefine 1; dominfo 1' > out1 2>&1
test $? = 0 || fail=1
sed '/^Persistent/n; /:/d' < out1 > out
cat <<\EOF > exp || fail=1
Persistent: yes
+
+Domain 1 has been undefined
+
Persistent: no
+
EOF
compare exp out || fail=1
# Succeed, now: first shut down, then undefine, both via name.
-$abs_top_builddir/tools/virsh -q -c test:///default \
+$abs_top_builddir/tools/virsh -c test:///default \
'shutdown test; undefine test; dominfo test' > out 2>&1
test $? = 1 || fail=1
cat <<\EOF > expout || fail=1
+Domain test is being shutdown
+
+Domain test has been undefined
+
error: failed to get domain 'test'
-error: Domain not found
+
EOF
compare expout out || fail=1
diff --git a/tools/vsh.c b/tools/vsh.c
index 9ea3c4b96a..de887a9e76 100644
--- a/tools/vsh.c
+++ b/tools/vsh.c
@@ -276,6 +276,7 @@ vshResetLibvirtError(void)
{
virFreeError(last_error);
last_error = NULL;
+ virResetLastError();
}
/*
--
2.17.1
6 years, 1 month
[libvirt] [RFC] Faster libvirtd restart with nwfilter rules
by Nikolay Shirokovskiy
Hi, all.
On fat hosts which are capable to run hundreds of VMs restarting libvirtd
makes it's services unavailable for a long time if VMs use network filters. In
my tests each of 100 VMs has no-promisc [1] and no-mac-spoofing filters and
executing virsh list right after daemon restart takes appoximately 140s if no
firewalld is running (that is ebtables/iptables/ip6tables commands are used to
configure kernel tables).
The problem is daemon does not even start to read from client connections
because state drivers are not initialized. Initialization is blocked in state
drivers autostart which grabs VMs locks. And VMs locks are hold by VMs
reconnection code. Each VM reloads network tables on reconnection and this
reloading is serialized on updateMutex in gentech nwfilter driver.
Workarounding autostart won't help much because even if state drivers will
initialize listing VM won't be possible because listing VMs takes each VM lock
one by one too. However managing VM that passed reconnection phase will be
possible which takes same 140s in worst case.
Note that this issue is only applicable if we use filters configuration that
don't need ip learning. In the latter case situation is different because
reconnection code spawns new thread that apply network rules only after ip is
learned from traffic and this thread does not grab VM lock. As result VMs are
managable but reloading filters in background takes appoximately those same
140s. I guess managing network filters during this period can have issues too.
Anyway this situation does not look good so fixing the described issue by
spawning threads even without ip learning does not look nice to me.
What speed up is possible on conservative approach? First we can remove for
test purpuses firewall ruleLock, gentech dirver updateMutex and filter object
mutex which do not serve function in restart scenario. This gives 36s restart
time. The speed up is archived because heavy fork/preexec steps are now run
concurrently.
Next we can try to reduce fork/preexec time. To estimate its contibution alone
let's bring back the above locks. It turns out the most time takes fork itself
and closing 8k (on my system) file descriptors in preexec. Using vfork gives
2x boost and so does dropping mass close. (I check this mass close contribution
because I not quite understand the purpose of this step - libvirt typically set
close-on-exec flag on it's descriptors). So this two optimizations alone can
result in restart time of 30s.
Unfortunately combining the above two approaches does not give boost multiple
of them along. The reason is due to concurrency and high number of VMs (100)
preexec boost does not have significant role and using vfork dininishes
concurrency as it freezes all parent threads before execve. So dropping locks
and closes gives 33s restart time and adding vfork to this gives 25s restart
time.
Another approach is to use --atomic-file option for ebtables
(iptables/ip6tables unfortunately does not have one). The idea is to save table
to file/edit file/commit table to kernel. I hoped this could give performance
boost because we don't need to load/store kernel network table for a single
rule update. In order to isolate approaches I also dropped all ip/ip6 updates
which can not be done this way. In this approach we can not drop ruleLock in
firewall because no other VM threads should change tables between save/commit.
This approach gives restart time 25s. But this approach is broken anyway as we
can not be sure another application doesn't change newtork table between
save/commit in which case these changes will be lost.
After all I think we need to move in a different direction. We can add API to
all binaries and firewalld to execute many commands in one run. We can pass
commands as arguments or wrote them into file which is then given to binary.
Then libvirt itself can update for example bridge network table in couple of
commands. The exact number depends on new API. For example if we add option to
delete chains recursively and an option not to fail on NOENT error we can
change table in one command (no listing current rules is required).
[1] no-promisc filter
<filter name='no-promisc' chain='root' priority='-750'>
<uuid>6d055022-1192-4a3d-ae1f-576baa5564b6</uuid>
<rule action='return' direction='in' priority='500'>
<mac dstmacaddr='ff:ff:ff:ff:ff:ff'/>
</rule>
<rule action='return' direction='in' priority='500'>
<mac dstmacaddr='$MAC'/>
</rule>
<rule action='return' direction='in' priority='500'>
<mac dstmacaddr='33:33:00:00:00:00' dstmacmask='ff:ff:00:00:00:00'/>
</rule>
<rule action='drop' direction='in' priority='500'>
<mac/>
</rule>
<rule action='return' direction='in' priority='500'>
<mac dstmacaddr='01:00:5e:00:00:00' dstmacmask='ff:ff:ff:80:00:00'/>
</rule>
</filter>
6 years, 1 month
[libvirt] [PATCH] qemu: Remove unused qemuProcessAutostartAll
by John Ferlan
The function was never defined in source, just the protoype.
Signed-off-by: John Ferlan <jferlan(a)redhat.com>
---
Pushed as trivial.
src/qemu/qemu_process.h | 1 -
1 file changed, 1 deletion(-)
diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h
index c2f7c2b5d2..2037467c94 100644
--- a/src/qemu/qemu_process.h
+++ b/src/qemu/qemu_process.h
@@ -47,7 +47,6 @@ int qemuProcessDestroyMemoryBackingPath(virQEMUDriverPtr driver,
virDomainObjPtr vm,
virDomainMemoryDefPtr mem);
-void qemuProcessAutostartAll(virQEMUDriverPtr driver);
void qemuProcessReconnectAll(virQEMUDriverPtr driver);
typedef struct _qemuProcessIncomingDef qemuProcessIncomingDef;
--
2.17.1
6 years, 1 month
