[libvirt] [PATCH 0/8] Refactor Intel IOMMU support
by Andrea Bolognani
Another prerequisite for vSMMUv3 support.
Well, technically I could go ahead and implement that without
refactoring Intel IOMMU support first, but I'd rather not add more
validation code to qemu_command.
Andrea Bolognani (8):
tests: Simplify Intel IOMMU testing
tests: Use DO_TEST_CAPS_*() for Intel IOMMU
tests: Add negative test for Intel IOMMU
conf: Allow NULL for virDomainDeviceInfoCallback
conf: Drop DOMAIN_DEVICE_ITERATE_GRAPHICS
conf: Add IOMMU support to virDomainDeviceInfoIterate()
qemu: Introduce qemuDomainDeviceDefValidateIOMMU()
qemu: Tweak Intel IOMMU command line generation
src/bhyve/bhyve_device.c | 4 ++
src/conf/domain_addr.c | 9 +++
src/conf/domain_conf.c | 59 ++++++++++++----
src/libxl/libxl_driver.c | 3 +
src/qemu/qemu_command.c | 69 +++----------------
src/qemu/qemu_domain.c | 66 +++++++++++++++++-
src/qemu/qemu_domain_address.c | 36 +++++++++-
src/qemu/qemu_hotplug.c | 3 +
...tel-iommu-caching-mode.x86_64-latest.args} | 15 ++--
.../intel-iommu-caching-mode.xml | 23 +------
...tel-iommu-device-iotlb.x86_64-latest.args} | 22 +++---
.../intel-iommu-device-iotlb.xml | 1 +
...rgs => intel-iommu-eim.x86_64-latest.args} | 15 ++--
tests/qemuxml2argvdata/intel-iommu-eim.xml | 1 +
... => intel-iommu-machine.x86_64-2.6.0.args} | 9 ++-
.../qemuxml2argvdata/intel-iommu-machine.xml | 3 +-
...hine.xml => intel-iommu-wrong-machine.xml} | 8 +--
...ne.args => intel-iommu.x86_64-latest.args} | 18 +++--
tests/qemuxml2argvdata/intel-iommu.xml | 1 +
tests/qemuxml2argvtest.c | 34 ++-------
...ntel-iommu-caching-mode.x86_64-latest.xml} | 0
...ntel-iommu-device-iotlb.x86_64-latest.xml} | 0
....xml => intel-iommu-eim.x86_64-latest.xml} | 0
...l => intel-iommu-machine.x86_64-2.6.0.xml} | 0
...ommu.xml => intel-iommu.x86_64-latest.xml} | 0
tests/qemuxml2xmltest.c | 15 ++--
26 files changed, 242 insertions(+), 172 deletions(-)
rename tests/qemuxml2argvdata/{intel-iommu-device-iotlb.args => intel-iommu-caching-mode.x86_64-latest.args} (61%)
rename tests/qemuxml2argvdata/{intel-iommu-caching-mode.args => intel-iommu-device-iotlb.x86_64-latest.args} (53%)
rename tests/qemuxml2argvdata/{intel-iommu-eim.args => intel-iommu-eim.x86_64-latest.args} (62%)
rename tests/qemuxml2argvdata/{intel-iommu.args => intel-iommu-machine.x86_64-2.6.0.args} (73%)
copy tests/qemuxml2argvdata/{intel-iommu-machine.xml => intel-iommu-wrong-machine.xml} (71%)
rename tests/qemuxml2argvdata/{intel-iommu-machine.args => intel-iommu.x86_64-latest.args} (55%)
rename tests/qemuxml2xmloutdata/{intel-iommu-caching-mode.xml => intel-iommu-caching-mode.x86_64-latest.xml} (100%)
rename tests/qemuxml2xmloutdata/{intel-iommu-device-iotlb.xml => intel-iommu-device-iotlb.x86_64-latest.xml} (100%)
rename tests/qemuxml2xmloutdata/{intel-iommu-eim.xml => intel-iommu-eim.x86_64-latest.xml} (100%)
rename tests/qemuxml2xmloutdata/{intel-iommu-machine.xml => intel-iommu-machine.x86_64-2.6.0.xml} (100%)
rename tests/qemuxml2xmloutdata/{intel-iommu.xml => intel-iommu.x86_64-latest.xml} (100%)
--
2.21.0
5 years, 4 months
[libvirt] (Dropping) OOM Handling in libvirt
by Daniel P. Berrangé
This is a long mail about ENOMEM (OOM) handling in libvirt. The executive
summary is that it is not worth the maint cost because:
- this code will almost never run on Linux hosts
- if it does run it will likely have bad behaviour silently dropping
data or crashing the process
- apps using libvirt often do so via a non-C language that aborts/exits
the app on OOM regardless, or use other C libraries that abort
- we can build a system that is more reliable when OOM happens by
not catching OOM, instead simply letting apps exit, restart and
carry on where they left off
The long answer follows...
The background
==============
Since the first commit libvirt has attempted to handle out of memory (OOM)
errors in the same way it does for any other problem. Namely, a virError will
be raised, and the method will jump to its cleanup/error label. The error
bubbles up the stack and in theory someone or something will catch this and do
something sensible/useful. This has long been considered "best practice" for
most C libraries, especially those used for system services. This mail makes
the argument that it is in fact /not/ "best practice" to try to handle OOM.
OOM handling is very difficult to get correct because it is something that
developers almost never encounter and thus the code paths are rarely run. We
designed our memory allocation APIs such that we get compile time errors if
code forgets to check the return value for failure. This is good as it
eliminates an entire class of bugs. Our error handling goto label pattern
tries to align OOM handling with other general error handling which is more
commonly tested.
We have code in the allocators which lets us run unit tests simulating the
failure of any allocation that is made during the test. Executing this is
extraordinarily time consuming as some of our unit tests have many 1000's or
even 10's of 1000's of allocations. The running time to simulate OOM for each
allocation is O(N^2) which does not scale well. As a result we've probably
only run these OOM tests a handful of times over the years.
The tests show we generally do remarkly well at OOM handling, but none the
less we have *always* found bugs in our handling where we fail to report the
OOM (as in, continue to run but with missing data), or we crash when cleaning
up from OOM. Our unit tests don't cover anywhere near all of our codebase
though. We have great coverage of XML parsing/formatting, and sporadic coverage
of everything else.
IOW, despite our clever API designs & coding patterns designed to improve our
OOM handling, we can not have confidence that we will actually operate correctly
in an OOM scenario.
Of course this applies to all error conditions that may arise, but OOM should
be considered special. With other error conditions from syscalls or API calls,
the effects are largely isolated to the site of the call. With OOM, the OOM
condition may well persist and so during cleanup we will hit further OOM
problems. We may well fail to even allocate the memory needed for raise a
virErrorPtr. All threads can see the OOM concurrently so the effect spreads
across the whole process very quickly.
Who benefits from OOM handling
==============================
Lets pretend for a minute though that our OOM handling is perfect and instead
ask who/what is benefitting from it ?
Libvirt's own processes. aka libvirtd, virtlockd, virtlogd
----------------------------------------------------------
For libvirtd we have essentially zero confidence that it will handle OOM
at all. It is running all virt driver code over which we have massively
inadequate unit testing cover to have any confidence that OOM will be well
handled. Even if OOM is handled in a worker thread, every iteration of the
event loop does an allocation to hold the poll FD array, so we can easily
see the event loop hitting OOM which will make the entire process shutdown,
or worse, hang during cleanup as worker threads block waiting for the event
loop todo something despite not being running anymore.
We already expect that bugs will cause libvirtd to crash and so have designed
the drivers to be robust such that it can restart and attempt to carry on as
normal afterwards. So arguably it would be fine to handle OOM by simply doing
an abort and let systemd restart the daemon.
For virtlockd and virtlogd we again have little confidence that they will
handle OOM correctly. They are, however, more critical processes that we badly
need to stay running at all times. We go to great effort to make it possible
to re-exec on upgrades keeping state open.
For virtlogd we could potentially change the way we deal with stdout/err for
QEMU. Instead of using an anonymous pipe, we could create a named fifo on disk
for each QEMU process. stdout/err would be connected to one end, and virtlogd
to the other end. This would enable us to have virtlogd restarted and re-open
the stdout/err for QEMU. This would mean we no longer need the re-exec logic
either, which is good as that's another thing that's rarely tested. This would
make abort on OOM acceptable.
For virtlockd we have the hardest problem. It fundamentally has to keep open
the FDs in order to maintain active locks. If it does crash / abort, then all
open locks are lost. It would have to manually re-acquire locks when it starts
up again, provided it had a record of which locks it was supposed to have. In
theory nothing bad should happen in this window where virtlockd is dead, as if
libvirtd tries to start another VM it will trigger auto-start of virtlockd via
its systemd socket unit, which could then cause it to reacquire previous locks
it had. IOW, it should be possible to make virtlockd robust enough that doing
abort on OOM is tolerable. We really need this robustness regardless, because
virtlockd can of course already crash due to code bugs. If we make it robust
to all crashes, then by implication it will be robust enough for aborton OOM.
Clients of libvirt. aka oVirt, OpenStack, KubeVirt, virt-manager, cockpit
-------------------------------------------------------------------------
Clients of libvirt can consume it via a number of different APIs, all
eventually calling into libvirt.so
- Core C API aka libvirt.so
Catches and propagates OOM to callers.
virsh just reports it as it would any other error. In single shot
mode it will be exiting regardles with error code. In interactive
mode, in theory it can carry on with processing the next command.
In practice its CLI parsing will probably then hit OOM anyway
causing it to exit.
virt-viewer links to GTK and this already has abort-on-OOM behaviour
via GLib. So in all liklihood, even if it catches the OOM from libvirt,
it will none the less abort on OOM in a GTK/GLib call.
libvirt-dbus is written using glib, so again even if it catches the OOM
from libvirt it will none the less abort on OOM in a GLib call.
- Python binding
Python bindings will raise a MemoryError exception which propagates
back up to the python code. In theory the app can catch this and
take some behaviour. This mostly only works though if the cause was
a single huge allocation, such that most other "normal" sized allocations
continue working. In a true OOM scenario where arbitrary allocs start
failing, the python interpretor will fail too.
- Perl binding
It is hard to find clear info about Perl behaviour during ENOMEM. The
libvirt bindings assume the Perl allocation functions won't ever fail,
so if they do we're going to reference a NULL pointer. If normal Perl
code gets OOM, the interpretor will raise an error. In theory this can
be caught, but in practice few, if any, apps will try so the process
will likely quit.
- Go binding
Errors from libvirt are all turned into Golang errors and propagated
to the caller. If the Go runtime gets an OOM from an allocation it will
panic the process which can't be caught, so it will exit with stack
trace.
- Java binding
The JVM tends to allocate a fixed size heap for its own use. So Java
code can see OOM exceptions even if the OS has plenty of memory. If
the OS does run out of memory, assuming the JVM heap was already
fully allocated it shouldn't immediately be affect, but might suffer
collatoral damage. But in theory libvirt OOM could get turned into a
Java exception that an app can catch & handle nicely.
There are other bindings, but the above captures the most important usage
of libvirt.
The complication of Linux
=========================
Note that all of the above discussion had the implicit assumption that malloc
will actually return ENOMEM when memory is exhausted.
On Linux at least this is not the case in a default install. Linux tends
to enable memory overcommit causing the kernel to satisfy malloc requests
even if it exceeds available RAM + Swap. The allocated memory won't even be
paged in to physical RAM until the page is written to. If there is insufficient
to make a page resident in RAM, then the OOM killer is set free. Some poor
victim will get killed off.
It is possible to disable RAM overcommit and also disable or deprioritize the
OOM killer. This might make it possible to actually see ENOMEM from an
allocation, but few if any developers or applications run in this scenario
so it should be considered untested in reality.
With cgroups it is possible to RAM and swap usage limits. In theory you can
disable the OOM killer per-cgroup and this will cause ENOMEM to the app. In
my testing with a trivial C program that simply mallocs a massive array and
them memsets it, it is hard to get ENOMEM to happen reliably. Sometimes the
app ends up just hanging when testing this.
Other operating systems have different behaviour and so are more likely to
really report ENOMEM to application code, but even if you can get ENOMEM
on other OS the earlier points illustrate that its not worth worrying about.
The conclusion
==============
To repeat the top of this document, attempts to handle OOM are not worth
the maint cost they incur:
- this code will almost never run on Linux hosts
- if it does run it will likely have bad behaviour silently dropping
data or crashing the process
- apps using libvirt often do so via a non-C language that aborts/exits
the app on OOM regardless, or use other C libraries that abort
- we can build a system that is more reliable when OOM happens by
not catching OOM, instead simply letting apps exit, restart and
carry on where they left off.
The proposal is thus to change all libvirt code (ie both client and server
side), to abort on OOM.
This will allow us to simplify the control flow in many methods eliminating
~1500 checks for memory failure, their associated goto jumps, and many of
the cleanup/error labels. This makes the code easier to follow & maintain
and opens up new avenues for improving libvirt's future development.
The main blocking prequisite to making the change is to address the needs for
reliable restart of virtlockd and virtlogd daemons.
As a point of reference libguestfs has had abort-on-oom behaviour forever
and no one has complained about this behaviour. Apps using libvirt often
also use libguestfs in the same process.
The implementation
==================
Assuming a decision to abort on OOM, libvirt can nwo follow QEMU's lead and
make use of the GLib library. The initial impl would thus be to just link to
GLib and switch VIR_ALLOC related APIs to use g_new/g_malloc & friends
internally. Over time code can be changed to use g_new/g_malloc directly thus
removing the error handling incrementally.
Use of GLib brings a number of other significant opportunities / advantages
to ongoing development
- Access to many standard data structures (hash tables, linked lists,
queues, growable arrays, growable strings).
- Access to platform portability APIs for threads, event loops, file I/O,
filename parsing and more.
- Access to GObject to replace our own virObject clone with something that is
far more feature rich. This is especially true of its callback facility
via signal handlers.
- Access to GIO to replace much of our socket I/O portability wrappers, and
DBus client APIs.
- Opens the possibility of calling to code in a non-C language via the GObject
introspection bindings for GObject & other helper APIs.
- Ability to drop use of gnulib once we use GLib for all portability problems
we currently rely on gnulib for, once a critical set of functionality is
ported to the GLib APIs. Most important is the sockets / event loop stuff
due to Windows portability.
- Ability to drop use of autoconf/automake in favour of a more maintainable
option like meson/ninja, once we eliminate use of gnulib.
Ultimately the big picture benefit is that we can spend less time working on
low level generic code related to platform portability, data structures, or
system services.
Note this is not claiming that all GLib's APIs are better than stuff we have
implemented in libvirt already. The belief is that even if some of the GLib
APIs are worse, it is still a benefit to the project to use them, as it will
reduce the code we maintain ourselves. This in turns lets us spend more time
working on high level code directly related to virtualization features and
be more productive when doing so.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
5 years, 4 months
[libvirt] Why virsh domxml-to-native qemu-argv cut VN name
by Tal Attaly
Hi,
I think there is a bug when running virsh domxml-to-native qemu-argv with VM with long name, it getting cut -
.. -name guest=l-dev-vrt-140-022-RH-7.0-Tal,debug-threads=on -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain--1-l-dev-vrt-140-022-RH/master-key.aes -machine pc-i440fx-2.12,accel=kvm,usb=off,dump-guest-core=off -m 8192 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid cb48f75c-40f6-432d-964b-906ee0bdcf48 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain--1-l-dev-vrt-140-022-RH/monitor.sock,server,nowait ..
-name is ok, but -object and -chardev are cut, which cause QEMU to file on -
qemu-system-x86_64: -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/do
main--1-l-dev-vrt-140-022-RH/monitor.sock,server,nowait: Failed to bind socket to /var/lib/libvirt/qemu/doma
in--1-l-dev-vrt-140-022-RH/monitor.sock: No such file or directory
Libvirt version is 3.9.0, here is the full domain XML -
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>l-dev-vrt-140-022-RH-7.0-Tal</name>
<uuid>cb48f75c-40f6-432d-964b-906ee0bdcf48</uuid>
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>4194304</currentMemory>
<vcpu placement='static'>2</vcpu>
<os>
<type arch='x86_64' machine='pc-i440fx-2.12'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>x86_64-softmmu/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/images/l-dev-vrt-140-022/l-dev-vrt-140-022.img'/>
<target dev='hda' bus='ide'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<controller type='usb' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<interface type='bridge'>
<mac address='00:50:56:18:8c:16'/>
<source bridge='br0:'/>
<model type='e1000'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<serial type='pty'>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes'>
<listen type='address'/>
</graphics>
<video>
<model type='cirrus' vram='16384' heads='1' primary='yes'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</memballoon>
</devices>
</domain>
Thanks,
Tal
5 years, 4 months
[libvirt] [PATCH 0/3] docs: hacking: Discourage some patterns to help code readability
by Peter Krempa
Peter Krempa (3):
docs: hacking: Document few practices for creating error messages
docs: hacking: Add good practices for shortening conditional
expressions
docs: hacking: Discourage use of the ternary operator and ban it's
abuse
docs/hacking.html.in | 61 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 61 insertions(+)
--
2.21.0
5 years, 4 months
[libvirt] [PATCH 0/3] Fix access controls for virtlockd/virtlogd sockets
by Daniel P. Berrangé
This series fixes CVE-2019-10132
It is pushed to master as it was reviewed on the security list
ahead of time.
Daniel P. Berrangé (3):
admin: reject clients unless their UID matches the current UID
locking: restrict sockets to mode 0600
logging: restrict sockets to mode 0600
src/admin/admin_server_dispatch.c | 22 ++++++++++++++++++++++
src/locking/virtlockd-admin.socket.in | 1 +
src/locking/virtlockd.socket.in | 1 +
src/logging/virtlogd-admin.socket.in | 1 +
src/logging/virtlogd.socket.in | 1 +
5 files changed, 26 insertions(+)
--
2.21.0
5 years, 4 months
[libvirt] [PATCH v2 for 5.3.0] storage_backend_rbd: Ignore rbd_diff_iterate error() if needed
by Michal Privoznik
When cloning an RBD volume we try to find a snapshot which is
not different to the image we're trying to clone. This boils down
to calling rbd_diff_iterate() or rbd_diff_iterate2() on systems
with newer ceph. These two are passed a callback -
virStorageBackendRBDIterateCb() which sets @diff to 1 and returns
-1. The idea is to stop iterating as soon as possible (i.e. we're
iterating over a snapshot with some deltas). Unfortunately,
returning a negative value from the callback means that iterate()
function fails and thus we report an error:
virsh # vol-clone --pool rbd_image_root coreos_2023 coreos00.disk
error: Failed to clone vol from coreos_2023
error: failed to iterate RBD snapshot coreos_2023@base: Operation not
permitted
Therefore, report an error if and only if it doesn't originates
in the callback.
Reported on libvirt-users list:
https://www.redhat.com/archives/libvirt-users/2019-April/msg00060.html
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
diff to v1:
- Ignore error as suggested by Jason as it allows us to exit iteration
early.
src/storage/storage_backend_rbd.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c
index f8c968e682..69f4fa2117 100644
--- a/src/storage/storage_backend_rbd.c
+++ b/src/storage/storage_backend_rbd.c
@@ -1101,7 +1101,9 @@ virStorageBackendRBDSnapshotFindNoDiff(rbd_image_t image,
virStorageBackendRBDIterateCb, (void *)&diff);
#endif
- if (r < 0) {
+ /* We care for errors only if the callback wasn't called at all. If it
+ * was, rbd_diff_iterate() returns an error which we have to ignore. */
+ if (r < 0 && diff == 0) {
virReportSystemError(-r, _("failed to iterate RBD snapshot %s@%s"),
imgname, snaps[i].name);
goto cleanup;
--
2.21.0
5 years, 4 months
[libvirt] [PATCH 0/4] qemu: Don't use API flags in monitor code (blockdev-add saga)
by Peter Krempa
Peter Krempa (4):
qemu: migration: Don't pass around flags for different API
qemu: monitor: Don't pass full flags to qemuMonitorJSONBlockdevMirror
qemu: monitor: Don't pass full flags to qemuMonitorJSONDriveMirror
qemu: monitor: Use VIR_AUTOPTR in
qemuMonitorJSON(Drive/Blockdev)Mirror
src/qemu/qemu_driver.c | 4 +-
src/qemu/qemu_migration.c | 21 ++++------
src/qemu/qemu_monitor.c | 17 ++++----
src/qemu/qemu_monitor.h | 5 ++-
src/qemu/qemu_monitor_json.c | 79 ++++++++++++++----------------------
src/qemu/qemu_monitor_json.h | 5 ++-
tests/qemumonitorjsontest.c | 6 +--
7 files changed, 60 insertions(+), 77 deletions(-)
--
2.21.0
5 years, 4 months
[libvirt] [PATCH 00/12] qemu: cleanups and improvements of blockjob handling (blockdev-add saga)
by Peter Krempa
Peter Krempa (12):
qemu: blockjob: Remove 'started' from struct _qemuBlockJobData
qemu: blockjob: Fix documentation for 'newstate' of _qemuBlockJobData
qemu: driver: Don't try to update blockjob status in
qemuDomainGetBlockJobInfo
qemu: driver: Set mirror state after successful command
qemu: Modernize memory cleaning in qemuDomainBlockCopyCommon
qemu: Modernize memory cleaning in qemuDomainBlockPullCommon
qemu: Modernize memory cleaning in qemuDomainBlockCommit
qemu: Remove unecessary error keeping in qemuDomainBlockCopyCommon
qemu: Remove unnecessary calls to qemuDomainStorageSourceAccessRevoke
qemu: Validate backing store of 'mirror' for block copy
qemu: Simplify allowing access to storage file for block copy
qemu: blockcopy: sanitize permission handling for 'mirror'
src/qemu/qemu_blockjob.h | 3 +-
src/qemu/qemu_driver.c | 185 ++++++++++++++++++++-------------------
2 files changed, 97 insertions(+), 91 deletions(-)
--
2.21.0
5 years, 4 months
[libvirt] [PATCH] i18n: simplify mini po generation
by Marty E. Plummer
If you pass --for-msgfmt to msgmerge it will automatically do most of
what you had the perl script doing.
Signed-off-by: Marty E. Plummer <hanetzer(a)startmail.com>
---
The only downside I can see is the loss of the comment headers with the
copyright year and such. A global note in the README.md or so may be an
acceptable change.
Makefile.am | 1 -
build-aux/minimize-po.pl | 37 -------------------------------------
po/Makefile.am | 8 +++-----
3 files changed, 3 insertions(+), 43 deletions(-)
delete mode 100755 build-aux/minimize-po.pl
diff --git a/Makefile.am b/Makefile.am
index dedd8d2ff8..ca7e012512 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -46,7 +46,6 @@ EXTRA_DIST = \
build-aux/check-spacing.pl \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
- build-aux/minimize-po.pl \
build-aux/mock-noinline.pl \
build-aux/prohibit-duplicate-header.pl \
build-aux/useless-if-before-free \
diff --git a/build-aux/minimize-po.pl b/build-aux/minimize-po.pl
deleted file mode 100755
index 497533a836..0000000000
--- a/build-aux/minimize-po.pl
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/perl
-
-my @block;
-my $msgstr = 0;
-my $empty = 0;
-my $unused = 0;
-my $fuzzy = 0;
-while (<>) {
- if (/^$/) {
- if (!$empty && !$unused && !$fuzzy) {
- print @block;
- }
- @block = ();
- $msgstr = 0;
- $fuzzy = 0;
- push @block, $_;
- } else {
- if (/^msgstr/) {
- $msgstr = 1;
- $empty = 1;
- }
- if (/^#.*fuzzy/) {
- $fuzzy = 1;
- }
- if (/^#~ msgstr/) {
- $unused = 1;
- }
- if ($msgstr && /".+"/) {
- $empty = 0;
- }
- push @block, $_;
- }
-}
-
-if (@block && !$empty && !$unused) {
- print @block;
-}
diff --git a/po/Makefile.am b/po/Makefile.am
index da117edbd5..9ac0bf3357 100644
--- a/po/Makefile.am
+++ b/po/Makefile.am
@@ -55,11 +55,9 @@ update-gmo: $(GMOFILES)
update-mini-po: $(POTFILE)
for lang in $(LANGS); do \
echo "Minimizing $$lang content" && \
- $(MSGMERGE) --no-location --no-fuzzy-matching --sort-output \
- $$lang.po $(POTFILE) | \
- $(SED) $(SED_PO_FIXUP_ARGS) | \
- $(PERL) $(top_srcdir)/build-aux/minimize-po.pl > \
- $(srcdir)/$$lang.mini.po ; \
+ $(MSGMERGE) --for-msgfmt --sort-output \
+ $$lang.po $(POTFILE) \
+ -o $(srcdir)/$$lang.mini.po ; \
done
push-pot: $(POTFILE)
--
2.21.0
5 years, 4 months
[libvirt] [PATCH] virt-xml-validate: Allow input to be read from stdin
by Johannes Holmberg
---
Hello,
This is an updated version of a patch I submitted on 2015-06-10. I got
some feedback on it but then moved on to a different project and
forgot about it. Anyway, I've updated the patch according to the
feedback so if you are still interested, here it is! :)
/Johannes
tools/virt-xml-validate.in | 44 ++++++++++++++++++++++++++++----------
1 file changed, 33 insertions(+), 11 deletions(-)
diff --git a/tools/virt-xml-validate.in b/tools/virt-xml-validate.in
index 64aeaaaa33..2d2afb74ec 100644
--- a/tools/virt-xml-validate.in
+++ b/tools/virt-xml-validate.in
@@ -16,6 +16,16 @@
set -e
+TMPFILE=
+
+cleanup() {
+ if [ $TMPFILE ]; then
+ rm -f $TMPFILE
+ fi
+}
+
+trap cleanup EXIT
+
case $1 in
-h | --h | --he | --hel | --help)
cat <<EOF
@@ -34,7 +44,7 @@ $0 (libvirt) @VERSION@
EOF
exit ;;
--) shift ;;
- -*)
+ -?*)
echo "$0: unrecognized option '$1'" >&2
exit 1 ;;
esac
@@ -42,18 +52,27 @@ esac
XMLFILE="$1"
TYPE="$2"
-if [ -z "$XMLFILE" ]; then
- echo "syntax: $0 XMLFILE [TYPE]" >&2
- exit 1
-fi
+if [ "$XMLFILE" = "-" ]; then
+ TMPFILE=`mktemp --tmpdir virt-xml.XXXX`
+ cat > $TMPFILE
+else
+ if [ -z "$XMLFILE" ]; then
+ echo "syntax: $0 XMLFILE [TYPE]" >&2
+ exit 1
+ fi
-if [ ! -f "$XMLFILE" ]; then
- echo "$0: document $XMLFILE does not exist" >&2
- exit 2
+ if [ ! -f "$XMLFILE" ]; then
+ echo "$0: document $XMLFILE does not exist" >&2
+ exit 2
+ fi
fi
if [ -z "$TYPE" ]; then
- ROOT=`xmllint --stream --debug "$XMLFILE" 2>/dev/null | grep "^0 1 " | awk '{ print $3 }'`
+ if [ $TMPFILE ]; then
+ ROOT=`xmllint --stream --debug - < "$TMPFILE" 2>/dev/null | grep "^0 1 " | awk '{ print $3 }'`
+ else
+ ROOT=`xmllint --stream --debug "$XMLFILE" 2>/dev/null | grep "^0 1 " | awk '{ print $3 }'`
+ fi
case "$ROOT" in
*domainsnapshot*) # Must come first, since *domain* is a substring
TYPE="domainsnapshot"
@@ -101,6 +120,9 @@ if [ ! -f "$SCHEMA" ]; then
exit 4
fi
-xmllint --noout --relaxng "$SCHEMA" "$XMLFILE"
-
+if [ $TMPFILE ]; then
+ xmllint --noout --relaxng "$SCHEMA" - < "$TMPFILE"
+else
+ xmllint --noout --relaxng "$SCHEMA" "$XMLFILE"
+fi
exit
--
2.17.1
5 years, 4 months