- Devel - libvirt List Archives

[libvirt] java api bindings for detecting domain events
by Avanti Ajay 19 Feb '14

19 Feb '14

Hi.. I am doing a project in which I need to detect KVM domain events and I am using java for my project. On the net I found many patches for the libvirt Java API bindings for event detection. Does the latest version 0.5 have these patches included ie can I use the method Domain.domainEventRegisterAny() directly. Also can u please tell me exactly how to use this method? Thank you Avanti

2 1

[libvirt] [bug-report] libvirtd 1.1.4-maint with lastest patches can't be connected
by Wangpan 19 Feb '14

19 Feb '14

Hi experts, Is there anybody could help me to fix this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1066801 my libvirt version is 1.1.4-manit without the newest patches after this one: 13 days ago Eric Blake event: move event filtering to daemon (regression fix) Description of problem: After I cherry-picked these patches of 1.1.4-maint branch(I want to fix the dead lock and a libvirtd crash bugs of libvirtd, https://bugzilla.redhat.com/show_bug.cgi?id=929412 https://bugzilla.redhat.com/show_bug.cgi?id=1058149) my libvirtd can be started, but the connection can't be established, and libvirtd failed to stop after 'service libvirt-bin stop', as well as 'ctrl+c' when I run 'libvirtd -v' on the foreground. Steps to Reproduce: 1. create many VMs(40 in my env) on the host 2. kill -9 `pid of libvirtd` 3. start libvirtd by using service libvirt-bin start 4. virsh list or virsh version Thanks a lot!

1 0

[libvirt] [PATCH v9 0/7] Support keyboard device
by Li Zhang 19 Feb '14

19 Feb '14

From: Li Zhang <zhlcindy(a)linux.vnet.ibm.com> This patchset is to add keyboard input device. For PPC64, it doesn't support a default keyboard device when the graphic is enabled. Libvirt supports QEMU command line as "-device VGA" which won't create any keyboard device for it. So it requires libvirt to add a default USB keyboard device for it. This patchset is to add keyboard input device and a default USB keyboard for PPC64. The related discussion in QEMU community: http://lists.nongnu.org/archive/html/qemu-devel/2013-11/msg01734.html Li Zhang (7): conf: Add one interface to add default input devices conf: Add a keyboard input device type conf: Remove the implicit PS2 devices for non-X86 platforms qemu_cap: Add USB keyboard capability qemu: format qemu command line for USB keyboard xen: format xen command line for USB keyboard Add a default USB keyboard and USB mouse for PPC64 v9 -> v8: * Move keyboard XML format to the first patch. (Jan Tomko) * Correct one logic error. (Jan Tomko) v8 -> v7: * Add USB keyboard output XML format when VIR_DOMAIN_XML_MIGRATABLE isn't set. (Daniel P. && Jan Tomko) * Fix test cases because of XML changes. v7 -> v6: * Corret indention errors in code. (Jan Tomko) * Correct make check error in xen driver. (Jan Tomko) * Split 5/6 to 2 parts, qemu and xen. (Jan Tomko) v6 -> v5: * Change kbd to keyboard, and correct -usbdevice parameter. * Remove implicit PS/2 keyboard output in XML in case older libvirtd can read it. (Jan Tomko) * Remove implicit keyboard for XEN to make it safer. v5 -> v4: * Add PS/2 KBD definition in XML file. (Daniel.P.Berrange) * Allow PS/2 KBD devices and set KBD default bus as PS2 for X86, USB for non-x86. (Daniel.P.Berrange) * Add an implicit PS/2 KBD as PS/2 mouse. (Danniel.P.Berrange) v4 -> v3: * Don't remove PS2 mouse device for other virtualization drivers (Jan Tomko). v3 -> v2: * Handle the KBD device type in xen and QEMU driver. (Daniel.P.Berrange) * Remove PS2 mouse device for non-X86 platforms. * Move virDomainDefMaybeAddInput to a new patch. (Jan Tomko) * Replace VIR_REALLOC_N with VIR_APPEND_ELEMENT. (Jan Tomoko) * Fix several typos. (Jan Tomoko) * Add a virReportError when QEMU_CAPS_DEVICE_USB_KBD can't be gotten. (Jan Tomoko) v2 -> v1: * change ifs to switch clause. * reconstruct the patches docs/schemas/domaincommon.rng | 1 + src/conf/domain_conf.c | 116 ++++++++++++++------- src/conf/domain_conf.h | 5 + src/libvirt_private.syms | 1 + src/qemu/qemu_capabilities.c | 3 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 41 ++++++-- src/qemu/qemu_domain.c | 23 +++- src/util/virarch.h | 2 + src/xenxs/xen_sxpr.c | 27 +++-- src/xenxs/xen_xm.c | 30 ++++-- tests/qemucapabilitiesdata/caps_1.2.2-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.3.1-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.4.2-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.5.3-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.6.0-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.6.50-1.caps | 1 + tests/qemuhelptest.c | 8 ++ ...qemuhotplug-console-compat-2+console-virtio.xml | 1 + .../qemuxml2argv-console-compat-2.xml | 1 + .../qemuxml2argv-graphics-listen-network.xml | 1 + .../qemuxml2argv-graphics-listen-network2.xml | 1 + .../qemuxml2argv-graphics-sdl-fullscreen.xml | 1 + .../qemuxml2argvdata/qemuxml2argv-graphics-sdl.xml | 1 + .../qemuxml2argv-graphics-spice-compression.xml | 1 + .../qemuxml2argv-graphics-spice-qxl-vga.xml | 1 + .../qemuxml2argv-graphics-spice-timeout.xml | 2 + .../qemuxml2argv-graphics-spice.xml | 1 + .../qemuxml2argv-graphics-vnc-policy.xml | 1 + .../qemuxml2argv-graphics-vnc-sasl.xml | 1 + .../qemuxml2argv-graphics-vnc-socket.xml | 1 + .../qemuxml2argv-graphics-vnc-tls.xml | 1 + .../qemuxml2argv-graphics-vnc-websocket.xml | 1 + .../qemuxml2argvdata/qemuxml2argv-graphics-vnc.xml | 1 + tests/qemuxml2argvdata/qemuxml2argv-input-xen.xml | 1 + .../qemuxml2argv-net-bandwidth.xml | 1 + tests/qemuxml2argvdata/qemuxml2argv-pci-bridge.xml | 1 + .../qemuxml2argvdata/qemuxml2argv-pseries-disk.xml | 3 +- .../qemuxml2argv-pseries-usb-kbd.args | 9 ++ .../qemuxml2argv-pseries-usb-kbd.xml | 19 ++++ .../qemuxml2argv-serial-spiceport.xml | 1 + tests/qemuxml2argvtest.c | 3 + .../qemuxml2xmlout-graphics-listen-network2.xml | 1 + .../qemuxml2xmlout-graphics-spice-timeout.xml | 1 + tests/vmx2xmldata/vmx2xml-graphics-vnc.xml | 1 + 45 files changed, 256 insertions(+), 66 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-usb-kbd.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-usb-kbd.xml -- 1.8.2.1

2 18

[libvirt] [Question] Is it ok to occupy large amount of cache when save?
by Wangyufei (James) 19 Feb '14

19 Feb '14

Hello, When I call virDomainManagedSave, libvirtd will fork a child process libvirt_iohelper to write save file, and the system cache will increase soon just like this: Swap: 0M total, 0M used, 0M free, 1668M cached Swap: 0M total, 0M used, 0M free, 1715M cached But I have cgroup to control the memory of libvirtd below 50 M, so in this case, libvirt_iohelper will be killed, and virDomainManagedSave failed. So my question is: 1. Is it ok for libvirtd to occupy large amount of cache when save? Can we flush cache in time? 2. In my situation, is it any good idea to help me out? Increase the memory limit value? Pass through the cache? Flush cache? Or any one better? Which one do you preffer? Best Regards, -WangYufei

2 1

[libvirt] ANNOUNCE: libvirt 1.1.3.4 maintenance release
by Cole Robinson 19 Feb '14

19 Feb '14

libvirt 1.1.3.4 maintenance release is now available. This is libvirt 1.1.3 with additional bugfixes that have accumulated upstream since the initial release. This release can be downloaded at: http://libvirt.org/sources/stable_updates/libvirt-1.1.3.4.tar.gz Changes in this version: * CVE-2013-6456: Avoid unsafe use of /proc/$PID/root in LXC hotunplug code * CVE-2013-6456: Avoid unsafe use of /proc/$PID/root in LXC chardev hostdev hotplug * CVE-2013-6456: Avoid unsafe use of /proc/$PID/root in LXC block hostdev hotplug * CVE-2013-6456: Avoid unsafe use of /proc/$PID/root in LXC USB hotplug * CVE-2013-6456: Avoid unsafe use of /proc/$PID/root in LXC disk hotplug * CVE-2013-6456: Avoid unsafe use of /proc/$PID/root in LXC shutdown/reboot code * Add helper for running code in separate namespaces * Add virFileMakeParentPath helper function * Move check for cgroup devices ACL upfront in LXC hotplug * Disks are always block devices, never character devices * Fix reset of cgroup when detaching USB device from LXC guests * Record hotplugged USB device in LXC live guest config * Fix path used for USB device attach with LXC * Don't block use of USB with containers * storage: avoid short reads while chasing backing chain * event: move event filtering to daemon (regression fix) * Push nwfilter update locking up to top level * Add a read/write lock implementation * tests: Add more tests for virConnectBaselineCPU * cpu: Try to use source CPU model in virConnectBaselineCPU * cpu: Fix VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES * tests: Better support for VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES * qemu: Change the default unix monitor timeout For info about past maintenance releases, see: http://wiki.libvirt.org/page/Maintenance_Releases Thanks, Cole

1 0

[libvirt] [PATCH] bridge_driver.h: Fix build --without-network
by Michal Privoznik 18 Feb '14

18 Feb '14

The networkNotifyActualDevice function is accepting two arguments, not one: qemu/qemu_process.c: In function 'qemuProcessNotifyNets': qemu/qemu_process.c:2776:47: error: macro "networkNotifyActualDevice" passed 2 arguments, but takes just 1 if (networkNotifyActualDevice(def, net) < 0) ^ Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/network/bridge_driver.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/network/bridge_driver.h b/src/network/bridge_driver.h index 0f80556..1c2a4a7 100644 --- a/src/network/bridge_driver.h +++ b/src/network/bridge_driver.h @@ -55,7 +55,7 @@ int networkDnsmasqConfContents(virNetworkObjPtr network, # else /* Define no-op replacements that don't drag in any link dependencies. */ # define networkAllocateActualDevice(dom, iface) 0 -# define networkNotifyActualDevice(iface) (iface=iface, 0) +# define networkNotifyActualDevice(dom, iface) (dom=dom, iface=iface, 0) # define networkReleaseActualDevice(dom, iface) (dom=dom, iface=iface, 0) # define networkGetNetworkAddress(netname, netaddr) (-2) # define networkDnsmasqConfContents(network, pidfile, configstr, \ -- 1.9.0

2 1

[libvirt] [PATCH] Fix conflicting types of virInitctlSetRunLevel
by Ján Tomko 18 Feb '14

18 Feb '14

aebbcdd didn't change the non-linux definition of the function, breaking the build on FreeBSD: ../../src/util/virinitctl.c:164: error: conflicting types for 'virInitctlSetRunLevel' ../../src/util/virinitctl.h:40: error: previous declaration of 'virInitctlSetRunLevel' was here --- Pushed as trivial. src/util/virinitctl.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/util/virinitctl.c b/src/util/virinitctl.c index 579268f..012200c 100644 --- a/src/util/virinitctl.c +++ b/src/util/virinitctl.c @@ -159,8 +159,7 @@ cleanup: return ret; } #else -int virInitctlSetRunLevel(virInitctlRunLevel level ATTRIBUTE_UNUSED, - const char *vroot ATTRIBUTE_UNUSED) +int virInitctlSetRunLevel(virInitctlRunLevel level ATTRIBUTE_UNUSED) { virReportUnsupportedError(); return -1; -- 1.8.3.2

1 0

[libvirt] [PATCH v4 0/3] Network hooks
by Michal Privoznik 18 Feb '14

18 Feb '14

Tier four. Michal Privoznik (3): network_conf: Expose virNetworkDefFormatInternal network: Introduce network hooks network: Taint networks that are using hook script docs/hooks.html.in | 98 ++++++++++++++++++++++++++++++++------ src/conf/network_conf.c | 64 ++++++++++++++++++++++--- src/conf/network_conf.h | 20 ++++++++ src/libvirt_private.syms | 4 ++ src/lxc/lxc_driver.c | 4 +- src/lxc/lxc_process.c | 6 +-- src/network/bridge_driver.c | 111 ++++++++++++++++++++++++++++++++++++++++++-- src/network/bridge_driver.h | 19 ++++---- src/qemu/qemu_command.c | 2 +- src/qemu/qemu_hotplug.c | 14 +++--- src/qemu/qemu_process.c | 4 +- src/util/virhook.c | 13 +++++- src/util/virhook.h | 11 +++++ 13 files changed, 321 insertions(+), 49 deletions(-) -- 1.8.5.3

2 10

[libvirt] [PATCH v9] bhyve: add a basic driver
by Roman Bogorodskiy 18 Feb '14

18 Feb '14

Changes from v8: - Implement domainUndefine and two more functions it depends on: domainIsActive and domainIsPersistent Changes from v7: - Squashed in ACL support - Check for disk and bus type for bhyve and disk type for bhyveload - Handle case when URI == NULL in ConnectOpen - Call bhyveload only after we've built bhyve command to avoid unneeded load/reload for wrong domain configuration - Cleanup unload calls on errors - Minor style fixes Changes from v6: - Fix typo: s/LIBIVRT_DRIVER_RESULT_BHYVE/LIBVIRT_DRIVER_RESULT_BHYVE/ - Report domain state in 'dominfo' - Add a patch which implements ACL support Now both 'make check' and 'make syntax-check' pass. Changes from v5: - Obtain version using uname(3) - Cleanup driver global objects in StateCleanup instead of ConnectClose Changes from v4: - Set acpi and apic flags based on domain definition - Add more detailed description about -H and -P flags of bhyve to justify theirs usage Roman Bogorodskiy (1): bhyve: add a basic driver configure.ac | 7 + daemon/libvirtd.c | 9 + include/libvirt/virterror.h | 1 + m4/virt-driver-bhyve.m4 | 57 ++++ po/POTFILES.in | 3 + src/Makefile.am | 31 ++ src/bhyve/bhyve_command.c | 314 ++++++++++++++++++++ src/bhyve/bhyve_command.h | 41 +++ src/bhyve/bhyve_driver.c | 707 ++++++++++++++++++++++++++++++++++++++++++++ src/bhyve/bhyve_driver.h | 28 ++ src/bhyve/bhyve_process.c | 227 ++++++++++++++ src/bhyve/bhyve_process.h | 36 +++ src/bhyve/bhyve_utils.h | 49 +++ src/conf/domain_conf.c | 3 +- src/conf/domain_conf.h | 1 + src/driver.h | 1 + src/libvirt.c | 3 + src/util/virerror.c | 1 + 18 files changed, 1518 insertions(+), 1 deletion(-) create mode 100644 m4/virt-driver-bhyve.m4 create mode 100644 src/bhyve/bhyve_command.c create mode 100644 src/bhyve/bhyve_command.h create mode 100644 src/bhyve/bhyve_driver.c create mode 100644 src/bhyve/bhyve_driver.h create mode 100644 src/bhyve/bhyve_process.c create mode 100644 src/bhyve/bhyve_process.h create mode 100644 src/bhyve/bhyve_utils.h -- 1.8.4.3

3 6

[libvirt] [PATCH REPOST] Introduce new OOM testing support
by Daniel P. Berrange 18 Feb '14

18 Feb '14

The previous OOM testing support would re-run the entire "main" method each iteration, failing a different malloc each time. When a test suite has 'n' allocations, the number of repeats requires is (n * (n + 1) ) / 2. This gets very large, very quickly. This new OOM testing support instead integrates at the virtTestRun level, so each individual test case gets repeated, instead of the entire test suite. This means the values of 'n' are orders of magnitude smaller. The simple usage is $ VIR_TEST_OOM=1 ./qemuxml2argvtest ... 29) QEMU XML-2-ARGV clock-utc ... OK Test OOM for nalloc=36 .................................... OK 30) QEMU XML-2-ARGV clock-localtime ... OK Test OOM for nalloc=36 .................................... OK 31) QEMU XML-2-ARGV clock-france ... OK Test OOM for nalloc=38 ...................................... OK ... the second lines reports how many mallocs have to be failed, and thus how many repeats of the test will be run. If it crashes, then running under valgrind will often show the problem $ VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest When debugging problems it is also helpful to select an individual test case $ VIR_TEST_RANGE=30 VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest When things get really tricky, it is possible to request that just specific allocs are failed. eg to fail allocs 5 -> 12, use $ VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-12 ../run valgrind ./qemuxml2argvtest In the worse case, you might want to know the stack trace of the alloc which was failed then VIR_TEST_OOM_TRACE can be set. If it is set to 1 then it will only print if it thinks a mistake happened. This is often not reliable, so setting it to 2 will make it print the stack trace for every alloc that is failed. $ VIR_TEST_OOM_TRACE=2 VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-5 ../run valgrind ./qemuxml2argvtest 30) QEMU XML-2-ARGV clock-localtime ... OK Test OOM for nalloc=36 !virAllocN /home/berrange/src/virt/libvirt/src/util/viralloc.c:180 virHashCreateFull /home/berrange/src/virt/libvirt/src/util/virhash.c:144 virDomainDefParseXML /home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11745 virDomainDefParseNode /home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12646 virDomainDefParse /home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12590 testCompareXMLToArgvFiles /home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:106 virtTestRun /home/berrange/src/virt/libvirt/tests/testutils.c:250 mymain /home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:418 (discriminator 2) virtTestMain /home/berrange/src/virt/libvirt/tests/testutils.c:750 ?? ??:0 _start ??:? FAILED Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- cfg.mk | 2 +- docs/internals/oomtesting.html.in | 213 ++++++++++++++++++++++++++++++++++++++ docs/sitemap.html.in | 4 + tests/qemuargv2xmltest.c | 14 +-- tests/qemuxml2argvtest.c | 12 ++- tests/qemuxmlnstest.c | 18 ++-- tests/testutils.c | 195 +++++++++++++++++++++++++++++++++- tests/testutils.h | 2 + 8 files changed, 438 insertions(+), 22 deletions(-) create mode 100644 docs/internals/oomtesting.html.in diff --git a/cfg.mk b/cfg.mk index 207dfeb..bbc966b 100644 --- a/cfg.mk +++ b/cfg.mk @@ -948,7 +948,7 @@ exclude_file_name_regexp--sc_libvirt_unmarked_diagnostics = \ exclude_file_name_regexp--sc_po_check = ^(docs/|src/rpc/gendispatch\.pl$$) exclude_file_name_regexp--sc_prohibit_VIR_ERR_NO_MEMORY = \ - ^(include/libvirt/virterror\.h|daemon/dispatch\.c|src/util/virerror\.c)$$ + ^(include/libvirt/virterror\.h|daemon/dispatch\.c|src/util/virerror\.c|docs/internals/oomtesting\.html\.in)$$ exclude_file_name_regexp--sc_prohibit_access_xok = ^src/util/virutil\.c$$ diff --git a/docs/internals/oomtesting.html.in b/docs/internals/oomtesting.html.in new file mode 100644 index 0000000..c5edacf --- /dev/null +++ b/docs/internals/oomtesting.html.in @@ -0,0 +1,213 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + <h1>Out of memory testing</h1> + + <ul id="toc"></ul> + + + <p> + This page describes how to use the test suite todo out of memory + testing. + </p> + + <h2>Building with OOM testing</h2> + + <p> + Since OOM testing requires hooking into the malloc APIs, it is + not enabled by default. The flag <code>--enable-test-oom</code> + must be given to <code>configure</code>. When this is done the + libvirt allocation APIs will have some hooks enabled. + </p> + + <pre> +$ ./configure --enable-test-oom +</pre> + + + <h2><a name="basicoom">Basic OOM testing support</a></h2> + + <p> + The first step in validating OOM usage is to run a test suite + with full OOM testing enabled. This is done by setting the + <code>VIR_TEST_OOM=1</code> environment variable. The way this + works is that it runs the test once normally to "prime" any + static memory allocations. Then it runs it once more counting + the total number of memory allocations. Then it runs it in a + loop failing a different memory allocation each time. For every + memory allocation failure triggered, it expects the test case + to return an error. OOM testing is quite slow requiring each + test case to be executed O(n) times, where 'n' is the total + number of memory allocations. This results in a total number + of memory allocations of '(n * (n + 1) ) / 2' + </p> + + <pre> +$ VIR_TEST_OOM=1 ./qemuxml2argvtest + 1) QEMU XML-2-ARGV minimal ... OK + Test OOM for nalloc=42 .......................................... OK + 2) QEMU XML-2-ARGV minimal-s390 ... OK + Test OOM for nalloc=28 ............................ OK + 3) QEMU XML-2-ARGV machine-aliases1 ... OK + Test OOM for nalloc=38 ...................................... OK + 4) QEMU XML-2-ARGV machine-aliases2 ... OK + Test OOM for nalloc=38 ...................................... OK + 5) QEMU XML-2-ARGV machine-core-on ... OK + Test OOM for nalloc=37 ..................................... OK +...snip... +</pre> + + <p> + In this output, the first line shows the normal execution and + the test number, and the second line shows the total number + of memory allocations from that test case. + </p> + + <h3><a name="valgrind">Tracking failures with valgrind</a></h3> + + <p> + The test suite should obviously *not* crash during OOM testing. + If it does crash, then to assist in tracking down the problem + it is worth using valgrind and only running a single test case. + For example, supposing test case 5 crashed. Then re-run the + test with + </p> + + <pre> +$ VIR_TEST_OOM=1 VIR_TEST_RANGE=5 ../run valgrind ./qemuxml2argvtest +...snip... + 5) QEMU XML-2-ARGV machine-core-on ... OK + Test OOM for nalloc=37 ..................................... OK +...snip... + </pre> + + <p> + Valgrind should report the cause of the crash - for example a + double free or use of uninitialized memory or NULL pointer + access. + </p> + + <h3><a name="stacktraces">Tracking failures with stack traces</a></h3> + + <p> + With some really difficult bugs valgrind is not sufficient to + identify the cause. In this case, it is useful to identify the + precise allocation which was failed, to allow the code path + to the error to be traced. The <code>VIR_TEST_OOM</code> + env variable can be given a range of memory allocations to + test. So if a test case has 150 allocations, it can be told + to only test allocation numbers 7-10. The <code>VIR_TEST_OOM_TRACE</code> + variable can be used to print out stack traces. + </p> + + <pre> +$ VIR_TEST_OOM_TRACE=2 VIR_TEST_OOM=1:7-10 VIR_TEST_RANGE=5 \ + ../run valgrind ./qemuxml2argvtest + 5) QEMU XML-2-ARGV machine-core-on ... OK + Test OOM for nalloc=37 !virAllocN +/home/berrange/src/virt/libvirt/src/util/viralloc.c:180 +virDomainDefParseXML +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11786 (discriminator 1) +virDomainDefParseNode +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677 +virDomainDefParse +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621 +testCompareXMLToArgvFiles +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107 +virtTestRun +/home/berrange/src/virt/libvirt/tests/testutils.c:266 +mymain +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2) +virtTestMain +/home/berrange/src/virt/libvirt/tests/testutils.c:791 +__libc_start_main +??:? +_start +??:? +!virAlloc +/home/berrange/src/virt/libvirt/src/util/viralloc.c:133 +virDomainDiskDefParseXML +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:4790 +virDomainDefParseXML +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11797 +virDomainDefParseNode +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677 +virDomainDefParse +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621 +testCompareXMLToArgvFiles +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107 +virtTestRun +/home/berrange/src/virt/libvirt/tests/testutils.c:266 +mymain +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2) +virtTestMain +/home/berrange/src/virt/libvirt/tests/testutils.c:791 +__libc_start_main +??:? +_start +??:? +!virAllocN +/home/berrange/src/virt/libvirt/src/util/viralloc.c:180 +virXPathNodeSet +/home/berrange/src/virt/libvirt/src/util/virxml.c:609 +virDomainDefParseXML +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11805 +virDomainDefParseNode +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677 +virDomainDefParse +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621 +testCompareXMLToArgvFiles +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107 +virtTestRun +/home/berrange/src/virt/libvirt/tests/testutils.c:266 +mymain +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2) +virtTestMain +/home/berrange/src/virt/libvirt/tests/testutils.c:791 +__libc_start_main +??:? +_start +??:? +!virAllocN +/home/berrange/src/virt/libvirt/src/util/viralloc.c:180 +virDomainDefParseXML +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11808 (discriminator 1) +virDomainDefParseNode +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677 +virDomainDefParse +/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621 +testCompareXMLToArgvFiles +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107 +virtTestRun +/home/berrange/src/virt/libvirt/tests/testutils.c:266 +mymain +/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2) +virtTestMain +/home/berrange/src/virt/libvirt/tests/testutils.c:791 +__libc_start_main +??:? +_start +??:? + </pre> + + <h3><a name="noncrash">Non-crash related problems</a></h3> + + <p> + Not all memory allocation bugs result in code crashing. Sometimes + the code will be silently ignoring the allocation failure, resulting + in incorrect data being produced. For example the XML parser may + mistakenly treat an allocation failure as indicating that an XML + attribute was not set in the input document. It is hard to identify + these problems from the test suite automatically. For this, the + test suites should be run with <code>VIR_TEST_DEBUG=1</code> set + and then stderr analysed for any unexpected data. For example, + the XML conversion may show an embedded "(null)" literal, or the + test suite might complain about missing elements / attributes + in the actual vs expected data. These are all signs of bugs in + OOM handling. In the future the OOM tests will be enhanced to + validate that an error VIR_ERR_NO_MEMORY is returned for each + allocation failed, rather than some other error. + </p> + </body> +</html> diff --git a/docs/sitemap.html.in b/docs/sitemap.html.in index 08c2fbd..7d0610b 100644 --- a/docs/sitemap.html.in +++ b/docs/sitemap.html.in @@ -342,6 +342,10 @@ <a href="internals/locking.html">Lock managers</a> <span>Use lock managers to protect disk content</span> </li> + <li> + <a href="internals/oomtesting.html">Out of memory testing</a> + <span>Simulating OOM conditions in the test suite</span> + </li> </ul> </li> <li> diff --git a/tests/qemuargv2xmltest.c b/tests/qemuargv2xmltest.c index 1b13fcc..e75c85d 100644 --- a/tests/qemuargv2xmltest.c +++ b/tests/qemuargv2xmltest.c @@ -42,7 +42,6 @@ static int testCompareXMLToArgvFiles(const char *xml, char *cmd = NULL; int ret = -1; virDomainDefPtr vmdef = NULL; - char *log; if (virtTestLoadFile(cmdfile, &cmd) < 0) goto fail; @@ -53,13 +52,16 @@ static int testCompareXMLToArgvFiles(const char *xml, cmd, NULL, NULL, NULL))) goto fail; - if ((log = virtTestLogContentAndReset()) == NULL) - goto fail; - if ((*log != '\0') != expect_warning) { + if (!virtTestOOMActive()) { + char *log; + if ((log = virtTestLogContentAndReset()) == NULL) + goto fail; + if ((*log != '\0') != expect_warning) { + VIR_FREE(log); + goto fail; + } VIR_FREE(log); - goto fail; } - VIR_FREE(log); if (!virDomainDefCheckABIStability(vmdef, vmdef)) { fprintf(stderr, "ABI stability check failed on %s", xml); diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 7a5b50d..26d63d3 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -284,7 +284,8 @@ static int testCompareXMLToArgvFiles(const char *xml, if (!(vmdef = virDomainDefParseFile(xml, driver.caps, driver.xmlopt, QEMU_EXPECTED_VIRT_TYPES, VIR_DOMAIN_XML_INACTIVE))) { - if (flags & FLAG_EXPECT_PARSE_ERROR) + if (!virtTestOOMActive() && + (flags & FLAG_EXPECT_PARSE_ERROR)) goto ok; goto out; } @@ -357,7 +358,8 @@ static int testCompareXMLToArgvFiles(const char *xml, migrateFrom, migrateFd, NULL, VIR_NETDEV_VPORT_PROFILE_OP_NO_OP, &testCallbacks))) { - if (flags & FLAG_EXPECT_FAILURE) { + if (!virtTestOOMActive() && + (flags & FLAG_EXPECT_FAILURE)) { ret = 0; if (virTestGetDebug() > 1) fprintf(stderr, "Got expected error: %s\n", @@ -371,7 +373,8 @@ static int testCompareXMLToArgvFiles(const char *xml, goto out; } - if (!!virGetLastError() != !!(flags & FLAG_EXPECT_ERROR)) { + if (!virtTestOOMActive() && + (!!virGetLastError() != !!(flags & FLAG_EXPECT_ERROR))) { if (virTestGetDebug() && (log = virtTestLogContentAndReset())) fprintf(stderr, "\n%s", log); goto out; @@ -392,7 +395,8 @@ static int testCompareXMLToArgvFiles(const char *xml, } ok: - if (flags & FLAG_EXPECT_ERROR) { + if (!virtTestOOMActive() && + (flags & FLAG_EXPECT_ERROR)) { /* need to suppress the errors */ virResetLastError(); } diff --git a/tests/qemuxmlnstest.c b/tests/qemuxmlnstest.c index 9426bf7..20a5ccd 100644 --- a/tests/qemuxmlnstest.c +++ b/tests/qemuxmlnstest.c @@ -122,15 +122,17 @@ static int testCompareXMLToArgvFiles(const char *xml, &testCallbacks))) goto fail; - if (!!virGetLastError() != expectError) { - if (virTestGetDebug() && (log = virtTestLogContentAndReset())) - fprintf(stderr, "\n%s", log); - goto fail; - } + if (!virtTestOOMActive()) { + if (!!virGetLastError() != expectError) { + if (virTestGetDebug() && (log = virtTestLogContentAndReset())) + fprintf(stderr, "\n%s", log); + goto fail; + } - if (expectError) { - /* need to suppress the errors */ - virResetLastError(); + if (expectError) { + /* need to suppress the errors */ + virResetLastError(); + } } if (!(actualargv = virCommandToString(cmd))) diff --git a/tests/testutils.c b/tests/testutils.c index 32fe374..52d07eb 100644 --- a/tests/testutils.c +++ b/tests/testutils.c @@ -47,6 +47,13 @@ #include "virprocess.h" #include "virstring.h" +#ifdef TEST_OOM +# ifdef TEST_OOM_TRACE +# include <dlfcn.h> +# include <execinfo.h> +# endif +#endif + #ifdef HAVE_PATHS_H # include <paths.h> #endif @@ -64,12 +71,37 @@ static unsigned int testDebug = -1; static unsigned int testVerbose = -1; static unsigned int testExpensive = -1; +#ifdef TEST_OOM +static unsigned int testOOM = 0; +static unsigned int testOOMStart = -1; +static unsigned int testOOMEnd = -1; +static unsigned int testOOMTrace = 0; +# ifdef TEST_OOM_TRACE +void *testAllocStack[30]; +int ntestAllocStack; +# endif +#endif +static bool testOOMActive = false; + static size_t testCounter = 0; static size_t testStart = 0; static size_t testEnd = 0; char *progname; +bool virtTestOOMActive(void) +{ + return testOOMActive; +} + +#ifdef TEST_OOM_TRACE +static void virTestAllocHook(int nalloc ATTRIBUTE_UNUSED, + void *opaque ATTRIBUTE_UNUSED) +{ + ntestAllocStack = backtrace(testAllocStack, ARRAY_CARDINALITY(testAllocStack)); +} +#endif + void virtTestResult(const char *name, int ret, const char *msg, ...) { va_list vargs; @@ -108,6 +140,35 @@ void virtTestResult(const char *name, int ret, const char *msg, ...) va_end(vargs); } +#ifdef TEST_OOM_TRACE +static void +virTestShowTrace(void) +{ + size_t j; + for (j = 2; j < ntestAllocStack; j++) { + Dl_info info; + char *cmd; + + dladdr(testAllocStack[j], &info); + if (info.dli_fname && + strstr(info.dli_fname, ".so")) { + if (virAsprintf(&cmd, "addr2line -f -e %s %p", + info.dli_fname, + ((void*)((unsigned long long)testAllocStack[j] + - (unsigned long long)info.dli_fbase))) < 0) + continue; + } else { + if (virAsprintf(&cmd, "addr2line -f -e %s %p", + (char*)(info.dli_fname ? info.dli_fname : "<unknown>"), + testAllocStack[j]) < 0) + continue; + } + ignore_value(system(cmd)); + VIR_FREE(cmd); + } +} +#endif + /* * Runs test * @@ -154,7 +215,7 @@ virtTestRun(const char *title, !((testCounter-1) % 40)) { fprintf(stderr, " %-3zu\n", (testCounter-1)); fprintf(stderr, " "); - } + } if (ret == 0) fprintf(stderr, "."); else if (ret == EXIT_AM_SKIP) @@ -163,6 +224,77 @@ virtTestRun(const char *title, fprintf(stderr, "!"); } +#ifdef TEST_OOM + if (testOOM && ret != EXIT_AM_SKIP) { + int nalloc; + int oomret; + int start, end; + size_t i; + virResetLastError(); + virAllocTestInit(); +# ifdef TEST_OOM_TRACE + virAllocTestHook(virTestAllocHook, NULL); +# endif + oomret = body(data); + nalloc = virAllocTestCount(); + fprintf(stderr, " Test OOM for nalloc=%d ", nalloc); + if (testOOMStart == -1 || + testOOMEnd == -1) { + start = 0; + end = nalloc; + } else { + start = testOOMStart; + end = testOOMEnd + 1; + } + testOOMActive = true; + for (i = start; i < end; i++) { + bool missingFail = false; +# ifdef TEST_OOM_TRACE + memset(testAllocStack, 0, ARRAY_CARDINALITY(testAllocStack)); + ntestAllocStack = 0; +# endif + virAllocTestOOM(i + 1, 1); + oomret = body(data); + + /* fprintf() disabled because XML parsing APIs don't allow + * distinguish between element / attribute not present + * in the XML (which is non-fatal), vs OOM / malformed + * which should be fatal. Thus error reporting for + * optionally present XML is mostly broken. + */ + if (oomret == 0) { + missingFail = true; +# if 0 + fprintf(stderr, " alloc %zu failed but no err status\n", i + 1); +# endif + } else { + virErrorPtr lerr = virGetLastError(); + if (!lerr) { +# if 0 + fprintf(stderr, " alloc %zu failed but no error report\n", i + 1); +# endif + missingFail = true; + } + } + if ((missingFail && testOOMTrace) || (testOOMTrace > 1)) { + fprintf(stderr, "%s", "!"); +# ifdef TEST_OOM_TRACE + virTestShowTrace(); +# endif + ret = -1; + } else { + fprintf(stderr, "%s", "."); + } + } + testOOMActive = false; + if (ret == 0) + fprintf(stderr, " OK\n"); + else + fprintf(stderr, " FAILED\n"); + virAllocTestInit(); + } +#endif /* TEST_OOM */ + return ret; } @@ -191,7 +323,6 @@ virtTestLoadFile(const char *file, char **buf) tmplen = buflen = st.st_size + 1; if (VIR_ALLOC_N(*buf, buflen) < 0) { - fprintf(stderr, "%s: larger than available memory (> %d)\n", file, buflen); VIR_FORCE_FCLOSE(fp); return -1; } @@ -467,7 +598,8 @@ virtTestLogOutput(virLogSource source ATTRIBUTE_UNUSED, { struct virtTestLogData *log = data; virCheckFlags(VIR_LOG_STACK_TRACE,); - virBufferAsprintf(&log->buf, "%s: %s", timestamp, str); + if (!testOOMActive) + virBufferAsprintf(&log->buf, "%s: %s", timestamp, str); } static void @@ -535,6 +667,9 @@ int virtTestMain(int argc, { int ret; char *testRange = NULL; +#ifdef TEST_OOM + char *oomstr; +#endif if (!virFileExists(abs_srcdir)) return EXIT_AM_HARDFAIL; @@ -590,6 +725,60 @@ int virtTestMain(int argc, } } +#ifdef TEST_OOM + if ((oomstr = getenv("VIR_TEST_OOM")) != NULL) { + char *next; + if (testDebug == -1) + testDebug = 1; + testOOM = 1; + if (oomstr[0] != '\0' && + oomstr[1] == ':') { + if (virStrToLong_ui(oomstr + 2, &next, 10, &testOOMStart) < 0) { + fprintf(stderr, "Cannot parse range %s\n", oomstr); + return EXIT_FAILURE; + } + if (*next == '\0') { + testOOMEnd = testOOMStart; + } else { + if (*next != '-') { + fprintf(stderr, "Cannot parse range %s\n", oomstr); + return EXIT_FAILURE; + } + if (virStrToLong_ui(next+1, NULL, 10, &testOOMEnd) < 0) { + fprintf(stderr, "Cannot parse range %s\n", oomstr); + return EXIT_FAILURE; + } + } + } else { + testOOMStart = -1; + testOOMEnd = -1; + } + } + +# ifdef TEST_OOM_TRACE + if ((oomstr = getenv("VIR_TEST_OOM_TRACE")) != NULL) { + if (virStrToLong_ui(oomstr, NULL, 10, &testOOMTrace) < 0) { + fprintf(stderr, "Cannot parse oom trace %s\n", oomstr); + return EXIT_FAILURE; + } + } +# else + if (getenv("VIR_TEST_OOM_TRACE")) { + fprintf(stderr, "%s", "OOM test tracing not enabled in this build\n"); + return EXIT_FAILURE; + } +# endif +#else /* TEST_OOM */ + if (getenv("VIR_TEST_OOM")) { + fprintf(stderr, "%s", "OOM testing not enabled in this build\n"); + return EXIT_FAILURE; + } + if (getenv("VIR_TEST_OOM_TRACE")) { + fprintf(stderr, "%s", "OOM test tracing not enabled in this build\n"); + return EXIT_FAILURE; + } +#endif /* TEST_OOM */ + ret = (func)(); virResetLastError(); diff --git a/tests/testutils.h b/tests/testutils.h index 674d3df..29eb9d9 100644 --- a/tests/testutils.h +++ b/tests/testutils.h @@ -44,6 +44,8 @@ extern char *progname; # error Fix Makefile.am # endif +bool virtTestOOMActive(void); + void virtTestResult(const char *name, int ret, const char *msg, ...) ATTRIBUTE_FMT_PRINTF(3,4); int virtTestRun(const char *title, -- 1.8.5.3

2 1