October 2009 - Devel - libvirt List Archives

[libvirt] Safe? /etc/libvirt/qemu on shared storage?
by James Brackinshaw 05 Oct '09

05 Oct '09

Hello, I have a directory /mnt/shared-storage where the disk files for kvm live. Live migration works well, but I have a problem: after a migration I see two guests, one on the original host "shutoff" and one on the new host "running". I'd like to see the guest only on the host it is running on. Would it be safe/recommended to put the /etc/libvirt/qemu directory on shared storage? What would the best way of doing this be? Is this lock safe? Thanks JB

5 7

[libvirt] Update libvirt API and refs XML files
by Matthias Bolte 02 Oct '09

02 Oct '09

Commit 47c8709564ac3f2ec5b4a3830ac213e69b4a44d5 "Fix up a few typos in the tree" triggered changes in docs/libvirt-api.xml and docs/libvirt-refs.xml. The attached patch contains this changes. The files docs/libvirt-api.xml and docs/libvirt-refs.xml are generated by apibuild.py, why are this generated files under version control? Matthias

6 8

[libvirt] Performance statistics
by James Brackinshaw 02 Oct '09

02 Oct '09

Hello, Is it possible to get at the (historical) performance data provided by libvirtd? Is it stored for a period of time, or only available to clients polling it? Thanks JB

3 3

[libvirt] [PATCH 00/11] Generic data stream handling
by Daniel P. Berrange 02 Oct '09

02 Oct '09

The following series of patches introduce support for generic data streams in the libvirt API, the remote protocol, client & daemon. The public API takes the form of a new object virStreamPtr and methods to read/write/close it The remote protocol was the main hard bit. Since the protocol allows multiple concurrent API calls on a single connection, this needed to also allow concurrent data streams. It is also not acceptable for a large data stream to block other traffic while it is transferring. Thus, we introduce a new protocol message type 'REMOTE_STREAM' to handle transfer for the stream data. A method involving a data streams starts off in the normal way with a REMOTE_CALL to the server, and a REMOTE_REPLY response message. If this was successful, there now follows the data stream traffic. For outgoing streams (data from client to server), the client will send zero or more REMOTE_STREAM packets containing the data with status == REMOTE_CONTINUE. These are asynchronous and not acknowledged by the server. At any time the server may send an async message with a type of REMOTE_STREAM and status of REMOTE_ERROR. This indicates to the client that the transfer is aborting at server request. If the client wishes to abort, it can send the server a REMOTE_STREAM+REMOTE_ERROR message. If the client finishes its data transfer, it will send a final REMOTE_STREAM+REMOTE_OK message, and the server will respond with the same. This full roundtrip handshake ensures any async error messages are guarenteed to be flushed For incoming data streams (data from server to client), the server sends zero or more REMOTE_STREAM packets containing the data with status == REMOTE_CONTINUE. These are asynchronous and not acknowledged by the client. At any time the client may send an async message with a type of REMOTE_STREAM and status of REMOTE_ERROR. This indicates to the server that the transfer is aborting at client request. If the server wishes to abort, it can send the server a REMOTE_STREAM+REMOTE_ERROR message. When the server finishes its data transfer, it will send a final REMOTE_STREAM+REMOTE_CONTINUE message ewith a data length of zero (ie EOF). The client will then send a REMOTE_STREAM+REMOTE_OK packet and the server will respond with the same. This full roundtrip handshake ensures any async error messages are guarenteed to be flushed This all ensures that multiple data streams can be active in parallel, and with a maximum data packet size of 256 KB, no single stream can cause too much latency on the connection for other API calls/streams. The only thing it does not allow for is one API method to use two or more streams. These may be famous last words, but I don't think that use case will be neccessary for any of our APIs... The last 5 patches with a subject of [DEMO] are *NOT* intended to be committed to the repository. They merely demonstrate the use of data streams for a couple of hypothetical file upload and download APIs. Actually they were mostly to allow me to test the code streams code without messing around with the QEMU migration code. The immediate use case for this data stream code is Chris' QEMU migration patchset. The next use case is to allow serial console access to be tunnelled over libvirtd, eg to make 'virsh console GUEST' work remotely. This use case is why I included the support for non-blocking data streams and event loop integration (not required for Chris' migration use case) Anyway, assuming Chris confirms that I've not broken his code, then patches 1-6 are targetted for this next release. .x-sc_avoid_write | 3 include/libvirt/libvirt.h | 101 ++++ include/libvirt/libvirt.h.in | 101 ++++ po/POTFILES.in | 1 qemud/Makefile.am | 1 qemud/dispatch.c | 131 ++++++ qemud/dispatch.h | 17 qemud/event.c | 54 +- qemud/qemud.c | 56 +- qemud/qemud.h | 33 + qemud/remote.c | 237 ++++++++++- qemud/remote.h | 1 qemud/remote_dispatch_args.h | 2 qemud/remote_dispatch_prototypes.h | 153 +++++++ qemud/remote_dispatch_table.h | 10 qemud/remote_generate_stubs.pl | 1 qemud/remote_protocol.c | 18 qemud/remote_protocol.h | 20 qemud/remote_protocol.x | 57 ++ qemud/stream.c | 605 +++++++++++++++++++++++++++++ qemud/stream.h | 53 ++ src/datatypes.c | 59 ++ src/datatypes.h | 33 + src/driver.h | 40 + src/esx/esx_driver.c | 2 src/libvirt.c | 769 +++++++++++++++++++++++++++++++++++++ src/libvirt_private.syms | 2 src/libvirt_public.syms | 18 src/lxc_driver.c | 2 src/opennebula/one_driver.c | 2 src/openvz_driver.c | 2 src/qemu_driver.c | 2 src/remote_internal.c | 601 ++++++++++++++++++++++++++++ src/test.c | 477 ++++++++++++++++++++++ src/uml_driver.c | 2 src/vbox/vbox_tmpl.c | 3 src/virsh.c | 164 +++++++ src/xen_unified.c | 2 tests/eventtest.c | 19 39 files changed, 3783 insertions(+), 71 deletions(-) Regards, Daniel

5 43

[libvirt] [PATCH 1/2] Fix ordering of <exports> in API description file
by Daniel P. Berrange 02 Oct '09

02 Oct '09

* docs/apibuild.py: Make uniq() function sort keys before returning them to ensure consitent ordering of <exports> in XML file --- docs/apibuild.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/docs/apibuild.py b/docs/apibuild.py index b00619f..c5257f3 100755 --- a/docs/apibuild.py +++ b/docs/apibuild.py @@ -55,7 +55,9 @@ def uniq(items): d = {} for item in items: d[item]=1 - return d.keys() + k = d.keys() + k.sort() + return k class identifier: def __init__(self, name, header=None, module=None, type=None, lineno = 0, -- 1.6.2.5

2 3

[libvirt] Tunnelled migration
by Chris Lalancette 02 Oct '09

02 Oct '09

At long last, now that the stream stuff is committed to the tree, I can repost the tunnelled migration patches. Note that these patches will conflict in some places with danpb's previously posted "PEER2PEER" patches, but it should be easy to resolve the differences. V2: Rework the monitor patch to be an API Use the new monitor API Remove silly use of qemuEscapeMonitorArgs()

2 6

[libvirt] Silently ignored virDomainRestore failures
by Charles Duffy 02 Oct '09

02 Oct '09

Howdy, all. I maintain a test infrastructure which makes heavy use of virDomainSave and virDomainRestore, and have been seeing occasional cases where my saved images are for some reason not restored correctly -- and, indeed, the incoming migration streams are not even read in their entirety. While this generally appears to be caused by issues outside of libvirt's purview, one unfortunate issue is that libvirt can report success performing a restore even when the operation is effectively an abject failure. Consider the following snippet, taken from one of my /var/log/libvirt/qemu/<domain>.log files: LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin USER=root LOGNAME=root /usr/bin/qemu-kvm -S -M pc-0.11 -m 512 -smp 1 <...lots of arguments here...> -incoming exec:cat cat: write error: Broken pipe This leaves a running qemu hosting a catatonic guest -- but the libvirt client (connecting through the Python bindings) received a status of success for the operation given here. libvirt's mechanism for validating a successful restore consists of running a "cont" command on the guest, and then checking virGetLastError(); AIUI, it is expected that the "cont" will not be able to run until the restore is completed, as the monitor should not be responsive until that time. Browsing through qemudMonitorSendCont (and qemudMonitorCommandWithHandler, which it calls), I don't see anything which looks at the log file with the stderr output from qemu to determine whether an error actually occurred. (As an aside, "info history" on the guest's monitor socket indicates that it was indeed issued this "cont"). Should the existing cont+virGetLastError() approach be sufficient to handle this class of error? If not, is there any guidance on what would comprise a better system? (I suppose we could add something to the exec: to affirmatively indicate on stderr that the decompressor [or cat, if not using one] exited successfully, and check for that marker in the log file... but that seems quite a dirty hack). Thanks!

2 7

[libvirt] [PATCH 00/27] Refactor QEMU monitor command handling
by Daniel P. Berrange 01 Oct '09

01 Oct '09

In the QEMU driver source code the methods which talk to the QEMU monitor currently all just call qemudMonitorCommand() directly with the raw command string, and then parse the raw reply. In the not too distant future QEMU is introducing a new machine parsable monitor syntax. With the current way the code is structured supporting this new mode will be seriously unpleasant. This large series of patches, moves all the monitor command formatting and parsing code out into a separate source file src/qemu/qemu_monitor_text.c. There is one API in that file for each logical monitor command we wish to issue, accepting (mostly) strongly typed arguments. The exception is the NIC hotplug method which still takes the raw NIC string for now. The main qemu_driver.c file now directly calls the appropriate monitor command APIs, making logic there much cleaner. When we add support for the new QEMU monitor syntax we'll gain another file src/qemu/qemu_monitor_json.c which implements all the same APIs as src/qemu/qemu_monitor_text.c, but using the new JSON syntax instead of raw text strings This patch series is soooooo large, because I did it in many small steps, one command at a time. It is also now much easier to debug the monitor by just setting the env variables LIBVIRT_LOG_OUTPUTS="1:stderr" LIBVIRT_LOG_FILTERS="1:qemu_monitor" And you'll get the command & reply of each monitor interaction printed I've tested basic handling of every new method with the exception of the migration ones, since I don't have a convenient target host when on my laptop. Overall we get a small increase in code size, but huge increase in readability ! Makefile.am | 1 libvirt_private.syms | 1 qemu/qemu_conf.c | 20 qemu/qemu_conf.h | 2 qemu/qemu_driver.c | 1669 +++++--------------------------------------- qemu/qemu_monitor_text.c | 1777 +++++++++++++++++++++++++++++++++++++++++++++++ qemu/qemu_monitor_text.h | 173 ++++ 7 files changed, 2171 insertions(+), 1472 deletions(-) Daniel

4 63

[libvirt] Re: [virt-tools-list] Some VMs not starting at host boot
by Cole Robinson 01 Oct '09

01 Oct '09

On 10/01/2009 11:07 AM, Daniel Huhardeaux wrote: > Hi, > > 3 VMs on a host, all in autostart: the both where > > <disk type='block' device='disk'> > <source dev='/dev/vg0/DomU1'/> > <target dev='vda' bus='virtio'/> > </disk> > > are not started automagically on host startup. They start without > problem from virsh. The one having > > <disk type='file' device='disk'> > <source file='/media/vm2/vdisk.qcow2'/> > <target dev='hda' bus='ide'/> > </disk> > > has no problem. Any idea why? FYI, libvirt/virsh specific questions should go to libvirt-list(a)redhat.com. CCing Try 'virsh dominfo <vmname>' for each VM to ensure autostart is actually enabled. To see why autostart is failing, you want to launch libvirtd by hand. Stop all your VMs, do a service libvirtd stop, then (as root) run 'libvirtd'. Any VM startup error messages should be printed on the console. - Cole

2 1

[libvirt] [PATCH] Fix build in separate build directory
by Jiri Denemark 01 Oct '09

01 Oct '09

Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- cfg.mk | 2 +- examples/domain-events/events-c/Makefile.am | 2 +- examples/hellolibvirt/Makefile.am | 2 +- tools/Makefile.am | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/cfg.mk b/cfg.mk index c217332..44cfd17 100644 --- a/cfg.mk +++ b/cfg.mk @@ -101,7 +101,7 @@ sc_prohibit_VIR_ERR_NO_MEMORY: msg='use virReportOOMError, not V'IR_ERR_NO_MEMORY \ $(_prohibit_regexp) -include Makefile.nonreentrant +include $(srcdir)/Makefile.nonreentrant sc_prohibit_nonreentrant: @fail=0 ; \ for i in $(NON_REENTRANT) ; \ diff --git a/examples/domain-events/events-c/Makefile.am b/examples/domain-events/events-c/Makefile.am index 60b1589..64457c2 100644 --- a/examples/domain-events/events-c/Makefile.am +++ b/examples/domain-events/events-c/Makefile.am @@ -1,4 +1,4 @@ -INCLUDES = -I@top_srcdir@/include +INCLUDES = -I$(top_builddir)/include -I$(top_srcdir)/include noinst_PROGRAMS = event-test event_test_CFLAGS = $(WARN_CFLAGS) event_test_SOURCES = event-test.c diff --git a/examples/hellolibvirt/Makefile.am b/examples/hellolibvirt/Makefile.am index 6ef2cc8..8d108a7 100644 --- a/examples/hellolibvirt/Makefile.am +++ b/examples/hellolibvirt/Makefile.am @@ -1,4 +1,4 @@ -INCLUDES = -I@top_srcdir@/include +INCLUDES = -I$(top_builddir)/include -I$(top_srcdir)/include noinst_PROGRAMS = hellolibvirt hellolibvirt_CFLAGS = $(WARN_CFLAGS) hellolibvirt_SOURCES = hellolibvirt.c diff --git a/tools/Makefile.am b/tools/Makefile.am index c48b203..941e93e 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -43,7 +43,7 @@ virsh_LDADD = \ $(VIRSH_LIBS) virsh_CFLAGS = \ -I$(top_srcdir)/gnulib/lib -I../gnulib/lib \ - -I../include \ + -I../include -I$(top_srcdir)/include \ -I$(top_srcdir)/src \ -I$(top_srcdir)/src/util \ -DGETTEXT_PACKAGE=\"$(PACKAGE)\" \ -- 1.6.5.rc2

2 1