- Devel - libvirt List Archives

[Libvir] Making QEMU use the real internal driver API
by Daniel P. Berrange 10 May '07

10 May '07

So currently, as we all know, the QEMU driver is just a shim through to the QEMU daemon. Looking at the code for Rich's remote driver I quite like the way the remote driver / server works like, and don't like the ugliness of having the special case QEMU code in there. We always had a long term plan to try & make the QEMU driver fit into the regular driver API more cleanly but not really tackled it yet. Similarly we've periodically brought up the idea of having asynchronous notifications of 'interesting' state changes in the drivers, to eliminate the need for polling. After a little thinking I have an idea that may let us address both issues in one go. Or to be more precise, put infrastructure in place to let us move closer to allowing async notiifcations. First let me consider the files under qemud/ .... Self-contained helper routines, ignore for rest of discussion bridge.c bridge.h iptables.c iptables.h uuid.c uuid.h buf.c buf.h The on the wire protcol definition, this is the bit we want to kill in favour of the generic remote wire protocol: protocol.h The code to convert from wire -> function calls. Again this ought to die in favour of the generic remote dispatcher. dispatch.c dispatch.h The QEMU config handling routines. conf.c conf.h The impl of core APIs: driver.c driver.h Finally the 'server': internal.h qemud.c The end-goal is that 'qemud/driver.c' would become 'src/qemu_internal.c' implementing the official internal driver API. The 'qemud/conf.c' would likely become 'src/qemu_config.c'. The key difference between the internal driver API, and the QEMU specific driver 'API' inside the daemon is the prescense of some global state via the 'qemud_server' struct. If we can figure out a way to store this state inside the regular driver API we can refactor that without much problem. Then there is the need for an event loop for monitoring the file handles of the QEMU processes, which at the moment is fairly tied into the event loop used to monitor the client/server socket handles. This existing code is rather ugly & has bad structure IMHO. The result is that code which ought to be isolated in driver.c, is instead part of qemud.c because of event loop integration. Finally, there is some code which runs on startup, and shutdown of the daemon, to do stuff like auto-start of VMs / networks, and clean shutdown of the same. Oh, and we need to make sure the QEMU driver is never activated locally, only when libvirt.so is being used as part of 'libvirtd'. So at a high level what I think we'd need to add to the internal driver API to make this possible is In driver.h: typedef int (*virDrvStartup)(void); typedef int (*virDrvShutdown)(void); Adding these to the 'virDriver' struct. Regular apps using libvirt.so would never trigger these APIs - they're only intended for the daemon, so I figure we'd have to private APIs in the libivrt.so (but not in the libvirt.h) __virStartup(void); __virShutdown(void); Since these are only called by the daemon, we now conveniently also have a way to stop the QEMU driver being locally activated. Next up, event loops. First, we don't want to mandate a particular impl of an event loop. If you get into the argument about QT vs GLib vs libevent and libvirt picks one, then you make it hard for libvirt to integrate with apps using the other. Second, we don't want to assume that 1 connection == 1 file handle. We also don't want to assume that the number of file handles used by a driver stays the same over time. Basically we need to expect that any single driver connection will have zero or more file handles that it is interested in. The DBus library deals with this exact same issue & the solution is very simple. The app using the libvirt registers a callback which the internal driver than then use to (un)register file handles. So in src/internal.h we'd add methods for internal drivers to call to add / remove file handles to/from the event loop: /* * @fd: the file handle on which the event occurred * @event: bitset of one or more of POLLxxx constants from poll.h * @opaque: data associated with 'fd' * * Return 0 to continue monitoring, -1 if an error occurred * and the handle should no longer be monitored */ typedef int (*virEventCallback)(int fd, int events, void *opaque); /* * @fd: the file handle to start monitoring * @events: bitset of POLLxxx contants to monitor * @cb: callback to invoke when an event ocurrs * @opaque: data to pass into callback's opaque parameter */ int virEventAddHandle(int fd, int events, virEventCallback cb, void *opaque); /* * @fd: the file handle to stop monitoring */ int virEventRemoveHandle(int fd); And we'd also add methods for a user of libvirt to register an event loop implementation: typedef int (*virEventAddHandleImpl)(int fd, int events, opaque); typedef int (*virEventRemoveHandleImpl)(int fd); void virEventInitialize(virEventAddHandleImpl add, virEventRemoveHandleImpl remove); If we want to add support for async events into the public API, then the virEventInitialize method could instead be in libvirt.h, and exported in the libvirt.so. Otherwise we could mark it private __virEventInitialize in libvirt.so. That I think should be enough to let us move QEMU into a regular style libvirt driver. NB, what follows now is a quick brain-dump - I'm not suggesting implementing this yet - its really a rough idea of general architecture. We'd definitely want to prototype it in a test app before committing to something like this. There's also many many scenarios to be thought about beyond just watching create/destroy events Thinking about a public API we could have: enum { VIR_CONN_DOMAIN_CREATED, VIR_CONN_DOMAIN_DESTROYED; } virConnectEvent; typedef int (*virConnectNotifyCallback)(virConnectPtr dom, unsigned char uuid[16], int event); virConnectNotify(virConnectPtr conn, virConnectNofifyCallback cb); An application using this would thus be able to do something like /* Assume there's a convenient libvirtglib.so addon providing these for apps */ extern virEventAddHandleImpl virGLibEventAddHandle; extern virEventRemoveHandleImpl virGLibEventRemoveHandle; int domainWatcher(virConnectPtr dom, unsigned char uuid[16], int event) { ... do something ... } int main () { virConnectPtr con; con = virConnectOpen(NULL); /* Setup an event loop via glib */ virEventInitilize(virGLibEventAddHandle, virGLibEventRemoveHandle); virConnectNotify(conn, domainWatcher); Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 1

[Libvir] [PATCH] Fields in virDomainPtr which appear to be never used
by Richard W.M. Jones 08 May '07

08 May '07

Below is the definition of struct _virDomain from internal.h. I may be missing something big (well, it wouldn't be the first time), but as far as I can see some of these fields are never used, and some are set but can never be read. struct _virDomain { unsigned int magic; /* specific value to check */ OK int uses; /* reference count */ OK virConnectPtr conn; /* pointer back to the connection */ OK char *name; /* the domain external name */ OK char *path; /* the domain internal path */ This field is set in src/xs_internal.c, but nowhere in libvirt does it appear to be read. The only other time it is reference is in hash.c where it is freed if non-NULL. int id; /* the domain ID */ OK int flags; /* extra flags */ Apparently this field could be set to DOMAIN_IS_SHUTDOWN or DOMAIN_IS_DEFINED. However neither of these symbols are used, and moreover this structure field looks like it is never written or read. unsigned char uuid[VIR_UUID_BUFLEN]; /* the domain unique identifier */ OK char *xml; /* the XML description for defined domains */ This field is, as far as I can tell, never set anywhere. However the code in hash.c does check whether it is set and will free it upon destruction of the _virDomain object. }; With this analysis in mind, attached is a patch which removes these three fields (path, flags, xml) and the unused virDomainFlags, with no apparent ill-effects. Rich. -- Emerging Technologies, Red Hat http://et.redhat.com/~rjones/ 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (USA), Charlie Peters (USA) and David Owens (Ireland)

2 2

[Libvir] [PATCH] Fix the value of XEN_V2_OP_GETDOMAININFOLIST
by Saori Fukuta 08 May '07

08 May '07

Hi, The value of "Max memory" shown by using "virsh dominfo" is different from xm command. # virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 300000 kB Used memory: 264064 kB # xm li -l HVM_RH5_sda6 | grep mem (maxmem 450) (memory 292) (shadow_memory 5) So, I changed the definition as follows, (before) #define XEN_V2_OP_GETDOMAININFOLIST 6 (changed) #define XEN_V2_OP_GETDOMAININFOLIST 5 and I could get the value that I want. # ./virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 460800 kB Used memory: 299008 kB # xm li -l HVM_RH5_sda6 | grep mem (maxmem 450) (memory 292) (shadow_memory 5) I think that is not mistake because the XEN_V2_OP_GETDOMAININFOLIST is a system operation, so the definition is not exactly wrong. /usr/include/xen/sysctl.h:#define XEN_SYSCTL_getdomaininfolist 6 Do you know any reason for this? Signed-off-by: Saori Fukuta <fukuta.saori(a)jp.fujitsu.com> Thanks, Saori Fukuta. Index: xen_internal.c =================================================================== RCS file: /data/cvs/libvirt/src/xen_internal.c,v retrieving revision 1.71 diff -u -p -r1.71 xen_internal.c --- xen_internal.c 13 Apr 2007 00:43:57 -0000 1.71 +++ xen_internal.c 26 Apr 2007 11:41:03 -0000 @@ -117,7 +117,7 @@ static regex_t xen_cap_rec; #define XEN_V0_OP_GETDOMAININFOLIST 38 #define XEN_V1_OP_GETDOMAININFOLIST 38 -#define XEN_V2_OP_GETDOMAININFOLIST 6 +#define XEN_V2_OP_GETDOMAININFOLIST 5 struct xen_v0_getdomaininfo { domid_t domain; /* the domain number */

3 7

[Libvir] Request for minimum memory level of dom0 (xen)
by Jan Michael 08 May '07

08 May '07

Hi LibVirt-Mailreaders, Is it possible to provide information about the minimum memory level of domain 0 (xen)? The value is located in the xen deamon config file as "dom0-min-mem". I would like to calculate on the basis of this value if there is enough memory left to create additional domains. Thanks in advance, Jan Michael

2 1

[Libvir] The fist problem to compile libvirt in a Debian etch.
by Marco Sinhoreli 07 May '07

07 May '07

Hello list, I'm writing a how-to about libvirt compilation in Debian Etch. Well, I tried to compile version 0.1.11 in a Debian etch completly newly built from debootstrap. These are the steps that I took: ----- init commands apt-get install make gcc # cd /usr/src && wget http://libvirt.org/sources/libvirt-cvs-snapshot.tar.gz&& tar -xzvf libvirt-cvs-snapshot.tar.gz # cd libvirt-0.1.11 # ./configure checking build system type... i686-pc-linux-gnulibc1 checking host system type... i686-pc-linux-gnulibc1 checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking for C compiler default output file name... configure: error: C compiler cannot create executables See `config.log' for more details. # tail config.log #define PACKAGE "libvirt" #define PACKAGE_BUGREPORT "" #define PACKAGE_NAME "" #define PACKAGE_STRING "" #define PACKAGE_TARNAME "" #define PACKAGE_VERSION "" #define VERSION "0.1.11" configure: exit 77 ----- finish commands I don't know what's the problem. Ideas please... thanks -- Marco Sinhoreli

4 7

Re: [Libvir] The fist problem to compile libvirt in a Debian etch.
by Marco Sinhoreli 05 May '07

05 May '07

Hello list, I changed the libvirt version and now I'm a one step forward ;-) So, I installed these debian packages and I runned the commands above: ------ # apt-get install gcc make g++ autogen gettext autoconf automake libncurses5-dev libxml2 libxml2-dev libreadline5-dev # cd /usr/src # wget http://libvirt.org/sources/libvirt-0.2.2.tar.gz # tar -xzvf libvirt-0.2.2.tar.gz && cd libvirt-0.2.2 # ./configure -with-libxml=/usr # make In this step the make command retur a error: [...] gcc -g -O2 -o .libs/virsh virsh-virsh.o virsh-console.o ./.libs/libvirt.so -L/usr/lib /usr/lib/libxml2.so -lcurses -lreadline -Wl,--rpath -Wl,/usr/local/lib virsh-virsh.o: In function `cmdVcpuinfo': /usr/src/libvirt-0.2.2/src/virsh.c:1230: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1294: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1294: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdDominfo': /usr/src/libvirt-0.2.2/src/virsh.c:1153: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1211: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1211: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdNetworkUuid': /usr/src/libvirt-0.2.2/src/virsh.c:2227: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:2244: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:2244: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdNetworkDefine': /usr/src/libvirt-0.2.2/src/virsh.c:1854: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1890: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1890: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdNetworkCreate': /usr/src/libvirt-0.2.2/src/virsh.c:1798: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1834: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1834: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdDomuuid': /usr/src/libvirt-0.2.2/src/virsh.c:1712: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1728: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1728: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdNodeinfo': /usr/src/libvirt-0.2.2/src/virsh.c:1542: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1562: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1562: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdDefine': /usr/src/libvirt-0.2.2/src/virsh.c:733: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:769: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:769: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdCreate': /usr/src/libvirt-0.2.2/src/virsh.c:678: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:714: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:714: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `vshCmddefHelp': /usr/src/libvirt-0.2.2/src/virsh.c:2563: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:2609: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:2609: undefined reference to `__stack_chk_fail' virsh-virsh.o: In function `cmdVcpupin': /usr/src/libvirt-0.2.2/src/virsh.c:1315: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1384: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/virsh.c:1384: undefined reference to `__stack_chk_fail' virsh-console.o: In function `vshRunConsole': /usr/src/libvirt-0.2.2/src/console.c:45: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/console.c:179: undefined reference to `__stack_chk_guard' /usr/src/libvirt-0.2.2/src/console.c:179: undefined reference to `__stack_chk_fail' ./.libs/libvirt.so: undefined reference to `__stack_chk_fail_local' collect2: ld returned 1 exit status make[2]: ** [virsh] Erro 1 make[2]: Saindo do diretório `/usr/src/libvirt-0.2.2/src' make[1]: ** [all-recursive] Erro 1 make[1]: Saindo do diretório `/usr/src/libvirt-0.2.2' make: ** [all] Erro 2 # ------------------------- I don't know why this error. Ideas? regards, Marco Sinhoreli

2 2

[Libvir] build problems with latest cvs version
by Jan Michael 04 May '07

04 May '07

Hi libvirt-list, I run into the same build problem with version libvirt version 0.2.1, 0.2.2 and the latest cvs-version. In libvirt directory I did the following (for cvs version) ./autogen.sh --disable-shared --disable-QEMU I'd like to disable shared libraries and QEMU support (I have only installed xen 3.0.3 envrionment). The make process breaks while building in [/opt/test/libvirt/qemud], while doing the following: <make output> if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/include/libxml2 -I../ include -I../include -I/usr/include/libxml2 -Wp,-D_FORTIFY_SOURCE=2 - fexceptions -fasynchronous-unwind-tables -Wall -Wmissing-prototypes - Wnested-externs -Wpointer-arith -Wextra -Wshadow -Wcast-align -Wwrite- strings -Waggregate-return -Wstrict-prototypes -Winline -Wredundant- decls -Wno-sign-compare -DLOCAL_STATE_DIR="\"/usr/local/var\"" - DSYSCONF_DIR="\"/usr/local/etc\"" -DQEMUD_PID_FILE="\"/usr/local/var/ run/libvirt_qemud.pid\"" -g -O2 -MT libvirt_qemud-bridge.o -MD -MP - MF ".deps/libvirt_qemud-bridge.Tpo" -c -o libvirt_qemud-bridge.o `test -f 'bridge.c' || echo './'`bridge.c; \ then mv -f ".deps/libvirt_qemud-bridge.Tpo" ".deps/libvirt_qemud- bridge.Po"; else rm -f ".deps/libvirt_qemud-bridge.Tpo"; exit 1; fi bridge.c: In function `brAddBridge': bridge.c:130: error: `SIOCBRADDBR' undeclared (first use in this function) bridge.c:130: error: (Each undeclared identifier is reported only once bridge.c:130: error: for each function it appears in.) bridge.c: In function `brDeleteBridge': bridge.c:148: error: `SIOCBRDELBR' undeclared (first use in this function) bridge.c: In function `brAddInterface': bridge.c:182: error: `SIOCBRADDIF' undeclared (first use in this function) bridge.c: In function `brDeleteInterface': bridge.c:190: error: `SIOCBRDELIF' undeclared (first use in this function) bridge.c: In function `brSetForwardDelay': bridge.c:481: error: `SYSFS_BRIDGE_ATTR' undeclared (first use in this function) bridge.c:481: error: syntax error before string constant bridge.c:481: warning: empty body in an if-statement bridge.c: In function `brGetForwardDelay': bridge.c:503: error: `SYSFS_BRIDGE_ATTR' undeclared (first use in this function) bridge.c:503: error: syntax error before string constant bridge.c:503: warning: empty body in an if-statement bridge.c: In function `brSetEnableSTP': bridge.c:531: error: `SYSFS_BRIDGE_ATTR' undeclared (first use in this function) bridge.c:531: error: syntax error before string constant bridge.c:531: warning: empty body in an if-statement bridge.c: In function `brGetEnableSTP': bridge.c:553: error: `SYSFS_BRIDGE_ATTR' undeclared (first use in this function) bridge.c:553: error: syntax error before string constant bridge.c:553: warning: empty body in an if-statement </make output> I get the same even on the older versions 0.2.2 and 0.2.1. Can you please help me to solve this compilation error? Many thanks in advance, Jan Michael

4 12

[Libvir] PATCH: Remove duplicate graphics for HVM on 3.0.5
by Daniel P. Berrange 03 May '07

03 May '07

In the Xen 3.0.5 series, the HVM graphics config was moved to use the same syntax as Paravirt (ie the pvfb devices). I previously added support for this, but didn't realize that the old syntax was kept as back-compat when doing 'xm list --long' or equiv. So we're ending up with 2 <graphics> elements with the same information. This patch makes us ignore the old style config if running on new XenD. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 1

[Libvir] PATCH: Disable xm_internal.c on newer XenD
by Daniel P. Berrange 02 May '07

02 May '07

The xm_internal.c driver (that processes /etc/xen config files) is only ever intended to be used on XenD <= 3.0.3 There are impls of all the methods in xend_internal.c that take priority on newer XenD, since the xend driver is before it in the stack. The only flaw is that upon failure of one of the xend method libvirt will carry on & invoke the xm_internal driver method. The attached patch addresses this by completely disabling the xm_internal driver on XenD >= 3.0.4 Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 3

[Libvir] PATCH: Use --strict-order with dnsmasq
by Daniel P. Berrange 02 May '07

02 May '07

A bug reported by David Lutterkort... The 'man resolv.conf' docs for 'nameserver' say [quote] nameserver Name server IP address Internet address (in dot notation) of a name server that the resolver should query. Up to MAXNS (currently 3, see <resolv.h>) name servers may be listed, one per keyword. If there are multi- ple servers, the resolver library queries them in the order listed. If no nameserver entries are present, the default is to use the name server on the local machine. (The algorithm used is to try a name server, and if the query times out, try the next, until out of name servers, then repeat trying all the name servers until a maximum number of retries are made.) [/quote] While 'man dnsmasq' docs for 'strict-order' say: [quote] -o, --strict-order By default, dnsmasq will send queries to any of the upstream servers it knows about and tries to favour servers to are known to be up. Setting this flag forces dnsmasq to try each query with each server strictly in the order they appear in /etc/resolv.conf [/quote] So by default, the algorithm dnsmasq uses for DNS lookups is a) Different from that use by GLibC b) Wrong Thus I think we should always use --strict-order when running dnsmasq. The attached patch adds this Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 4