October 2007 - Devel - libvirt List Archives

[Libvir] Concepts in storage management
by Daniel P. Berrange 22 Oct '07

22 Oct '07

So we've had many email threads on the subject of storage, but none have resulted in a satisfactory way forward to implementing any storage mgmt apis. Part of the problem I think is that we've not tried to understand all the various concepts / technologies which are available & how they relate to each other. This mail attempts to outline all the different technologies. There's a short list of API operations, but I don't want to particularly get into API details until we have a good undersatnding of the concepts. First and foremost I don't believe it is acceptable to say we're only going to allow one kind of storage. Storage is the key piece of infrastructure for any serious network, and we have to be able to adapt to deployment scenarios that present themselves. Second, there is clearly a huge number of storage technologies here and there's no way we'll implement support for all of them in one go. So we need to prioritize getting the conceptual model correct, to allow us to incrementally support new types of storage backend. Taxonomy of storage types ========================= | +- Block | | | +- Disk | | | | | +- Direct attached | | | | | | | +- IDE/ATA disk | | | +- SCSI disk | | | +- FibreChannel disk | | | +- USB disk/flash | | | +- FireWire disk/flash | | | | | +- Remote attached | | | | | +- iSCSI disk | | +- GNBD disk | | | +- Partition | | | +- Virtual | | | +- Direct attached | | | | | +- LVM | | +- ZFS | | | +- Remote attached | | | +- Cluster LVM | +- FileSystem | | | +- Directed attached | | | | | +- ext2/3/4 | | +- xfs | | +- ZFS | | | +- Remote attached | | | +- NFS | +- GFS | +- OCFS2 | +- Directory | +- File | +- Raw allocated +- Raw sparse +- QCow2 +- VMDK Storage attributes ================== - Local vs network (ext3 vs NFS, SCSI vs iSCSI) - Private vs shared (IDE vs FibreChannel) - Pool vs volume (LVM VG vs LV, Directory vs File, Disk vs Partition) - Container vs guest (OpenVZ vs Xen) - Attributes - Compressed - Encrypted - Auto-extend - Snapshots - RO - RW - Partition table - MBR - GPT - UUID - 16 hex digits - Unique string - SCSI WWID (world wide ID) - Local Path(s) (/dev/sda, /var/lib/xen/images/foo.img) - Server Hostname - Server Identifier (export path/target) - MAC security label (SELinux) - Redundancy - Mirrored - Striped - Multipath - Pool operation - RO - RW Nesting hierarchy ================= Many possibilities... - 1 x Host -> N x iSCSI target -> N x LUN -> N x Partition - N x Disk/Partition -> 1 x LVM VG -> B x LVM LV - 1 x Filesystem -> N x directory -> N x file - 1 x File -> 1 x Block (loopback) Application users ================= - virt-manager / virt-install - Enumerate available pools - Allocate volume from pool - Create guest with volume - virt-clone - Copy disks - Snapshot disks - virt-df - Filesystem usage - pygrub - Extract kernel/initrd from filesystem - virt-factory - Manage storage pools - Pre-migration sanity checks - virt-backup - Snapshot disks - virt-p2v - Snapshot disks Storage representation ====================== Two core concepts - Volume - a chunk of storage - assignable to a guest - assignable to a pool - optionally part of a pool - Pool - a chunk of storage - contains free space - allocate to provide volumes - compromised of volumes Recursive! n x Volume -> Pool -> n x Volume Nesting to many levels... Do we need an explicit Filesystem concept ? Operations ========== Limited set of operations to perform - List host volumes (physical attached devices) - List pools (logical volume groups, partitioned devs, filesystems) - List pool volumes (dev partitions, LVM logical volumes, files) - Define pool (eg create directory, or define iSCSI target) - Undefine pool (delete directory, undefine iSCSI config - Activate pool (mount NFS volume, login to iSCSI target) - Deactivate pool (unmount volume, logout of iSCSI) - Dump pool XML (get all the metadata) - Lookup by path - Lookup by UUID - Lookup by name - Create volume (create a file, allocate a LVM LV, etc) - Destroy volume (delete a file, deallocate a LVM LV) - Resize volume (grow or shrink volume) - Copy volume (copy data between volumes) - Snapshot volume (snapshot a volume) - Dump volume XML (get all the metadata) - Lookup by path - Lookup by UUID - Lookup by name http://www.redhat.com/archives/libvir-list/2007-February/msg00010.html http://www.redhat.com/archives/libvir-list/2007-September/msg00119.html Do we also need some explicit Filesystem APIs ? XML description =============== The horrible recursiveness & specific attributes are all in the XML description for different storage pool / volume types. This is where we define things like what physical volume are in a volume group, iSCSI server / target names, login details, etc, etc XXX fill in the hard stuff for metadata description here Implementation backends ======================= - FileSystem/Directory/File - POSIX APIs - LVM - LVM tools, or libLVM - Disk/partitions - sysfs / parted - iSCSI - sysfs / iscsi utils - ZFS - ZFS tools Implementation strategy ======================= Should prioritize implementation according to immediate application needs Initial goal to support remote guest creation on par with current capabilities: - Directory + allocateing raw sparse files - Enumerate existing disks, partitions & LVM volumes Further work: - Allocating LVM volumes - Defining LVM volume groups - Partitioning disks - Mounting networked filesystems - Accessing iSCSI volumes - Copying existing volumes - Snapshotting volumes - Cluster aware filesystems (GFS) - Various file formats (QCow, VMDK, etc) Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

4 13

Re: [Libvir] Problem to compile virt-manager in debian etch
by Richard W.M. Jones 20 Oct '07

20 Oct '07

Marco Sinhoreli wrote: > The configure script was found, but the Makefile hasn't been created. Need to see the full messages. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

4 8

[Libvir] [PATCH] fix negative value handling of virConfParseLong()
by Tatsuro Enokura 19 Oct '07

19 Oct '07

Hi, I found a bug about negative value at virConfParseLong() in conf.c. This patch fixes it. Signed-off-by: Tatsuro Enokura <fj7716hz(a)aa.jp.fujitsu.com> Thanks, Tatsuro Enokura. -------------------------------------------------------------------------- Index: libvirt/src/conf.c =================================================================== RCS file: /data/cvs/libvirt/src/conf.c,v retrieving revision 1.11 diff -u -p -r1.11 conf.c --- libvirt/src/conf.c 4 Jul 2007 09:16:23 -0000 1.11 +++ libvirt/src/conf.c 19 Oct 2007 08:37:36 -0000 @@ -345,6 +345,8 @@ virConfParseLong(virConfParserCtxtPtr ct l = l * 10 + (CUR - '0'); NEXT; } + if (neg) + l *= -1; *val = l; return(0); } --------------------------------------------------------------------------

3 2

[Libvir] Opening KVM guest in xterm rather than vnc
by David Mueller 19 Oct '07

19 Oct '07

I'm trying to setup my virtual machines to display in an xterm window rather than a VNC window. The guests and host are Fedora 7; the guest only needs to run in a console mode (no X11). I see a bit on http://libvirt.org/format.html about using a console tag instead of graphics in my XML, but I can't figure out how to connect to that. With our older Xen on Fedora Core 5 setup, we used the following command to create a guest and it worked great; I'm trying to approximate something similar with libvirt/KVM on Fedora 7: xterm -e "/usr/sbin/xm create -c guest.xen" A notable advantage of using an xterm over VNC is that we can resize the window; with VNC as far as I've been able to see, we've been stuck with the default 80x24 window. We also can't copy and paste text from the VNC window to other applications. Any suggestions would be appreciated. I am currently using libvirt-0.3.2-1.fc7; if this is something that we'll have to wait for a newer version that will be in Fedora 8, that's okay. - David

3 2

[Libvir] memory error in another test
by Jim Meyering 19 Oct '07

19 Oct '07

Running tests under valgrind exposed this: (line numbers are slightly off, due to local changes) ==27876== ERROR SUMMARY: 32 errors from 2 contexts (suppressed: 5 from 2) ==27876== ==27876== 16 errors in context 1 of 2: ==27876== Conditional jump or move depends on uninitialised value(s) ==27876== at 0x41BA18: qemudBuildCommandLine (qemu_conf.c:1871) ==27876== by 0x406764: testCompareXMLToArgvFiles (qemuxml2argvtest.c:46) ==27876== by 0x4069DA: testCompareXMLToArgvHelper (qemuxml2argvtest.c:97) ==27876== by 0x406D60: virtTestRun (testutils.c:69) ==27876== by 0x406A46: main (qemuxml2argvtest.c:114) ==27876== ==27876== 16 errors in context 2 of 2: ==27876== Conditional jump or move depends on uninitialised value(s) ==27876== at 0x41AA45: qemudBuildCommandLine (qemu_conf.c:1606) ==27876== by 0x406764: testCompareXMLToArgvFiles (qemuxml2argvtest.c:46) ==27876== by 0x4069DA: testCompareXMLToArgvHelper (qemuxml2argvtest.c:97) ==27876== by 0x406D60: virtTestRun (testutils.c:69) ==27876== by 0x406A46: main (qemuxml2argvtest.c:114) Both of those are due to uses of uninitialized vm->migrateFrom[0]. Here's the fix: Avoid memory error in a test program. * tests/qemuxml2argvtest.c (testCompareXMLToArgvFiles): Initialize vm.migrateFrom[0], to avoid "read-uninitialized" error from within qemudBuildCommandLine. Index: tests/qemuxml2argvtest.c =================================================================== RCS file: /data/cvs/libvirt/tests/qemuxml2argvtest.c,v retrieving revision 1.5 diff -u -p -r1.5 qemuxml2argvtest.c --- tests/qemuxml2argvtest.c 21 Sep 2007 21:20:32 -0000 1.5 +++ tests/qemuxml2argvtest.c 17 Oct 2007 13:42:15 -0000 @@ -40,6 +40,7 @@ static int testCompareXMLToArgvFiles(con vm.qemuVersion = 0 * 1000 * 100 + (8 * 1000) + 1; vm.qemuCmdFlags = QEMUD_CMD_FLAG_VNC_COLON | QEMUD_CMD_FLAG_NO_REBOOT; + vm.migrateFrom[0] = '\0'; vmdef->vncActivePort = vmdef->vncPort;

2 2

[Libvir] avoid reading uninitialized memory in test program
by Jim Meyering 19 Oct '07

19 Oct '07

valgrind complained about the printf in conftest.c. Here's the fix: * src/conf.c (__virConfWriteMem): Fix typos in comment. * tests/conftest.c (main): Use fwrite, not printf, since the result buffer is not NUL-terminated. =================================================================== RCS file: /data/cvs/libvirt/src/conf.c,v retrieving revision 1.11 diff -u -p -r1.11 conf.c --- src/conf.c 4 Jul 2007 09:16:23 -0000 1.11 +++ src/conf.c 17 Oct 2007 11:31:54 -0000 @@ -1,7 +1,7 @@ /** * conf.c: parser for a subset of the Python encoded Xen configuration files * - * Copyright (C) 2006 Red Hat, Inc. + * Copyright (C) 2006, 2007 Red Hat, Inc. * * See COPYING.LIB for the License of this software * @@ -916,7 +916,7 @@ error: /** * __virConfWriteMem: * @memory: pointer to the memory to store the config file - * @len: pointer to the lenght in byte of the store, on output the size + * @len: pointer to the length in bytes of the store, on output the size * @conf: the conf * * Writes a configuration file back to a memory area. @len is an IN/OUT =================================================================== RCS file: /data/cvs/libvirt/tests/conftest.c,v retrieving revision 1.1 diff -u -p -r1.1 conftest.c --- tests/conftest.c 29 Aug 2006 22:27:07 -0000 1.1 +++ tests/conftest.c 17 Oct 2007 11:31:54 -0000 @@ -24,7 +24,10 @@ int main(int argc, char **argv) { fprintf(stderr, "Failed to serialize %s back\n", argv[1]); exit(3); } - printf("%s", buffer); virConfFree(conf); + if (fwrite(buffer, len, 1, stdout) != len) { + fprintf(stderr, "Write failed\n"); + exit(1); + } exit(0); }

2 2

[Libvir] PATCH: RFC supporting SASL authentication
by Daniel P. Berrange 19 Oct '07

19 Oct '07

History: we currently provide two TCP sockets, one clear text, no auth, the other TLS with x509 client certificate verification for auth. We also provide a UNIX domain socket relying on file perms to restrict access. I have previously provided a policykit patch for the latter allowing admin defined auth policy for UNIX socket access. The PolicyKit patch was flawed in that it did not provide a way for a client app to determine whether the UNIX socket needed PolicyKit auth ahead of time or not. This required apps to make assumptions prior to connecting which is not really viable. This mail is working towards a more flexible authentication solution, and goes straight for the big picture by integrating SASL authentication. This gives us integration with Kerberos (GSSAPI) and PAM and whatever the hell else SAS supports[1]. The critical decision in all this is the wire protocol & how to adapt it to slot in authentication in a way that is compatible with existing clients. As it stands the decision to use clear data vs TLS on the wire is based on the socket the client connects to - we do a TLS handshake at the very start. We need to be able to support SASL on all sockets, TCP, TLS & UNIX since they can all benefit from stuff like Kerberos. So I decided we need to integrate at the RPC layer for this to work. The general protocol picture ---------------------------- It helps to look at qemud/remote_protocol.x when reading this next ... Every API basically has an RPC call & reply pair. The reply messages have a status field, either 'REMOTE_OK' or 'REMOTE_ERROR'. In the former case the normal API return values follow on the wire. In the latter case a virErrroPtr object is serialized on the wire. For this patch I decided to add a 3rd code, REMOTE_AUTH. If an application tries to make an RPC call on a socket requiring authentication, and has not yet authenticated the server will return REMOTE_AUTH code. It then also returns an int (remote_auth_type) specifying the authentication method to use. I have defined two: enum remote_auth_type { REMOTE_AUTH_NONE = 0, REMOTE_AUTH_SASL = 1 }; With plans to add REMOTE_AUTH_POLKIT in a future patch. A legacy client getting back REMOTE_AUTH code will just quit the connection attempt since they don't support authentication. If the admin so desires they can still provide the TLS socket in a non-authenticated mode and only turn on SASL for the TCP socket. So the decision about whether to enable legacy clients is admin controlled. This is the best we can do. A new client getting back REMOTE_AUTH code will then read the remote_auth_type off the wire. If the requested type is one that the client supports then it can begin the authentication process, otherwise we virRaiseError and stop connecting. The SASL specific picture ------------------------- For the SASL auth the process involves a multi-phase handshake looking something like: client server 1. -> ask for mechanism list -> new ctx 2. new ctx <- list of mechanisms <- 3. start auth -> initial auth data -> start auth 4. step auth <- reply auth data <- 5. -> step auth data -> step auth goto 4. <- reply auth data <- Authentication can complete at step 4 in this process, or steps 4 & 5 can repeat an arbitrary number of times. So, to implement this if it neccessary to define 3 new RPC calls internal to the remote driver/daemon (aka not exposed to public API like the rest of the RPC calls). These are: REMOTE_PROC_AUTH_SASL_INIT = 66, REMOTE_PROC_AUTH_SASL_START = 67, REMOTE_PROC_AUTH_SASL_STEP = 68 These are basically just punting back & forth the data going in & out of the appropriate SASL apis. sasl_{server,client}_{new,start,step} See the man pages for more info. So on the server end, if a socket is configured to require SASL auth, the server will reject all RPC calls *except* for those three above with the REMOTE_AUTH code. Once SASL auth has completed, it will allow all the normal RPC calls. The effect is basically that the client is not able to call virConnectOpen & friends until auth has completed. On the client end, the fun is in the 'call' method of remote_internal.c. This has been split in two. The original method is now 'onecall', and there is a thin wrapper named 'call'. 'call' simply invokves onecall, and if it gets back a REMOTE_AUTH code, will do the SASL handshake & then re-run the original call. So again the effect is basically that the first virConnectOpen will cause the auth handshake to be performed. The SASL implementation details ------------------------------- This is the bare minimum SASL integration. I have not attempted to hook up any callbacks for gathering credentials. This basically means that the only SASL mechanism which works is Kerberos GSSAPI - its credentials are fetched out-of-band & so don't require callbacks. We do need to consider callbacks later so we can do username/password auth, and all the various other methods SASL has. As well as authentication, some SASL mechanisms provide a way to negotiate a data encryption layer for the subsequent session. GSSAPI is the only commonly used mechanism which supports this. I have not implemented this yet though. What we would do though is to enable this capability on the the plain TCP socket only. This would make the TCP socket truely secure, and avoid any extra overhead on the TLS socket or UNIX domain socket. I have set the wire packet size for the SASL negotiation to 8192 bytes at a time. This has been sufficient so far, but I need to validate this before we commit, because this will be wire ABI sensitive. Or I could change the XDR spec to be variable length. Anyway needs re-visiting This only deals with authentication. I have not attempted any authoriztion controls. So anyone who has a valid kerberos principle can connect to the server. We clearly need a local group list as we do for the x509 client certificates. Ultimately we could try LDAP lookups & other intersting suggestions. The SASL stuff is detected in configure & enabled/disabled as appropriate. I need to add extra config file params though to let the sysadmin control what socket uses what auth mechanism. The setup / usage details ------------------------- As I said, I only used GSSAPI so far. To use this all your clients need to be able to kinit & get a ticket. For testing I have setup my own personal Kerberos server using Fedora 7 & FreeIPA (http://freeipa.org/) Each libvirt server needs to be issued with a Kerberos service principle. I am using the word 'libvirt' as the service name. The service principle must match the FQDN of the server host. So on your Kerberos server you can issue a service principle with kadmin.local > addprinc libvirt/cherry.virt.boston.redhat.com(a)VIRT.BOSTON.REDHAT.COM > ktadd -k /tmp/cherry-libvirt.tab libvirt/cherry.virt.boston.redhat.com(a)VIRT.BOSTON.REDHAT.COM > quit Then copy the /tmp/cherry-libvirt.tab file to /etc/libvirt/krb5.tab on the server in question. (Change the hostnames & REALM as needed of course). If you want to change the GSSAPI mechanism, the settings are in the file /etc/sasl2/libvirt.conf. Though again only GSSAPI works so far. If you edit the /etc/libvirt/libvirtd.conf file you can enable listen_tcp=1 and run run libvirtd --listen. >From the client machine it should now be possible to do $ kinit berrange(a)VIRT.BOSTON.REDHAT.COM $ virsh --connect xen+tcp://cherry.virt.boston.redhat.com/ You probably screwed something up though along the way because Kerberos is nasty like that. If you use --verbose with libvirtd it'll show details of any errors during authentication. On the client end you can add in a param on the connect URI of ?debug=stderr and it'll print some info about the auth process to stderr. Check that it shows 'GSSAPI' as a valid mechanism. If it doesn't, then your server is not configured as it should be - check keytab file - check you have cyrus-sasl-gssapi RPM. If that's ok, then check 'klist' on the client - if the first phase of chatter between the client & the KDC worked you should see the principle of the server cached on the client. If you don't, then check the KDC logs in /var/log/krb5kdc.log Kerberos/GSSAPI error reporting sucks really bad. That said, I'm sure I'm missing something, because the errors I get out of SASL are even worse than normal. BTW, you can see various 'XXX' in my patch. Basically all of them need to be addressed before I'd consider this patch suitable to merge. configure.in | 39 +++ include/libvirt/virterror.h | 1 libvirt.spec.in | 3 qemud/Makefile.am | 21 +- qemud/internal.h | 8 qemud/libvirtd.init.in | 3 qemud/libvirtd.sysconf | 3 qemud/qemud.c | 29 ++ qemud/remote.c | 370 ++++++++++++++++++++++++++++++++++-- qemud/remote_dispatch_localvars.h | 5 qemud/remote_dispatch_proc_switch.h | 24 ++ qemud/remote_dispatch_prototypes.h | 3 qemud/remote_protocol.c | 164 +++++++++++++++ qemud/remote_protocol.h | 59 +++++ qemud/remote_protocol.x | 53 ++++- src/Makefile.am | 3 src/remote_internal.c | 307 +++++++++++++++++++++++++++++ src/virsh.c | 4 src/virterror.c | 6 tests/Makefile.am | 2 20 files changed, 1073 insertions(+), 34 deletions(-) Regards, Dan. [1] Not entirely true. Some auth methods require credentials to be fetched from the user, so we can only support methods for which we have callbacks. This patch ignores the callback question, so only support GSSAPI where the credentials are fetched out-of-band. This can be addressed later. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 3

Re: [Libvir] Problem to compile virt-manager in debian etch
by Richard W.M. Jones 19 Oct '07

19 Oct '07

Marco Sinhoreli wrote: > The autogen.sh output: > > ./autogen.sh > libtoolize: `config.guess' exists: use `--force' to overwrite > libtoolize: `config.sub' exists: use `--force' to overwrite > libtoolize: `ltmain.sh' exists: use `--force' to overwrite > hecking for a BSD-compatible install... /usr/bin/install -c > checking whether build environment is sane... yes > checking for a thread-safe mkdir -p... /bin/mkdir -p > checking for gawk... no > checking for mawk... mawk > [...] > > The remaining seems okay. So looks like it's working then? Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

2 1

[Libvir] storage API and ZFS
by John Levon 19 Oct '07

19 Oct '07

Accidentally deleted your reply Richard, sorry. ZFS 'pools' are aggregates composed of some number of real devices. LVM confuses me so I don't know what they map on to, but the basic idea is that you can carve filesystems out of such pools at will. You can also carve out 'volumes' - these appear as a disk device in /dev/zvol/dsk/poolname/volumename It's the latter that you use in Xen to hold domU root filesystems (or you can use a normal file on whatever file system, or a real device, as desired). What might be tricky is anything that claims to return "space left", depending exactly on what you want it to mean. But (I think) certain LVM setups have similar issues. regards john

2 1

[Libvir] Is it possible to add an interface to intercept the migration process?
by Meng Ran 19 Oct '07

19 Oct '07

Hi All, I'm thinking about if it possible to add an interface to let user specified action to happen on destination during the migration, this should be useful for migrating things that have different paths on source and destination. Allowing specified action(callback) to happen at a time point, e.g., after the config file is transfered and before starting the vm, can bring in a lot of flexibilities considering the migration of devices which are shared but have different paths/views. Giving the dest a chance to recognize these devices and update the config, and then start the vm probably is helpful there. Comment? Thanks, Jack

5 6

[Libvir] CPU pinning of domains at creation time
by Daniel Veillard 19 Oct '07

19 Oct '07

There are a few things I gathered on this issue. This affects NUMA setups, where basically if a domain must be placed on a given cell it is not good to let the hypervisor place it first with its own heuristics and then later migrate it to a different set of CPU, but better to instruct the hypervisor to start said domain on the given set. - For Xen it is possible to instruct the hypervisor by passing (cpus '2,3') in the SExpr where the argument is a list of the physical processors allowed - For KVM I think the standard way would be to select the cpuset using sched_setaffinity() between the fork of the current process and the exec of the qemu process - there is no need (from a NUMA perspective) to do fine grained allocation at that point, as long as the domain can be restricted to a given cell at startup, then if needed virDomainPinVcpu() can be used later to do more precise pinning in order to try to optimize placement - to be able to instruct the hypervisor at creation time adding the information in the domain XML description looks the more natural way (another option would be to force to use virDomainDefineXML, add a call using the resulting virDomainPtr to define the set, and then virDomainCreate would be used to do the actual start) + the good point of having this embedded in the XML is that we still have all informations about the domain settings in the XML, if we want to restart it later + the bad point is that we need to fetch and carry this extra information when doing XML dumps to not loose it for example when manipulating the domain to add or remove devices - extracting a cpuset can still be an heavy operation, for example if using xend on need one RPC per vcpu in the domain, the cpuset being constructed by OR'ing logically all cpumaps used by the vcpus of the domain (though in most case this will be the full map after the first CPU and can be stopped immediately) - for the mapping at the XML level I suggest to use a simple extension to the <vcpu>n</vcpu> and extend it to <vcpu cpuset='2,3'>n</vcpu> with a limited syntax which is just the comma separated list of allowed CPU numbers (if the code actually detects such a cpuset is in effect i.e. in general this won't be added). Internally implementing this should not be too hard, I would probably refactor some of the existing parsing code, provide functions to get the cpuset and the number of physical processors. Does this sounds okay ? Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

6 13

[Libvir] Problem to compile virt-manager in debian etch
by Marco Sinhoreli 19 Oct '07

19 Oct '07

Hello all, I have a problem to compile the virt-manager in a Debian etch (64 bit version). I got the virt-manager source from mercurial repository. Bellow the output autogen.sh command: ./autogen.sh libtoolize: `config.guess' exists: use `--force' to overwrite libtoolize: `config.sub' exists: use `--force' to overwrite libtoolize: `ltmain.sh' exists: use `--force' to overwrite configure.ac:7: required file `./config.rpath' not found man/Makefile.am:6: `%'-style pattern rules are a GNU make extension pixmaps/Makefile.am:3: wildcard $(srcdir: non-POSIX variable name pixmaps/Makefile.am:3: (probably a GNU make extension) src/Makefile.am:15: wildcard $(srcdir: non-POSIX variable name src/Makefile.am:15: (probably a GNU make extension) src/Makefile.am:35: `%'-style pattern rules are a GNU make extension src/Makefile.am:38: `%'-style pattern rules are a GNU make extension src/Makefile.am:41: `%'-style pattern rules are a GNU make extension src/Makefile.am:44: `%'-style pattern rules are a GNU make extension src/Makefile.am:26: variable `schema_SOURCES' is defined but no program or src/Makefile.am:26: library has `schema' as canonic name (possible typo) src/Makefile.am:11: variable `libexec_SOURCES' is defined but no program or src/Makefile.am:11: library has `libexec' as canonic name (possible typo) src/Makefile.am:4: variable `bin_SOURCES' is defined but no program or src/Makefile.am:4: library has `bin' as canonic name (possible typo) src/Makefile.am:18: variable `desktop_SOURCES' is defined but no program or src/Makefile.am:18: library has `desktop' as canonic name (possible typo) src/Makefile.am:8: variable `python_SOURCES' is defined but no program or src/Makefile.am:8: library has `python' as canonic name (possible typo) src/Makefile.am:22: variable `dbus_SOURCES' is defined but no program or src/Makefile.am:22: library has `dbus' as canonic name (possible typo) src/graphWidgets/Makefile.am:5: `:='-style assignments are not portable src/graphWidgets/Makefile.am:5: shell pkg-config --variable=codegendir pygtk-2.0: non-POSIX variable name src/graphWidgets/Makefile.am:5: (probably a GNU make extension) src/graphWidgets/Makefile.am:6: `:='-style assignments are not portable src/graphWidgets/Makefile.am:6: shell pkg-config --variable=defsdir pygtk-2.0: non-POSIX variable name src/graphWidgets/Makefile.am:6: (probably a GNU make extension) src/virtManager/Makefile.am:3: wildcard $(srcdir: non-POSIX variable name src/virtManager/Makefile.am:3: (probably a GNU make extension) src/virtManager/Makefile.am:7: python_DATA:$(srcdir: non-POSIX variable name src/virtManager/Makefile.am:15: `%'-style pattern rules are a GNU make extension Any idea? regards. -- Marco Sinhoreli

2 4

[Libvir] PATCH: Remote protocol receive+dispatch cleanup
by Daniel P. Berrange 18 Oct '07

18 Oct '07

Dating the from time when we had both the QEMU & generic remote protocol there are tiny qemud/protocol.{x,c,h} files basically just containing one struct for the packet header. This is more or less duplicated in the remote protocol & with a few small tweaks can simplify this code. So the attached patch removes the protocol.{x,c,h} files. The qemud.c file in the 'receive header' branch now merely reads a single 4 byte unsigned int to discover the packet length. The rest of the packet is dealt with in the 'receive payload' branch. It immediately dispatches to the remote program. There is no wire format change here - shouuld just be code cleanup a/qemud/protocol.c | 17 -------------- a/qemud/protocol.h | 39 -------------------------------- a/qemud/protocol.x | 40 --------------------------------- qemud/Makefile.am | 5 ---- qemud/internal.h | 1 qemud/qemud.c | 57 +++++++++++++++--------------------------------- qemud/remote.c | 2 - qemud/remote_protocol.h | 1 qemud/remote_protocol.x | 3 ++ src/Makefile.am | 1 10 files changed, 24 insertions(+), 142 deletions(-) Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 1

[Libvir] BuildRequires check
by Jim Meyering 17 Oct '07

17 Oct '07

Hi, I tried to configure libvirt from cvs on a just-built rawhide system and it failed like this, since I hadn't installed the "check" package, which defines PKG_CHECK_EXISTS: checking for init script flavor... redhat checking for iptables... /sbin/iptables ./configure: line 12187: syntax error near unexpected token `libxml-2.0,LIBXML_FOUND=yes' ./configure: line 12187: ` PKG_CHECK_EXISTS(libxml-2.0,LIBXML_FOUND=yes)' Here's a fix: 2007-10-17 Jim Meyering <meyering(a)redhat.com> * libvirt.spec.in (BuildRequires): Add "check". Index: libvirt.spec.in =================================================================== RCS file: /data/cvs/libvirt/libvirt.spec.in,v retrieving revision 1.65 diff -u -p -r1.65 libvirt.spec.in --- libvirt.spec.in 12 Oct 2007 19:54:15 -0000 1.65 +++ libvirt.spec.in 17 Oct 2007 10:03:22 -0000 @@ -16,6 +16,7 @@ Requires: ncurses Requires: dnsmasq Requires: bridge-utils Requires: iptables +BuildRequires: check BuildRequires: xen-devel BuildRequires: libxml2-devel BuildRequires: readline-devel

2 2

[Libvir] "make check" requires qemu
by Jim Meyering 17 Oct '07

17 Oct '07

So perhaps libvirt should BuildRequire qemu, too? When I ran "make check" on a system without qemu, I got lots of failures: ... QEMU XML-2-ARGV Misc ACPI ... FAILED libvir: QEMU error : Cannot find QEMU binary /usr/bin/qemu: No such file or directory QEMU XML-2-ARGV Misc No Reboot ... FAILED libvir: QEMU error : Cannot find QEMU binary /usr/bin/qemu: No such file or directory QEMU XML-2-ARGV Net User ... FAILED FAIL: qemuxml2argvtest Here's a patch: 2007-10-17 Jim Meyering <meyering(a)redhat.com> * libvirt.spec.in (BuildRequires): Add "qemu". Index: libvirt.spec.in =================================================================== RCS file: /data/cvs/libvirt/libvirt.spec.in,v retrieving revision 1.65 diff -u -p -r1.65 libvirt.spec.in --- libvirt.spec.in 12 Oct 2007 19:54:15 -0000 1.65 +++ libvirt.spec.in 17 Oct 2007 10:11:33 -0000 @@ -25,6 +25,7 @@ BuildRequires: gnutls-devel BuildRequires: avahi-devel BuildRequires: dnsmasq BuildRequires: bridge-utils +BuildRequires: qemu Obsoletes: libvir ExclusiveArch: i386 x86_64 ia64

2 1

[Libvir] sexpr_get: add __printf__ attribute
by Jim Meyering 17 Oct '07

17 Oct '07

I noticed sexpr_get takes a printf-style format, yet it is declared with no printf attribute. Here's the fix: 2007-10-17 Jim Meyering <meyering(a)redhat.com> * src/xend_internal.c (sexpr_get): Declare with gcc's printf attribute. Index: src/xend_internal.c =================================================================== RCS file: /data/cvs/libvirt/src/xend_internal.c,v retrieving revision 1.148 diff -u -p -r1.148 xend_internal.c --- src/xend_internal.c 15 Oct 2007 21:38:56 -0000 1.148 +++ src/xend_internal.c 17 Oct 2007 08:29:26 -0000 @@ -692,6 +692,10 @@ xend_op(virConnectPtr xend, const char * */ static struct sexpr * sexpr_get(virConnectPtr xend, const char *fmt, ...) + __attribute__ ((__format__ (__printf__, 2, 3))); + +static struct sexpr * +sexpr_get(virConnectPtr xend, const char *fmt, ...) { char buffer[4096]; char path[1024];

2 2

[Libvir] FYI: bug in remote code
by Mark Johnson 17 Oct '07

17 Oct '07

I ran into a bug in the remote code... I was doing # virsh -c xen_tcp://<ip_addr>/ It you don't have qemu compiled in, you will hang in in libvirtd in do_open() res = virNetworkDriverTab[i]->open (ret, name, flags); when trying to open the remote network driver. Usually qemu returns success and you fall out of the loop before you call this (which is why you won't see it with qemu support built in). You hang in libvirtd trying to read from the libvirtd PF_UNIX socket. You read from the socket because you hit this code path in doRemoteOpen() if (!uri->server && !transport_str) { if (flags & VIR_DRV_OPEN_REMOTE_UNIX) { transport = trans_unix; No idea what it should be doing? :-) I reproduced it on todays CVS bits on FC7 by commenting out qemu's network register.. int qemudRegister(void) { virRegisterDriver(&qemuDriver); /* virRegisterNetworkDriver(&qemuNetworkDriver); */ virRegisterStateDriver(&qemuStateDriver); return 0; } MRJ

2 4

[Libvir] [for discussion only] virDomainBlockPeek preliminary implementation
by Richard W.M. Jones 17 Oct '07

17 Oct '07

The attached patch (for discussion only) adds a virDomainBlockPeek call, allowing callers to peek into the block devices of domains. +/** + * virDomainBlockPeek: + * @dom: pointer to the domain object + * @path: path to the block device + * @offset: offset within block device + * @size: size to read + * @buffer: return buffer (must be at least size bytes) + * + * This function allows you to read the contents of a domain's + * disk device. + * + * Typical uses for this are to determine if the domain has + * written a Master Boot Record (indicating that the domain + * has completed installation), or to try to work out the state + * of the domain's filesystems. + * + * (Note that in the local case you might try to open the + * block device or file directly, but that won't work in the + * remote case, nor if you don't have sufficient permission. + * Hence the need for this call). + * + * 'path' must be a device or file corresponding to the domain. + * In other words it must be the precise string returned in + * a <disk><source dev='...'/></disk> from + * virDomainGetXMLDesc. + * + * 'offset' and 'size' represent an area which must lie entirely + * within the device or file. 'size' may be 0 to test if the + * call would succeed. + * + * 'buffer' is the return buffer and must be at least 'size' bytes. + * + * Returns: 0 in case of success or -1 in case of failure. + */ +int +virDomainBlockPeek (virDomainPtr dom, + const char *path, + long long offset /* really 64 bits */, + size_t size, + void *buffer) The patch doesn't include: - remote - qemu - Xen < 3.0.3 inactive domains I will add these if I get favourable feedback. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

2 3

[Libvir] PATCH: Implement CDROM media change for QEMU/KVM driver
by Daniel P. Berrange 17 Oct '07

17 Oct '07

Hugh recently added support for CDROM media change to the Xen driver, so I thought it was time I did the same for the QEMU/KVM driver. The attached patch implements the virDomainAttachDevice API to support changing the media of the CDROM device. To re-use the existing QEMU code for parsing I had to re-factor a couple of the internal QEMU device parsing apis so that they get supplied with a pre-allocated struct for the device info, rather than allocating it themselves. Although this isn't strictly needed for CDROM media change, I have prepared the virDomainAttachDevice method so that I can easily add USB device hotplut in the near future. It will also be useful when KVM gets paravirt driver support sooon... The actual implementation just runs the 'change cdrom path' command over the QEMU monitor console. diffstat libvirt-qemu-cdrom-change.patch qemu_conf.c | 135 +++++++++++++++++++++++++++++++++++++++------------------- qemu_conf.h | 21 +++++++++ qemu_driver.c | 132 ++++++++++++++++++++++++++++++++++++++++++++++++++------ 3 files changed, 230 insertions(+), 58 deletions(-) Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 7

[Libvir] Thoughts on remote storage support
by Tóth István 17 Oct '07

17 Oct '07

Hello! I had a hobby project where I needed to manipulate xen disk images on remote systems, that used a model similar to libirt's remote support. Based on what I learnt from it, I came up with a possible model for libvirt's remote storage support. I present it here for discussion. We typically store the images in volumes on LVM, or in dedicated file system folders. The folders and volume groups usable by libvirt can be limited in a config file. It is probably not neccessary to differentiate between defined and created files, as you can not stop and start a file like a domain, you either have it on disk, or not. Libvirt should not store information on these files, everything should be checked/listed on the fly, so that if you just copy an image to a directory, libvirt can deduct all information (well, all it can) on it, and handle it just as if the file was created by it. The handle for the file is its path, plus its virConnect object (i.e. the host it is on). For consistency, it may be possible to create an object for it, but as disk images have no persistent properties apart from what is on the disk, and it can always be checked from there, it provides no extra functionality. I think there is no need to support remote files explicitly, as the domains mount local files/volumes. The file/volume may actually be mounted from a NAS or SAN, of course, but it does not matter because we use the local path names, and AFAIK all virtualization tools use local files or local devices as blockdevs. I have added compression to the mix because it is immensely useful. I have used lzop in my project, and a full backup and restore was much faster when using a compressed backup file, than with and uncrompessed one. It conserves disk space, as well as cpu/bus capacity. Zeroing out newly allocated files, helps with compressed backups, as well as security. It also means that no holey files can be used. The objects we are dealing with are disk images. They have the following properties: -Path: The unix path of the file ( /mnt/images/fc7.img or /dev/VG/fc7) -Compression: Mountable/compressed -Type: Plain file/LVM volume/ What else? -Size -Filesystem: swap/ext3/xfs/.... -Is it mounted? We can do the following operations on the images: Create -connection -filepath -size Allocates a new image of the given type, size, and name. Libvirt should parse the filepath, and determine the base path, check if it's a directory or a VG, check if libvirt is allowed to operate on the path/VG, then create the file/volume. For security reasons zeroing out the allocated space should be a non-optional step of the allocation. DirectoryList -connection -directorypath Plain ls functionality, that returns the list of files, and any subdirectories. If called on a VG, it returns the volumes in it. Info -connection -filepath Returns information on the given file/volume, including size, type, filesystem (if available), whether it is a snapshot (if a volume), and whether it is mounted or not. size can be determined by ls or lvinfo, filesystem by 'file' command. Delete -connection -filepath Delete the file/volume. Find out if it's a file or volume, and rm or lvremove it. Grow -connection -path -filename -newsize Grows the specified image to the given size. The newly added space is zeroed out. Shrink? -connection -path -filename -newsize Shrinks the specified image the the given size. It's very tricky, because to avoid data loss, we need to analyze the file system size. Of course, we can just say that it's the reponsibility of the user, after all, we allow him to outright delete the file as well. We may combine it with Grow, and call it Resize. Growfs -connection -path -filename -newsize? Grows the filesystem on the image to fill the size of the image, or the given newsize. Can only be used on umounted images. It is neccesary because some filesystems may not be grown while mounted, so the guest can not do it on its own. Shrinkfs -connection -path -filname -newsize Shrinks the filesystem to the given size. Same coniderations apply as with growfs, may be combined to Resizefs? Snapshot -connection -filepath -filesize Creates a snapshot of the given image. It is only possible with images on LVM. Should return the snapshot image name. The snapshot can later be deleted with Delete. CopyTo -connection -source filepath -target connection -target filepath -snapshot flag -archive flag -overwrite flag Copy the source image to the target image. If connection is on another machine, then it's a network copy. If the snapshot flag is true, then first create a snapshot of the source image, copy that, then delete the snapshot. If the archive flag is active, then the target file will be archive file (compressed). If the overwite flag is active, then the target file is overwitten, if it exists. Otherwise existing files are not changed. Even if the source file is compressed, the target file is uncompressed, unless the archive flag is set. CopyContents -connection -source filepath -target connection -target filepath -snapshot flag Copies the contents of the source file to the target file. The target file must already exist, and be no smaller than the source file. The contents of the target file are overwitten, and any extra space is zeroed out. Archive -connection -filepath compresses the given file. Makes sense only on files, not volumes. Unarchive -connection -filepath uncompresses the given file. Makes sense only on archved(compressed) files. StorageInfo - connection Returns information on the node's storage configuration. What kind of filesystems it can handle, What are the accessible file / VG paths, what's the free space on them, etc. A typical usage scenario could be something like this: Aconn=getVirconn("ssh:Ahost"); //Open the connection to host A Bconn=getVirconn("ssh:Bhost"); //Host B will hold our backup image Ainfo=StorageInfo(Aconn); //Get Ainfo AVGPath = <get the first usable VG path from VG info> Newimage = Create(Aconn, concat("AVGPath", "newimage", 100000); //A 100Mb volume is created and zeroed out. CopyContents(Aconn, NewImage, Aconn, "/images/ghost/FC7default.img", no); //Copy our pre-created FC7 image to the new image Growfs(Aconn, NewImage, 0); //Grow the copied filesystem to fill the whole volume. <Here we define a new domain, and use NewImage as the name of the backing image for the guest's block device> <Start the new domain> Copy(Aconn, NewImage, Bconn, "/mnt/backups/backup23image", snapshot=yes, archive=yes, overwrite=no); //Make an LVM snapshot of NewImage, and copy it to Host B on the given filename, compressing it on the fly, then remove the snapshot <Stop the domain> CopyContents(Bconn, "/mnt/backups/backup23image", Aconn, NewImage, snapshot=no); //Restore the backed-up image to NewImage, decompress it on the fly. <Start the domain> <Stop the domain> Grow(Aconn, NewImage, 200000); //Grow the volume to 200MBs Growfs(Aconn, NewImage); //Grow the fs on the volume to fill the volume <Start the domain> ..... Best regards István

3 4

[Libvir] [PATCH] Refactor xend_internal.c block device detection
by Richard W.M. Jones 17 Oct '07

17 Oct '07

This patch is a little bit esoteric, but I need it for something I'm working on at the moment. At the moment the code in xend_internal.c: xend_parse_sexp_desc parses the domain sexpr directly into XML. This makes it rather hard to just get a list of block devices without repeating the same code. So here I've factored out the common code for parsing block devices into a separate function and callback. There are a couple of small changes that you should be aware of: (1) The <devices> list may be returned in a different order (specifically, block devices are always returned first). (2) We iterate over the root nodes of the sexpr twice (once for block devices, once for vifs and vfbs). But the sexpr is small and in-memory so this shouldn't be a problem compared to having to do the HTTP request to xend to get it in the first place. 'make check' passes. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

3 4

[Libvir] remote access ?
by Mark Johnson 16 Oct '07

16 Oct '07

I was looking through some of the remote code.. I was wondering if a 32-bit virsh client is supposed to be able to connect to a 64-bit libvirtd? The reason I ask is because I saw some references to longs.. For example, in xdr_remote_node_get_info_ret() ... objp->cpus = IXDR_GET_LONG(buf); objp->mhz = IXDR_GET_LONG(buf); objp->nodes = IXDR_GET_LONG(buf); objp->sockets = IXDR_GET_LONG(buf); objp->cores = IXDR_GET_LONG(buf); objp->threads = IXDR_GET_LONG(buf); Thanks, MRJ

4 17

[Libvir] [PATCH] [REPOST] Remove virDomainRestart typedef from the public API
by Richard W.M. Jones 16 Oct '07

16 Oct '07

... however it is still used internally by the test driver, so the typedef is moved into src/test.c. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

3 2

[Libvir] [PATCH] Topology fix for "no cpus"
by beth kon 15 Oct '07

15 Oct '07

I was able to test on a 128-way NUMA box and found a bug. My code did not handle the case of no cpus being associated with a node. I decided to represent (pretty straightforward decision :-) no cpus as follows in the xml... <cell id='2'> <cpus num='0'> </cpus> </cell> Here is the patch... Signed-off-by: Beth Kon <eak(a)us.ibm.com> -- Elizabeth Kon (Beth) IBM Linux Technology Center Open Hypervisor Team email: eak(a)us.ibm.com

3 3

[Libvir] PATCH: Fix const-ness for attach/detach device APIs
by Daniel P. Berrange 15 Oct '07

15 Oct '07

virDomainAttachDevice and virDomainDetachDevice both take a char * for the XML description, but this is mistakenly not declared to be const. This patch fixes the public header files & all the internal drivers. NB, yes this is in the public API, no it won't break any apps since these are input params only. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 2