- Devel - libvirt List Archives

[Libvir] [Announce] libvirt-cim : An open-source CIM provider for libvirt
by Daniel Veillard 05 Nov '07

05 Nov '07

libvirt-cim : An open-source CIM provider for libvirt This is the announcement of a new open-source project called libvirt-cim based on libvirt and aiming at offering the complete functionality of libvirt via a CIM provider implementing the DMTF SVPC virtualization model and released under an LGPL licence. A CIM [1] provider is an implementation of a set of standardized interfaces (Common Information Model) whose goal are to provide well defined entry points allowing easier and interoperable management tools to be built. In the case of libvirt-cim, the goal is of course to export the SVPC virtualization model, which then can be used to manage storage, hosts and domains remotely. Currently CIM based management targets mostly enterprise deployments but has the potential of solving the more generic problem of remote control interfaces which affects nearly all virtualization use cases and many other aspects of the computing environment. The current state of the project is based on an initial implementation provided by IBM, and released for the libvirt project under an LGPL licence. It currently targets only Xen virtualization but the goal is to expand support to the other mechanisms supported by libvirt, like KVM. It is made of a library libcmpiutil of library functions which are not specific to libvirt and useful for CMPI, the programming interfaces used to develop CIM providers. The main part is in the libvirt-cim module. The CIM provider is currently tested with the Pegasus [2] and SFCB [3] environment and the goal is to ensure compatibility with the various CIM based management systems. The project is hosted on libvirt.org, the source code is managed using Mercurial [4] and Dan Smith [5] is the project maintainer doing the commits. The project home page is available at: http://libvirt.org/CIM/ source code download is available at ftp://libvirt.org/libvirt-cim/ and the Mercurial tree is located at: http://libvirt.org/hg/ A dedicated mailing-list has been set up, libvirt-cim(a)redhat.com with the usual list page and archive mailman pages at: http://www.redhat.com/mailman/listinfo/libvirt-cim Daniel [1] http://en.wikipedia.org/wiki/Common_Information_Model_%28computing%29 [2] http://www.openpegasus.org [3] http://sblim.wiki.sourceforge.net/Sfcb [4] Mercurial: http://www.selenic.com/mercurial/wiki/ [5] Dan Smith <danms(a)us.ibm.com> -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

1 0

[Libvir] collectd libvirtstats plugin (version 1, for discussion only)
by Richard W.M. Jones 05 Nov '07

05 Nov '07

Attached is a plugin allowing collectd to use libvirt to collect CPU, disk and network statistics for domains. There are a few problems with it at the moment, described below. Firstly just to say that this is a patch against the released collectd 4.2.0. I need to pull out the GIT repository and rebase the patch against that. Any other comments, most welcome! Usage ----- With this patch you should be able to collect stats by adding the following to your collectd.conf file: LoadPlugin libvirtstats <Plugin libvirtstats> Connection "xen:///" </Plugin> 'collectd' only needs to be running on the host. It uses libvirt to collect stats for each guest, via mechanisms in the hypervisor or host operating system. Requirements ------------ libvirt >= 0.3.3 and libxml2. The configure file should auto-detect the requirements and configure libvirtstats plugin only if they are available. Problems -------- I can't seem to get the ignorelist to do anything at all. It seems like it includes all the domains and devices, no matter what I specify in the configuration file and what the state of IgnoreSelected is set to. In the types.db, I have set the max for virt_cpu_total to be 256,000,000,000. The max CPU usage for a single CPU is 1 billion nanoseconds per second, but virt_cpu_total is for all CPUs in the system, and there could be (in theory) an unlimited number of CPUs, so an unlimited max. Therefore I have assumed here that max CPUs = 256. I'm using CSV output, which works, but it seems to report the aggregate counter values instead of the differences. Is that to be expected? Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

1 0

[Libvir] [PATCH] finish NUMA code reorg, plug cpuset at creat time support
by Daniel Veillard 05 Nov '07

05 Nov '07

The following patch finishes the cleanup for NUMA parsing code: - the cpuset parsing is moved to xml.c - some comments and cleanups of the include then add the output of a (cpus '...') line based on the /domain/vcpu/@cpuset attributes, this is parsed and reserialized as ranges to avoid any possibility of misinterpretation of say ^ or any special syntax we may want to add in the future. A few things to note: - dependant on the tiny patch I sent earlier today - if we notice that the set covers all CPU maybe we should avoid outputing (cpus '...'), trivial to add - mostly untested yet Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

3 10

[Libvir] [PATCH] Fix failure of HVM domain definition on Xen 3.0.3 or earlier(BZ328841)
by Masayuki Sunou 01 Nov '07

01 Nov '07

Hi VirDomainDefineXML() fails, when HVM domain using CD-ROM is created on Xen 3.0.3 or earlier. -->BZ328841(https://bugzilla.redhat.com/show_bug.cgi?id=328841) One of attached patches (define.patch) fixes it. But, when domain is started by virsh start using the configuration file created by fixing this, the problem that the CD-ROM device is missed occurs. So, another patch (start.patch) fixes this problem. Signed-off-by: Masayuki Sunou <fj1826dm(a)aa.jp.fujitsu.com> Thanks, Masayuki Sunou.

2 5

[Libvir] PATCH: 0/7 Implementation of storage APIs
by Daniel P. Berrange 31 Oct '07

31 Oct '07

Since the previous discussions didn't really end up anywhere conclusive I decided it would be better to have a crack at getting some working code to illustrate my ideas. Thus, the following series of 7 patches provide an implementation of storage APIs which follow the scheme outlined in the previous mail on storage concepts http://www.redhat.com/archives/libvir-list/2007-October/msg00195.html I have only addressed storage pools & volumes. I am not tackling the host device enumeration APIs at this time, since it is not a blocker for the rest of the work. The mails which follow, in particular the last one, will explain the patches in more details. The focus is primarily on providing the public API, and wiring up the remote driver, daemon and protocol, and the generic storage driver. The generic storage driver then calls out to specific backends for different types of storage. implementation provides a fully functional backend for managing local directories containing raw files (aka equivalent of /var/lib/xen/images). There are empty stubs for LVM, iSCSI, Disks/Partitions, etc to be filled in as we decide... Some open questions - Is it worth bothering with a UUID for individual storage volumes. It is possible to construct a globally unique identifier for most volumes, by combing various bits of metadata we have such as device unique ID, inode, iSCSI target & LUN, etc There isn't really any UUID that fits into the classic libvirt 16 byte UUID. I've implemented (randomly generated) UUIDs for the virStorageVolPtr object, but I'm inclined to remove them, since its not much use if they change each time the libvirtd daemon is restarted. The 'name' field provides a unique identifier scoped to the storage pool. I think we could add a 'char *key' field, as an arbitrary opaque string, forming a globally unique identifier for the volume. This would serve same purpose as UUID, but without the 16 bytes constraint which we can't usefully provide. - For the local directory backend, I've got the ability to choose between file formats on a per-volume basis. eg, /var/lib/xen/images can contain a mix of raw, qcow, vmdk, etc files. This is nice & flexible for the user, but a more complex thing to implement, since it means we have to probe each volume and try & figure out its format each time we list volumes. If we constrained the choice between formats to be at the pool level instead of the volume level we could avoid probing & thus simplify the code. This is what XenAPI does. - If creating non-sparse files, it can take a very long time to do the I/O to fill in the image. In virt-intsall/virt-manager we have nice incremental progress display. The API I've got currently though is blocking. This blocks the calling application. It also blocks the entire libvirtd daemon since we are single process. There are a couple of ways we can address this: - Allow the libvirtd daemon to serve each client connection in a separate thread. We'd need to adding some mutex locking to the QEMU driver and the storage driver to handle this. It would have been nice to avoid threads, but I don't think we can much longer. - For long running operations, spawn off a worker thread (or process) to perform the operation. Don't send a reply to the RPC message, instead just put the client on a 'wait queue', and get on with serving other clients. When the worker thread completes, send the RPC reply to the original client. - Having the virStorageVolCreate() method return immediately, giving back the client app some kind of 'job id'. The client app can poll on another API virStorageVolJob() method to determine how far through the task has got. The implementation in the driver would have to spawn a worker thread to do the actual long operation. Possibly we can allow creation to be async or blocking by making use of the 'flags' field to virStorageVolCreate() method, eg VIR_STORAGE_ASYNC. If we make async behaviour optional, we still need some work in the daemon to avoid blocking all clients. This problem will also impact us when we add cloning of existing volumes. It already sort of hits us when saving & restoring VMs if they have large amounts of memory. So perhaps we need togo for the general solution of making the daemon threaded per client connection. The ASYNC flag may still be useful anyway to get the incremental progress feedback in the UI. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

4 26

[Libvir] PATCH: Support NIC model selection for QEMU/KVM
by Jim Paris 31 Oct '07

31 Oct '07

Hi, Sometime between kvm-36 and kvm-46 I ran into problems with the default QEMU network card (ne2k-pci). Switching it fixed the problems, but libvirt doesn't support changing the NIC model. These patches add support for: <interface> <nic model="rtl8139"/> </interface> which becomes qemu -net nic,model=rtl8139,mac=... By default, no model is appended to the qemu command line, as before. Documentation update & some fixes are included too. -jim

3 11

[Libvir] read-only git mirror of libvirt CVS repository
by Jim Meyering 26 Oct '07

26 Oct '07

I've set up an automatically-sync'd git mirror of the libvirt.org's libvirt CVS repository. The mirror is updated approximately every 30 minutes. You can view the change-set summary via this URL: http://git.et.redhat.com/?p=libvirt.git You can get a copy of the repository with the following command: (this uses the efficient "git://" protocol, and is sort of analogous to CVS pserver, in that you get anonymous, read-only access) git clone git://et.redhat.com/libvirt That command creates a libvirt/ directory. It's not big, just 4.2MB for the .git/ repository (containing the entire history) and the usual 10MB of code, docs and .po files. Note, if you're new to "git", you typically run "git clone" only once, initially, and from then on, you "cd" into the directory and download/merge the latest sources via "git pull". But note that if you do development or otherwise modify a version-controlled file, your next "git pull" won't work if the pull would have to merge changes to one of the modified files.

5 8

[Libvir] Build error without Xen
by Jim Paris 26 Oct '07

26 Oct '07

Hi, The recent changes to src/xml.c breaks the build without xen: $ ./autogen.sh \ --prefix=/usr \ --mandir=\${prefix}/share/man \ --infodir=\${prefix}/share/info \ --sysconfdir=/etc \ --localstatedir=/var \ --without-xen $ make ... gcc -g -O2 -o .libs/virsh -Wall -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wextra -Wshadow -Wcast-align -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Winline -Wredundant-decls -Wno-sign-compare -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fasynchronous-unwind-tables virsh-virsh.o virsh-console.o -Wall -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wextra -Wshadow -Wcast-align -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Winline -Wredundant-decls -Wno-sign-compare -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fasynchronous-unwind-tables ./.libs/libvirt.so -lcurses -lreadline ./.libs/libvirt.so: undefined reference to `xenNbCpus' collect2: ld returned 1 exit status make[2]: *** [virsh] Error 1 make[2]: Leaving directory `/usr/local/src/libvirt/libvirt/src' -jim

3 4

[Libvir] problems using libvirt to mange "defined" domains.
by Evan Bigall 26 Oct '07

26 Oct '07

I'm having a problem using libvirt to manage "defined" domains (ie: domains for which it has XML, but are not running.) I have tried several debugging scenarios, but the simplest one is this. In a libvirt C program I define the following XML: char domxml[]="<domain type='xen' id='99'>" "<name>rhel5vm</name>" "<uuid>f948d7f4f36c4bfae8053bca535183b3</uuid>" "<os>" "<type>linux</type>" "<kernel>/var/lib/xen/boot/rhel5vm/vmlinuz-2.6.18-8.el5xen</kernel>" "<initrd>/var/lib/xen/boot/rhel5vm/initrd-2.6.18-8.el5xen.img</initrd>" "<root>/dev/xvda1</root>" "</os>" "<memory>512000</memory>" "<vcpu>1</vcpu>" "<on_poweroff>destroy</on_poweroff>" "<on_reboot>restart</on_reboot>" "<on_crash>restart</on_crash>" "<devices>" "<interface type='bridge'>" "<source bridge='xenbr0'/>" "<mac address='00:16:3e:09:ef:b0'/>" "<script path='vif-bridge'/>" "</interface>" "<disk type='block' device='disk'>" "<driver name='phy'/>" "<source dev='/dev/sda3'/>" "<target dev='xvda'/>" "</disk>" "<console tty='/dev/pts/2'/>" "</devices>" "</domain>"; (I've played with several values for the domain id). What I find is that when I use virDomainDefineXML() to define the domain, for some reason it drops, the "root" element from the "os" specification. Because of this the domain then crashes when I try to start it. I've also tried, using virsh to create the domain from an XML file, capture the XML with virDomainGetXMLDesc(), pause the C program, and then try to redefine the domain with the domain with the exact XML virDomainGetXMLDesc() gave me (after destroying the original of course), and I get the same behavior, it strips the root element from the OS specification. Here is the version information: [root@rhel5-xen tmp]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 5 (Tikanga) [root@rhel5-xen tmp]# uname -a Linux rhel5-xen 2.6.18-8.1.8.el5xen #1 SMP Mon Jun 25 17:19:38 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux [root@rhel5-xen tmp]# virsh version Compiled against library: libvir 0.3.1 Using library: libvir 0.3.1 Using API: Xen 3.0.1 Running hypervisor: Xen 3.0.0 I am using rhel5 "out of the box" with libvirt upgraded to: libvirt-0.3.1-1.x86_64.rpm libvirt-devel-0.3.1-1.x86_64.rpm libvirt-python-0.3.1-1.x86_64.rpm I tried using libvirt-0.3.2-1, but sample code segv'd in virConnectClose(). All I'm really trying to do is build a primitive interface to list domains (both running, and non-running), and start and stop them. I'm open to suggestions as to whether I have this misconfigured or if there would be a better version with which to do this. Thanks, Evan

2 1

Re: [Libvir] Creating a Domain
by Richard W.M. Jones 26 Oct '07

26 Oct '07

chetan saundankar wrote: > "the API should not be targetted to a single virtualization environment > though Xen is the current default...." > Does this mean that other virtualization environments say VMWare ESX > server management can also be done with future releases of Libvirt? If > yes then when do you expect to roll out that feature? That's certainly an aspiration. However there is no firm date for VMWare support at this time. Currently we have support for Xen, QEMU, KVM and OpenVZ. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

1 0