- Devel - libvirt List Archives

[Libvir] virtinst 0.100.0 and virt-manager 0.3.0 releases
by Daniel P. Berrange 23 Jan '07

23 Jan '07

Following on from the recent libvirt 0.1.11 release, I'm pleased to announce new releases of the virtinst & virt-manager applications. The virtinst 0.100.0 release is a major re-factoring to make use of the libvirt inactive domain management APIs when provisioning guests. It also tries to clean up many of the Xen specific bits to facilitate future use with the QEMU backend of libvirt. From a usability point of view it also now displays progress information when downloading kernel & initrd and creating the filesystem images. The virt-manager 0.3.0 release brings a major functionality update, enabling of inactive domain management. This requires libvirt at least version 0.1.11 to provide implementations of inactive domain management for Xen 3.0.3 and Xen 3.0.4. With this new functionality the display will list all guests which are in the 'shutoff' state. The guest can be started with the 'Run' button in the virtual console window. The vistinst package must also be updated to at least version 0.100.0 to ensure that during provisioning of guests it uses the new inactive domain management APIs. Finally there have been a variety of minor UI fixes & enhancements such as progress bars during guest creation, reliability fixes to the virtual console and even greater coverage for translations. As of today, virtinst is now formally a part of the virt-manager project, so downloads for both virtinst & virt-manager are available from the same download page: http://virt-manager.et.redhat.com/downloads.html All historical release from virtinst 0.95.0 onwards are available there. In addition, to encourage / facilitate broader community development & feedback on the virtinst/virt-manager applications we have decided to designate et-mgmt-tools(a)redhat.com as the project primary mailing list: http://www.redhat.com/mailman/listinfo/et-mgmt-tools We welcome feedback on what additional capabilities / features people would like to see in future virtinst / virt-manager releases. As a taste of things to come, we have active developments plans for: - Secure authenticated remote management - Support for QEMU & KVM virtualization Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

1 1

[Libvir] Release of libvirt-0.1.11
by Daniel Veillard 22 Jan '07

22 Jan '07

I just made a release, last one was one month ago, there is a leak fixed and this make it simpler to commit new code for QEmu and remote support: * Finish XML XM config files support (Daniel Berrange) * Remove memory leak when freeing virConf objects (Daniel Berrange) * Finishing inactive domain support (Daniel Berrange) * Added a Relax-NG schemas to check XML instances Available as usual at ftp://libvirt.org/libvirt/ thanks everybody ! Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

1 0

[Libvir] Tricky provisioning problem with inactive domains
by Daniel P. Berrange 17 Jan '07

17 Jan '07

Adding support for inactive domains was supposed to make everyone's life easier, but as luck would have it, its actually made one thing very much harder. In the virt-inst/virt-manager tools provisioning works something like this: In paravirt case: - Create a guest using an explicit kernel/initrd from the images/xen directory on the install CD - Write a config file to /etc/xen setup to boot using pygrub In fullvirt case: - Create a guest booting directly off a CDROM - Write a config file to /etc/xen setup to boot off the harddisk So in both these cases, the libvirt XML config for the very first boot of the guest is different, from the XML config for subsequent boots. With the new inactive domain support in libvirt & xend, we can't write out config files directly, instead there is the virDomainDefine() API, which calls to appropriate APIs in XenD. And this is where the problem arises: 1. If we call virDomainDefine() to write the long term config, then virDomainStart() will not be using the correct boot method for the initial install. 2. If we call virDomainDefine() to write the initial install config, then virDomainStart() will kick off the install correctly, but on subsequent boots we'll end up booting the installer again instead of the just installed OS. 3. We could just use virDomainCreate() to start installer, and try to use virDomainDefine() to write the long term config - the latter call will fail though because there will already be a running guest with that name. 4. Conversely if we use virDomainDefine() to write the config, and then tried to create a one-off guest with virDomainCreate() the latter will fail due to duplicate names. So, thus far the only way out of the trap I can think of is: 1. Use virDomainCreate() to kick off the initial install 2. Poll virDomainLookupByXXX() to watch for this initial guest shutting down 3. Write out the persistent config using virDomainDefine() The big problem with this, is that if the user were to exit virt-manager sometime after the guest install starts, but before step 3, the config for the guest will never be written, even though it has successfully installed. There are two further ideas I've had - both requiring additional APIs in libvirt & probably XenD too. - Make it possible change the boot configuration of an existing guest. This would let us do: 1. Use virDomainDefine() to define a config file suitable for installing the guest, ie using explicit kernel/initrd 2. Use virDomainStart() to kick off the installer 3. Uew new API to change the guest config to remove explicit kernel & initrd config, and add a bootloader for pygrub. Or in HVM case, switch boot order to use harddisk instead of CDROM & detach the CDROM device. - Make it possible to start an existing inactive guest using an alternative one-off configuration. This would let us do: 1. Use virDomainDefine() to define a config file suitable for running the guest during normal operation. 2. Use virDomainStartConfig(xml) to start the guest with a special config suitable for installing the guest. Ultimately I think we do need the means to change arbitrary parts of a guest configuration, so I think the first option would be the preferred approach. The trouble is that I think implementing this would require using the new XenAPI, or adding a number of new methods to the legacy SXPR API which is not really very desirable. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

5 7

Re: [Libvir] Virtual networking
by Richard W.M. Jones 16 Jan '07

16 Jan '07

Hugh Brock wrote: > Daniel P. Berrange wrote: > >> 3. The way I think you re suggesting - a libvirt server on every remote >> host which calls into the regular libvirt internal driver model to >> proxy remote calls. So even if the hypervisor in question provides a >> remote network management API, we will always use the local API and >> do *all* remote networking via the libvirt server >> >> http://people.redhat.com/berrange/libvirt/libvirt-arch-remote-2.png >> > This strikes me as *much* easier to manage, and the most consistent > thus far with the idea that libvirt should remain as > hypervisor-neutral as possible. I guess the management issue is going to be versioning the protocol. If the protocol is just a direct mapping of vir* calls and structures then you'll quickly end up in a situation where even the smallest change requires you to upgrade the world or old versions have to be maintained indefinitely. That's not saying I don't like the idea. Rich. -- Red Hat UK Ltd. 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 (will change soon)

2 1

Re: [Libvir] Virtual networking
by Hugh Brock 16 Jan '07

16 Jan '07

Richard W.M. Jones wrote: > Hugh Brock wrote: >> Daniel P. Berrange wrote: >>> 3. The way I think you re suggesting - a libvirt server on every remote >>> host which calls into the regular libvirt internal driver model to >>> proxy remote calls. So even if the hypervisor in question provides a >>> remote network management API, we will always use the local API and >>> do *all* remote networking via the libvirt server >>> >>> http://people.redhat.com/berrange/libvirt/libvirt-arch-remote-2.png >>> >> This strikes me as *much* easier to manage, and the most consistent >> thus far with the idea that libvirt should remain as >> hypervisor-neutral as possible. > > I guess the management issue is going to be versioning the protocol. If > the protocol is just a direct mapping of vir* calls and structures then > you'll quickly end up in a situation where even the smallest change > requires you to upgrade the world or old versions have to be maintained > indefinitely. > > That's not saying I don't like the idea. > True enough... we're guaranteeing we're going to have backwards compatibility problems. On the other hand the libvirt API is supposed to be held pretty stable. DV, any thoughts? --H

1 0

Re: [Libvir] Certificate management APIs ?
by Richard W.M. Jones 16 Jan '07

16 Jan '07

[Apologies also that this is not threaded with the original post] > $HOME/.libvirt/tls/ > | > +- ca > | | > | +- cert.pem > | +- ca-crl.pem Note that there are standard locations for CA certs. On my Debian box the standard locations appear to be /etc/ca-certificates.conf and /usr/share/ca-certificates. Not sure yet about Fedora/RHEL. I suppose you hope that people will be using formal CA's rather than their own, or at least have a CA certificate issued by a formal CA from which they can issue their own client & server certs. Rich. -- Red Hat UK Ltd. 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 (will change soon)

3 4

Re: [Libvir] Authenticate APIs ?
by Richard W.M. Jones 15 Jan '07

15 Jan '07

[Apologies that this is not threaded with the original post] > Following on from the issue of certificate management, is the issue of > authentication. This hasn't been an issue thus far, because Xen has zero > authentication. I'm not planning to make this same mistake with the QEMU > management daemon though - its going to have a secure data transport and > real authentication from day-1. Thus we need to consider how authentication > is exposed at the libvirt client API layer. > > First off, there are many possible authentication approaches: > > - Username + password > - Username + one time key > - Username + password digest > - Kerberos tickets > - x509 certificates > - ...etc I would definitely avoid over-engineering a solution. I suspect that "large corp" users will understand certificate infrastructures -- they probably already use X.509 client certificates to authenticate desktops -- and will be able to manage those. Everyone else will want to use ssh, the model for that being the way cvs allows you to flexibly hand over authentication problems to an external program through setting the $CVS_RSH environment variable. It's so easy to set up ssh to get passwordless remote logins. If they haven't set that up, and they're using libvirt through a command line tool like virsh then they'll get a prompt from ssh to type a password. ssh also has the advantage that it is very widely installed. The only issue is what command to run on the remote system. A simple command line tool which talks to the daemon over a socket might be one option. nc (netcat), gnutls_cli or stunnel might work too. Rich. -- Red Hat UK Ltd. 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 (will change soon)

2 1

[Libvir] Certificate management APIs ?
by Daniel P. Berrange 15 Jan '07

15 Jan '07

I now have the QEMU backend working with full wire encryption using the TLS protocol, and so ready to start thinking about various authentication related issues. - The client needs to have the certificate of the CA in order to validate the signature on the remote server's certificate (aka, just like web browsers need a set of CA certificates) Currently I just have it loading 'cert.pem' out of the current working directory of the app using libvirt. Obviously this is not a real solution. - The client technically should have a certificate revocation list too. Again I currently load this out of current working dir - The server end can optionally require the client to present a certificate too, as its means of authenticating the client. This obviously requires a way of specifying the client certificate and secret key in libvirt when opening the connection. - The first time it connects to a server, the client validates the server's certificate against the CA certificate, and remote hostname. Web browsers will also typically display the certificate to the user and ask them to allow one-time, allow permanently, or reject it. This would require a way to pass the certificate back to the client app during the virConnectOpen call. The above is all related to x509 certificates. So we basically have 5 types of file we need to store / provide to libvirt when opening a connection: - Certificate Authority's certificate - Certificate Authority's certificate revocation list - Client secret key - Client certificate - Trusted server certificates And some form of callback to let the client app validate the server certificate. As a rough stab at an API my thoughts are: typedef enum { VIR_CERT_CA_CERT, VIR_CERT_CA_CRL, VIR_CERT_CLIENT_KEY, VIR_CERT_CLIENT_CERT, } virConnectCertDataType; /** * dataType: one of the virConnectCertDataType constants * data: base64, PEM encoded certificate/key data */ int virConnectAddCertData(int dataType, const char *data); /** * dataType: one of the virConnectCertDataType constants * dataFile: file containing base64, PEM encoded certificate/key data */ int virConnectAddCertDataFile(int dataType, const char *dataFile); Virt-manager would use the latter API to load in its client certificate, CA certs, etc to libvirt prior to opening a connection to a TLS enabled server. This means libvirt doesn't have to make the policy on where to store its client cert data. On the otherhand, we probably want the files to be in a common location for virsh & virt-manager so both can use the same data. On yet another hand, virt-manager will also likely end up using TLS to connect to VNC, so needs access to the files independant on libvirt / virsh for that. So we should probably have these flexible APIs, but recommend that all apps use a pre-defined directory unless they have a particular need not to, eg $HOME/.libvirt/tls/ | +- ca | | | +- cert.pem | +- ca-crl.pem | +- client | | | +- cert.pem | +- key.pem | +- server | +- trusted.pem /** * hostname: canonical name of remote host providing certiifcate * cert: base64, PEM encoded certificate data * * Invoked to let client application decide whether to accept a * server certificate. The certificate wil already have been * validated against the CA cert, and its expiration date, etc * checked. This callback is intended to allow the end user to * accept/reject the certificate. The trusted flag indicates * whether the server is present in the list of known servers. */ typedef int (*virConnectCertificateValidator)(const char *hostname, const char *cert); int virConnectSetCertificateValidator(virConnectCertificateValidator cb) The applications like virt-manager would probably want to present the certificate details to the user for validation, also optionally maintaining a list of previously trusted server certs. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Authenticate APIs ?
by Daniel P. Berrange 15 Jan '07

15 Jan '07

Following on from the issue of certificate management, is the issue of authentication. This hasn't been an issue thus far, because Xen has zero authentication. I'm not planning to make this same mistake with the QEMU management daemon though - its going to have a secure data transport and real authentication from day-1. Thus we need to consider how authentication is exposed at the libvirt client API layer. First off, there are many possible authentication approaches: - Username + password - Username + one time key - Username + password digest - Kerberos tickets - x509 certificates - ...etc So of these require prompting the user for data (eg username / password), others will pick up their credentials from known sources (eg, the keberos ticket cache file in /tmp/krbXXX), or have to be provided ahead of time as part of the transport layer securit protocol (eg the x509 certificate file). We don't have to consider the latter case, since that's taken care of by the certificate management API - unless we want to figure out a more unified API, but I'm not sure it would be particularly sane. The interesting case from an API point of view is the first, which requires prompting the user for data. This is pretty much the PAM approach, with the added complication that you're in the middle of a network protocol and you may have several wire messages going back & forth, each time requesting more pieces of info. ie you can't possibly gather the neccessary info ahead of time. So we need some form of callback to collect the neccessary auth data. The network protocol equivalent of PAM is SASL. This actually has a number of different types of callback function yuou can register, depending on what sort of data you're prepared to provide as a client - Challenge / response: - const char *challenge - const char *prompt - const char *defresult, - const char **result, - unsigned *len - Password, username, language - const char **result - unsigned *len - AUthentication realm - const char **availrealms, - const char **result There are a few more auth callback types defined in SASL spec, which I've omitted here for brevity. We could map this onto an API something like this: enum { VIR_CONN_AUTH_PASSWORD, VIR_CONN_AUTH_USERNAME, VIR_CONN_AUTH_LANG, VIR_CONN_AUTH_CHALLENGE, VIR_CONN_AUTH_REALM, } virConnectAuthToken; typedef int (virConnectAuthCBSimple)(const char **result, unsigned *len); typedef int (virConnectAuthCBPrompt)(const char *challenge, const char *prompt, const char *defresult, const char *result, unsigned *len); typedef int (virConnetAuthCBRealm)(const char **availrealms, const char *result); typedef struct { int token; union { virConnectAuthCBSimple simple; virConnectAuthCBPrompt prompt; virConnectAuthCBRealm realm; } cb; } virConnectAuthCallback; int virConnectSetAuthCallbacks(virConnectAuthCallback *cbs, int ncbs); Now, of course not everything is going to be SASL based - Xen may well end up using TLS/SSL + HTTP Basic auth. There's a couple of other HTTP related auth protocols too. They are all basically a subset of SASL provides, so the above API ought to be sufficient to serve their auth needs too. Finally, one could simply say, this is all rather complicated, why don't we just use a simple username+password for everything. While this would be nice from a coding POV, I think we need to be forward looking and ensure we're setup to cope with things like Kerberos single-sign-on. This is why I'm looking at SASL for the QEMU authentication process - if you use libsasl.so you're app doesn't even need to know what auth method it is using - the admin can simple create an appropriate config file for sasl, and bingo you're fully kerberized & single sign-on capable. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Availability of the Relax-NG schemas for libvirt XML instances
by Daniel Veillard 13 Jan '07

13 Jan '07

As hinted previously in the QEmu thread, I developped a Relax-NG schemas to check the XML instances used to define Domains in libvirt. It's in CVS in the docs directory and also available online at http://libvirt.org/libvirt.rng For those not familiar with Relax-NG there is a nice tutorial and docs at http://relaxng.org/ Libxml2 on which libvirt depends has support for Relax-NG so we could easilly add checking at domain creation time within the library, but I decided to not add this at the moment, at least the schemas need to get more testing itself. The simplest to validate existing XML instances is to use the command line tool of libxml2 named xmllint in the following way: paphio:~/libvirt/tests -> xmllint --noout --relaxng ../docs/libvirt.rng xml2sexprdata/xml2sexpr-disk-drv-blkback.xml xml2sexprdata/xml2sexpr-disk-drv-blkback.xml validates paphio:~/libvirt/tests -> The schemas try to validate content as much as possible, but it's not possible to validate really in a complete fashion, for example to validate an IP address I used the following type: <define name='addrIP'> <data type='string'> <param name="pattern">([0-2]?[0-9]?[0-9]\.){3}[0-2]?[0-9]?[0-9]</param> </data> </define> which will catch '192.618.0.254' as erroneous but not '192.168.0.264' . There is also a number of cases which are structure related but hard to express cleanly in a schemas for example the fact that we may have an emulator description in the device list but that it's should only be allowed in an hvm kind of configuration. Still despite the limitations I found 2 weirdness: - we allow both <os> and <bootloader> blocks to be present, but I think they are redundant, I remember that Xen will ignore the bootloader informations if the os description is there, shouldn't we ban having both in a domain description ? e.g. tests/sexpr2xmldata/sexpr2xml-curmem.xml in the tree I still allowed this in the schemas but this may be wrong. - I though an interface of type ethernet needed a source description, but the output tests/sexpr2xmldata/sexpr2xml-net-routed.xml seems to miss both the source and target of the interface though the device is present in tests/sexpr2xmldata/sexpr2xml-net-routed.sexpr as (dev 'eth3') this probably indicates our conversion from sexpr to xml is lossy there. currently tests/sexpr2xmldata/sexpr2xml-net-routed.xml is the only example from our regression tests which doesn't pass the schemas testing. In general the schemas is both stricter than libvirt itself (as libvirt will only pick the parts of the XML it recognizes and need) and still allows some invalid constructs, I guess it's nearly impossible to make it a one to one match but I hope this can still be useful both for client tools, regression testings, and as another source of documentation. I still need to update the libvirt documentation, the XML page had some omissions and document the availability of the schemas and find its place when doing a 'make install', Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

2 1