[libvirt] Remote Xen and libvirt >= 0.5.0 Issues
Hi, I'm currently running a few servers with CentOS 5.3 (i386) with the stock Xen version: # rpm -q kernel-xen xen libvirt kernel-xen-2.6.18-92.1.22.el5 kernel-xen-2.6.18-128.1.6.el5 xen-3.0.3-80.el5_3.2 libvirt-0.6.3-1 libvirt was compiled from the src RPM from libvirt.org to run on i386 architecture without issue. I'm having a problem when trying to connect to remote Xen installations using versions of libvirt starting from 0.5.0 and above. When issuing virsh commands to query the local instance, it works without issue, but any remote sources gives 'error: failed to connect to the hypervisor'. Here's what LIBVIRT_DEBUG reveals: # LIBVIRT_DEBUG=6 virsh -d 5 -c xen://node3/ list command: "list " 13:53:17.132: debug : virInitialize:290 : register drivers 13:53:17.132: debug : virRegisterDriver:667 : registering Test as driver 0 13:53:17.132: debug : virRegisterNetworkDriver:567 : registering Test as network driver 0 13:53:17.132: debug : virRegisterStorageDriver:598 : registering Test as storage driver 0 13:53:17.132: debug : virRegisterDeviceMonitor:629 : registering Test as device driver 0 13:53:17.132: debug : xenHypervisorInit:1922 : Using new hypervisor call: 30001 13:53:17.132: debug : xenHypervisorInit:1991 : Using hypervisor call v2, sys ver3 dom ver5 13:53:17.132: debug : virRegisterDriver:667 : registering Xen as driver 1 13:53:17.132: debug : virRegisterDriver:667 : registering OPENVZ as driver 2 13:53:17.133: debug : vboxRegister:68 : VBoxCGlueInit failed: 0.0.0, errorval=-1 13:53:17.133: debug : virRegisterDriver:667 : registering VBOX as driver 3 13:53:17.133: debug : virRegisterDriver:667 : registering remote as driver 4 13:53:17.133: debug : virRegisterNetworkDriver:567 : registering remote as network driver 1 13:53:17.133: debug : virRegisterStorageDriver:598 : registering remote as storage driver 1 13:53:17.133: debug : virRegisterDeviceMonitor:629 : registering remote as device driver 1 13:53:17.133: debug : virConnectOpenAuth:1100 : name=xen://node3/, auth=0x67d0080, flags=0 13:53:17.133: debug : do_open:920 : name "xen://node3/" to URI components: scheme xen opaque (null) authority (null) server node3 user (null) port 0 path / 13:53:17.133: debug : do_open:930 : trying driver 0 (Test) ... 13:53:17.133: debug : do_open:936 : driver 0 Test returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 1 (Xen) ... 13:53:17.133: debug : do_open:936 : driver 1 Xen returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 2 (OPENVZ) ... 13:53:17.133: debug : do_open:936 : driver 2 OPENVZ returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 3 (VBOX) ... 13:53:17.133: debug : do_open:936 : driver 3 VBOX returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 4 (remote) ... 13:53:17.133: debug : do_open:936 : driver 4 remote returned DECLINED 13:53:17.133: debug : virUnrefConnect:210 : unref connection 0x93275d0 1 13:53:17.134: debug : virReleaseConnect:171 : release connection 0x93275d0 error: failed to connect to the hypervisor But when connecting to localhost, it works just fine: # LIBVIRT_DEBUG=6 virsh -d 5 -c xen:/// list command: "list " 13:53:23.558: debug : virInitialize:290 : register drivers 13:53:23.558: debug : virRegisterDriver:667 : registering Test as driver 0 13:53:23.558: debug : virRegisterNetworkDriver:567 : registering Test as network driver 0 13:53:23.558: debug : virRegisterStorageDriver:598 : registering Test as storage driver 0 13:53:23.558: debug : virRegisterDeviceMonitor:629 : registering Test as device driver 0 13:53:23.559: debug : xenHypervisorInit:1922 : Using new hypervisor call: 30001 13:53:23.559: debug : xenHypervisorInit:1991 : Using hypervisor call v2, sys ver3 dom ver5 13:53:23.559: debug : virRegisterDriver:667 : registering Xen as driver 1 13:53:23.559: debug : virRegisterDriver:667 : registering OPENVZ as driver 2 13:53:23.559: debug : vboxRegister:68 : VBoxCGlueInit failed: 0.0.0, errorval=-1 13:53:23.559: debug : virRegisterDriver:667 : registering VBOX as driver 3 13:53:23.559: debug : virRegisterDriver:667 : registering remote as driver 4 13:53:23.559: debug : virRegisterNetworkDriver:567 : registering remote as network driver 1 13:53:23.560: debug : virRegisterStorageDriver:598 : registering remote as storage driver 1 13:53:23.560: debug : virRegisterDeviceMonitor:629 : registering remote as device driver 1 13:53:23.560: debug : virConnectOpenAuth:1100 : name=xen:///, auth=0x67d0080, flags=0 13:53:23.560: debug : do_open:920 : name "xen:///" to URI components: scheme xen opaque (null) authority (null) server (null) user (null) port 0 path / 13:53:23.560: debug : do_open:930 : trying driver 0 (Test) ... 13:53:23.560: debug : do_open:936 : driver 0 Test returned DECLINED 13:53:23.560: debug : do_open:930 : trying driver 1 (Xen) ... 13:53:23.560: debug : xenUnifiedOpen:295 : Trying hypervisor sub-driver 13:53:23.560: debug : xenUnifiedOpen:298 : Activated hypervisor sub-driver 13:53:23.560: debug : xenUnifiedOpen:306 : Trying XenD sub-driver 13:53:23.563: debug : xenUnifiedOpen:309 : Activated XenD sub-driver 13:53:23.563: debug : xenUnifiedOpen:315 : Trying XM sub-driver 13:53:23.563: debug : xenUnifiedOpen:318 : Activated XM sub-driver 13:53:23.563: debug : xenUnifiedOpen:322 : Trying XS sub-driver 13:53:23.564: debug : xenStoreOpen:346 : Failed to add event handle, disabling events 13:53:23.564: debug : xenUnifiedOpen:325 : Activated XS sub-driver 13:53:23.570: debug : xenUnifiedOpen:361 : Trying Xen inotify sub-driver 13:53:23.571: debug : xenInotifyOpen:439 : Adding a watch on /etc/xen 13:53:23.571: debug : xenInotifyOpen:451 : Building initial config cache 13:53:23.571: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/centos 13:53:23.571: debug : xenXMConfigCacheAddFile:436 : Failed to read /etc/xen/centos 13:53:23.571: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/test3 13:53:23.571: debug : xenXMConfigCacheAddFile:465 : Added config test3 /etc/xen/test3 13:53:23.571: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/hany 13:53:23.571: debug : xenXMConfigCacheAddFile:465 : Added config hany /etc/xen/hany 13:53:23.572: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/hany2 13:53:23.572: debug : xenXMConfigCacheAddFile:465 : Added config hany4 /etc/xen/hany2 13:53:23.572: debug : xenInotifyOpen:458 : Registering with event loop 13:53:23.572: debug : xenInotifyOpen:462 : Failed to add inotify handle, disabling events 13:53:23.572: debug : virConnectRef:1165 : conn=0x996b5c8 refs=3 13:53:23.572: debug : xenUnifiedOpen:364 : Activated Xen inotify sub-driver 13:53:23.572: debug : do_open:936 : driver 1 Xen returned SUCCESS 13:53:23.572: debug : do_open:956 : network driver 0 Test returned DECLINED 13:53:23.572: debug : doRemoteOpen:511 : proceeding with name = xen:/// 13:53:23.572: debug : call:6543 : Doing call 66 (nil) 13:53:23.573: debug : call:6613 : We have the buck 66 0xb736c008 0xb736c008 13:53:23.573: debug : processCallRecvLen:6201 : Got length, now need 36 total (32 more) 13:53:23.573: debug : processCalls:6469 : Giving up the buck 66 0xb736c008 (nil) 13:53:23.573: debug : call:6644 : All done with our call 66 (nil) 0xb736c008 13:53:23.573: debug : call:6543 : Doing call 1 (nil) 13:53:23.573: debug : call:6613 : We have the buck 1 0x997f3b0 0x997f3b0 13:53:23.584: debug : processCallRecvLen:6201 : Got length, now need 28 total (24 more) 13:53:23.585: debug : processCalls:6469 : Giving up the buck 1 0x997f3b0 (nil) 13:53:23.585: debug : call:6644 : All done with our call 1 (nil) 0x997f3b0 13:53:23.585: debug : doRemoteOpen:822 : Adding Handler for remote events 13:53:23.585: debug : doRemoteOpen:829 : virEventAddHandle failed: No addHandleImpl defined. continuing without events. 13:53:23.585: debug : do_open:956 : network driver 1 remote returned SUCCESS 13:53:23.585: debug : do_open:978 : storage driver 0 Test returned DECLINED 13:53:23.585: debug : do_open:978 : storage driver 1 remote returned SUCCESS 13:53:23.585: debug : do_open:999 : node driver 0 Test returned DECLINED 13:53:23.585: debug : do_open:999 : node driver 1 remote returned SUCCESS 13:53:23.585: debug : virConnectNumOfDomains:1440 : conn=0x996b5c8 13:53:23.585: debug : virConnectListDomains:1401 : conn=0x996b5c8, ids=0x997a6d0, maxids=1 Id Name State ---------------------------------- 13:53:23.586: debug : virDomainLookupByID:1576 : conn=0x996b5c8, id=0 13:53:23.586: debug : virGetDomain:271 : New hash entry 0x997b300 13:53:23.586: debug : virDomainGetInfo:2569 : domain=0x997b300, info=0xbfc34c64 13:53:23.586: debug : virDomainGetName:2268 : domain=0x997b300 13:53:23.586: debug : virDomainGetID:2361 : domain=0x997b300 0 Domain-0 running 13:53:23.586: debug : virDomainFree:1806 : domain=0x997b300 13:53:23.586: debug : virUnrefDomain:345 : unref domain 0x997b300 Domain-0 1 13:53:23.586: debug : virReleaseDomain:303 : release domain 0x997b300 Domain-0 13:53:23.587: debug : virReleaseDomain:315 : unref connection 0x996b5c8 5 13:53:23.587: debug : virConnectClose:1118 : conn=0x996b5c8 13:53:23.587: debug : call:6543 : Doing call 2 (nil) 13:53:23.587: debug : call:6613 : We have the buck 2 0x997f3b0 0x997f3b0 13:53:23.588: debug : processCallRecvLen:6201 : Got length, now need 28 total (24 more) 13:53:23.588: debug : processCalls:6469 : Giving up the buck 2 0x997f3b0 (nil) 13:53:23.588: debug : call:6644 : All done with our call 2 (nil) 0x997f3b0 13:53:23.588: debug : virUnrefConnect:210 : unref connection 0x996b5c8 4 13:53:23.588: debug : virUnrefConnect:210 : unref connection 0x996b5c8 3 13:53:23.589: debug : virUnrefConnect:210 : unref connection 0x996b5c8 2 13:53:23.589: debug : virUnrefConnect:210 : unref connection 0x996b5c8 1 13:53:23.589: debug : virReleaseConnect:171 : release connection 0x996b5c8 Anybody else run into this? Any help would be greatly appreciated. Thanks, Hany
On Sun, Jun 07, 2009 at 02:02:20PM -0400, Hany Fahim wrote:
Hi, I'm currently running a few servers with CentOS 5.3 (i386) with the stock Xen version:
# rpm -q kernel-xen xen libvirt kernel-xen-2.6.18-92.1.22.el5 kernel-xen-2.6.18-128.1.6.el5 xen-3.0.3-80.el5_3.2 libvirt-0.6.3-1
libvirt was compiled from the src RPM from libvirt.org to run on i386 architecture without issue. I'm having a problem when trying to connect to remote Xen installations using versions of libvirt starting from 0.5.0 and above. When issuing virsh commands to query the local instance, it works without issue, but any remote sources gives 'error: failed to connect to the hypervisor'. Here's what LIBVIRT_DEBUG reveals:
# LIBVIRT_DEBUG=6 virsh -d 5 -c xen://node3/ list command: "list " 13:53:17.133: debug : do_open:930 : trying driver 0 (Test) ... 13:53:17.133: debug : do_open:936 : driver 0 Test returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 1 (Xen) ... 13:53:17.133: debug : do_open:936 : driver 1 Xen returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 2 (OPENVZ) ... 13:53:17.133: debug : do_open:936 : driver 2 OPENVZ returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 3 (VBOX) ... 13:53:17.133: debug : do_open:936 : driver 3 VBOX returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 4 (remote) ... 13:53:17.133: debug : do_open:936 : driver 4 remote returned DECLINED 13:53:17.133: debug : virUnrefConnect:210 : unref connection 0x93275d0 1 13:53:17.134: debug : virReleaseConnect:171 : release connection 0x93275d0 error: failed to connect to the hypervisor
It looks very much like the remote driver is failing to connect to the remote libvirtd daemon. Please verify that libvirtd on the remote host was started with the --listen flag, and that you have configured x509 certificates and have not disabled the 'listen_tls' config param in /etc/libvirt/libvirtd.conf Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Hey Daniel, Thanks for the reply. The strange thing is, libvirt isn't even attempting to establish a connection with the remote server. I've performed tcpdumps to verify this; no traffic is exchanged between the two hosts when executing the virsh command. If I switch back to a version of libvirt below 0.5.0 such as 0.4.6, it works like a charm. Is there anything else I could try? Hany On Mon, Jun 8, 2009 at 7:04 AM, Daniel P. Berrange <berrange@redhat.com>wrote:
Hi, I'm currently running a few servers with CentOS 5.3 (i386) with the stock Xen version:
# rpm -q kernel-xen xen libvirt kernel-xen-2.6.18-92.1.22.el5 kernel-xen-2.6.18-128.1.6.el5 xen-3.0.3-80.el5_3.2 libvirt-0.6.3-1
libvirt was compiled from the src RPM from libvirt.org to run on i386 architecture without issue. I'm having a problem when trying to connect to remote Xen installations using versions of libvirt starting from 0.5.0 and above. When issuing virsh commands to query the local instance, it works without issue, but any remote sources gives 'error: failed to connect to
On Sun, Jun 07, 2009 at 02:02:20PM -0400, Hany Fahim wrote: the
hypervisor'. Here's what LIBVIRT_DEBUG reveals:
# LIBVIRT_DEBUG=6 virsh -d 5 -c xen://node3/ list command: "list " 13:53:17.133: debug : do_open:930 : trying driver 0 (Test) ... 13:53:17.133: debug : do_open:936 : driver 0 Test returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 1 (Xen) ... 13:53:17.133: debug : do_open:936 : driver 1 Xen returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 2 (OPENVZ) ... 13:53:17.133: debug : do_open:936 : driver 2 OPENVZ returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 3 (VBOX) ... 13:53:17.133: debug : do_open:936 : driver 3 VBOX returned DECLINED 13:53:17.133: debug : do_open:930 : trying driver 4 (remote) ... 13:53:17.133: debug : do_open:936 : driver 4 remote returned DECLINED 13:53:17.133: debug : virUnrefConnect:210 : unref connection 0x93275d0 1 13:53:17.134: debug : virReleaseConnect:171 : release connection 0x93275d0 error: failed to connect to the hypervisor
It looks very much like the remote driver is failing to connect to the remote libvirtd daemon. Please verify that libvirtd on the remote host was started with the --listen flag, and that you have configured x509 certificates and have not disabled the 'listen_tls' config param in /etc/libvirt/libvirtd.conf
Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/:| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org:| |: http://autobuild.org -o- http://search.cpan.org/~danberr/:| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Mon, Jun 08, 2009 at 12:20:12PM -0400, Hany Fahim wrote:
Hey Daniel, Thanks for the reply. The strange thing is, libvirt isn't even attempting to establish a connection with the remote server. I've performed tcpdumps to verify this; no traffic is exchanged between the two hosts when executing the virsh command. If I switch back to a version of libvirt below 0.5.0 such as 0.4.6, it works like a charm.
Have you configured the neccessary x509/TLS certificates on the client side ? Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Hey Daniel, Sorry, I should have mentioned that. Yes, I did setup the x509/TLS certificates based on the instructions provided by the libvirt documentation. The setup with the certificates work flawlessly with 0.4.6. Here is a successful run of the virsh command using libvirt 0.4.6 with the certificates: # LIBVIRT_DEBUG=6 virsh -d 5 -c xen://node3/ list command: "list " DEBUG: libvirt.c: virInitialize (register drivers) DEBUG: xen_internal.c: xenHypervisorInit (Using new hypervisor call: 30003 ) DEBUG: xen_internal.c: xenHypervisorInit (Using hypervisor call v2, sys ver6 dom ver5 ) DEBUG: libvirt.c: virConnectOpenAuth (name=xen://node3/, auth=0x675b9c, flags=0) DEBUG: libvirt.c: do_open (name "xen://node3/" to URI components: scheme xen opaque (null) authority (null) server node3 user (null) port 0 path / ) DEBUG: libvirt.c: do_open (trying driver 0 (Test) ...) DEBUG: libvirt.c: do_open (driver 0 Test returned DECLINED) DEBUG: libvirt.c: do_open (trying driver 1 (QEMU) ...) DEBUG: libvirt.c: do_open (driver 1 QEMU returned DECLINED) DEBUG: libvirt.c: do_open (trying driver 2 (Xen) ...) DEBUG: libvirt.c: do_open (driver 2 Xen returned DECLINED) DEBUG: libvirt.c: do_open (trying driver 3 (remote) ...) DEBUG: remote_internal.c: doRemoteOpen (proceeding with name = xen:///) DEBUG: remote_internal.c: initialise_gnutls (loading CA file /etc/pki/CA/cacert.pem) DEBUG: remote_internal.c: initialise_gnutls (loading client cert and key from files /etc/pki/libvirt/clientcert.pem and /etc/pki/libvirt/private/clientkey.pem) DEBUG: libvirt.c: do_open (driver 3 remote returned SUCCESS) DEBUG: libvirt.c: do_open (network driver 0 Test returned DECLINED) DEBUG: libvirt.c: do_open (network driver 1 QEMU returned DECLINED) DEBUG: libvirt.c: do_open (network driver 2 remote returned SUCCESS) DEBUG: libvirt.c: do_open (storage driver 0 Test returned DECLINED) DEBUG: libvirt.c: do_open (storage driver 1 storage returned DECLINED) DEBUG: libvirt.c: do_open (storage driver 2 remote returned SUCCESS) DEBUG: libvirt.c: virConnectNumOfDomains (conn=0x8e681f0) DEBUG: libvirt.c: virConnectListDomains (conn=0x8e681f0, ids=0x8e76f58, maxids=1) Id Name State ---------------------------------- DEBUG: libvirt.c: virDomainLookupByID (conn=0x8e681f0, id=0) DEBUG: hash.c: __virGetDomain (New hash entry 0x8e8e330) DEBUG: libvirt.c: virDomainGetInfo (domain=0x8e8e330, info=0xbfce7cc4) DEBUG: libvirt.c: virDomainGetName (domain=0x8e8e330) DEBUG: libvirt.c: virDomainGetID (domain=0x8e8e330) 0 Domain-0 running DEBUG: libvirt.c: virDomainFree (domain=0x8e8e330) DEBUG: hash.c: virUnrefDomain (unref domain 0x8e8e330 Domain-0 1) DEBUG: hash.c: virReleaseDomain (release domain 0x8e8e330 Domain-0) DEBUG: hash.c: virReleaseDomain (unref connection 0x8e681f0 xen://node3/ 2) DEBUG: libvirt.c: virConnectClose (conn=0x8e681f0) DEBUG: hash.c: virUnrefConnect (unref connection 0x8e681f0 xen://node3/ 1) DEBUG: hash.c: virReleaseConnect (release connection 0x8e681f0 xen://node3/) Hany On Mon, Jun 8, 2009 at 12:34 PM, Daniel P. Berrange <berrange@redhat.com>wrote:
On Mon, Jun 08, 2009 at 12:20:12PM -0400, Hany Fahim wrote:
Hey Daniel, Thanks for the reply. The strange thing is, libvirt isn't even attempting to establish a connection with the remote server. I've performed tcpdumps to verify this; no traffic is exchanged between the two hosts when executing the virsh command. If I switch back to a version of libvirt below 0.5.0 such as 0.4.6, it works like a charm.
Have you configured the neccessary x509/TLS certificates on the client side ?
Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/:| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org:| |: http://autobuild.org -o- http://search.cpan.org/~danberr/:| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Mon, Jun 08, 2009 at 12:47:30PM -0400, Hany Fahim wrote:
Hey Daniel, Sorry, I should have mentioned that. Yes, I did setup the x509/TLS certificates based on the instructions provided by the libvirt documentation. The setup with the certificates work flawlessly with 0.4.6. Here is a successful run of the virsh command using libvirt 0.4.6 with the certificates:
# LIBVIRT_DEBUG=6 virsh -d 5 -c xen://node3/ list
I think i have a spotted a possible bug here. Try using an explicit transport scheme in the URI , eg xen+tls://node3/ Looks like we may have broken the implicit TLS support Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Yep, that did it. It works when using xen+tls as the transport. Thanks for your help Daniel. Hany On Mon, Jun 8, 2009 at 12:53 PM, Daniel P. Berrange <berrange@redhat.com>wrote:
Hey Daniel, Sorry, I should have mentioned that. Yes, I did setup the x509/TLS certificates based on the instructions provided by the libvirt documentation. The setup with the certificates work flawlessly with 0.4.6. Here is a successful run of the virsh command using libvirt 0.4.6 with
On Mon, Jun 08, 2009 at 12:47:30PM -0400, Hany Fahim wrote: the
certificates:
# LIBVIRT_DEBUG=6 virsh -d 5 -c xen://node3/ list
I think i have a spotted a possible bug here. Try using an explicit transport scheme in the URI , eg xen+tls://node3/ Looks like we may have broken the implicit TLS support
Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/:| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org:| |: http://autobuild.org -o- http://search.cpan.org/~danberr/:| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
participants (2)
-
Daniel P. Berrange -
Hany Fahim