Hi,<div><br></div><div>I&#39;m currently running a few servers with CentOS 5.3 (i386) with the stock Xen version:</div><div><br></div><div><div># rpm -q kernel-xen xen libvirt</div><div>kernel-xen-2.6.18-92.1.22.el5</div><div>
kernel-xen-2.6.18-128.1.6.el5</div><div>xen-3.0.3-80.el5_3.2</div><div>libvirt-0.6.3-1</div><div><br></div><div>libvirt was compiled from the src RPM from <a href="http://libvirt.org">libvirt.org</a> to run on i386 architecture without issue. I&#39;m having a problem when trying to connect to remote Xen installations using versions of libvirt starting from 0.5.0 and above. When issuing virsh commands to query the local instance, it works without issue, but any remote sources gives &#39;error: failed to connect to the hypervisor&#39;. Here&#39;s what LIBVIRT_DEBUG reveals:</div>
<div><br></div><div><div># LIBVIRT_DEBUG=6 virsh -d 5 -c xen://node3/ list</div><div>command: &quot;list &quot;</div><div>13:53:17.132: debug : virInitialize:290 : register drivers</div><div>13:53:17.132: debug : virRegisterDriver:667 : registering Test as driver 0</div>
<div>13:53:17.132: debug : virRegisterNetworkDriver:567 : registering Test as network driver 0</div><div>13:53:17.132: debug : virRegisterStorageDriver:598 : registering Test as storage driver 0</div><div>13:53:17.132: debug : virRegisterDeviceMonitor:629 : registering Test as device driver 0</div>
<div>13:53:17.132: debug : xenHypervisorInit:1922 : Using new hypervisor call: 30001</div><div><br></div><div>13:53:17.132: debug : xenHypervisorInit:1991 : Using hypervisor call v2, sys ver3 dom ver5</div><div><br></div>
<div>13:53:17.132: debug : virRegisterDriver:667 : registering Xen as driver 1</div><div>13:53:17.132: debug : virRegisterDriver:667 : registering OPENVZ as driver 2</div><div>13:53:17.133: debug : vboxRegister:68 : VBoxCGlueInit failed: 0.0.0, errorval=-1</div>
<div>13:53:17.133: debug : virRegisterDriver:667 : registering VBOX as driver 3</div><div>13:53:17.133: debug : virRegisterDriver:667 : registering remote as driver 4</div><div>13:53:17.133: debug : virRegisterNetworkDriver:567 : registering remote as network driver 1</div>
<div>13:53:17.133: debug : virRegisterStorageDriver:598 : registering remote as storage driver 1</div><div>13:53:17.133: debug : virRegisterDeviceMonitor:629 : registering remote as device driver 1</div><div>13:53:17.133: debug : virConnectOpenAuth:1100 : name=xen://node3/, auth=0x67d0080, flags=0</div>
<div>13:53:17.133: debug : do_open:920 : name &quot;xen://node3/&quot; to URI components:</div><div>  scheme xen</div><div>  opaque (null)</div><div>  authority (null)</div><div>  server node3</div><div>  user (null)</div>
<div>  port 0</div><div>  path /</div><div><br></div><div>13:53:17.133: debug : do_open:930 : trying driver 0 (Test) ...</div><div>13:53:17.133: debug : do_open:936 : driver 0 Test returned DECLINED</div><div>13:53:17.133: debug : do_open:930 : trying driver 1 (Xen) ...</div>
<div>13:53:17.133: debug : do_open:936 : driver 1 Xen returned DECLINED</div><div>13:53:17.133: debug : do_open:930 : trying driver 2 (OPENVZ) ...</div><div>13:53:17.133: debug : do_open:936 : driver 2 OPENVZ returned DECLINED</div>
<div>13:53:17.133: debug : do_open:930 : trying driver 3 (VBOX) ...</div><div>13:53:17.133: debug : do_open:936 : driver 3 VBOX returned DECLINED</div><div>13:53:17.133: debug : do_open:930 : trying driver 4 (remote) ...</div>
<div>13:53:17.133: debug : do_open:936 : driver 4 remote returned DECLINED</div><div>13:53:17.133: debug : virUnrefConnect:210 : unref connection 0x93275d0 1</div><div>13:53:17.134: debug : virReleaseConnect:171 : release connection 0x93275d0</div>
<div>error: failed to connect to the hypervisor</div><div><br></div><div>But when connecting to localhost, it works just fine:</div><div><br></div><div><div># LIBVIRT_DEBUG=6 virsh -d 5 -c xen:/// list</div><div>command: &quot;list &quot;</div>
<div>13:53:23.558: debug : virInitialize:290 : register drivers</div><div>13:53:23.558: debug : virRegisterDriver:667 : registering Test as driver 0</div><div>13:53:23.558: debug : virRegisterNetworkDriver:567 : registering Test as network driver 0</div>
<div>13:53:23.558: debug : virRegisterStorageDriver:598 : registering Test as storage driver 0</div><div>13:53:23.558: debug : virRegisterDeviceMonitor:629 : registering Test as device driver 0</div><div>13:53:23.559: debug : xenHypervisorInit:1922 : Using new hypervisor call: 30001</div>
<div><br></div><div>13:53:23.559: debug : xenHypervisorInit:1991 : Using hypervisor call v2, sys ver3 dom ver5</div><div><br></div><div>13:53:23.559: debug : virRegisterDriver:667 : registering Xen as driver 1</div><div>13:53:23.559: debug : virRegisterDriver:667 : registering OPENVZ as driver 2</div>
<div>13:53:23.559: debug : vboxRegister:68 : VBoxCGlueInit failed: 0.0.0, errorval=-1</div><div>13:53:23.559: debug : virRegisterDriver:667 : registering VBOX as driver 3</div><div>13:53:23.559: debug : virRegisterDriver:667 : registering remote as driver 4</div>
<div>13:53:23.559: debug : virRegisterNetworkDriver:567 : registering remote as network driver 1</div><div>13:53:23.560: debug : virRegisterStorageDriver:598 : registering remote as storage driver 1</div><div>13:53:23.560: debug : virRegisterDeviceMonitor:629 : registering remote as device driver 1</div>
<div>13:53:23.560: debug : virConnectOpenAuth:1100 : name=xen:///, auth=0x67d0080, flags=0</div><div>13:53:23.560: debug : do_open:920 : name &quot;xen:///&quot; to URI components:</div><div>  scheme xen</div><div>  opaque (null)</div>
<div>  authority (null)</div><div>  server (null)</div><div>  user (null)</div><div>  port 0</div><div>  path /</div><div><br></div><div>13:53:23.560: debug : do_open:930 : trying driver 0 (Test) ...</div><div>13:53:23.560: debug : do_open:936 : driver 0 Test returned DECLINED</div>
<div>13:53:23.560: debug : do_open:930 : trying driver 1 (Xen) ...</div><div>13:53:23.560: debug : xenUnifiedOpen:295 : Trying hypervisor sub-driver</div><div>13:53:23.560: debug : xenUnifiedOpen:298 : Activated hypervisor sub-driver</div>
<div>13:53:23.560: debug : xenUnifiedOpen:306 : Trying XenD sub-driver</div><div>13:53:23.563: debug : xenUnifiedOpen:309 : Activated XenD sub-driver</div><div>13:53:23.563: debug : xenUnifiedOpen:315 : Trying XM sub-driver</div>
<div>13:53:23.563: debug : xenUnifiedOpen:318 : Activated XM sub-driver</div><div>13:53:23.563: debug : xenUnifiedOpen:322 : Trying XS sub-driver</div><div>13:53:23.564: debug : xenStoreOpen:346 : Failed to add event handle, disabling events</div>
<div><br></div><div>13:53:23.564: debug : xenUnifiedOpen:325 : Activated XS sub-driver</div><div>13:53:23.570: debug : xenUnifiedOpen:361 : Trying Xen inotify sub-driver</div><div>13:53:23.571: debug : xenInotifyOpen:439 : Adding a watch on /etc/xen</div>
<div>13:53:23.571: debug : xenInotifyOpen:451 : Building initial config cache</div><div>13:53:23.571: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/centos</div><div>13:53:23.571: debug : xenXMConfigCacheAddFile:436 : Failed to read /etc/xen/centos</div>
<div>13:53:23.571: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/test3</div><div>13:53:23.571: debug : xenXMConfigCacheAddFile:465 : Added config test3 /etc/xen/test3</div><div>13:53:23.571: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/hany</div>
<div>13:53:23.571: debug : xenXMConfigCacheAddFile:465 : Added config hany /etc/xen/hany</div><div>13:53:23.572: debug : xenXMConfigCacheAddFile:387 : Adding file /etc/xen/hany2</div><div>13:53:23.572: debug : xenXMConfigCacheAddFile:465 : Added config hany4 /etc/xen/hany2</div>
<div>13:53:23.572: debug : xenInotifyOpen:458 : Registering with event loop</div><div>13:53:23.572: debug : xenInotifyOpen:462 : Failed to add inotify handle, disabling events</div><div>13:53:23.572: debug : virConnectRef:1165 : conn=0x996b5c8 refs=3</div>
<div>13:53:23.572: debug : xenUnifiedOpen:364 : Activated Xen inotify sub-driver</div><div>13:53:23.572: debug : do_open:936 : driver 1 Xen returned SUCCESS</div><div>13:53:23.572: debug : do_open:956 : network driver 0 Test returned DECLINED</div>
<div>13:53:23.572: debug : doRemoteOpen:511 : proceeding with name = xen:///</div><div>13:53:23.572: debug : call:6543 : Doing call 66 (nil)</div><div>13:53:23.573: debug : call:6613 : We have the buck 66 0xb736c008 0xb736c008</div>
<div>13:53:23.573: debug : processCallRecvLen:6201 : Got length, now need 36 total (32 more)</div><div>13:53:23.573: debug : processCalls:6469 : Giving up the buck 66 0xb736c008 (nil)</div><div>13:53:23.573: debug : call:6644 : All done with our call 66 (nil) 0xb736c008</div>
<div>13:53:23.573: debug : call:6543 : Doing call 1 (nil)</div><div>13:53:23.573: debug : call:6613 : We have the buck 1 0x997f3b0 0x997f3b0</div><div>13:53:23.584: debug : processCallRecvLen:6201 : Got length, now need 28 total (24 more)</div>
<div>13:53:23.585: debug : processCalls:6469 : Giving up the buck 1 0x997f3b0 (nil)</div><div>13:53:23.585: debug : call:6644 : All done with our call 1 (nil) 0x997f3b0</div><div>13:53:23.585: debug : doRemoteOpen:822 : Adding Handler for remote events</div>
<div>13:53:23.585: debug : doRemoteOpen:829 : virEventAddHandle failed: No addHandleImpl defined. continuing without events.</div><div>13:53:23.585: debug : do_open:956 : network driver 1 remote returned SUCCESS</div><div>
13:53:23.585: debug : do_open:978 : storage driver 0 Test returned DECLINED</div><div>13:53:23.585: debug : do_open:978 : storage driver 1 remote returned SUCCESS</div><div>13:53:23.585: debug : do_open:999 : node driver 0 Test returned DECLINED</div>
<div>13:53:23.585: debug : do_open:999 : node driver 1 remote returned SUCCESS</div><div>13:53:23.585: debug : virConnectNumOfDomains:1440 : conn=0x996b5c8</div><div>13:53:23.585: debug : virConnectListDomains:1401 : conn=0x996b5c8, ids=0x997a6d0, maxids=1</div>
<div> Id Name                 State</div><div>----------------------------------</div><div>13:53:23.586: debug : virDomainLookupByID:1576 : conn=0x996b5c8, id=0</div><div>13:53:23.586: debug : virGetDomain:271 : New hash entry 0x997b300</div>
<div>13:53:23.586: debug : virDomainGetInfo:2569 : domain=0x997b300, info=0xbfc34c64</div><div>13:53:23.586: debug : virDomainGetName:2268 : domain=0x997b300</div><div>13:53:23.586: debug : virDomainGetID:2361 : domain=0x997b300</div>
<div>  0 Domain-0             running</div><div>13:53:23.586: debug : virDomainFree:1806 : domain=0x997b300</div><div>13:53:23.586: debug : virUnrefDomain:345 : unref domain 0x997b300 Domain-0 1</div><div>13:53:23.586: debug : virReleaseDomain:303 : release domain 0x997b300 Domain-0</div>
<div>13:53:23.587: debug : virReleaseDomain:315 : unref connection 0x996b5c8 5</div><div><br></div><div>13:53:23.587: debug : virConnectClose:1118 : conn=0x996b5c8</div><div>13:53:23.587: debug : call:6543 : Doing call 2 (nil)</div>
<div>13:53:23.587: debug : call:6613 : We have the buck 2 0x997f3b0 0x997f3b0</div><div>13:53:23.588: debug : processCallRecvLen:6201 : Got length, now need 28 total (24 more)</div><div>13:53:23.588: debug : processCalls:6469 : Giving up the buck 2 0x997f3b0 (nil)</div>
<div>13:53:23.588: debug : call:6644 : All done with our call 2 (nil) 0x997f3b0</div><div>13:53:23.588: debug : virUnrefConnect:210 : unref connection 0x996b5c8 4</div><div>13:53:23.588: debug : virUnrefConnect:210 : unref connection 0x996b5c8 3</div>
<div>13:53:23.589: debug : virUnrefConnect:210 : unref connection 0x996b5c8 2</div><div>13:53:23.589: debug : virUnrefConnect:210 : unref connection 0x996b5c8 1</div><div>13:53:23.589: debug : virReleaseConnect:171 : release connection 0x996b5c8</div>
<div><br></div><div>Anybody else run into this? Any help would be greatly appreciated.</div><div><br></div><div>Thanks,</div><div><br></div><div>Hany</div></div></div></div>