Hi,

I am installing OpenStack on Red Hat 7.5. For Nova compute component, I choose libvirt and Xen.

However, I am running into some problems. It seems libvirt is unable to communicate with Xen.

Xen installation:

I installed Xen410 using "Xen make easy" https://xen.crc.id.au/support/guides/install/. Here is a list of packages installed and verification that Xen is running.

# lsb_release -a

LSB Version: :core-4.1-amd64:core-4.1-noarch

Distributor ID: RedHatEnterpriseServer

Description: Red Hat Enterprise Linux Server release 7.5 (Maipo)

Release: 7.5

Codename: Maipo

# yum list | grep xen

* kernel-xen: us3.mirror.crc.id.au

* kernel-xen-extra: us3.mirror.crc.id.au

kernel-xen.x86_64 4.14.94-6.el7xen @kernel-xen

kernel-xen.x86_64 4.14.97-6.el7xen @kernel-xen

kernel-xen-devel.x86_64 4.14.97-6.el7xen @kernel-xen

kernel-xen-firmware.x86_64 4.14.97-6.el7xen @kernel-xen

kernel-xen-headers.x86_64 4.14.97-6.el7xen @kernel-xen

kernel-xen-release.noarch 1:7-13 @kernel-xen

xen410.x86_64 4.10.2-7.el7 @kernel-xen

xen410-devel.x86_64 4.10.2-7.el7 @kernel-xen

xen410-hypervisor.x86_64 4.10.2-7.el7 @kernel-xen

xen410-libs.x86_64 4.10.2-7.el7 @kernel-xen

xen410-licenses.x86_64 4.10.2-7.el7 @kernel-xen

xen410-ocaml.x86_64 4.10.2-7.el7 @kernel-xen

xen410-runtime.x86_64 4.10.2-7.el7 @kernel-xen

xen46-licenses.x86_64 4.6.6-16.el7 @kernel-xen

# xl info

host : nfv-sec-cn01-pa.labs.hpecorp.net

release : 4.14.97-6.el7xen.x86_64

version : #2 SMP Fri Feb 1 00:33:52 AEDT 2019

machine : x86_64

nr_cpus : 32

max_cpu_id : 63

nr_nodes : 2

cores_per_socket : 8

threads_per_core : 2

cpu_mhz : 1995

hw_caps : bfebfbff:17bee3ff:2c100800:00000001:00000001:00000000:00000000:00000100

virt_caps : hvm hvm_directio

total_memory : 131034

free_memory : 96587

sharing_freed_memory : 0

sharing_used_memory : 0

outstanding_claims : 0

free_cpus : 0

xen_major : 4

xen_minor : 10

xen_extra : .3-pre

xen_version : 4.10.3-pre

xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64

xen_scheduler : credit

xen_pagesize : 4096

platform_params : virt_start=0xffff800000000000

xen_changeset : Tue Nov 20 15:45:04 2018 +0100 git:b6e203b-dirty

xen_commandline : placeholder dom0_mem=32768M cpufreq=xen dom0_max_vcpus=8 dom0_vcpus_pin console=tty0 console=com1 com1=115200,8n1

cc_compiler : gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)

cc_compile_by : mockbuild

cc_compile_domain :

cc_compile_date : Wed Nov 21 02:09:26 AEDT 2018

build_id : 2d53c8b15aaf737b0351e78ad863eb5a530e7df0

xend_config_format : 4

libvirt was installed as dependencies of openstack-nova-compute from yum.

==================================================================================================================================================================================================

Package Arch Version Repository Size

==================================================================================================================================================================================================

Installing:

openstack-nova-compute noarch 1:17.0.9-1.el7 openstack-queens 7.9 k

Installing for dependencies:

OpenIPMI-modalias x86_64 2.0.23-2.el7 RedHat-7Server-x86_64-errata 16 k

attr x86_64 2.4.46-13.el7 RedHat-7Server-x86_64-errata 66 k

dosfstools x86_64 3.0.20-9.el7 RedHat-7.1Server-x86_64 101 k

fuse x86_64 2.9.2-10.el7 RedHat-7Server-x86_64-errata 86 k

gdisk x86_64 0.8.6-5.el7 RedHat-7.1Server-x86_64 187 k

glusterfs-cli x86_64 3.8.4-53.el7 RedHat-7Server-x86_64-errata 176 k

hivex x86_64 1.3.10-6.9.el7 RedHat-7Server-x86_64-errata 101 k

ipmitool x86_64 1.8.18-7.el7 RedHat-7Server-x86_64-errata 441 k

libguestfs x86_64 1:1.36.10-6.el7_5.2 RedHat-7Server-x86_64-errata 1.9 M

libreport-filesystem x86_64 2.1.11-40.el7 RedHat-7Server-x86_64-errata 39 k

libvirt-daemon x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 852 k

libvirt-daemon-driver-interface x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 220 k

libvirt-daemon-driver-network x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 389 k

libvirt-daemon-driver-nodedev x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 221 k

libvirt-daemon-driver-nwfilter x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 243 k

libvirt-daemon-driver-qemu x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 714 k

libvirt-daemon-driver-secret x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 210 k

libvirt-daemon-driver-storage x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 173 k

libvirt-daemon-driver-storage-core x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 405 k

libvirt-daemon-driver-storage-disk x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 182 k

libvirt-daemon-driver-storage-gluster x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 183 k

libvirt-daemon-driver-storage-iscsi x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 179 k

libvirt-daemon-driver-storage-logical x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 183 k

libvirt-daemon-driver-storage-mpath x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 178 k

libvirt-daemon-driver-storage-rbd x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 185 k

libvirt-daemon-driver-storage-scsi x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 179 k

libvirt-daemon-kvm x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 173 k

libvirt-python x86_64 3.9.0-1.el7 RedHat-7Server-x86_64-errata 335 k

mdadm x86_64 4.0-13.el7 RedHat-7Server-x86_64-errata 431 k

mtools x86_64 4.0.18-5.el7 RedHat-7.1Server-x86_64 203 k

perl-hivex x86_64 1.3.10-6.9.el7 RedHat-7Server-x86_64-errata 41 k

python-libguestfs x86_64 1:1.36.10-6.el7_5.2 RedHat-7Server-x86_64-errata 203 k

scrub x86_64 2.5.2-7.el7 RedHat-7Server-x86_64-errata 41 k

squashfs-tools x86_64 4.3-0.21.gitaae0aff4.el7 RedHat-7.1Server-x86_64 101 k

supermin5 x86_64 5.1.19-1.el7 RedHat-7Server-x86_64-errata 631 k

syslinux x86_64 4.05-13.el7 RedHat-7Server-x86_64-errata 990 k

syslinux-extlinux x86_64 4.05-13.el7 RedHat-7Server-x86_64-errata 363 k

However, after installation, Nova complains "no connection driver available for xen:///".

Also, virsh cannot connect Xen.

/var/log/nova/nova-compute.log

2019-02-01 15:13:29.965 5091 INFO nova.service [-] Starting compute node (version 18.1.0-1.el7)

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host [-] Connection to libvirt failed: no connection driver available for xen:///: libvirtError: no connection driver available for xen:///

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host Traceback (most recent call last):

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 460, in get_connection

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host conn = self._get_connection()

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 443, in _get_connection

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host {'msg': ex})

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host self.force_reraise()

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host six.reraise(self.type_, self.value, self.tb)

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 432, in _get_connection

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host self._wrapped_conn = self._get_new_connection()

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 386, in _get_new_connection

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host wrapped_conn = self._connect(self._uri, self._read_only)

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 242, in _connect

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host libvirt.openAuth, uri, auth, flags)

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 148, in proxy_call

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host rv = execute(f, *args, **kwargs)

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 129, in execute

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host six.reraise(c, e, tb)

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host rv = meth(*args, **kwargs)

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib64/python2.7/site-packages/libvirt.py", line 105, in openAuth

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host if ret is None:raise libvirtError('virConnectOpenAuth() failed')

2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host libvirtError: no connection driver available for xen:///

# virsh -c xen:/// list

error: failed to connect to the hypervisor

error: no connection driver available for xen:///

Then I check the connection drivers of libvirt. It seems the daemon driver for xen is missing.

# ls -l /usr/lib64/libvirt/connection-driver/

total 2664

-rwxr-xr-x 1 root root 115976 Jun 5 2018 libvirt_driver_interface.so

-rwxr-xr-x 1 root root 292008 Jun 5 2018 libvirt_driver_lxc.so

-rwxr-xr-x 1 root root 169424 Jun 5 2018 libvirt_driver_network.so

-rwxr-xr-x 1 root root 119720 Jun 5 2018 libvirt_driver_nodedev.so

-rwxr-xr-x 1 root root 174216 Jun 5 2018 libvirt_driver_nwfilter.so

-rwxr-xr-x 1 root root 1545344 Jun 5 2018 libvirt_driver_qemu.so

-rwxr-xr-x 1 root root 86616 Jun 5 2018 libvirt_driver_secret.so

-rwxr-xr-x 1 root root 202568 Jun 5 2018 libvirt_driver_storage.so

But I couldn't find the xen driver (libvirt-daemon-driver-xen) from yum. I do find it driver on some websites (http://rpmfind.net/linux/rpm2html/search.php?query=libvirt-daemon-driver-xen), but I am not sure which is the right version to choose.

I also found this email https://www.spinics.net/lists/centos-virt/msg05163.html.

Then I decided to compile libvirt from source code. So I checked out the source code from github and tried to configure and compile it.

However, from the output of ./autogen.sh --system, I don't find driver for xen is enabled. Actually, xen is not listed as driver in the output.

configure:

configure: Configuration summary

configure: =====================

configure:

configure: Drivers

configure:

configure: QEMU: yes

configure: OpenVZ: yes

configure: VMware: yes

configure: VBox: yes

configure: XenAPI: no

configure: libxl: yes

configure: LXC: yes

configure: PHYP: no

configure: ESX: no

configure: Hyper-V: no

configure: vz: no

configure: Bhyve: no

configure: Test: yes

configure: Remote: yes

configure: Network: yes

configure: Libvirtd: yes

configure: Interface: yes

configure:

configure: Storage Drivers

configure:

configure: Dir: yes

configure: FS: yes

configure: NetFS: yes

configure: LVM: yes

configure: iSCSI: yes

configure: iscsi-direct: no

configure: SCSI: yes

configure: mpath: yes

configure: Disk: no

configure: RBD: no

configure: Sheepdog: no

configure: Gluster: no

configure: ZFS: no

configure: Virtuozzo storage: no

configure:

configure: Security Drivers

configure:

configure: SELinux: yes

configure: AppArmor: no

configure:

configure: Driver Loadable Modules

configure:

configure: driver_modules: yes (CFLAGS='' LIBS='-ldl')

configure:

configure: Libraries

configure:

configure: acl: no

configure: apparmor: no

configure: attr: no

configure: audit: no

configure: avahi: no

configure: bash_completion: no

configure: blkid: no

configure: capng: no

configure: curl: no

configure: dbus: no

configure: dlopen: yes (CFLAGS='' LIBS='-ldl')

configure: firewalld: no

configure: firewalld-zone: no

configure: fuse: no

configure: glusterfs: no

configure: gnutls: yes (CFLAGS='-I/usr/include/p11-kit-1 ' LIBS='-lgnutls ')

configure: hal: no

configure: libiscsi: no

configure: libnl: yes (CFLAGS=' ' LIBS='-lnl ')

configure: libpcap: no

configure: libssh: no

configure: libxl: yes (CFLAGS=' -DLIBXL_API_VERSION=0x040500' LIBS='-lxenlight -lxlutil -lxenstore -lxentoollog')

configure: libxml: yes (CFLAGS='-I/usr/include/libxml2 ' LIBS='-lxml2 ')

configure: macvtap: yes (CFLAGS='' LIBS='')

configure: netcf: no

configure: NLS: yes

configure: nss: yes

configure: numactl: no

configure: openwsman: no

configure: pciaccess: yes (CFLAGS=' ' LIBS='-lpciaccess ')

configure: pm_utils: yes (CFLAGS='' LIBS='')

configure: polkit: no

configure: rbd: no

configure: readline: no

configure: sanlock: no

configure: sasl: yes (CFLAGS='' LIBS='-lsasl2')

configure: selinux: yes (CFLAGS='' LIBS='-lselinux')

configure: ssh2: no

configure: udev: yes (CFLAGS=' ' LIBS='-ludev ')

configure: virtualport: yes (CFLAGS='' LIBS='')

configure: xdr: yes (CFLAGS='' LIBS='')

configure: xenapi: no

configure: yajl: yes (CFLAGS='' LIBS='-lyajl')

configure:

configure: Windows

configure:

configure: Cygwin: no

configure: MinGW: no

configure: MSVC: no

configure: windres: no

configure:

configure: Test suite

configure:

configure: Coverage: no

configure: Alloc OOM: no

configure:

configure: Miscellaneous

configure:

configure: Debug: yes

configure: Use -Werror: yes

configure: Warning Flags: -fno-common -W -Waddress -Waggressive-loop-optimizations -Wall -Wattributes -Wbad-function-cast -Wbuiltin-macro-redefined -Wcast-align -Wchar-subscripts -Wclobbered -Wcomment -Wcomments -Wcoverage-mismatch -Wcpp -Wdeprecated-declarations -Wdiv-by-zero -Wdouble-promotion -Wempty-body -Wendif-labels -Wextra -Wformat-contains-nul -Wformat-extra-args -Wformat-security -Wformat-y2k -Wformat-zero-length -Wfree-nonheap-object -Wignored-qualifiers -Wimplicit -Wimplicit-function-declaration -Wimplicit-int -Winit-self -Winline -Wint-to-pointer-cast -Winvalid-memory-model -Winvalid-pch -Wlogical-op -Wmain -Wmaybe-uninitialized -Wmissing-braces -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs -Wmissing-parameter-type -Wmissing-prototypes -Wmultichar -Wnarrowing -Wnested-externs -Wnonnull -Wold-style-declaration -Wold-style-definition -Woverflow -Woverride-init -Wpacked-bitfield-compat -Wparentheses -Wpointer-arith -Wpointer-sign -Wpointer-to-int-cast -Wpragmas -Wpsabi -Wreturn-local-addr -Wreturn-type -Wsequence-point -Wshadow -Wsizeof-pointer-memaccess -Wstrict-aliasing -Wstrict-prototypes -Wsuggest-attribute=const -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wswitch -Wsync-nand -Wtrampolines -Wtrigraphs -Wtype-limits -Wuninitialized -Wunknown-pragmas -Wunused -Wunused-but-set-parameter -Wunused-but-set-variable -Wunused-function -Wunused-label -Wunused-local-typedefs -Wunused-parameter -Wunused-result -Wunused-value -Wunused-variable -Wvarargs -Wvariadic-macros -Wvector-operation-performance -Wvolatile-register-var -Wwrite-strings -Wnormalized=nfc -Wno-sign-compare -Wjump-misses-init -Wswitch-enum -Wno-format-nonliteral -fstack-protector-strong -fexceptions -fasynchronous-unwind-tables -fipa-pure-const -Wno-suggest-attribute=pure -Wno-suggest-attribute=const -Werror

configure: DTrace: no

configure: numad: no

configure: Init script: systemd

configure: Char device locks: /var/lock

configure: Default Editor: vi

configure: Loader/NVRAM:

configure: virt-login-shell: yes

configure: virt-host-validate: yes

configure: TLS priority: NORMAL

configure:

configure: Developer Tools

configure:

configure: wireshark_dissector: no

configure:

configure: Privileges

configure:

configure: QEMU: root:root

configure:

My question is how should I install the libvirt-daemon-driver-xen driver and make Nova work with Xen?

Thanks a lot in advance!

Best regards,

Lianjie Cao