[libvirt-users] Installing libvirt and Xen on Red Hat 7.5

Hi, I am installing OpenStack on Red Hat 7.5. For Nova compute component, I choose libvirt and Xen. However, I am running into some problems. It seems libvirt is unable to communicate with Xen. Xen installation: I installed Xen410 using "Xen make easy" https://xen.crc.id.au/support/guides/install/. Here is a list of packages installed and verification that Xen is running. # lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 7.5 (Maipo) Release: 7.5 Codename: Maipo # yum list | grep xen * kernel-xen: us3.mirror.crc.id.au * kernel-xen-extra: us3.mirror.crc.id.au kernel-xen.x86_64 4.14.94-6.el7xen @kernel-xen kernel-xen.x86_64 4.14.97-6.el7xen @kernel-xen kernel-xen-devel.x86_64 4.14.97-6.el7xen @kernel-xen kernel-xen-firmware.x86_64 4.14.97-6.el7xen @kernel-xen kernel-xen-headers.x86_64 4.14.97-6.el7xen @kernel-xen kernel-xen-release.noarch 1:7-13 @kernel-xen xen410.x86_64 4.10.2-7.el7 @kernel-xen xen410-devel.x86_64 4.10.2-7.el7 @kernel-xen xen410-hypervisor.x86_64 4.10.2-7.el7 @kernel-xen xen410-libs.x86_64 4.10.2-7.el7 @kernel-xen xen410-licenses.x86_64 4.10.2-7.el7 @kernel-xen xen410-ocaml.x86_64 4.10.2-7.el7 @kernel-xen xen410-runtime.x86_64 4.10.2-7.el7 @kernel-xen xen46-licenses.x86_64 4.6.6-16.el7 @kernel-xen # xl info host : nfv-sec-cn01-pa.labs.hpecorp.net release : 4.14.97-6.el7xen.x86_64 version : #2 SMP Fri Feb 1 00:33:52 AEDT 2019 machine : x86_64 nr_cpus : 32 max_cpu_id : 63 nr_nodes : 2 cores_per_socket : 8 threads_per_core : 2 cpu_mhz : 1995 hw_caps : bfebfbff:17bee3ff:2c100800:00000001:00000001:00000000:00000000:00000100 virt_caps : hvm hvm_directio total_memory : 131034 free_memory : 96587 sharing_freed_memory : 0 sharing_used_memory : 0 outstanding_claims : 0 free_cpus : 0 xen_major : 4 xen_minor : 10 xen_extra : .3-pre xen_version : 4.10.3-pre xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : Tue Nov 20 15:45:04 2018 +0100 git:b6e203b-dirty xen_commandline : placeholder dom0_mem=32768M cpufreq=xen dom0_max_vcpus=8 dom0_vcpus_pin console=tty0 console=com1 com1=115200,8n1 cc_compiler : gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28) cc_compile_by : mockbuild cc_compile_domain : cc_compile_date : Wed Nov 21 02:09:26 AEDT 2018 build_id : 2d53c8b15aaf737b0351e78ad863eb5a530e7df0 xend_config_format : 4 libvirt was installed as dependencies of openstack-nova-compute from yum. ================================================================================================================================================================================================== Package Arch Version Repository Size ================================================================================================================================================================================================== Installing: openstack-nova-compute noarch 1:17.0.9-1.el7 openstack-queens 7.9 k Installing for dependencies: OpenIPMI-modalias x86_64 2.0.23-2.el7 RedHat-7Server-x86_64-errata 16 k attr x86_64 2.4.46-13.el7 RedHat-7Server-x86_64-errata 66 k dosfstools x86_64 3.0.20-9.el7 RedHat-7.1Server-x86_64 101 k fuse x86_64 2.9.2-10.el7 RedHat-7Server-x86_64-errata 86 k gdisk x86_64 0.8.6-5.el7 RedHat-7.1Server-x86_64 187 k glusterfs-cli x86_64 3.8.4-53.el7 RedHat-7Server-x86_64-errata 176 k hivex x86_64 1.3.10-6.9.el7 RedHat-7Server-x86_64-errata 101 k ipmitool x86_64 1.8.18-7.el7 RedHat-7Server-x86_64-errata 441 k libguestfs x86_64 1:1.36.10-6.el7_5.2 RedHat-7Server-x86_64-errata 1.9 M libreport-filesystem x86_64 2.1.11-40.el7 RedHat-7Server-x86_64-errata 39 k libvirt-daemon x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 852 k libvirt-daemon-driver-interface x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 220 k libvirt-daemon-driver-network x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 389 k libvirt-daemon-driver-nodedev x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 221 k libvirt-daemon-driver-nwfilter x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 243 k libvirt-daemon-driver-qemu x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 714 k libvirt-daemon-driver-secret x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 210 k libvirt-daemon-driver-storage x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 173 k libvirt-daemon-driver-storage-core x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 405 k libvirt-daemon-driver-storage-disk x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 182 k libvirt-daemon-driver-storage-gluster x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 183 k libvirt-daemon-driver-storage-iscsi x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 179 k libvirt-daemon-driver-storage-logical x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 183 k libvirt-daemon-driver-storage-mpath x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 178 k libvirt-daemon-driver-storage-rbd x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 185 k libvirt-daemon-driver-storage-scsi x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 179 k libvirt-daemon-kvm x86_64 3.9.0-14.el7_5.6 RedHat-7Server-x86_64-errata 173 k libvirt-python x86_64 3.9.0-1.el7 RedHat-7Server-x86_64-errata 335 k mdadm x86_64 4.0-13.el7 RedHat-7Server-x86_64-errata 431 k mtools x86_64 4.0.18-5.el7 RedHat-7.1Server-x86_64 203 k perl-hivex x86_64 1.3.10-6.9.el7 RedHat-7Server-x86_64-errata 41 k python-libguestfs x86_64 1:1.36.10-6.el7_5.2 RedHat-7Server-x86_64-errata 203 k scrub x86_64 2.5.2-7.el7 RedHat-7Server-x86_64-errata 41 k squashfs-tools x86_64 4.3-0.21.gitaae0aff4.el7 RedHat-7.1Server-x86_64 101 k supermin5 x86_64 5.1.19-1.el7 RedHat-7Server-x86_64-errata 631 k syslinux x86_64 4.05-13.el7 RedHat-7Server-x86_64-errata 990 k syslinux-extlinux x86_64 4.05-13.el7 RedHat-7Server-x86_64-errata 363 k However, after installation, Nova complains "no connection driver available for xen:///". Also, virsh cannot connect Xen. /var/log/nova/nova-compute.log 2019-02-01 15:13:29.965 5091 INFO nova.service [-] Starting compute node (version 18.1.0-1.el7) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host [-] Connection to libvirt failed: no connection driver available for xen:///: libvirtError: no connection driver available for xen:/// 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host Traceback (most recent call last): 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 460, in get_connection 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host conn = self._get_connection() 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 443, in _get_connection 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host {'msg': ex}) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host self.force_reraise() 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host six.reraise(self.type_, self.value, self.tb) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 432, in _get_connection 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host self._wrapped_conn = self._get_new_connection() 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 386, in _get_new_connection 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host wrapped_conn = self._connect(self._uri, self._read_only) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 242, in _connect 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host libvirt.openAuth, uri, auth, flags) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 148, in proxy_call 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host rv = execute(f, *args, **kwargs) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 129, in execute 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host six.reraise(c, e, tb) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host rv = meth(*args, **kwargs) 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host File "/usr/lib64/python2.7/site-packages/libvirt.py", line 105, in openAuth 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host if ret is None:raise libvirtError('virConnectOpenAuth() failed') 2019-02-01 15:13:30.023 5091 ERROR nova.virt.libvirt.host libvirtError: no connection driver available for xen:/// # virsh -c xen:/// list error: failed to connect to the hypervisor error: no connection driver available for xen:/// Then I check the connection drivers of libvirt. It seems the daemon driver for xen is missing. # ls -l /usr/lib64/libvirt/connection-driver/ total 2664 -rwxr-xr-x 1 root root 115976 Jun 5 2018 libvirt_driver_interface.so -rwxr-xr-x 1 root root 292008 Jun 5 2018 libvirt_driver_lxc.so -rwxr-xr-x 1 root root 169424 Jun 5 2018 libvirt_driver_network.so -rwxr-xr-x 1 root root 119720 Jun 5 2018 libvirt_driver_nodedev.so -rwxr-xr-x 1 root root 174216 Jun 5 2018 libvirt_driver_nwfilter.so -rwxr-xr-x 1 root root 1545344 Jun 5 2018 libvirt_driver_qemu.so -rwxr-xr-x 1 root root 86616 Jun 5 2018 libvirt_driver_secret.so -rwxr-xr-x 1 root root 202568 Jun 5 2018 libvirt_driver_storage.so But I couldn't find the xen driver (libvirt-daemon-driver-xen) from yum. I do find it driver on some websites ( http://rpmfind.net/linux/rpm2html/search.php?query=libvirt-daemon-driver-xen), but I am not sure which is the right version to choose. I also found this email https://www.spinics.net/lists/centos-virt/msg05163.html. Then I decided to compile libvirt from source code. So I checked out the source code from github and tried to configure and compile it. However, from the output of ./autogen.sh --system, I don't find driver for xen is enabled. Actually, xen is not listed as driver in the output. configure: configure: Configuration summary configure: ===================== configure: configure: Drivers configure: configure: QEMU: yes configure: OpenVZ: yes configure: VMware: yes configure: VBox: yes configure: XenAPI: no configure: libxl: yes configure: LXC: yes configure: PHYP: no configure: ESX: no configure: Hyper-V: no configure: vz: no configure: Bhyve: no configure: Test: yes configure: Remote: yes configure: Network: yes configure: Libvirtd: yes configure: Interface: yes configure: configure: Storage Drivers configure: configure: Dir: yes configure: FS: yes configure: NetFS: yes configure: LVM: yes configure: iSCSI: yes configure: iscsi-direct: no configure: SCSI: yes configure: mpath: yes configure: Disk: no configure: RBD: no configure: Sheepdog: no configure: Gluster: no configure: ZFS: no configure: Virtuozzo storage: no configure: configure: Security Drivers configure: configure: SELinux: yes configure: AppArmor: no configure: configure: Driver Loadable Modules configure: configure: driver_modules: yes (CFLAGS='' LIBS='-ldl') configure: configure: Libraries configure: configure: acl: no configure: apparmor: no configure: attr: no configure: audit: no configure: avahi: no configure: bash_completion: no configure: blkid: no configure: capng: no configure: curl: no configure: dbus: no configure: dlopen: yes (CFLAGS='' LIBS='-ldl') configure: firewalld: no configure: firewalld-zone: no configure: fuse: no configure: glusterfs: no configure: gnutls: yes (CFLAGS='-I/usr/include/p11-kit-1 ' LIBS='-lgnutls ') configure: hal: no configure: libiscsi: no configure: libnl: yes (CFLAGS=' ' LIBS='-lnl ') configure: libpcap: no configure: libssh: no configure: libxl: yes (CFLAGS=' -DLIBXL_API_VERSION=0x040500' LIBS='-lxenlight -lxlutil -lxenstore -lxentoollog') configure: libxml: yes (CFLAGS='-I/usr/include/libxml2 ' LIBS='-lxml2 ') configure: macvtap: yes (CFLAGS='' LIBS='') configure: netcf: no configure: NLS: yes configure: nss: yes configure: numactl: no configure: openwsman: no configure: pciaccess: yes (CFLAGS=' ' LIBS='-lpciaccess ') configure: pm_utils: yes (CFLAGS='' LIBS='') configure: polkit: no configure: rbd: no configure: readline: no configure: sanlock: no configure: sasl: yes (CFLAGS='' LIBS='-lsasl2') configure: selinux: yes (CFLAGS='' LIBS='-lselinux') configure: ssh2: no configure: udev: yes (CFLAGS=' ' LIBS='-ludev ') configure: virtualport: yes (CFLAGS='' LIBS='') configure: xdr: yes (CFLAGS='' LIBS='') configure: xenapi: no configure: yajl: yes (CFLAGS='' LIBS='-lyajl') configure: configure: Windows configure: configure: Cygwin: no configure: MinGW: no configure: MSVC: no configure: windres: no configure: configure: Test suite configure: configure: Coverage: no configure: Alloc OOM: no configure: configure: Miscellaneous configure: configure: Debug: yes configure: Use -Werror: yes configure: Warning Flags: -fno-common -W -Waddress -Waggressive-loop-optimizations -Wall -Wattributes -Wbad-function-cast -Wbuiltin-macro-redefined -Wcast-align -Wchar-subscripts -Wclobbered -Wcomment -Wcomments -Wcoverage-mismatch -Wcpp -Wdeprecated-declarations -Wdiv-by-zero -Wdouble-promotion -Wempty-body -Wendif-labels -Wextra -Wformat-contains-nul -Wformat-extra-args -Wformat-security -Wformat-y2k -Wformat-zero-length -Wfree-nonheap-object -Wignored-qualifiers -Wimplicit -Wimplicit-function-declaration -Wimplicit-int -Winit-self -Winline -Wint-to-pointer-cast -Winvalid-memory-model -Winvalid-pch -Wlogical-op -Wmain -Wmaybe-uninitialized -Wmissing-braces -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs -Wmissing-parameter-type -Wmissing-prototypes -Wmultichar -Wnarrowing -Wnested-externs -Wnonnull -Wold-style-declaration -Wold-style-definition -Woverflow -Woverride-init -Wpacked-bitfield-compat -Wparentheses -Wpointer-arith -Wpointer-sign -Wpointer-to-int-cast -Wpragmas -Wpsabi -Wreturn-local-addr -Wreturn-type -Wsequence-point -Wshadow -Wsizeof-pointer-memaccess -Wstrict-aliasing -Wstrict-prototypes -Wsuggest-attribute=const -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wswitch -Wsync-nand -Wtrampolines -Wtrigraphs -Wtype-limits -Wuninitialized -Wunknown-pragmas -Wunused -Wunused-but-set-parameter -Wunused-but-set-variable -Wunused-function -Wunused-label -Wunused-local-typedefs -Wunused-parameter -Wunused-result -Wunused-value -Wunused-variable -Wvarargs -Wvariadic-macros -Wvector-operation-performance -Wvolatile-register-var -Wwrite-strings -Wnormalized=nfc -Wno-sign-compare -Wjump-misses-init -Wswitch-enum -Wno-format-nonliteral -fstack-protector-strong -fexceptions -fasynchronous-unwind-tables -fipa-pure-const -Wno-suggest-attribute=pure -Wno-suggest-attribute=const -Werror configure: DTrace: no configure: numad: no configure: Init script: systemd configure: Char device locks: /var/lock configure: Default Editor: vi configure: Loader/NVRAM: configure: virt-login-shell: yes configure: virt-host-validate: yes configure: TLS priority: NORMAL configure: configure: Developer Tools configure: configure: wireshark_dissector: no configure: configure: Privileges configure: configure: QEMU: root:root configure: My question is how should I install the libvirt-daemon-driver-xen driver and make Nova work with Xen? Thanks a lot in advance! Best regards, Lianjie Cao