April 2020 - Devel - libvirt List Archives

[libvirt] [PATCH] qemu: fix hang in p2p + xbzrle compression + parallel migration
by Lin Ma 16 Apr '20

16 Apr '20

When we do parallel migration, The multifd-channels migration parameter needs to be set on the destination side as well before incoming migration URI, unless we accept the default number of connections(2). Usually, This can be correctly handled by libvirtd. But in this case if we use p2p + xbzrle compression without parameter '--comp-xbzrle-cache', qemuMigrationParamsDump returns too early, The corresponding migration parameter will not be set on the destination side, It results QEMU hangs. Reproducer: virsh migrate --live --p2p --comp-methods xbzrle \ --parallel --parallel-connections 3 GUEST qemu+ssh://dsthost/system or virsh migrate --live --p2p --compressed \ --parallel --parallel-connections 3 GUEST qemu+ssh://dsthost/system Signed-off-by: Lin Ma <lma(a)suse.com> --- src/qemu/qemu_migration_params.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd5e2ce1b6..810199370f 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -630,7 +630,6 @@ qemuMigrationParamsDump(qemuMigrationParamsPtr migParams, if (migParams->compMethods == 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE && !migParams->params[QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE].set) { *flags |= VIR_MIGRATE_COMPRESSED; - return 0; } for (i = 0; i < QEMU_MIGRATION_COMPRESS_LAST; ++i) { -- 2.23.0

2 1

[libvirt-php] libvirt_domain_interface_addresses() only returns only on IP address for NICs with multiple IPs
by Fernando Casas Schössow 15 Apr '20

15 Apr '20

Hi, While using the function libvirt_domain_interface_addresses() with qemu guest agent as the source, for interfaces with more than one IP address, the function will only return one IP. Example code: $netdetails = libvirt_domain_interface_addresses($vmres, 2); foreach ($netdetails as $nic) { print_r($nic); } Example output: Array ( [name] => eth0 [hwaddr] => 52:54:00:12:b9:22 [naddrs] => 2 [addrs] => Array ( [addr] => fe80::5054:ff:fe12:b922 [prefix] => 64 [type] => 1 ) ) Array ( [name] => eth1 [hwaddr] => 52:54:00:8e:b0:35 [naddrs] => 2 [addrs] => Array ( [addr] => fe80::5054:ff:fe8e:b035 [prefix] => 64 [type] => 1 ) ) As you can see for this guest the function returns two interfaces (eth0 and eth1). For both interfaces naddrs => 2 which is correct because both interfaces have an ipv4 and an ipv6 address but the addrs array only contains the ipv6 address information while the ipv4 address information is missing. System information: OS: Debian Bullseye PHP version (fast cgi): 7.3.15 Libvirt PHP version: 0.5.5 Libvirt version: 6.0.0 KVM host libvirt version: 5.9.0 If you need any other information please let me know. Thanks. Fernando

1 0

[libvirt-php] libvirt_domain_interface_addresses causing segfault
by Fernando Casas Schössow 15 Apr '20

15 Apr '20

Hi, While executing libvirt_domain_interface_addresses() with the qemu-agent running in the guest as the source, the function causes PHP to segfault. If I change the source to either dhcp or arp, the function executes correctly. Example: $netdetails = libvirt_domain_interface_addresses($vmres, 0); <-- executes correctly (dhcp source) $netdetails = libvirt_domain_interface_addresses($vmres, 1); <-- segfault (qemu agent source) $netdetails = libvirt_domain_interface_addresses($vmres, 2); <-- executes correctly (arp source) I tested the equivalent function using virsh against the same host and it executes correctly with any source. I have a php-cgi core file that I can share if that may help on finding the problem. gdb output: GNU gdb (Debian 9.1-2) 9.1 Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/bin/php-cgi... (No debugging symbols found in /usr/bin/php-cgi) [New LWP 2594] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `php-cgi index.php'. Program terminated with signal SIGSEGV, Segmentation fault. #0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:85 85 ../sysdeps/x86_64/multiarch/../strlen.S: No such file or directory. (gdb) bt #0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:85 #1 0x000055c6a4ceacd2 in add_assoc_string_ex () #2 0x00007f4d36175253 in ?? () from /usr/lib/php/20180731/libvirt-php.so #3 0x000055c6a4d6b477 in execute_ex () #4 0x000055c6a4d705c7 in zend_execute () #5 0x000055c6a4ce8ed4 in zend_execute_scripts () #6 0x000055c6a4c8be38 in php_execute_script () #7 0x000055c6a4b53da8 in ?? () #8 0x00007f4d39efce0b in __libc_start_main (main=0x55c6a4b51cf0, argc=2, argv=0x7fff35718588, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff35718578) at ../csu/libc-start.c:308 #9 0x000055c6a4b544ca in _start () System information: OS: Debian Bullseye PHP version (fast cgi): 7.3.15 Libvirt PHP version: 0.5.5 Libvirt version: 6.0.0 KVM host libvirt version: 5.9.0 If you need any other information please let me know. Thanks. Fernando

1 0

[libvirt PATCH 0/3] travis: Drop xcode10.3 build, clean up
by Andrea Bolognani 15 Apr '20

15 Apr '20

Andrea Bolognani (3): travis: Deduplicate build instructions travis: Reduce test matrix travis: Remove usage of 'sudo' .travis.yml | 52 ++++++++++++++++++---------------------------------- 1 file changed, 18 insertions(+), 34 deletions(-) -- 2.25.2

2 10

[libvirt PATCH 0/2] gitlab: Clean up and improve ccache usage
by Andrea Bolognani 15 Apr '20

15 Apr '20

Test pipeline: https://gitlab.com/abologna/libvirt/pipelines/131128028 Andrea Bolognani (2): gitlab: Don't define $MAKE gitlab: Enable improved ccache usage .gitlab-ci.yml | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) -- 2.25.1

2 6

[PATCH] qemu: Revoke access to mirror on failed blockcopy
by Michal Privoznik 15 Apr '20

15 Apr '20

When preparing to do a blockcopy, the mirror image is modified so that QEMU can access it. For instance, the mirror has seclabels set, if it is a NVMe disk it is detached from the host and so on. And usually, the restore is done upon successful finish of the blockcopy operation. But, if something fails then we need to explicitly revoke the access to the mirror image (and thus reattach NVMe disk back to the host). Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1822538 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_driver.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 31f199fdef..9475235f01 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -17950,6 +17950,7 @@ qemuDomainBlockCopyCommon(virDomainObjPtr vm, virDomainDiskDefPtr disk = NULL; int ret = -1; bool need_unlink = false; + bool need_revoke = false; g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); const char *format = NULL; bool mirror_reuse = !!(flags & VIR_DOMAIN_BLOCK_COPY_REUSE_EXT); @@ -18124,6 +18125,7 @@ qemuDomainBlockCopyCommon(virDomainObjPtr vm, if (qemuDomainStorageSourceChainAccessAllow(driver, vm, mirror) < 0) goto endjob; + need_revoke = true; if (blockdev) { if (mirror_reuse) { @@ -18240,6 +18242,8 @@ qemuDomainBlockCopyCommon(virDomainObjPtr vm, if (crdata) qemuBlockStorageSourceAttachRollback(priv->mon, crdata->srcdata[0]); ignore_value(qemuDomainObjExitMonitor(driver, vm)); + if (need_revoke) + qemuDomainStorageSourceChainAccessRevoke(driver, vm, mirror); } if (need_unlink && virStorageFileUnlink(mirror) < 0) VIR_WARN("%s", _("unable to remove just-created copy target")); -- 2.24.1

3 3

what a correct use for virConnectDomainEventRegisterAny API, how to Obtain a stable expected result
by thomas.kuang 15 Apr '20

15 Apr '20

HI, everyone: My target deal with network hotplug use virDomainDetachDeviceFlags. Because when the API return ,the network maybe doesn’t remove from my vm guest os. So I use virConnectDomainEventRegisterAny to register an event ID: VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED , my process as follow: cb_para->call_id=virConnectDomainEventRegisterAny(cb_para->conn,cb_para->dom,VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED, VIR_DOMAIN_EVENT_CALLBACK(vnf_control_del_network_cb), cb_para, vnf_control_del_network_cb_free); flags |= VIR_DOMAIN_AFFECT_CONFIG; if (virDomainIsActive(dom) == 1) { flags |= VIR_DOMAIN_AFFECT_LIVE; } ret = virDomainDetachDeviceFlags(dom, xml, flags); above code write in thread loop ,then in the same loop : while (1) { mission = vnf_mission_queue_get(task); if (mission == NULL) { sleep(1); continue; } vnf_op_process(&mission->info); // this will deal with network hotplug,will call virConnectDomainEventRegisterAny then call virDomainDetachDeviceFlags if (mission) { vnf_mission_free(mission); } if(virEventRunDefaultImpl() < 0) { // at here process the registered callback for event-registered printf();.... } } My problem is: some time , the virEventRunDefaultImpl can trigger the vnf_control_del_network_cb callback ,but some time there is nothing ,as if the VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED has lost. what cause the Unpredictable behavior ? what is a correct use for virConnectDomainEventRegisterAny ? if can't trigger the vnf_control_del_network_cb callback , the memory :cb_para will mem-leak, so in order to deal the execpt ,i register a timer to process the VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED timeout cb_para->timer_id = virEventAddTimeout(cb_para->time_out, vnf_control_del_network_timeout_cb, cb_para, vnf_control_del_network_cb_free); thought use the timer ,can i avoid the cb_para mem-leak, but fail to achive hotplug network .

2 2

[PATCH 00/43] Replace virMutex & co. by the Glib equivalent
by Rafael Fonseca 15 Apr '20

15 Apr '20

This patch series replaces virMutex and virRWLock by GMutex and GRWLock, respectively. And because virCond usage is connected to virCond, it also replaces the latter by GCond. Rafael Fonseca (43): admin: convert virMutex to GMutex bhyve: convert virMutex to GMutex conf: nwfilter_ipaddrmap: convert virMutex to GMutex conf: virchrdev: convert virMutex to GMutex esx: convert virMutex to GMutex util: virtpm: convert virMutex to GMutex vbox: convert virMutex to GMutex util: virnodesuspend: convert virMutex to GMutex libxl: convert virMutex to GMutex in libxlPrivate vz: vz_driver: convert virMutex to GMutex locking: convert virMutex to GMutex logging: convert virMutex to GMutex conf: nwfilter_conf: convert virRWLock to GRWLock util: virlog: convert virMutex to GMutex test: convert virMutex to GMutex nwfilter: nwfilter_dhcpsnoop: convert virMutex to GMutex nwfilter: nwfilter_gentech_driver: convert virMutex to GRecMutex lxc: lxc_fuse: convert virMutex to GMutex lxc: convert virMutex to GMutex network: bridge_driver: convert virMutex to GMutex node_device: convert virMutex to GMutex vmware: convert virMutex to GMutex tests: qemusecuritymock: convert virMutex to GMutex tests: qemumonitortestutils: convert virMutex to GMutex qemu: convert virMutex to GMutex remote: remote_driver: convert virMutex to GMutex nwfilter: convert virMutex to GMutex openvz: convert virMutex to GMutex storage: convert virMutex to GMutex secret: convert virMutex to GMutex remote: convert virMutex to GMutex in daemonClientPrivate util: virfirewall: convert virMutex to GMutex util: virlockspace: convert virMutex to GMutex util: virnetdevmacvlan: convert virMutex to GMutex util: virnetdevveth: convert virMutex to GMutex util: virnetlink: convert virMutex to GMutex tools: virsh: convert virMutex to GMutex nwfilter: nwfilter_learnipaddr: convert virMutex to GMutex util: virthreadpool: convert virMutex to GMutex util: virObjectRWLockable: convert virRWLock to GRWLock util: virthread: remove virRWLock util: virObjectLockable: convert virMutex to GMutex util: virthread: remove GMutex and Gcond examples/systemtap/lock-debug.stp | 4 +- src/admin/admin_server_dispatch.c | 13 +-- src/bhyve/bhyve_driver.c | 11 +- src/bhyve/bhyve_utils.h | 2 +- src/conf/domain_conf.c | 27 +---- src/conf/domain_conf.h | 5 +- src/conf/nwfilter_conf.c | 21 ++-- src/conf/nwfilter_conf.h | 5 +- src/conf/nwfilter_ipaddrmap.c | 14 +-- src/conf/virchrdev.c | 27 ++--- src/conf/virdomainobjlist.c | 35 +++--- src/conf/virinterfaceobj.c | 18 +-- src/conf/virnetworkobj.c | 20 ++-- src/conf/virnodedeviceobj.c | 16 +-- src/conf/virnodedeviceobj.h | 4 +- src/conf/virnwfilterbindingobjlist.c | 14 +-- src/conf/virnwfilterobj.c | 15 +-- src/conf/virnwfilterobj.h | 2 +- src/conf/virsecretobj.c | 16 +-- src/conf/virstorageobj.c | 50 ++++----- src/conf/virstorageobj.h | 2 +- src/esx/esx_stream.c | 25 ++--- src/esx/esx_vi.c | 68 ++++------- src/esx/esx_vi.h | 6 +- src/libvirt_private.syms | 22 +--- src/libxl/libxl_conf.h | 6 +- src/libxl/libxl_domain.c | 18 +-- src/libxl/libxl_domain.h | 2 +- src/libxl/libxl_driver.c | 9 +- src/locking/lock_daemon.c | 32 ++---- src/locking/lock_daemon.h | 2 +- src/locking/lock_daemon_dispatch.c | 32 ++---- src/logging/log_daemon.c | 26 +---- src/logging/log_daemon.h | 2 +- src/lxc/lxc_conf.h | 6 +- src/lxc/lxc_controller.c | 25 ++--- src/lxc/lxc_domain.c | 18 +-- src/lxc/lxc_domain.h | 2 +- src/lxc/lxc_driver.c | 11 +- src/lxc/lxc_fuse.c | 13 +-- src/lxc/lxc_fuse.h | 2 +- src/network/bridge_driver.c | 11 +- src/network/bridge_driver_platform.h | 2 +- src/node_device/node_device_driver.c | 11 +- src/node_device/node_device_hal.c | 20 +--- src/node_device/node_device_udev.c | 43 ++----- src/nwfilter/nwfilter_dhcpsnoop.c | 48 ++++---- src/nwfilter/nwfilter_driver.c | 15 ++- src/nwfilter/nwfilter_gentech_driver.c | 40 +++---- src/nwfilter/nwfilter_learnipaddr.c | 43 ++++--- src/openvz/openvz_conf.h | 2 +- src/openvz/openvz_driver.c | 4 +- src/qemu/qemu_agent.c | 50 ++++----- src/qemu/qemu_conf.c | 4 +- src/qemu/qemu_conf.h | 2 +- src/qemu/qemu_domain.c | 95 +++++++--------- src/qemu/qemu_domain.h | 4 +- src/qemu/qemu_driver.c | 21 ++-- src/qemu/qemu_hotplug.c | 28 ++--- src/qemu/qemu_hotplug.h | 2 +- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_monitor.c | 27 ++--- src/qemu/qemu_process.c | 4 +- src/remote/remote_daemon.h | 2 +- src/remote/remote_daemon_dispatch.c | 100 ++++------------- src/remote/remote_daemon_stream.c | 29 ++--- src/remote/remote_driver.c | 15 +-- src/rpc/virnetclient.c | 25 ++--- src/secret/secret_driver.c | 13 +-- src/storage/storage_driver.c | 11 +- src/test/test_driver.c | 12 +- src/util/virfdstream.c | 26 ++--- src/util/virfirewall.c | 6 +- src/util/virlockspace.c | 83 ++++---------- src/util/virlog.c | 9 +- src/util/virnetdevmacvlan.c | 20 ++-- src/util/virnetdevveth.c | 6 +- src/util/virnetlink.c | 15 +-- src/util/virnodesuspend.c | 6 +- src/util/virobject.c | 40 +++---- src/util/virobject.h | 10 +- src/util/virthread.c | 134 ---------------------- src/util/virthread.h | 57 ---------- src/util/virthreadpool.c | 149 +++++++++---------------- src/util/virtpm.c | 18 +-- src/vbox/vbox_common.c | 12 +- src/vmware/vmware_conf.c | 2 +- src/vmware/vmware_conf.h | 2 +- src/vmware/vmware_driver.c | 7 +- src/vz/vz_driver.c | 26 ++--- src/vz/vz_utils.c | 41 ++----- src/vz/vz_utils.h | 6 +- tests/qemuhotplugmock.c | 6 +- tests/qemuhotplugtest.c | 2 + tests/qemumonitortestutils.c | 42 +++---- tests/qemusecuritymock.c | 30 ++--- tests/qemusecuritytest.c | 10 +- tests/qemuxml2argvtest.c | 1 + tests/testutilslxc.c | 9 +- tests/testutilsqemu.c | 5 +- tests/testutilsxen.c | 9 +- tools/virsh-console.c | 23 +--- tools/virsh.c | 11 +- tools/virt-admin.c | 11 +- tools/vsh.c | 4 +- tools/vsh.h | 2 +- 106 files changed, 747 insertions(+), 1426 deletions(-) -- 2.25.2

5 64

Wiki page audit - possible page deleteion
by Daniel P. Berrangé 15 Apr '20

15 Apr '20

Ideally I would like to decomission the current wiki.libvirt.org site. It is based on mediawiki running on openshift and has myself as a single point of failure. GitLab provides a wiki, but I don't think we need to use that either, as I think that it is desirable to bring the content into our main website. By putting an "edit this page" link on each page of our website, users can quickly see what source to change. There is still the burden of submitting a merge request & having review feedback, but I think we could mitigate this by having the reviewer actually make the changes they want directly, avoiding the tedious back & forth updates in easy cases. GitLab allows this if the person opening the merge request selects the option to allow maintainers to edit code. In any case, to remove wiki.libvirt.org we need to do something with its current content. The majority of content was first created 5-10 years ago, only a handful of pages get frequent edits right now: https://wiki.libvirt.org/index.php?title=Special:RecentChanges&limit=500&da… Looking at the pages I see some key groups, so I'll talk about them separately... A set of "Troubleshooting" guides supposedly describing common problems and their suggested solution. Some of these still apply but others will be outdated, so we need to decide which to keep. Yes we really do have multiple spellings of the same page in many cases After import a guest from an existing disk image using virt-install, the guest starting stalls with "No boot device" Common XML errors Could not add rule to fixup DHCP response checksums on network 'default' Creating VMWare ESXi domain failed with error "this function is not supported by the connection driver: virDomainCreateXML" Determining version information Determining version information, dealing with "unknown procedure" Different Processor Model Determined Different processor model determined Domain cannot be installed Domain starting fails with Error "monitor socket did not show up" Error "internal error cannot find character device" when trying to connect a domain's console Failed to connect to the hypervisor Guest can reach host, but can't reach outside network Guest can reach outside network, but can't reach host (macvtap) Guest won't start - warning: could not open /dev/net/tun ('generic ethernet' interface) I created an external snapshot, but libvirt will not let me delete or revert to it I created an external snapshot, but libvirt won't let me delete or revert to it Libvirt daemon is not listening on tcp ports although configured to Libvirt identifies host processor as a different model from the hardware documentation Migration fails because disk image cannot be found Migration fails with "Unable to resolve address" error No guest machines are present PXE boot (or dhcp) on guest failed The daemon cannot be started The domain cannot be started when specifying different processor TroubleshootMacvtapHostFail Troubleshooting Unable to add bridge br0 port vnet0: No such device Unable to connect to console of a running domain Virtual network "default" has not been started Virtual network 'default' has not been started Some illustrated guides to TLS cert creation. We already have a page https://libvirt.org/tlscerts.html and the wiki duplicates much of the info there. One difference is that the wiki also describes the gtk-vnc/spice-gtk/qemu cert setup, not merely libvirt. We should really consolidate into our main website though, as its confusing to have two separate docs for the same tasks TLSCreateCACert TLSCreateCACertSteps TLSCreateClientCerts TLSCreateServerCerts TLSDaemonConfiguration TLSFurtherReferences TLSSetup HostCommTLSSetup VNCTLSSetup Various GSoC pages, the per-year pages, and for 2016 only some reports from the contributors. Most of the latter is outdated info that's no longer really relevant, so I'd probablyjust delete the last 4 pages, and move the rest to the main site. Google Summer of Code 2016 Google Summer of Code 2017 Google Summer of Code 2018 Google Summer of Code 2019 Google Summer of Code FAQ Google Summer of Code Ideas Google Summer of Code 2016/Abstracting device address allocation Google Summer of Code 2016/Asynchronous lifecycle events for storage objects Google Summer of Code 2016/Making virsh more bash like Google Summer of Code 2016/lxc migration Some graphically illustrated guides to virtmanager. These never belonged here, and should be part of virt-manager website if they're really still desired. The screenshots are from an ancient version though, so I figure we can just delete them. TaskIsolatedNetworkSetupVirtManager TaskNATSetupVirtManager TaskRoutedNetworkSetupVirtManager CreatingNewVM in VirtualMachineManager DeletingVirtualMachine in VirtualMachineManager VirtualNetworking SSHSetup Some old pages containing a "todo" list, which we stopped updating a long time ago. Many of the ideas are not things we'd want to implement today so serve to mislead people. Propose to delete them all Todo TodoAMQPAgent TodoAsynchronousJobs TodoAvahiTXTSupport TodoConcurrentConnectionPtr TodoDaemonMultithreading TodoDaemonRestart TodoDriverKVM TodoEvents TodoFineGrainedSecurity TodoHostDevicePassthru TodoHostDevices TodoModules TodoNICBonding TodoNICMultipath TodoNetworkTopologyDiscovery TodoPackages TodoPoolBasedConfigs TodoPreMigrationChecks TodoSecureMigration TodoStorageSCSI TodoVMSnapshots TodoVirtViewerSecurity TodoWindowsSupport Stuff related to Cimtest, largely pages containing results form test runs from ancient distros. I propose to delete all of this as its irrelevant even if someone was still interested in cimtest today. Cimtest Cimtest buglist Cimtest setup Cimtest test info Cimtest testruns Cimtest todo KVM F10 KVM F11 KVM F12 KVM F13 KVM F9 KVM current KVM current Fedora rawhide KVM current SLES11 KVM current sources on RHEL5.4 KVM on current sources - older test results LXC current XenFV current XenFV current 53 XenFV current Fedora XenFV rpm XenPV current XenPV current 53 XenPV current Fedora XenPV rpm XenPV rpm 54 Various misc pages that I've not spent too much time looking at each page, but as a rough approx I'd keep the following list, pulling it into the main website BiteSizedTasks DebugLogs Debugging FAQ Libvirt-snmp Libvirtd and dnsmasq Live-disk-backup-with-active-blockcommit Live-merge-an-entire-disk-image-chain-including-current-active-disk Maintenance Releases NPIV in libvirt Net.bridge-nf-call and sysctl.conf Net.bridge.bridge-nf-call and sysctl.conf Networking Qemu guest agent SSHPolicyKitSetup VM lifecycle Vhost-scsi target And probably delete the following (though some might want preserving upon closer inspection) AprilFools' AreasToFocusOnTesting DocsToDo Features/virSimple HowToPopulateLibosinfoDB IntroductoryGuides KVMGapPriorityList Libvirt-CIM Libvirt-CIM/Issues Libvirt-cim setup Libvirt-gconfig Libvirt-qpid LibvirtCim/Bugs LibvirtConsoleManagement Main Page Making virsh more bash like NBD storage migration OVS and PVLANS OldTodo QEMUSwitchToLibvirt SVGImages Snapshots Stable Releases StorageDocsPerfectWorldScenario Tips UbuntuKVMWalkthrough VirshHelpV2 Virtio Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

4 5

[libvirt PATCH] docs: Add pci-addresses.rst
by Andrea Bolognani 15 Apr '20

15 Apr '20

This document describes the relationship between PCI addresses as seen in the domain XML and by the guest OS, which is a topic that people get confused by time and time again. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- docs/formatdomain.html.in | 6 +- docs/pci-addresses.rst | 184 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 189 insertions(+), 1 deletion(-) create mode 100644 docs/pci-addresses.rst diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 6f43976815..0077666862 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -4286,7 +4286,11 @@ element with no other attributes as an explicit request to assign a PCI address for the device rather than some other type of address that may also be appropriate for that same - device (e.g. virtio-mmio). + device (e.g. virtio-mmio).<br/> + The relationship between the PCI addresses configured in the domain + XML and those seen by the guest OS can sometime seem confusing: a + separate document describes <a href="pci-addresses.html">how PCI + addresses work</a> in more detail. </dd> <dt><code>drive</code></dt> <dd>Drive addresses have the following additional diff --git a/docs/pci-addresses.rst b/docs/pci-addresses.rst new file mode 100644 index 0000000000..96c6466899 --- /dev/null +++ b/docs/pci-addresses.rst @@ -0,0 +1,184 @@ +======================================== +PCI addresses in domain XML and guest OS +======================================== + +.. contents:: + +When discussing PCI addresses, it's important to understand the the +relationship between the addresses that can be seen in the domain XML +and those that are visible inside the guest OS. + + +Simple cases +============ + +When the PCI topology of the VM is very simple, the PCI addresses +will usually match. + +For example, the domain XML snippet + +:: + + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='1' port='0x8'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <interface type='network'> + <source network='default'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </interface> + +will result in the PCI topology + +:: + + 0000:00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller + 0000:00:01.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port + 0000:01:00.0 Ethernet controller: Red Hat, Inc. Virtio network device (rev 01) + +showing up in the guest OS. + +The PCI address of the ``virtio-net`` adapter, ``0000:01:00.0``, is +the same in both cases, so there's no confusion. + + +More complex cases +================== + +In more complex cases, the PCI address visible in the domain XML will +correlate to the one seen by the guest OS in a less obvious way. + +pcie-expander-bus +----------------- + +This fairly uncommon device, which can be used with ``x86_64/q35`` +guests, will help illustrate one such scenario. + +For example, the domain XML snippet + +:: + + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='254'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='2' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <interface type='network'> + <source network='default'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </interface> + +will result in the PCI topology + +:: + + 0000:00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller + 0000:00:01.0 Host bridge: Red Hat, Inc. QEMU PCIe Expander bridge + 0000:fe:00.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port + 0000:ff:00.0 Ethernet controller: Red Hat, Inc. Virtio network device (rev 01) + +showing up in the guest OS. + +This time the addresses don't match: this is because the ``busNr`` +property for the ``pcie-expander-bus`` controller causes it to show +up as bus 254 (``0xfe`` in hexadecimal) instead of bus 1 as one might +expect based on its ``index`` property. + +How can the domain XML shown above work at all, then? Surely the +``pcie-root-port`` controller and the ``virtio-net`` adapter should +use ``bus=0xfe`` and ``bus=0xff`` respectively for the configuration +to be accepted by libvirt? + +As it turns out, that's not the case. The reason for this is that +QEMU, and consequently libvirt, uses the ``bus`` property of a +device's PCI address only to match it with the PCI controller that +has the same ``index`` property, and not to set the actual PCI +address, which is decided by the guest OS. + +So, by looking at the XML snippet above, we can see that the +``virtio-net`` adapter plugs into the ``pcie-root-port`` controller, +which plugs into the ``pcie-expander-bus`` controller, which plugs +into ``pcie-root``: the guest OS sees the same topology, but assigns +different PCI addresses to some of its component. + +The takeaway is that the *relationship* between controllers are the +very same whether you look at the domain XML or at the guest OS, but +the *actual PCI addresses* are not guaranteed to match and in fact, +except for the very simplest cases, they usually will not. + +spapr-pci-host-bridge +--------------------- + +This device, which is unique to ``ppc64/pseries`` guests, will help +illustrate another scenario. + +For example, the domain XML snippet + +:: + + <controller type='pci' index='0' model='pci-root'> + <model name='spapr-pci-host-bridge'/> + <target index='0'/> + </controller> + <controller type='pci' index='1' model='pci-root'> + <model name='spapr-pci-host-bridge'/> + <target index='1'/> + </controller> + <interface type='network'> + <source network='default'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> + </interface> + +will result in the PCI topology + +:: + + 0001:00:01.0 Ethernet controller: Red Hat, Inc. Virtio network device + +showing up in the guest OS. Note that the two +``spapr-pci-host-bridge`` controllers are not listed. + +This time, in addition to the bus not matching just like in the +previous example, the interesting part is that the domain doesn't +match either: this is because each ``spapr-pci-host-bridge`` +controller creates a separate PCI domain. + +Once again, while the PCI addresses seen in the domain XML and those +seen by the guest OS do not match, the relationships between the +various devices are preserved. + + +Device assignment +================= + +When using VFIO to assign host devices to a guest, an additional +caveat to keep in mind that the guest OS will base its decisions upon +the *target address* rather than the *source address*. + +For example, the domain XML snippet + +:: + + <hostdev mode='subsystem' type='pci' managed='yes'> + <driver name='vfio'/> + <source> + <address domain='0x0001' bus='0x08' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </hostdev> + +will result in the device showing up as ``0000:00:01.0`` in the +guest OS rather than as ``0001:08:00.1``. + +Of course, all the rules and behaviors described above still apply. -- 2.25.2

4 5