- Devel - libvirt List Archives

[libvirt] [jenkins-ci PATCH] Add support for cross compiling libvirt via Debian
by Daniel P. Berrangé 05 Feb '19

05 Feb '19

Debian's filesystem layout has a nice advantage over Fedora which is that it can install non-native RPMs in the main root filesystem. It is thus possible to prepare an x86_64 filesystem containing -dev packages for a foreign architecture, along with a GCC cross compiler. QEMU has used this technique to facilitate developer build testing of non-x86 architectures, since few people have access to physical hardware for most of these architectures. For the same reason it would be helpful to libvirt developers. This docker file allows creation of containers buildenv-cross-$ARCH for arm64, armel, armhf, mips, mipsel, mips64el, ppc64el, s390x, based off Debian 9, in which libvirt can be cross-compiled. Debian can support further architectures, but only by using the unstable 'sid' repos. This does not enable use of 'sid' but it could be added fairly easily at a later date if desired. Most of the other architectures are fairly old & obscure, the main interesting exception being riscv64. If qemu-user emulators are registered with the kernel binfmt, then it is theoretically possible to run "make check" tool, though this has not been tested. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .gitignore | 1 + cross-build/arch-config.csv | 8 +++ cross-build/buildenv-cross.docker.in | 96 ++++++++++++++++++++++++++++ 3 files changed, 105 insertions(+) create mode 100644 cross-build/arch-config.csv create mode 100644 cross-build/buildenv-cross.docker.in diff --git a/.gitignore b/.gitignore index b25c15b..4884352 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ *~ +cross-build/*docker diff --git a/cross-build/arch-config.csv b/cross-build/arch-config.csv new file mode 100644 index 0000000..36fcb5f --- /dev/null +++ b/cross-build/arch-config.csv @@ -0,0 +1,8 @@ +arm64,crossbuild-essential-arm64,aarch64-linux-gnu +armel,crossbuild-essential-armel,arm-linux-gnueabi +armhf,crossbuild-essential-armhf,arm-linux-gnueabihf +mips64el,gcc-mips64el-linux-gnuabi64,mips64el-linux-gnuabi64 +mips,gcc-mips-linux-gnu,mips-linux-gnu +mipsel,gcc-mipsel-linux-gnu,mipsel-linux-gnu +ppc64el,crossbuild-essential-ppc64el,powerpc64le-linux-gnu +s390x,gcc-multilib-s390x-linux-gnu,s390x-linux-gnu diff --git a/cross-build/buildenv-cross.docker.in b/cross-build/buildenv-cross.docker.in new file mode 100644 index 0000000..97a9e69 --- /dev/null +++ b/cross-build/buildenv-cross.docker.in @@ -0,0 +1,96 @@ +# +# ::ARCH:: cross-compiler target +# +FROM debian:stretch-slim + +# Grab the updated list of packages +RUN apt update && apt dist-upgrade -yy + +# First some common native binaries. Any program that the +# build will execute needs to have a native binary package +# installed. We don't want to rely on qemu-user emulation +# for executing non-native binaries as that is slow & flaky +RUN DEBIAN_FRONTEND=noninteractive \ + apt-get install -y --no-install-recommends \ + git \ + build-essential \ + ca-certificates \ + libtool\ + autoconf \ + automake \ + python3 \ + xsltproc \ + parted \ + radvd \ + qemu-system-common \ + dwarves \ + libxml2-utils \ + dnsmasq-base \ + openssh-client \ + netcat-openbsd \ + ebtables \ + iptables \ + sheepdog \ + qemu-utils \ + qemu-system-x86 \ + pkg-config + + +# Add the ::ARCH:: architecture +RUN dpkg --add-architecture ::ARCH:: + + +# Grab the freshly updated list of packages +RUN apt update + + +# Pull in the native package for the cross-compiler +RUN DEBIAN_FRONTEND=noninteractive \ + apt-get install -y --no-install-recommends \ + ::COMPILER:: + +# Finally pull in the foreign architecture library +# packages that we're going to link against. Everything +# here should be a -dev package. Binaries belong earlier +RUN DEBIAN_FRONTEND=noninteractive \ + apt-get install -y --no-install-recommends \ + libxml2-dev:::ARCH:: \ + libncurses5-dev:::ARCH:: \ + libreadline-dev:::ARCH:: \ + zlib1g-dev:::ARCH:: \ + libgcrypt20-dev:::ARCH:: \ + libgnutls28-dev:::ARCH:: \ + libavahi-client-dev:::ARCH:: \ + libsasl2-dev:::ARCH:: \ + libparted-dev:::ARCH:: \ + libdevmapper-dev:::ARCH:: \ + uuid-dev:::ARCH:: \ + libudev-dev:::ARCH:: \ + libpciaccess-dev:::ARCH:: \ + libpolkit-gobject-1-dev:::ARCH:: \ + libcap-ng-dev:::ARCH:: \ + libnl-3-dev:::ARCH:: \ + libnl-route-3-dev:::ARCH:: \ + libyajl-dev:::ARCH:: \ + libpcap0.8-dev:::ARCH:: \ + libsanlock-dev:::ARCH:: \ + libaudit-dev:::ARCH:: \ + libselinux1-dev:::ARCH:: \ + libapparmor-dev:::ARCH:: \ + libdbus-1-dev:::ARCH:: \ + librbd-dev:::ARCH:: \ + librados-dev:::ARCH:: \ + libfuse-dev:::ARCH:: \ + libssh2-1-dev:::ARCH:: \ + libcurl4-gnutls-dev:::ARCH:: \ + libacl1-dev:::ARCH:: \ + libiscsi-dev:::ARCH:: \ + xfslibs-dev:::ARCH:: \ + libssh-dev:::ARCH:: \ + libgnutls28-dev:::ARCH:: + +# Tell libvirt which architecture we want to target +ENV LIBVIRT_CONFIGURE_OPTS "--host=::ABI:: --build=x86_64-linux-gnu --target=::ABI::" + +# Tell pkg-config where to find the non-native .pc files +ENV PKG_CONFIG_LIBDIR=/usr/lib/::ABI::/pkgconfig -- 2.20.1

2 2

[libvirt] [PULL 00/14] Ui 20190205 patches
by Gerd Hoffmann 05 Feb '19

05 Feb '19

The following changes since commit 773c4a6228fd910556cee2d477ee56c591a30000: test-filter-mirror: pass UNIX domain socket through fd (2019-02-04 16:03:20 +0000) are available in the git repository at: git://git.kraxel.org/qemu tags/ui-20190205-pull-request for you to fetch changes up to 19c1b9fd3dd5955893c0d3c187a4180313e9a0f1: keymap: fix keyup mappings (2019-02-05 10:45:44 +0100) ---------------------------------------------------------------- ui: add kbd stats tracker. ui: gtk scroll fixes. ui: egl cursor scale fix. ui: more sdl1 cleanup. ---------------------------------------------------------------- Brendan Shanks (1): ui/cocoa.m: Fix macOS 10.14 deprecation warnings Chen Zhang (1): ui/egl-helpers: Augment parameter list of egl_texture_blend() to convey scales of viewport Gerd Hoffmann (8): kbd-state: add keyboard state tracker sdl2: remove sdl2_reset_keys() function kbd-state: use state tracker for sdl2 sdl2: use only QKeyCode in sdl2_process_key() kbd-state: use state tracker for gtk kbd-state: use state tracker for vnc keymap: pass full keyboard state to keysym2scancode keymap: fix keyup mappings Sergio Lopez (2): ui: don't send any event if delta_y == 0 ui: listen for GDK_SMOOTH_SCROLL events Thomas Huth (2): Remove deprecated -no-frame option ui/sdl_keysym: Remove obsolete SDL1.2 related code include/sysemu/sysemu.h | 1 - include/ui/egl-helpers.h | 2 +- include/ui/gtk.h | 2 + include/ui/kbd-state.h | 101 +++++++++++++++++ include/ui/sdl2.h | 3 +- ui/keymaps.h | 3 +- ui/sdl_keysym.h | 278 ----------------------------------------------- ui/vnc.h | 5 +- ui/curses.c | 2 +- ui/egl-headless.c | 3 +- ui/egl-helpers.c | 9 +- ui/gtk-egl.c | 3 +- ui/gtk.c | 43 ++------ ui/kbd-state.c | 130 ++++++++++++++++++++++ ui/keymaps.c | 55 ++++++---- ui/sdl2-input.c | 50 +-------- ui/sdl2.c | 12 +- ui/spice-display.c | 2 +- ui/vnc.c | 119 ++++++-------------- vl.c | 23 +--- qemu-deprecated.texi | 6 - qemu-options.hx | 11 -- ui/Makefile.objs | 2 +- ui/cocoa.m | 28 ++++- 24 files changed, 360 insertions(+), 533 deletions(-) create mode 100644 include/ui/kbd-state.h delete mode 100644 ui/sdl_keysym.h create mode 100644 ui/kbd-state.c -- 2.9.3

2 15

[libvirt] [PATCH 0/3] qemu: Report better error on dump/migrate failure
by Andrea Bolognani 05 Feb '19

05 Feb '19

A detailed explanation of why this is needed can be found at https://bugzilla.redhat.com/show_bug.cgi?id=1578741#c8 Andrea Bolognani (3): vircommand: Ensure buffers are NULL-terminated virfile: Report error in virFileWrapperFdFree() iohelper: Don't include newlines in error messages src/util/iohelper.c | 2 +- src/util/vircommand.c | 2 ++ src/util/virfile.c | 6 +++++- 3 files changed, 8 insertions(+), 2 deletions(-) -- 2.20.1

2 5

[libvirt] [PATCH 00/10] snapshot: misc fixes
by Nikolay Shirokovskiy 05 Feb '19

05 Feb '19

The patch series aims to uniformly handle snapshot attribute for disks from multiple places - snapshot definitions, domain definitions, snapshot operation and defaults which is done in [1] utimately. Patch [2] makes a step towards this goal. Other patches are fixes along the way that disables some invalid cases on conversely enable previously disable valid cases. The patches with most reasoning ([2] and [3]) are effectively revert Eric's patch f9670bf thus I hope Eric will take a look. Nikolay Shirokovskiy (10): conf: snapshot: fix comment in _virDomainSnapshotDef conf: snapshot: don't pass flags from different family conf: snapshot: check disk with path on parse conf: snapshot: align exernal/internal snapshot the same way [2] conf: snapshot: remove snapshot mode checking from disk align [3] conf: virDomainSnapshotAlignDisks: use convinient variable qemu: disable internal snapshot of readonly disk qemu: snapshot: fix for case of disk with empty source qemu: snapshot: align disks consistently [1] conf: snapshot: make disk aligns same on redefinition src/conf/domain_conf.c | 6 +-- src/conf/snapshot_conf.c | 103 ++++++++++++++++++++++------------------------- src/conf/snapshot_conf.h | 5 +-- src/qemu/qemu_driver.c | 27 ++++++++++--- src/test/test_driver.c | 5 +-- 5 files changed, 74 insertions(+), 72 deletions(-) -- 1.8.3.1

2 12

[libvirt] [PULL 1/8] qemu-nbd: Deprecate qemu-nbd --partition
by Eric Blake 05 Feb '19

05 Feb '19

The existing qemu-nbd --partition code claims to handle logical partitions up to 8, since its introduction in 2008 (commit 7a5ca86). However, the implementation is bogus (actual MBR logical partitions form a sort of linked list, with one partition per extended table entry, rather than four logical partitions in a single extended table), making the code unlikely to work for anything beyond -P5 on actual guest images. What's more, the code does not support GPT partitions, which are becoming more popular, and maintaining device subsetting in both NBD and the raw device is unnecessary duplication of effort (even if it is not too difficult). Note that obtaining the offsets of a partition (MBR or GPT) can be learned by using 'qemu-nbd -c /dev/nbd0 file.qcow2 && sfdisk --dump /dev/nbd0', but by the time you've done that, you might as well just mount /dev/nbd0p1 that the kernel creates for you instead of bothering with qemu exporting a subset. Or, keeping to just user-space code, use nbdkit's partition filter, which has already known both GPT and primary MBR partitions for a while, and was just recently enhanced to support arbitrary logical MBR parititions. Start the clock on the deprecation cycle, with examples of how to accomplish device subsetting without using -P. Signed-off-by: Eric Blake <eblake(a)redhat.com> Message-Id: <20190125234837.2272-1-eblake(a)redhat.com> Reviewed-by: Richard W.M. Jones <rjones(a)redhat.com> Reviewed-by: Stefano Garzarella <sgarzare(a)redhat.com> --- qemu-deprecated.texi | 33 +++++++++++++++++++++++++++++++++ qemu-nbd.texi | 6 ++++-- qemu-nbd.c | 2 ++ 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi index 9cc20b365c5..8a6174df0c1 100644 --- a/qemu-deprecated.texi +++ b/qemu-deprecated.texi @@ -160,3 +160,36 @@ Example of legacy encoding: The above, converted to the current supported format: @code{json:@{"file.driver":"rbd", "file.pool":"rbd", "file.image":"name"@}} + +@section Related binaries + +@subsection qemu-nbd --partition (since 4.0.0) + +The ``qemu-nbd --partition $digit'' code (also spelled @option{-P}) +can only handle MBR partitions, and has never correctly handled +logical partitions beyond partition 5. If you know the offset and +length of the partition (perhaps by using @code{sfdisk} within the +guest), you can achieve the effect of exporting just that subset of +the disk by use of the @option{--image-opts} option with a raw +blockdev using the @code{offset} and @code{size} parameters layered on +top of any other existing blockdev. For example, if partition 1 is +100MiB long starting at 1MiB, the old command: + +@code{qemu-nbd -t -P 1 -f qcow2 file.qcow2} + +can be rewritten as: + +@code{qemu-nbd -t --image-opts driver=raw,offset=1M,size=100M,file.driver=qcow2,file.backing.driver=file,file.backing.filename=file.qcow2} + +Alternatively, the @code{nbdkit} project provides a more powerful +partition filter on top of its nbd plugin, which can be used to select +an arbitrary MBR or GPT partition on top of any other full-image NBD +export. Using this to rewrite the above example results in: + +@code{qemu-nbd -t -k /tmp/sock -f qcow2 file.qcow2 &} +@code{nbdkit -f --filter=partition nbd socket=/tmp/sock partition=1} + +Note that if you are exposing the export via /dev/nbd0, it is easier +to just export the entire image and then mount only /dev/nbd0p1 than +it is to reinvoke @command{qemu-nbd -c /dev/nbd0} limited to just a +subset of the image. diff --git a/qemu-nbd.texi b/qemu-nbd.texi index 386bece4680..d0c51828149 100644 --- a/qemu-nbd.texi +++ b/qemu-nbd.texi @@ -56,8 +56,10 @@ auto-detecting. @item -r, --read-only Export the disk as read-only. @item -P, --partition=@var{num} -Only expose MBR partition @var{num}. Understands physical partitions -1-4 and logical partitions 5-8. +Deprecated: Only expose MBR partition @var{num}. Understands physical +partitions 1-4 and logical partition 5. New code should instead use +@option{--image-opts} with the raw driver wrapping a subset of the +original image. @item -B, --bitmap=@var{name} If @var{filename} has a qcow2 persistent bitmap @var{name}, expose that bitmap via the ``qemu:dirty-bitmap:@var{name}'' context diff --git a/qemu-nbd.c b/qemu-nbd.c index 1f7b2a03f5d..00c07fd27ea 100644 --- a/qemu-nbd.c +++ b/qemu-nbd.c @@ -787,6 +787,8 @@ int main(int argc, char **argv) flags &= ~BDRV_O_RDWR; break; case 'P': + warn_report("The '-P' option is deprecated; use --image-opts with " + "a raw device wrapper for subset exports instead"); if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 || partition < 1 || partition > 8) { error_report("Invalid partition '%s'", optarg); -- 2.20.1

1 0

[libvirt] [PATCH RFC] qemu: caps: Don't try to ask for CAP_DAC_OVERRIDE if non-root
by Peter Krempa 04 Feb '19

04 Feb '19

It will not work. This breaks qemu capabilities probing as a user. --- src/qemu/qemu_capabilities.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 7ed5f94803..81ef0357e7 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -4524,7 +4524,8 @@ virQEMUCapsInitQMPCommandRun(virQEMUCapsInitQMPCommandPtr cmd, #if WITH_CAPNG /* QEMU might run into permission issues, e.g. /dev/sev (0600), override * them just for the purpose of probing */ - virCommandAllowCap(cmd->cmd, CAP_DAC_OVERRIDE); + if (geteuid() == 0) + virCommandAllowCap(cmd->cmd, CAP_DAC_OVERRIDE); #endif virCommandSetGID(cmd->cmd, cmd->runGid); -- 2.20.1

3 3

[libvirt] [PATCH] qemu: Refresh state before starting the VCPUs
by Marc Hartmayer 04 Feb '19

04 Feb '19

For normal starts (no incoming migration) the refresh of the QEMU state must be done before the VCPUs getting started since otherwise there might be a race condition between a possible shutdown of the guest OS and the QEMU monitor queries. This fixes "qemu: migration: Refresh device information after transferring state" (93db7eea1b864). Signed-off-by: Marc Hartmayer <mhartmay(a)linux.ibm.com> --- src/qemu/qemu_process.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index dace5aaca102..2a3763f40d49 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6929,10 +6929,17 @@ qemuProcessStart(virConnectPtr conn, } relabel = true; - if (incoming && - incoming->deferredURI && - qemuMigrationDstRun(driver, vm, incoming->deferredURI, asyncJob) < 0) - goto stop; + if (incoming) { + if (incoming->deferredURI && + qemuMigrationDstRun(driver, vm, incoming->deferredURI, asyncJob) < 0) + goto stop; + } else { + /* Refresh state of devices from QEMU. During migration this + * needs to happen after the state information is fully + * transferred. */ + if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) + goto stop; + } if (qemuProcessFinishStartup(driver, vm, asyncJob, !(flags & VIR_QEMU_PROCESS_START_PAUSED), @@ -6945,11 +6952,6 @@ qemuProcessStart(virConnectPtr conn, /* Keep watching qemu log for errors during incoming migration, otherwise * unset reporting errors from qemu log. */ qemuMonitorSetDomainLog(priv->mon, NULL, NULL, NULL); - - /* Refresh state of devices from qemu. During migration this needs to - * happen after the state information is fully transferred. */ - if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) - goto stop; } ret = 0; -- 2.17.0

3 6

[libvirt] [jenkins-ci PATCH 0/2] guests: Install isoinfo for virt-manager
by Andrea Bolognani 04 Feb '19

04 Feb '19

Andrea Bolognani (2): guests: Add mapping for isoinfo guests: Install isoinfo for virt-manager guests/vars/mappings.yml | 4 ++++ guests/vars/projects/virt-manager.yml | 1 + 2 files changed, 5 insertions(+) -- 2.20.1

2 3

[libvirt] [PATCH v2] qemu: Assume migration with a network disk migration is safe
by Michal Privoznik 04 Feb '19

04 Feb '19

If a domain has a disk that is type='network' we require specific cache mode to allow migration with it (either 'directsync' or 'none'). This doesn't make much sense since network disks are supposed to be safe to migrate by default. At the same time, we should be checking for the actual source type, not apparent type set in the domain XML. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- v2 of: https://www.redhat.com/archives/libvir-list/2019-January/msg01070.html diff to v1: - Allow just every network disk - Check for actual disk type not the apparent one src/qemu/qemu_migration.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 52aa665640..1433b2c2f3 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1236,7 +1236,7 @@ qemuMigrationSrcIsSafe(virDomainDefPtr def, continue; /* However, disks on local FS (e.g. ext4) are not safe. */ - if (virDomainDiskGetType(disk) == VIR_STORAGE_TYPE_FILE) { + if (virStorageSourceGetActualType(disk->src) == VIR_STORAGE_TYPE_FILE) { if ((rc = virFileIsSharedFS(src)) < 0) { return false; } else if (rc == 0) { @@ -1248,8 +1248,8 @@ qemuMigrationSrcIsSafe(virDomainDefPtr def, return false; else if (rc == 1) continue; - } else if (disk->src->type == VIR_STORAGE_TYPE_NETWORK && - disk->src->protocol == VIR_STORAGE_NET_PROTOCOL_RBD) { + } else if (virStorageSourceGetActualType(disk->src) == VIR_STORAGE_TYPE_NETWORK) { + /* But network disks are safe again. */ continue; } -- 2.19.2

2 1

[libvirt] [PATCH 0/3] scsi-disk: Device Identification fixes
by Kevin Wolf 04 Feb '19

04 Feb '19

The vendor specific designator in the Device Identification VPD page has two problems: 1. It defaults to the BlockBackend name (-drive id=...), which everyone expected to be a host detail that the guest never sees 2. With -blockdev based setups it defaults to an empty string; if this default is used with more than one disk, the guest OS will interpret this as a single multipath disk. We can address problem 2 immediately, and start running the deprecation clock for problem 1. Related bug reports: https://bugzilla.redhat.com/show_bug.cgi?id=1669446 https://bugzilla.redhat.com/show_bug.cgi?id=1668248 Kevin Wolf (3): scsi-disk: Don't use empty string as device id scsi-disk: Add device_id property scsi-disk: Deprecate device_id fallback to BlockBackend name hw/scsi/scsi-disk.c | 41 ++++++++++++++++++++++++++++------------- qemu-deprecated.texi | 22 +++++++++++++++++++++- 2 files changed, 49 insertions(+), 14 deletions(-) -- 2.20.1

6 19