September 2014 - Devel - libvirt List Archives

[libvirt] ANNOUNCE: libguestfs 1.26 released
by Richard W.M. Jones 10 Dec '14

10 Dec '14

I'm pleased to announce libguestfs 1.26, a library and set of tools for accessing and modifying virtual machine disk images. This release took more than 6 months of work by a considerable number of people, and has many new features (see release notes below). You can get libguestfs 1.26 here: Main website: http://libguestfs.org/ Source: http://libguestfs.org/download/1.26-stable/ You will also need latest supermin from here: http://libguestfs.org/download/supermin/ Fedora 20/21: http://koji.fedoraproject.org/koji/packageinfo?packageID=8391 It will appear as an update for F20 in about a week. Debian/experimental coming soon, see: https://packages.debian.org/experimental/libguestfs0 The Fedora and Debian packages have split dependencies so you can download just the features you need. >From http://libguestfs.org/guestfs-release-notes.1.html : RELEASE NOTES FOR LIBGUESTFS 1.26 New features Tools virt-customize(1) is a new tool for customizing virtual machine disk images. It lets you install packages, edit configuration files, run scripts, set passwords and so on. virt-builder(1) and virt-sysprep(1) use virt-customize, and command line options across all these tools are now identical. virt-diff(1) is a new tool for showing the differences between the filesystems of two virtual machines. It is mainly useful when showing what files have been changed between snapshots. virt-builder(1) has been greatly enhanced. There are many more ways to customize the virtual machine. It can pull templates from multiple repositories. A parallelized internal xzcat implementation speeds up template decompression. Virt-builder uses an optimizing planner to choose the fastest way to build the VM. It is now easier to use virt-builder from other programs. Internationalization support has been added to metadata. More efficient SELinux relabelling of files. Can build guests for multiple architectures. Error messages have been improved. (Pino Toscano) virt-sparsify(1) has a new --in-place option. This sparsifies an image in place (without copying it) and is also much faster. (Lots of help provided by Paolo Bonzini) virt-sysprep(1) can delete and scrub files under user control. You can lock user accounts or set random passwords on accounts. Can remove more log files. Can unsubscribe a guest from Red Hat Subscription Manager. New flexible way to enable and disable operations. (Wanlong Gao, Pino Toscano) virt-win-reg(1) allows you to use URIs to specify remote disk images. virt-format(1) can now pass the extra space that it recovers back to the host. guestfish(1) has additional environment variables to give fine control over the ><fs> prompt. Guestfish reads its (rarely used) configuration file in a different order now so that local settings override global settings. (Pino Toscano) virt-make-fs(1) was rewritten in C, but is unchanged in terms of functionality and command line usage. Language bindings The OCaml bindings have a new Guestfs.Errno module, used to check the error number returned by Guestfs.last_errno. PHP tests now work. (Pino Toscano) Inspection Inspection can recognize Debian live images. Architectures ARMv7 (32 bit) now supports KVM acceleration. Aarch64 (ARM 64 bit) is supported, but the appliance part does not work yet. PPC64 support has been fixed and enhanced. Security Denial of service when inspecting disk images with corrupt btrfs volumes It was possible to crash libguestfs (and programs that use libguestfs as a library) by presenting a disk image containing a corrupt btrfs volume. This was caused by a NULL pointer dereference causing a denial of service, and is not thought to be exploitable any further. See commit d70ceb4cbea165c960710576efac5a5716055486 for the fix. This fix is included in libguestfs stable branches ≥ 1.26.0, ≥ 1.24.6 and ≥ 1.22.8, and also in RHEL ≥ 7.0. Earlier versions of libguestfs are not vulnerable. Better generation of random root passwords and random seeds When generating random root passwords and random seeds, two bugs were fixed which are possibly security related. Firstly we no longer read excessive bytes from /dev/urandom (most of which were just thrown away). Secondly we changed the code to avoid modulo bias. These issues were not thought to be exploitable. (Both changes suggested by Edwin Török) API GUID parameters are now validated when they are passed to API calls, whereas previously you could have passed any string. (Pino Toscano) New APIs guestfs_add_drive_opts: new discard parameter The new discard parameter allows fine-grained control over discard/trim support for a particular disk. This allows the host file to become more sparse (or thin-provisioned) when you delete files or issue the guestfs_fstrim API call. guestfs_add_domain: new parameters: cachemode, discard These parameters are passed through when adding the domain's disks. guestfs_blkdiscard Discard all blocks on a guestfs device. Combined with the discard parameter above, this makes the host file sparse. guestfs_blkdiscardzeroes Test if discarded blocks read back as zeroes. guestfs_compare_* guestfs_copy_* For each struct returned through the API, libguestfs now generates guestfs_compare_* and guestfs_copy_* functions to allow you to compare and copy structs. guestfs_copy_attributes Copy attributes (like permissions, xattrs, ownership) from one file to another. (Pino Toscano) guestfs_disk_create A flexible API for creating empty disk images from scratch. This avoids the need to call out to external programs like qemu-img(1). guestfs_get_backend_settings guestfs_set_backend_settings Per-backend settings (can also be set via the environment variable LIBGUESTFS_BACKEND_SETTINGS). The main use for this is forcing TCG mode in the qemu-based backends, for example: export LIBGUESTFS_BACKEND=direct export LIBGUESTFS_BACKEND_SETTINGS=force_tcg guestfs_part_get_name Get the label or name of a partition (for GPT disk images). Build changes The following extra packages are required to build libguestfs 1.26: supermin ≥ 5 Supermin version 5 is required to build this version of libguestfs. flex, bison Virt-builder now uses a real parser to parse its metadata file, so these tools are required. xz This is now a required build dependency, where previously it was (in theory) optional. Internals PO message extraction rewritten to be more robust. (Pino Toscano) podwrapper gives an error if the --insert or --verbatim argument pattern is not found. Libguestfs now passes the qemu -enable-fips option to enable FIPS, if qemu supports it. ./configure --without-qemu can be used if you don't want to specify a default hypervisor. Copy-on-write [COW] overlays, used for example for read-only drives, are now created through an internal backend API (.create_cow_overlay). Libvirt backend uses some funky C macros to generate XML. These are simpler and safer. The ChangeLog file format has changed. It is now just the same as git log, instead of using a custom format. Appliance start-up has changed: * The libguestfs appliance now initializes LVM the same way as it is done on physical machines. * The libguestfs appliance does not write an empty string to /proc/sys/kernel/hotplug when starting up. Note that you must configure your kernel to have CONFIG_UEVENT_HELPER_PATH="" otherwise you will get strange LVM errors (this applies as much to any Linux machine, not just libguestfs). (Peter Rajnoha) Libguestfs can now be built on arches that have ocamlc(1) but not ocamlopt(1). (Hilko Bengen, Olaf Hering) You cannot use ./configure --disable-daemon --enable-appliance. It made no sense anyway. Now it is expressly forbidden by the configure script. The packagelist file uses m4 for macro expansion instead of cpp. Bugs fixed https://bugzilla.redhat.com/1073906 java bindings inspect_list_applications2 throws java.lang.ArrayIndexOutOfBoundsException: https://bugzilla.redhat.com/1063374 [RFE] enable subscription manager clean or unregister operation to sysprep https://bugzilla.redhat.com/1060404 virt-resize does not preserve GPT partition names https://bugzilla.redhat.com/1057504 mount-local should give a clearer error if root is not mounted https://bugzilla.redhat.com/1056290 virt-sparsify overwrites block devices if used as output files https://bugzilla.redhat.com/1055452 libguestfs: error: invalid backend: appliance https://bugzilla.redhat.com/1054761 guestfs_pvs prints "unknown device" if a physical volume is missing https://bugzilla.redhat.com/1053847 Recommended default clock/timer settings https://bugzilla.redhat.com/1046509 ruby-libguestfs throws "expecting 0 or 1 arguments" on Guestfs::Guestfs.new https://bugzilla.redhat.com/1045450 Cannot inspect cirros 0.3.1 disk image fully https://bugzilla.redhat.com/1045033 LIBVIRT_DEFAULT_URI=qemu:///system breaks libguestfs https://bugzilla.redhat.com/1044585 virt-builder network (eg. --install) doesn't work if resolv.conf sets nameserver 127.0.0.1 https://bugzilla.redhat.com/1044014 When SSSD is installed, libvirt configuration requires authentication, but not clear to user https://bugzilla.redhat.com/1039995 virt-make-fs fails making fat/vfat whole disk: Device partition expected, not making filesystem on entire device '/dev/sda' (use -I to override) https://bugzilla.redhat.com/1039540 virt-sysprep to delete more logfiles https://bugzilla.redhat.com/1033207 RFE: libguestfs inspection does not recognize Free4NAS live CD https://bugzilla.redhat.com/1028660 RFE: virt-sysprep/virt-builder should have an option to lock a user account https://bugzilla.redhat.com/1026688 libguestfs fails examining libvirt guest with ceph drives: rbd: image name must begin with a '/' https://bugzilla.redhat.com/1022431 virt-builder fails if $HOME/.cache doesn't exist https://bugzilla.redhat.com/1022184 libguestfs: do not use versioned jar file https://bugzilla.redhat.com/1020806 All libguestfs LVM operations fail on Debian/Ubuntu https://bugzilla.redhat.com/1008417 Need update helpout of part-set-gpt-type https://bugzilla.redhat.com/953907 virt-sysprep does not correctly set the hostname on Debian/Ubuntu https://bugzilla.redhat.com/923355 guestfish prints literal "\n" in error messages https://bugzilla.redhat.com/660687 guestmount: "touch" command fails: touch: setting times of `timestamp': Invalid argument https://bugzilla.redhat.com/593511 [RFE] function to get partition name https://bugzilla.redhat.com/563450 list-devices returns devices of different types out of order --- Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

2 5

[libvirt] ANNOUNCE: virt-manager 1.1.0 released
by Cole Robinson 03 Dec '14

03 Dec '14

I'm happy to announce the release of virt-manager 1.1.0! virt-manager is a desktop application for managing KVM, Xen, and LXC virtualization via libvirt. The release can be downloaded from: http://virt-manager.org/download/ The direct download links are: http://virt-manager.org/download/sources/virt-manager/virt-manager-1.1.0.ta… This release includes: - Switch to libosinfo as OS metadata database (Giuseppe Scrivano) - Use libosinfo for OS detection from CDROM media labels (Giuseppe Scrivano) - Use libosinfo for improved OS defaults, like recommended disk size (Giuseppe Scrivano) - virt-image tool has been removed, as previously announced - Enable Hyper-V enlightenments for Windows VMs - Revert virtio-console default, back to plain serial console - Experimental q35 option in new VM 'customize' dialog - UI for virtual network QoS settings (Giuseppe Scrivano) - virt-install: --disk discard= support (Jim Minter) - addhardware: Add spiceport UI (Marc-André Lureau) - virt-install: --events on_poweroff etc. support (Chen Hanxiao) - cli --network portgroup= support and UI support - cli --boot initargs= and UI support - addhardware: allow setting controller model (Chen Hanxiao) - virt-install: support setting hugepage options (Chen Hanxiao) Thanks to everyone who has contributed to this release through testing, bug reporting, submitting patches, and otherwise sending in feedback! Thanks, Cole

2 7

Re: [libvirt] [GIT PULL] namespace updates for v3.17-rc1
by Richard Weinberger 30 Nov '14

30 Nov '14

On Wed, Aug 6, 2014 at 2:57 AM, Eric W. Biederman <ebiederm(a)xmission.com> wrote: > > Linus, > > Please pull the for-linus branch from the git tree: > > git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git for-linus > > HEAD: 344470cac42e887e68cfb5bdfa6171baf27f1eb5 proc: Point /proc/mounts at /proc/thread-self/mounts instead of /proc/self/mounts > > This is a bunch of small changes built against 3.16-rc6. The most > significant change for users is the first patch which makes setns > drmatically faster by removing unneded rcu handling. > > The next chunk of changes are so that "mount -o remount,.." will not > allow the user namespace root to drop flags on a mount set by the system > wide root. Aks this forces read-only mounts to stay read-only, no-dev > mounts to stay no-dev, no-suid mounts to stay no-suid, no-exec mounts to > stay no exec and it prevents unprivileged users from messing with a > mounts atime settings. I have included my test case as the last patch > in this series so people performing backports can verify this change > works correctly. > > The next change fixes a bug in NFS that was discovered while auditing > nsproxy users for the first optimization. Today you can oops the kernel > by reading /proc/fs/nfsfs/{servers,volumes} if you are clever with pid > namespaces. I rebased and fixed the build of the !CONFIG_NFS_FS case > yesterday when a build bot caught my typo. Given that no one to my > knowledge bases anything on my tree fixing the typo in place seems more > responsible that requiring a typo-fix to be backported as well. > > The last change is a small semantic cleanup introducing > /proc/thread-self and pointing /proc/mounts and /proc/net at it. This > prevents several kinds of problemantic corner cases. It is a > user-visible change so it has a minute chance of causing regressions so > the change to /proc/mounts and /proc/net are individual one line commits > that can be trivially reverted. Unfortunately I lost and could not find > the email of the original reporter so he is not credited. From at least > one perspective this change to /proc/net is a refgression fix to allow > pthread /proc/net uses that were broken by the introduction of the network > namespace. > > Eric > > Eric W. Biederman (11): > namespaces: Use task_lock and not rcu to protect nsproxy > mnt: Only change user settable mount flags in remount > mnt: Move the test for MNT_LOCK_READONLY from change_mount_flags into do_remount > mnt: Correct permission checks in do_remount This commit breaks libvirt-lxc. libvirt does in lxcContainerMountBasicFS(): /* * We can't immediately set the MS_RDONLY flag when mounting filesystems * because (in at least some kernel versions) this will propagate back * to the original mount in the host OS, turning it readonly too. Thus * we mount the filesystem in read-write mode initially, and then do a * separate read-only bind mount on top of that. */ bindOverReadonly = !!(mnt_mflags & MS_RDONLY); VIR_DEBUG("Mount %s on %s type=%s flags=%x", mnt_src, mnt->dst, mnt->type, mnt_mflags & ~MS_RDONLY); if (mount(mnt_src, mnt->dst, mnt->type, mnt_mflags & ~MS_RDONLY, NULL) < 0) { ^^^^ Here it fails for sysfs because with user namespaces we bind the existing /sys into the container and would have to read out all existing mount flags from the current /sys mount. Otherwise mount() fails with EPERM. On my test system /sys is mounted with "rw,nosuid,nodev,noexec,relatime" and libvirt misses the realtime... virReportSystemError(errno, _("Failed to mount %s on %s type %s flags=%x"), mnt_src, mnt->dst, NULLSTR(mnt->type), mnt_mflags & ~MS_RDONLY); goto cleanup; } if (bindOverReadonly && mount(mnt_src, mnt->dst, NULL, MS_BIND|MS_REMOUNT|MS_RDONLY, NULL) < 0) { ^^^ Here it fails because now we'd have to specify all flags as used for the first mount. For the procfs case MS_NOSUID|MS_NOEXEC|MS_NODEV. See lxcBasicMounts[]. In this case the fix is easy, add mnt_mflags to the mount flags. virReportSystemError(errno, _("Failed to re-mount %s on %s flags=%x"), mnt_src, mnt->dst, MS_BIND|MS_REMOUNT|MS_RDONLY); goto cleanup; } -- Thanks, //richard

5 12

[libvirt] [RFC: PATCH 0/2] Display allocation during dumpxml
by Eric Blake 18 Nov '14

18 Nov '14

I'm still working on code to populate the latest numbers for each disk of a domain, including getting numbers for offline domains, but have confirmed that with these two patches alone I'm able to see <capacity> and <allocation> numbers for block volumes of live domains (thanks to how we populate backing chain information). So while there are more patches to come, I'd like to get review started on my proposed API addition. Eric Blake (2): dumpxml: add flag to virDomainGetXMLDesc dumpxml: prepare to output block info docs/schemas/domaincommon.rng | 22 ++++++++++++++++++++++ include/libvirt/libvirt.h.in | 1 + src/conf/domain_conf.c | 16 +++++++++++++++- src/libvirt.c | 15 +++++++++++---- src/util/virstoragefile.h | 3 ++- tools/virsh-domain.c | 6 ++++++ tools/virsh.pod | 6 ++++-- 7 files changed, 61 insertions(+), 8 deletions(-) -- 1.9.3

2 4

[libvirt] [PATCH v3] leaseshelper: improvements to support all events
by Nehal J Wani 18 Nov '14

18 Nov '14

This patch enables the helper program to detect event(s) triggered when there is a change in lease length or expiry and client-id. This transfers complete control of leases database to libvirt and obsoletes use of the lease database file (<network-name>.leases). That file will not be created, read, or written. This is achieved by adding the option --leasefile-ro to dnsmasq and passing a custom env var to leaseshelper, which helps us map events related to leases with their corresponding network bridges, no matter what the event be. Also, this requires the addition of a new non-lease entry in our custom lease database: "server-duid". It is required to identify a DHCPv6 server. Now that dnsmasq doesn't maintain its own leases database, it relies on our helper program to tell it about previous leases and server duid. Thus, this patch makes our leases program honor an extra action: "init", in which it sends the known info in a particular format to dnsmasq by printing it to stdout. --- This is compatible with libvirt 1.2.6 as only additions have been introduced, and the old leases file (*.status) will still be supported. v3: * Add server-duid as an entry in the lease object for every ipv6 lease. * Remove unnecessary variables and double copies. * Take value from DNSMASQ_OLD_HOSTNAME if hostname is not known. v2: http://www.redhat.com/archives/libvir-list/2014-July/msg01109.html v1: https://www.redhat.com/archives/libvir-list/2014-July/msg00568.html src/network/bridge_driver.c | 3 + src/network/leaseshelper.c | 132 +++++++++++++++++++++++++++++++++++--------- 2 files changed, 109 insertions(+), 26 deletions(-) diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index 965fdec..b578b3a 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -1288,7 +1288,10 @@ networkBuildDhcpDaemonCommandLine(virNetworkObjPtr network, cmd = virCommandNew(dnsmasqCapsGetBinaryPath(caps)); virCommandAddArgFormat(cmd, "--conf-file=%s", configfile); + /* Libvirt gains full control of leases database */ + virCommandAddArgFormat(cmd, "--leasefile-ro"); virCommandAddArgFormat(cmd, "--dhcp-script=%s", leaseshelper_path); + virCommandAddEnvPair(cmd, "VIR_BRIDGE_NAME", network->def->bridge); *cmdout = cmd; ret = 0; diff --git a/src/network/leaseshelper.c b/src/network/leaseshelper.c index c8543a2..e984cbb 100644 --- a/src/network/leaseshelper.c +++ b/src/network/leaseshelper.c @@ -50,6 +50,12 @@ */ #define VIR_NETWORK_DHCP_LEASE_FILE_SIZE_MAX (32 * 1024 * 1024) +/* + * Use this when passing possibly-NULL strings to printf-a-likes. + * Required for unknown parameters during init call. + */ +#define EMPTY_STR(s) ((s) ? (s) : "*") + static const char *program_name; /* Display version information. */ @@ -65,7 +71,7 @@ usage(int status) if (status) { fprintf(stderr, _("%s: try --help for more details\n"), program_name); } else { - printf(_("Usage: %s add|old|del mac|clientid ip [hostname]\n" + printf(_("Usage: %s add|old|del|init mac|clientid ip [hostname]\n" "Designed for use with 'dnsmasq --dhcp-script'\n" "Refer to man page of dnsmasq for more details'\n"), program_name); @@ -89,6 +95,7 @@ enum virLeaseActionFlags { VIR_LEASE_ACTION_ADD, /* Create new lease */ VIR_LEASE_ACTION_OLD, /* Lease already exists, renew it */ VIR_LEASE_ACTION_DEL, /* Delete the lease */ + VIR_LEASE_ACTION_INIT, /* Tell dnsmasq of existing leases on restart */ VIR_LEASE_ACTION_LAST }; @@ -96,7 +103,7 @@ enum virLeaseActionFlags { VIR_ENUM_DECL(virLeaseAction); VIR_ENUM_IMPL(virLeaseAction, VIR_LEASE_ACTION_LAST, - "add", "old", "del"); + "add", "old", "del", "init"); int main(int argc, char **argv) @@ -112,20 +119,24 @@ main(int argc, char **argv) const char *interface = virGetEnvAllowSUID("DNSMASQ_INTERFACE"); const char *exptime_tmp = virGetEnvAllowSUID("DNSMASQ_LEASE_EXPIRES"); const char *hostname = virGetEnvAllowSUID("DNSMASQ_SUPPLIED_HOSTNAME"); + const char *server_duid = virGetEnvAllowSUID("DNSMASQ_SERVER_DUID"); const char *leases_str = NULL; long long currtime = 0; long long expirytime = 0; size_t i = 0; + size_t count_ipv6 = 0; + size_t count_ipv4 = 0; int action = -1; int pid_file_fd = -1; int rv = EXIT_FAILURE; int custom_lease_file_len = 0; - bool add = false; bool delete = false; virJSONValuePtr lease_new = NULL; virJSONValuePtr lease_tmp = NULL; virJSONValuePtr leases_array = NULL; virJSONValuePtr leases_array_new = NULL; + virJSONValuePtr *leases_ipv4 = NULL; + virJSONValuePtr *leases_ipv6 = NULL; virSetErrorFunc(NULL, NULL); virSetErrorLogPriorityFunc(NULL); @@ -156,16 +167,17 @@ main(int argc, char **argv) } } - if (argc != 4 && argc != 5) { + if (argc != 4 && argc != 5 && argc != 2) { /* Refer man page of dnsmasq --dhcp-script for more details */ usage(EXIT_FAILURE); } /* Make sure dnsmasq knows the interface. The interface name is not known - * when dnsmasq (re)starts and throws 'del' events for expired leases. - * So, if any old lease has expired, it will be automatically removed the - * next time this program is invoked */ - if (!interface) + * via env variable set by dnsmasq when dnsmasq (re)starts and throws 'del' + * events for expired leases. So, libvirtd sets another env var for this + * purpose */ + if (!interface && + !(interface = virGetEnvAllowSUID("VIR_BRIDGE_NAME"))) goto cleanup; ip = argv[3]; @@ -176,6 +188,10 @@ main(int argc, char **argv) if (argc == 5) hostname = argv[4]; + /* In case hostname is still unkown, use the last known one */ + if (!hostname) + hostname = virGetEnvAllowSUID("DNSMASQ_OLD_HOSTNAME"); + if (VIR_STRDUP(exptime, exptime_tmp) < 0) goto cleanup; @@ -185,7 +201,7 @@ main(int argc, char **argv) exptime[strlen(exptime) - 1] = '\0'; /* Check if it is an IPv6 lease */ - if (virGetEnvAllowSUID("DNSMASQ_IAID")) { + if (iaid) { mac = virGetEnvAllowSUID("DNSMASQ_MAC"); clientid = argv[2]; } @@ -235,7 +251,6 @@ main(int argc, char **argv) delete = true; if (action == VIR_LEASE_ACTION_ADD || action == VIR_LEASE_ACTION_OLD) { - add = true; /* Create new lease */ if (!(lease_new = virJSONValueNewObject())) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -260,11 +275,13 @@ main(int argc, char **argv) goto cleanup; if (clientid && virJSONValueObjectAppendString(lease_new, "client-id", clientid) < 0) goto cleanup; + if (server_duid && virJSONValueObjectAppendString(lease_new, "server-duid", server_duid) < 0) + goto cleanup; if (expirytime && virJSONValueObjectAppendNumberLong(lease_new, "expiry-time", expirytime) < 0) goto cleanup; } } - } else { + } else if (action != VIR_LEASE_ACTION_INIT) { fprintf(stderr, _("Unsupported action: %s\n"), virLeaseActionTypeToString(action)); exit(EXIT_FAILURE); @@ -294,7 +311,7 @@ main(int argc, char **argv) i = 0; while (i < virJSONValueArraySize(leases_array)) { const char *ip_tmp = NULL; - long long expirytime_tmp = -1; + const char *server_duid_tmp = NULL; if (!(lease_tmp = virJSONValueArrayGet(leases_array, i))) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -303,14 +320,13 @@ main(int argc, char **argv) } if (!(ip_tmp = virJSONValueObjectGetString(lease_tmp, "ip-address")) || - (virJSONValueObjectGetNumberLong(lease_tmp, "expiry-time", &expirytime_tmp) < 0)) { + (virJSONValueObjectGetNumberLong(lease_tmp, "expiry-time", &expirytime) < 0)) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to parse json")); goto cleanup; } - /* Check whether lease has expired or not */ - if (expirytime_tmp < currtime) { + if (expirytime < currtime) { i++; continue; } @@ -321,6 +337,30 @@ main(int argc, char **argv) continue; } + /* Store pointers to ipv4 and ipv6 leases */ + if (strchr(ip_tmp, ':')) { + /* This is an ipv6 lease */ + ignore_value(VIR_APPEND_ELEMENT_COPY(leases_ipv6, count_ipv6, lease_tmp)); + if ((server_duid_tmp + = virJSONValueObjectGetString(lease_tmp, "server-duid"))) { + if (!server_duid) { + /* Control reaches here when the 'action' is not for an + * ipv6 lease or, for some weird reason the env var + * DNSMASQ_SERVER_DUID wasn't set*/ + server_duid = server_duid_tmp; + } + } else { + /* Inject server-duid into those ipv6 leases which + * didn't have it previously, for example, those + * created by leaseshelper from libvirt 1.2.6 */ + if (virJSONValueObjectAppendString(lease_tmp, "server-duid", server_duid) < 0) + goto cleanup; + } + } else { + /* This is an ipv4 lease */ + ignore_value(VIR_APPEND_ELEMENT_COPY(leases_ipv4, count_ipv4, lease_tmp)); + } + /* Move old lease to new array */ if (virJSONValueArrayAppend(leases_array_new, lease_tmp) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -333,31 +373,71 @@ main(int argc, char **argv) } } - if (add) { + switch ((enum virLeaseActionFlags) action) { + case VIR_LEASE_ACTION_INIT: + /* Man page of dnsmasq says: the script (helper program, in our case) + * should write the saved state of the lease database, in dnsmasq + * leasefile format, to stdout and exit with zero exit code, when + * called with argument init. Format: + * $expirytime $mac $ip $hostname $clientid # For all ipv4 leases + * duid $server-duid # If DHCPv6 is present + * $expirytime $iaid $ip $hostname $clientduid # For all ipv6 leases */ + for (i = 0; i < count_ipv4; i++) { + lease_tmp = leases_ipv4[i]; + virJSONValueObjectGetNumberLong(lease_tmp, "expiry-time", &expirytime); + printf("%lld %s %s %s %s\n", + expirytime, + virJSONValueObjectGetString(lease_tmp, "mac-address"), + virJSONValueObjectGetString(lease_tmp, "ip-address"), + EMPTY_STR(virJSONValueObjectGetString(lease_tmp, "hostname")), + EMPTY_STR(virJSONValueObjectGetString(lease_tmp, "client-id"))); + } + if (server_duid) { + printf("duid %s\n", server_duid); + for (i = 0; i < count_ipv6; i++) { + lease_tmp = leases_ipv6[i]; + virJSONValueObjectGetNumberLong(lease_tmp, "expiry-time", &expirytime); + printf("%lld %s %s %s %s\n", + expirytime, + virJSONValueObjectGetString(lease_tmp, "iaid"), + virJSONValueObjectGetString(lease_tmp, "ip-address"), + EMPTY_STR(virJSONValueObjectGetString(lease_tmp, "hostname")), + EMPTY_STR(virJSONValueObjectGetString(lease_tmp, "client-id"))); + } + } + break; + + case VIR_LEASE_ACTION_OLD: + case VIR_LEASE_ACTION_ADD: if (virJSONValueArrayAppend(leases_array_new, lease_new) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to create json")); goto cleanup; } lease_new = NULL; - } - if (!(leases_str = virJSONValueToString(leases_array_new, true))) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("empty json array")); - goto cleanup; - } + default: + if (!(leases_str = virJSONValueToString(leases_array_new, true))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("empty json array")); + goto cleanup; + } - /* Write to file */ - if (virFileRewrite(custom_lease_file, 0644, - customLeaseRewriteFile, &leases_str) < 0) - goto cleanup; + /* Write to file */ + if (virFileRewrite(custom_lease_file, 0644, + customLeaseRewriteFile, &leases_str) < 0) + goto cleanup; + } rv = EXIT_SUCCESS; cleanup: if (pid_file_fd != -1) virPidFileReleasePath(pid_file, pid_file_fd); + for (i = 0; i < count_ipv4; i++) + VIR_FREE(leases_ipv4); + for (i = 0; i < count_ipv6; i++) + VIR_FREE(leases_ipv6); VIR_FREE(pid_file); VIR_FREE(exptime); -- 1.9.3

4 8

[libvirt] [PATCH v2 0/8] Post-copy live migration support
by Cristian Klein 06 Nov '14

06 Nov '14

Qemu currently implements pre-copy live migration. VM memory pages are first copied from the source hypervisor to the destination, potentially multiple times as pages get dirtied during transfer, then VCPU state is migrated. Unfortunately, if the VM dirties memory faster than the network bandwidth, then pre-copy cannot finish. `virsh` currently includes an option to suspend a VM after a timeout, so that migration may finish, but at the expense of downtime. A future version of qemu will implement post-copy live migration. The VCPU state is first migrated to the destination hypervisor, then memory pages are pulled from the source hypervisor. Post-copy has the potential to do migration with zero-downtime, despite the VM dirtying pages fast, with minimum performance impact. On the other hand, one post-copy is in progress, any network failure would render the VM unusable, as its memory is partitioned between the source and destination hypervisor. Therefore, post-copy should only be used when necessary. Post-copy migration in qemu will work as follows: (1) The `x-postcopy-ram` migration capability needs to be set. (2) Migration is started. (3) When the user decides so, post-copy migration is activated by sending the `migrate-start-postcopy` command. Qemu acknowledges by setting migration status to `postcopy-active`. v2: - Fixed formatting - Set target version to libvirt 1.2.10 - Only use JSON monitor - Renamed `qemuMigrateStartPostCopy` to `qemuDomainMigrateStartPostCopy` - Added parameter `flags` to domainMigrateStartPostCopy (currently unused) - Misc fixes required for `make check` - Stop perform phase, when post-copy starts - Wait for post-copy completion in confirm phase, before killing source VM Implementation note: `qemuMigrationWaitForCompletion` is overloaded. When called the first time it waits for post-copy to start, when called the second time it waits for post-copy to complete. I did so to reduce code duplication, but am not sure this is the best solution. Cristian Klein (8): Added public API to enable post-copy migration Added public API to start post-copy migration Added low-level API to qemu post-copy migration Implemented VIR_MIGRATE_POSTCOPY in qemu driver Added job type VIR_DOMAIN_JOB_PHASE1_COMPLETED Implemented post-copy migration logic in qemu Implement virDomainMigrateStartPostCopy in qemu virsh: add postcopy-after option to migrate command include/libvirt/libvirt.h.in | 5 +++ src/driver.h | 5 +++ src/libvirt.c | 46 ++++++++++++++++++++++++ src/libvirt_public.syms | 5 +++ src/qemu/qemu_driver.c | 58 ++++++++++++++++++++++++++++++ src/qemu/qemu_migration.c | 85 ++++++++++++++++++++++++++++++++++++++++++-- src/qemu/qemu_migration.h | 3 +- src/qemu/qemu_monitor.c | 24 +++++++++++-- src/qemu/qemu_monitor.h | 4 +++ src/qemu/qemu_monitor_json.c | 23 +++++++++++- src/qemu/qemu_monitor_json.h | 1 + src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 13 ++++++- src/remote_protocol-structs | 5 +++ tools/virsh-domain.c | 75 ++++++++++++++++++++++++++++++++++++-- tools/virsh.pod | 5 +++ 16 files changed, 347 insertions(+), 11 deletions(-) -- 1.9.1

6 29

[libvirt] NBD TLS support in QEMU
by Stefan Hajnoczi 03 Nov '14

03 Nov '14

Hi, QEMU offers both NBD client and server functionality. The NBD protocol runs unencrypted, which is a problem when the client and server communicate over an untrusted network. The particular use case that prompted this mail is storage migration in OpenStack. The goal is to encrypt the NBD connection between source and destination hosts during storage migration. I think we can integrate TLS into the NBD protocol as an optional flag. A quick web search does not reveal existing open source SSL/TLS NBD implementations. I do see a VMware NBDSSL protocol but there is no specification so I guess it is proprietary. The NBD protocol starts with a negotiation phase. This would be the appropriate place to indicate that TLS will be used. After client and server complete TLS setup the connection can continue as normal. Besides QEMU, the userspace NBD tools (http://nbd.sf.net/) can also be extended to support TLS. In this case the kernel needs a localhost socket and userspace handles TLS. Thoughts? Stefan

14 44

[libvirt] [libvirt-java] [PATCH 00/65]
by Claudio Bley 03 Nov '14

03 Nov '14

Hi. Here are a few patches that piled up in my local branch. Some of them I already submitted to this list, but there has been no reponse to them. Included are a few trivial fixes as well as memory leak fixes and additions to the public API. There had been some minor interest in my first version of domain event support in the Java wrapper[1], which I have reworked almost entirely. If nobody objects, say, within the next two weeks or so, I'll go ahead and push the whole series as was suggested to me by Daniel P. Berrange on this list[2]. I've made sure that all JUnit tests pass after every commit with JNA versions 3.4.1, 3.4.2, 3.5.0, 3.5.1, 3.5.2 and 4.0.0. I'd really appreciate your comments or suggestions. Thanks! [1]: https://www.redhat.com/archives/libvir-list/2013-January/msg01236.html [2]: https://www.redhat.com/archives/libvir-list/2014-January/msg01091.html Claudio Bley (65): Fix warnings about using raw types Fix warnings about accessing static methods Fix typos in Error.java test: fix typo in testConnection() test: ensure that exceptions are thrown when expected Make comments proper javadoc comments for enum constants Ignore editor backup files Depend on JNA versions 3.4.1 to 4.0.0 jna: load virt-0 or virt library depending on the platform Fix wrapping of native size_t data type Use virFree in order to release memory acquired from libvirt tests: remove obsolete test driver Make Device.listCapabilities return only valid array elements test: ensure the Device.listCapabilities method works Start refactoring of error handling Remove processError from Device class Remove processError from Domain class Remove processError from DomainSnapshot class Remove processError from Interface class Remove processError method from Network class Remove processError method from NetworkFilter class Remove processError method from Secret class Remove processError method from StoragePool class Remove processError method from StorageVol class Remove processError method from Stream class Remove processError method from Connect class Call processError only when virInitialize signalled an error Remove ErrorHandler.processError(Libvirt) method Implement equals and hashCode methods for Connect and Domain Fix Domain.getSchedulerParameters / getSchedulerType Fix memleak in Domain.snapshotListNames Fix memleak in StoragePool.listVolumes Fix memleak in DomainSnapshot.getXMLDesc Fix memleak in StorageVol.getPath Fix memleak in StorageVol.getXMLDesc jna: Wrap the virEvent(Add,Remove)Timeout libvirt functions Implement Connect.isAlive Implement Connect.setKeepAlive Introduce event loop support Add constants for enum virDomainEventID Prepare to define proper domain event callback support Add constructIncRef factory method to Domain class events: handle registration for IOError events events: handle registration for Reboot events events: handle registration of domain lifecycle events test: add unit test for domain lifecycle events events: handle registration for PMWakeup events events: add support for PMSuspend events Implement connection close callback support Implement Connect.getSysinfo Implement Domain.blockPeek Implement Domain.memoryPeek Implement Secret.getUsageType Implement Domain.isUpdated Implement Domain.reset Implement Domain.PMwakeup Implement Domain.sendKey Implement interface ByteChannel for Stream class Implement Domain.screenshot test: add testDomainScreenshot JUnit test Add helper for handling bit-flags Connect: add constructors using java.net.URI params Replace Connect.getLibVirVersion method with Library.getVersion Replace static connectionVersion method with getLibVersion Deprecate Connect.getHypervisorVersion .gitignore | 2 + pom.xml.in | 2 +- src/main/java/org/libvirt/BitFlags.java | 18 + src/main/java/org/libvirt/Connect.java | 731 +++++++++++++++++--- src/main/java/org/libvirt/Device.java | 57 +- src/main/java/org/libvirt/Domain.java | 672 +++++++++++------- src/main/java/org/libvirt/DomainSnapshot.java | 29 +- src/main/java/org/libvirt/Error.java | 409 +++++++---- src/main/java/org/libvirt/ErrorHandler.java | 54 +- src/main/java/org/libvirt/Interface.java | 43 +- src/main/java/org/libvirt/KeycodeSet.java | 46 ++ src/main/java/org/libvirt/Library.java | 134 +++- src/main/java/org/libvirt/MemoryAddressMode.java | 19 + src/main/java/org/libvirt/Network.java | 61 +- src/main/java/org/libvirt/NetworkFilter.java | 43 +- src/main/java/org/libvirt/Secret.java | 72 +- src/main/java/org/libvirt/SecretUsageType.java | 23 + src/main/java/org/libvirt/StoragePool.java | 106 ++- src/main/java/org/libvirt/StorageVol.java | 68 +- src/main/java/org/libvirt/Stream.java | 232 +++++-- src/main/java/org/libvirt/SuspendTarget.java | 15 + src/main/java/org/libvirt/event/CrashedDetail.java | 15 + src/main/java/org/libvirt/event/DefinedDetail.java | 17 + src/main/java/org/libvirt/event/DetailInfo.java | 5 + src/main/java/org/libvirt/event/DomainEvent.java | 71 ++ .../java/org/libvirt/event/DomainEventDetail.java | 7 + .../java/org/libvirt/event/DomainEventType.java | 60 ++ src/main/java/org/libvirt/event/EventListener.java | 7 + src/main/java/org/libvirt/event/IOErrorAction.java | 39 ++ .../java/org/libvirt/event/IOErrorListener.java | 21 + .../java/org/libvirt/event/LifecycleListener.java | 24 + .../java/org/libvirt/event/PMSuspendListener.java | 17 + .../java/org/libvirt/event/PMSuspendReason.java | 5 + .../java/org/libvirt/event/PMSuspendedDetail.java | 15 + .../java/org/libvirt/event/PMWakeupListener.java | 17 + .../java/org/libvirt/event/PMWakeupReason.java | 5 + .../java/org/libvirt/event/RebootListener.java | 15 + src/main/java/org/libvirt/event/ResumedDetail.java | 20 + .../java/org/libvirt/event/ShutdownDetail.java | 12 + src/main/java/org/libvirt/event/StartedDetail.java | 30 + src/main/java/org/libvirt/event/StoppedDetail.java | 40 ++ .../java/org/libvirt/event/SuspendedDetail.java | 40 ++ .../java/org/libvirt/event/UndefinedDetail.java | 7 + src/main/java/org/libvirt/jna/Libvirt.java | 121 +++- src/main/java/org/libvirt/jna/SizeT.java | 19 + .../java/org/libvirt/jna/SizeTByReference.java | 50 ++ src/main/java/org/libvirt/jna/virConnectAuth.java | 4 +- .../java/org/libvirt/jna/virConnectCredential.java | 4 +- .../java/org/libvirt/jna/virDomainBlockInfo.java | 4 +- .../java/org/libvirt/jna/virDomainBlockStats.java | 4 +- src/main/java/org/libvirt/jna/virDomainInfo.java | 4 +- .../org/libvirt/jna/virDomainInterfaceStats.java | 4 +- .../java/org/libvirt/jna/virDomainJobInfo.java | 4 +- .../java/org/libvirt/jna/virDomainMemoryStats.java | 4 +- src/main/java/org/libvirt/jna/virError.java | 4 +- src/main/java/org/libvirt/jna/virNodeInfo.java | 4 +- .../java/org/libvirt/jna/virSchedParameter.java | 4 +- .../java/org/libvirt/jna/virStoragePoolInfo.java | 4 +- .../java/org/libvirt/jna/virStorageVolInfo.java | 4 +- src/main/java/org/libvirt/jna/virVcpuInfo.java | 4 +- src/test/java/org/libvirt/TestJavaBindings.java | 139 +++- src/test/java/org/libvirt/TestLibvirtGlobals.java | 4 + src/test/java/test.java | 280 -------- 63 files changed, 2807 insertions(+), 1187 deletions(-) create mode 100644 src/main/java/org/libvirt/BitFlags.java create mode 100644 src/main/java/org/libvirt/KeycodeSet.java create mode 100644 src/main/java/org/libvirt/MemoryAddressMode.java create mode 100644 src/main/java/org/libvirt/SecretUsageType.java create mode 100644 src/main/java/org/libvirt/SuspendTarget.java create mode 100644 src/main/java/org/libvirt/event/CrashedDetail.java create mode 100644 src/main/java/org/libvirt/event/DefinedDetail.java create mode 100644 src/main/java/org/libvirt/event/DetailInfo.java create mode 100644 src/main/java/org/libvirt/event/DomainEvent.java create mode 100644 src/main/java/org/libvirt/event/DomainEventDetail.java create mode 100644 src/main/java/org/libvirt/event/DomainEventType.java create mode 100644 src/main/java/org/libvirt/event/EventListener.java create mode 100644 src/main/java/org/libvirt/event/IOErrorAction.java create mode 100644 src/main/java/org/libvirt/event/IOErrorListener.java create mode 100644 src/main/java/org/libvirt/event/LifecycleListener.java create mode 100644 src/main/java/org/libvirt/event/PMSuspendListener.java create mode 100644 src/main/java/org/libvirt/event/PMSuspendReason.java create mode 100644 src/main/java/org/libvirt/event/PMSuspendedDetail.java create mode 100644 src/main/java/org/libvirt/event/PMWakeupListener.java create mode 100644 src/main/java/org/libvirt/event/PMWakeupReason.java create mode 100644 src/main/java/org/libvirt/event/RebootListener.java create mode 100644 src/main/java/org/libvirt/event/ResumedDetail.java create mode 100644 src/main/java/org/libvirt/event/ShutdownDetail.java create mode 100644 src/main/java/org/libvirt/event/StartedDetail.java create mode 100644 src/main/java/org/libvirt/event/StoppedDetail.java create mode 100644 src/main/java/org/libvirt/event/SuspendedDetail.java create mode 100644 src/main/java/org/libvirt/event/UndefinedDetail.java create mode 100644 src/main/java/org/libvirt/jna/SizeT.java create mode 100644 src/main/java/org/libvirt/jna/SizeTByReference.java delete mode 100644 src/test/java/test.java -- 1.7.9.5

6 190

[libvirt] [PATCH 0/2] v1.1 bugfix: support dhcp network interfaces
by Gene Czarcinski 29 Oct '14

29 Oct '14

v1.1 adds some documentation changes. Support for a network such as -N dhcp,source=default was not working in that dhclient was not being started. Although I am not sure what the real problem is, the solution is to use g_spawn_sync() instead of g_spawn_async() to start /sbin/dhclient. The second patch addes "-v" to the dhclient arguments to improve debugging info. The dhclient into will be in /var/log/messages the Secure Contrainer host system and not in the container itself. Gene Czarcinski (2): v1.1 for dhclient use g_spawn_sync() v1.1 add -v to dhclient parameter arguments libvirt-sandbox/libvirt-sandbox-init-common.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) -- 1.9.3

3 5

[libvirt] [PATCH] Renamed internal __mon_yday into __vir_mon_yday to avoid conflicts
by Cédric Bosdonnat 29 Oct '14

29 Oct '14

libc has another constant with the same name, which leads to redefinition error when building against static libvirt --- src/util/virtime.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/util/virtime.c b/src/util/virtime.c index acbec41..7b3ec44 100644 --- a/src/util/virtime.c +++ b/src/util/virtime.c @@ -105,7 +105,7 @@ int virTimeFieldsNowRaw(struct tm *fields) #define DIV(a, b) ((a) / (b) - ((a) % (b) < 0)) #define LEAPS_THRU_END_OF(y) (DIV (y, 4) - DIV (y, 100) + DIV (y, 400)) -const unsigned short int __mon_yday[2][13] = { +const unsigned short int __vir_mon_yday[2][13] = { /* Normal years. */ { 0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334, 365 }, /* Leap years. */ @@ -160,7 +160,7 @@ void virTimeFieldsThen(unsigned long long when, struct tm *fields) fields->tm_year = y - 1900; fields->tm_yday = days; - ip = __mon_yday[is_leap_year(y)]; + ip = __vir_mon_yday[is_leap_year(y)]; for (y = 11; days < (long int) ip[y]; --y) continue; days -= ip[y]; -- 1.8.4.5

3 2