August 2013 - Devel - libvirt List Archives

[libvirt] [PATCH] maint: slightly reduce configure size
by Eric Blake 16 Aug '13

16 Aug '13

Rather than inlining gl_WARN_ADD loads of time, we can shave about 17k size off of the configure script by delaying it to a cleanup shell loop. * m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Track a list of things to check, rather than inlining multiple checks. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- m4/virt-compile-warnings.m4 | 72 ++++++++++++++++++++++----------------------- 1 file changed, 36 insertions(+), 36 deletions(-) diff --git a/m4/virt-compile-warnings.m4 b/m4/virt-compile-warnings.m4 index 6bf797f..938c8bb 100644 --- a/m4/virt-compile-warnings.m4 +++ b/m4/virt-compile-warnings.m4 @@ -133,47 +133,31 @@ AC_DEFUN([LIBVIRT_COMPILE_WARNINGS],[ # Remove the ones we don't want, blacklisted earlier gl_MANYWARN_COMPLEMENT([wantwarn], [$maybewarn], [$dontwarn]) - # Check for $CC support of each warning - for w in $wantwarn; do - gl_WARN_ADD([$w]) - done - # GNULIB uses '-W' (aka -Wextra) which includes a bunch of stuff. # Unfortunately, this means you can't simply use '-Wsign-compare' # with gl_MANYWARN_COMPLEMENT # So we have -W enabled, and then have to explicitly turn off... - gl_WARN_ADD([-Wno-sign-compare]) + wantwarn="$wantwarn -Wno-sign-compare" # GNULIB expects this to be part of -Wc++-compat, but we turn # that one off, so we need to manually enable this again - gl_WARN_ADD([-Wjump-misses-init]) + wantwarn="$wantwarn -Wjump-misses-init" # GNULIB turns on -Wformat=2 which implies -Wformat-nonliteral, # so we need to manually re-exclude it. Also, older gcc 4.2 # added an implied ATTRIBUTE_NONNULL on any parameter marked # ATTRIBUTE_FMT_PRINT, which causes -Wformat failure on our # intentional use of virReportError(code, NULL). - gl_WARN_ADD([-Wno-format-nonliteral]) + wantwarn="$wantwarn -Wno-format-nonliteral" if test $lv_cv_gcc_wformat_null_works = no; then - gl_WARN_ADD([-Wno-format]) + wantwarn="$wantwarn -Wno-format" fi # This should be < 256 really. Currently we're down to 4096, # but using 1024 bytes sized buffers (mostly for virStrerror) # stops us from going down further - gl_WARN_ADD([-Wframe-larger-than=4096]) - dnl gl_WARN_ADD([-Wframe-larger-than=256]) - - # Silence certain warnings in gnulib, and use improved glibc headers - AC_DEFINE([lint], [1], - [Define to 1 if the compiler is checking for lint.]) - AH_VERBATIM([FORTIFY_SOURCE], - [/* Enable compile-time and run-time bounds-checking, and some warnings, - without upsetting newer glibc. */ - #if !defined _FORTIFY_SOURCE && defined __OPTIMIZE__ && __OPTIMIZE__ - # define _FORTIFY_SOURCE 2 - #endif - ]) + wantwarn="$wantwarn -Wframe-larger-than=4096" + dnl wantwarn="$wantwarn -Wframe-larger-than=256" # Extra special flags dnl -fstack-protector stuff passes gl_WARN_ADD with gcc @@ -182,43 +166,59 @@ AC_DEFUN([LIBVIRT_COMPILE_WARNINGS],[ *-*-linux*) dnl Fedora only uses -fstack-protector, but doesn't seem to dnl be great overhead in adding -fstack-protector-all instead - dnl gl_WARN_ADD([-fstack-protector]) - gl_WARN_ADD([-fstack-protector-all]) - gl_WARN_ADD([--param=ssp-buffer-size=4]) + dnl wantwarn="$wantwarn -fstack-protector" + wantwarn="$wantwarn -fstack-protector-all" + wantwarn="$wantwarn --param=ssp-buffer-size=4" dnl Even though it supports it, clang complains about dnl use of --param=ssp-buffer-size=4 unless used with dnl the -c arg. It doesn't like it when used with args dnl that just link together .o files. Unfortunately dnl we can't avoid that with automake, so we must turn dnl off the following clang specific warning - gl_WARN_ADD([-Wno-unused-command-line-argument]) + wantwarn="$wantwarn -Wno-unused-command-line-argument" ;; *-*-freebsd*) dnl FreeBSD ships old gcc 4.2.1 which doesn't handle dnl -fstack-protector-all well - gl_WARN_ADD([-fstack-protector]) + wantwarn="$wantwarn -fstack-protector" - gl_WARN_ADD([-Wno-unused-command-line-argument]) + wantwarn="$wantwarn -Wno-unused-command-line-argument" ;; esac - gl_WARN_ADD([-fexceptions]) - gl_WARN_ADD([-fasynchronous-unwind-tables]) - gl_WARN_ADD([-fdiagnostics-show-option]) - gl_WARN_ADD([-funit-at-a-time]) + wantwarn="$wantwarn -fexceptions" + wantwarn="$wantwarn -fasynchronous-unwind-tables" + wantwarn="$wantwarn -fdiagnostics-show-option" + wantwarn="$wantwarn -funit-at-a-time" # Need -fipa-pure-const in order to make -Wsuggest-attribute=pure # fire even without -O. - gl_WARN_ADD([-fipa-pure-const]) + wantwarn="$wantwarn -fipa-pure-const" # We should eventually enable this, but right now there are at # least 75 functions triggering warnings. - gl_WARN_ADD([-Wno-suggest-attribute=pure]) - gl_WARN_ADD([-Wno-suggest-attribute=const]) + wantwarn="$wantwarn -Wno-suggest-attribute=pure" + wantwarn="$wantwarn -Wno-suggest-attribute=const" if test "$set_werror" = "yes" then - gl_WARN_ADD([-Werror]) + wantwarn="$wantwarn -Werror" fi + # Check for $CC support of each warning + for w in $wantwarn; do + gl_WARN_ADD([$w]) + done + + # Silence certain warnings in gnulib, and use improved glibc headers + AC_DEFINE([lint], [1], + [Define to 1 if the compiler is checking for lint.]) + AH_VERBATIM([FORTIFY_SOURCE], + [/* Enable compile-time and run-time bounds-checking, and some warnings, + without upsetting newer glibc. */ + #if !defined _FORTIFY_SOURCE && defined __OPTIMIZE__ && __OPTIMIZE__ + # define _FORTIFY_SOURCE 2 + #endif + ]) + dnl Needed to keep compile quiet on python 2.4 save_WARN_CFLAGS=$WARN_CFLAGS WARN_CFLAGS= -- 1.8.3.1

2 2

[libvirt] [PATCH] libxl: refactor capabilities code
by Jim Fehlig 16 Aug '13

16 Aug '13

Cleanup the libxl capabilities code to be a bit more extensible, splitting out the creation of host and guest capabilities. This should make it easier to implement additional capabilities in the future, such as NUMA topology reporting. --- Although I can't see how changes here would affect other drivers, I nonetheless ensured libvirtd loaded when not booted to Xen :). src/libxl/libxl_conf.c | 256 +++++++++++++++++++++++-------------------------- 1 file changed, 121 insertions(+), 135 deletions(-) diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index 827dfdd..5ea4163 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -64,126 +64,62 @@ static const char *xen_cap_re = "(xen|hvm)-[[:digit:]]+\\.[[:digit:]]+-(x86_32|x static regex_t xen_cap_rec; -static virCapsPtr -libxlBuildCapabilities(virArch hostarch, - int host_pae, - struct guest_arch *guest_archs, - int nr_guest_archs) +static int +libxlCapsInitHost(libxl_ctx *ctx, virCapsPtr caps) { - virCapsPtr caps; - size_t i; - - if ((caps = virCapabilitiesNew(hostarch, 1, 1)) == NULL) - goto no_memory; - - if (host_pae && - virCapabilitiesAddHostFeature(caps, "pae") < 0) - goto no_memory; - - for (i = 0; i < nr_guest_archs; ++i) { - virCapsGuestPtr guest; - char const *const xen_machines[] = {guest_archs[i].hvm ? "xenfv" : "xenpv"}; - virCapsGuestMachinePtr *machines; - - if ((machines = virCapabilitiesAllocMachines(xen_machines, 1)) == NULL) - goto no_memory; - - if ((guest = virCapabilitiesAddGuest(caps, - guest_archs[i].hvm ? "hvm" : "xen", - guest_archs[i].arch, - ((hostarch == VIR_ARCH_X86_64) ? - "/usr/lib64/xen/bin/qemu-dm" : - "/usr/lib/xen/bin/qemu-dm"), - (guest_archs[i].hvm ? - "/usr/lib/xen/boot/hvmloader" : - NULL), - 1, - machines)) == NULL) { - virCapabilitiesFreeMachines(machines, 1); - goto no_memory; - } - machines = NULL; - - if (virCapabilitiesAddGuestDomain(guest, - "xen", - NULL, - NULL, - 0, - NULL) == NULL) - goto no_memory; - - if (guest_archs[i].pae && - virCapabilitiesAddGuestFeature(guest, - "pae", - 1, - 0) == NULL) - goto no_memory; - - if (guest_archs[i].nonpae && - virCapabilitiesAddGuestFeature(guest, - "nonpae", - 1, - 0) == NULL) - goto no_memory; - - if (guest_archs[i].ia64_be && - virCapabilitiesAddGuestFeature(guest, - "ia64_be", - 1, - 0) == NULL) - goto no_memory; - - if (guest_archs[i].hvm) { - if (virCapabilitiesAddGuestFeature(guest, - "acpi", - 1, - 1) == NULL) - goto no_memory; + int err; + libxl_physinfo phy_info; + int host_pae; - if (virCapabilitiesAddGuestFeature(guest, "apic", - 1, - 0) == NULL) - goto no_memory; + err = regcomp(&xen_cap_rec, xen_cap_re, REG_EXTENDED); + if (err != 0) { + char error[100]; + regerror(err, &xen_cap_rec, error, sizeof(error)); + regfree(&xen_cap_rec); + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Failed to compile regex %s"), error); + return -1; + } - if (virCapabilitiesAddGuestFeature(guest, - "hap", - 0, - 1) == NULL) - goto no_memory; - } + if (libxl_get_physinfo(ctx, &phy_info) != 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Failed to get node physical info from libxenlight")); + return -1; } - return caps; + /* hw_caps is an array of 32-bit words whose meaning is listed in + * xen-unstable.hg/xen/include/asm-x86/cpufeature.h. Each feature + * is defined in the form X*32+Y, corresponding to the Y'th bit in + * the X'th 32-bit word of hw_cap. + */ + host_pae = phy_info.hw_cap[0] & LIBXL_X86_FEATURE_PAE_MASK; + if (host_pae && + virCapabilitiesAddHostFeature(caps, "pae") < 0) + return -1; - no_memory: - virObjectUnref(caps); - return NULL; + return 0; } -static virCapsPtr -libxlMakeCapabilitiesInternal(virArch hostarch, - libxl_physinfo *phy_info, - char *capabilities) +static int +libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) { + const libxl_version_info *ver_info; char *str, *token; regmatch_t subs[4]; char *saveptr = NULL; size_t i; + virArch hostarch = caps->host.arch; - int host_pae = 0; struct guest_arch guest_archs[32]; int nr_guest_archs = 0; - virCapsPtr caps = NULL; memset(guest_archs, 0, sizeof(guest_archs)); - /* hw_caps is an array of 32-bit words whose meaning is listed in - * xen-unstable.hg/xen/include/asm-x86/cpufeature.h. Each feature - * is defined in the form X*32+Y, corresponding to the Y'th bit in - * the X'th 32-bit word of hw_cap. - */ - host_pae = phy_info->hw_cap[0] & LIBXL_X86_FEATURE_PAE_MASK; - + if ((ver_info = libxl_get_version_info(ctx)) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Failed to get version info from libxenlight")); + return -1; + } /* Format of capabilities string is documented in the code in * xen-unstable.hg/xen/arch/.../setup.c. * @@ -209,7 +145,7 @@ libxlMakeCapabilitiesInternal(virArch hostarch, /* Split capabilities string into tokens. strtok_r is OK here because * we "own" the buffer. Parse out the features from each token. */ - for (str = capabilities, nr_guest_archs = 0; + for (str = ver_info->capabilities, nr_guest_archs = 0; nr_guest_archs < sizeof(guest_archs) / sizeof(guest_archs[0]) && (token = strtok_r(str, " ", &saveptr)) != NULL; str = NULL) { @@ -274,17 +210,80 @@ libxlMakeCapabilitiesInternal(virArch hostarch, } } - if ((caps = libxlBuildCapabilities(hostarch, - host_pae, - guest_archs, - nr_guest_archs)) == NULL) - goto error; + for (i = 0; i < nr_guest_archs; ++i) { + virCapsGuestPtr guest; + char const *const xen_machines[] = {guest_archs[i].hvm ? "xenfv" : "xenpv"}; + virCapsGuestMachinePtr *machines; - return caps; + if ((machines = virCapabilitiesAllocMachines(xen_machines, 1)) == NULL) + return -1; - error: - virObjectUnref(caps); - return NULL; + if ((guest = virCapabilitiesAddGuest(caps, + guest_archs[i].hvm ? "hvm" : "xen", + guest_archs[i].arch, + ((hostarch == VIR_ARCH_X86_64) ? + "/usr/lib64/xen/bin/qemu-dm" : + "/usr/lib/xen/bin/qemu-dm"), + (guest_archs[i].hvm ? + "/usr/lib/xen/boot/hvmloader" : + NULL), + 1, + machines)) == NULL) { + virCapabilitiesFreeMachines(machines, 1); + return -1; + } + machines = NULL; + + if (virCapabilitiesAddGuestDomain(guest, + "xen", + NULL, + NULL, + 0, + NULL) == NULL) + return -1; + + if (guest_archs[i].pae && + virCapabilitiesAddGuestFeature(guest, + "pae", + 1, + 0) == NULL) + return -1; + + if (guest_archs[i].nonpae && + virCapabilitiesAddGuestFeature(guest, + "nonpae", + 1, + 0) == NULL) + return -1; + + if (guest_archs[i].ia64_be && + virCapabilitiesAddGuestFeature(guest, + "ia64_be", + 1, + 0) == NULL) + return -1; + + if (guest_archs[i].hvm) { + if (virCapabilitiesAddGuestFeature(guest, + "acpi", + 1, + 1) == NULL) + return -1; + + if (virCapabilitiesAddGuestFeature(guest, "apic", + 1, + 0) == NULL) + return -1; + + if (virCapabilitiesAddGuestFeature(guest, + "hap", + 0, + 1) == NULL) + return -1; + } + } + + return 0; } static int @@ -874,35 +873,22 @@ error: virCapsPtr libxlMakeCapabilities(libxl_ctx *ctx) { - int err; - libxl_physinfo phy_info; - const libxl_version_info *ver_info; + virCapsPtr caps; - err = regcomp(&xen_cap_rec, xen_cap_re, REG_EXTENDED); - if (err != 0) { - char error[100]; - regerror(err, &xen_cap_rec, error, sizeof(error)); - regfree(&xen_cap_rec); - virReportError(VIR_ERR_INTERNAL_ERROR, - _("Failed to compile regex %s"), error); + if ((caps = virCapabilitiesNew(virArchFromHost(), 1, 1)) == NULL) return NULL; - } - if (libxl_get_physinfo(ctx, &phy_info) != 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Failed to get node physical info from libxenlight")); - return NULL; - } + if (libxlCapsInitHost(ctx, caps) < 0) + goto error; - if ((ver_info = libxl_get_version_info(ctx)) == NULL) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Failed to get version info from libxenlight")); - return NULL; - } + if (libxlCapsInitGuests(ctx, caps) < 0) + goto error; + + return caps; - return libxlMakeCapabilitiesInternal(virArchFromHost(), - &phy_info, - ver_info->capabilities); +error: + virObjectUnref(caps); + return NULL; } int -- 1.8.1.4

3 4

[libvirt] [PATCHv2 0/2] Fix starting of VM's on hosts without DBus compiled in or enabled
by Peter Krempa 16 Aug '13

16 Aug '13

Hi, this series fixes regression on hosts without DBus that was introduced by the refactor to use systemd.machined to create cgroups. Version 2: As errors can't be unlogged, this version enhances the error reporting code to add conditional error logging. The only remaining issue is that if a user is running DBus but doesn't use systemd, the logs are still spammed with errors. Peter Krempa (2): virerror: Add support for suppressing reporting of errors cgroup: Don't fail to start a VM when DBus isn't compiled in or running daemon/libvirtd.c | 2 +- daemon/remote.c | 2 +- src/libvirt_private.syms | 2 +- src/node_device/node_device_hal.c | 2 +- src/nwfilter/nwfilter_driver.c | 4 ++-- src/rpc/virnetserver.c | 2 +- src/util/virdbus.c | 15 ++++++++------- src/util/virdbus.h | 2 +- src/util/virerror.c | 31 ++++++++++++++++++++----------- src/util/virerror.h | 21 +++++++++++++++------ src/util/virsystemd.c | 4 ++-- 11 files changed, 53 insertions(+), 34 deletions(-) -- 1.8.3.2

2 3

Re: [libvirt] [openstack-dev] [nova] live-snapshot/cloning of virtual machines
by Andres Lagar-Cavilla 16 Aug '13

16 Aug '13

> On Fri, Aug 16, 2013 at 11:05:19AM +0100, Daniel P. Berrange wrote: >> On Wed, Aug 14, 2013 at 04:53:01PM -0700, Vishvananda Ishaya wrote: >>> Hi Everyone, >>> >>> I have been trying for some time to get the code for the live-snapshot blueprint[1] >>> in. Going through the review process for the rpc and interface code[2] was easy. I >>> suspect the api-extension code[3] will also be relatively trivial to get in. The >>> main concern is with the libvirt driver implementation[4]. I'd like to discuss the >>> concerns and see if we can make some progress. >>> >>> Short Summary (tl;dr) >>> ===================== >>> >>> I propose we merge live-cloning as an experimental feature for havanna and have the >>> api extension disabled by default. >>> >>> Overview >>> ======== >>> >>> First of all, let me express the value of live snapshoting. The >>> slowest part of the vm provisioning process is generally booting >>> of the OS. > > Like Dan I'm dubious about this whole plan. But this ^^ statement in > particular. I would like to see hard data to back this up. What we need to keep in mind is that "boot" is a small part of the picture, at least "boot" as commonly referred to in Linux. Consider a web sphere-like Java bundle of code. These things take a while to load. JiT-ed methods provide a tremendous performance boost. Nevermind if the the server constructs secondary indices to perform fast lookups of data. That is just Linux. Windows is well known for pounding storage fabrics with thousands of small reads during boot storms. Certainly a boot Windows sequence has baked in a lot of service startup sequences that prime a lot of memory content for performance objectives. Boot here means "ready to rock-n-roll", not "Cirros is up." We have live deployments that are based on bypassing the entire *application startup* sequence and have a server ready to provide high-performance responses to queries once spawned from a live saved image. > > You should be able to boot an OS pretty quickly, and furthermore it's > (a) much safer for all the reasons Dan outlines, and (b) improvements > that you make to boot times help everyone. > > [...] >>> 2. Security Concerns >>> ==================== >>> >>> There are a number of security issues with loading state from another vm. Here is a >>> short list of things that need to be done just to make a cloned vm usable: >>> >>> a) mac address needs to be recreated >>> b) entropy pool needs to be reset >>> c) host name must be reset >>> d) host keys bust be regenerated >>> >>> There are others, and trying to clone a running application as well may expose other >>> sensitive data, especially if users are snaphsoting vms and making them public. > > Are we talking about cloning VMs that you already trust, or cloning > random VMs and allowing random other users to use them? These would > lead to very different solutions. In the first case, you only care > about correctness, not security. In the second case, you care about > security as well as correctness. Case number one. The correctness issues are a hard problem, and a particularly hard one in Windows, but it is pragmatically solvable. For a common scenario in Linux, renewing dhcp leases and leveling your entropy pool are what you need. > > I highly doubt the second case is possible because scrubbing the disk > is going to take far too long for any supposed time-saving to matter. That would be very counter-productive, so yes, focusing on the first case. > > As Dan says, even the first case is dubious because it won't be correct. > >> The libguestfs project provide tools to perform offline cloning of >> VM disk images. Its virt-sysprep knows how to delete alot (but by >> no means all possible) sensitive file data for common Linux & >> Windows OS. It still has to be combined with use of the >> virt-sparsify tool though, to ensure the deleted data is actually >> purged from the VM disk image as well as the filesystem, by >> releasing all unused VM disk sectors back to the host storage (and >> not all storage supports that). > > Links to the tools that Dan mentions: > > http://libguestfs.org/virt-sysprep.1.html > http://libguestfs.org/virt-sparsify.1.html Virt-sparsify is not strictly relevant here. The disk side of live images is carried out with qcow2. Virt-sysprep is great work and highly relevant. But virt-sysprep allows us to see the argument in a different light. Have you noticed nova does not run virt-sysprep before booting an ephemeral instance from an image? (AFAIK, could be wrong, not even regenerating host ssh keys is part of the assured workflow). Furthermore, one can create arbitrary (cold, non-live) images at any time, from live instances This isn't necessarily wrong. It underpins massive deployments, it pragmatically adds value. The fundamental semantics at play with live-instances are the same: know what you are doing, ephemeral instances, bound to your tenant. So as long as we take care of not weakening cryptographic foundations and correctly reconfiguring the identity, all the principles above still apply: know what you are doing, ephemeral instances, bound to your tenant. We are working on proof-of-concept using the qemu guest agent to undertake all these tasks. It is PoC to show it's doable. It's neither clean nor asking for mainline merge (at the moment). It's not the only solution for the guest side problem. An one final very important note for the libvirt list and those who may be tuning in just now: no changes to libvirt are being asked for here. This is a nova-side tech preview feature. Best, Andres > > Note these tools can only be used on offline machines. > > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones > virt-top is 'top' for virtual machines. Tiny program with many > powerful monitoring features, net stats, disk stats, logging, etc. > http://people.redhat.com/~rjones/virt-top

1 0

Re: [libvirt] [openstack-dev] [nova] live-snapshot/cloning of virtual machines
by Daniel P. Berrange 16 Aug '13

16 Aug '13

On Wed, Aug 14, 2013 at 04:53:01PM -0700, Vishvananda Ishaya wrote: > Hi Everyone, > > I have been trying for some time to get the code for the live-snapshot blueprint[1] > in. Going through the review process for the rpc and interface code[2] was easy. I > suspect the api-extension code[3] will also be relatively trivial to get in. The > main concern is with the libvirt driver implementation[4]. I'd like to discuss the > concerns and see if we can make some progress. > > Short Summary (tl;dr) > ===================== > > I propose we merge live-cloning as an experimental feature for havanna and have the > api extension disabled by default. > > Overview > ======== > > First of all, let me express the value of live snapshoting. The slowest part of the > vm provisioning process is generally booting of the OS. The advantage of live- > snapshotting is that it allows the possibility of bringing up application servers > while skipping the overhead of vm (and application startup). For Linux at least I think bootup time is a problem that is being solved by the distros. It is possible to boot up many modern Linux distros in a couple of seconds even in physical hardware - VMs can be even faster since they don't have such stupid BIOS to worry about & have a restricted set of possible hardware. This is on a par with, or better than, the overheads imposed by Nova itself in the boot up process. Windows may be a different story, but I've not used it in years so don't know what its boot performance is like. > I recognize that this capability comes with some security concerns, so I don't expect > this feature to go in and be ready to for use in production right away. Similarly, > containers have a lot of the same benefit, but have had their own security issues > which are gradually being resolved. My hope is that getting this feature in would > allow people to start experimenting with live-booting so that we could uncover some > of these security issues. > > There are two specific concerns that have been raised regarding my patch. The first > concern is related to my use of libvirt. The second concern is related to the security > issues above. Let me address them separately. > > 1. Libvirt Issues > ================= > > The only feature I require from the hypervisor is to load memory/processor state for > a vm from a file. Qemu supports this directly. The only way that libvirt exposes this > functionality is via its restore command which is specifically for restoring the > previous state of an existing vm. "Cloning", or restoring the memory state of a > cloned vm is considered unsafe (which I will address in the second point, below). > > The result of the limited api is that I must include some hacks to make the restore > command actually allow me to restore the state of the new vm. I recognize that this > is using an undocumented libvirt api and isn't the ideal solution, but it seemed > "better" then avoiding libvirt and talking directly to qemu. > > This is obviously not ideal. It is my hope that this 0.1 version of the feature will > allow us to iteratively improve the live-snapshot/clone proccess and get the security > to a point where the libvirt maintainers would be willing to accept a patch to directly > expose an api to load memory from a file. To characterize this as a libvirt issue is somewhat misleading. The reason why libvirt does not explicitly allow this, is that from discussions with the upstream QEMU/KVM developers, the recommendation/advise that this is not a safe operation and should not be exposed to application developers. The expectation is that the functionality in QEMU is only targetted for taking point in time snapshots & allowing rollback of a VM to those snapshots, not creating clones of active VMs. > 2. Security Concerns > ==================== > > There are a number of security issues with loading state from another vm. Here is a > short list of things that need to be done just to make a cloned vm usable: > > a) mac address needs to be recreated > b) entropy pool needs to be reset > c) host name must be reset > d) host keys bust be regenerated > > There are others, and trying to clone a running application as well may expose other > sensitive data, especially if users are snaphsoting vms and making them public. > > The only issue that I address on the driver side is the mac addresses. This is the > minimum that needs to be done just to be able to access the vm over the network. This > is implemented by unplugging all network devices before the snapshot and plugging new > network devices in on clone. This isn't the most friendly thing to guest applications, > but it seems like the safest option for the first version of this feature. This is not really as safe as you portray. When restoring from the snapshot the VM will initially be running with virtual NIC with a different MAC address from the one associated with the in memory OS kernel state. Even if you hotunplug the device and plug in a new one, you still have a period where the virtual hardware exposed to the guest does not match the memory state of the guest OS. Perhaps you will be lucky and not hit problems with some OS, but equally you can be unlucky and do bad things to the OS kernel or application state. Relying on luck in this way does not lead to a supportable solution IMHO. There are other unique identifiers exposed in the virtual hardware that will/should change when you clone VMs too, the host UUID, the storage serial keys and you cannot easily fix those by just unplugging hardware & replugging it. > So cloning vms must be done with care. Sensitive data must be removed from the vm > pre-clone and new data needs to be generated post-clone. Ultimately this should all > be done via guest-agent of some sort. I have found some volunteers to make the guest > agent a reality, but it will take a bit of time to get something workable, and it > will be much more difficult if there isn't a way to test the feature. Note that even if you think you have removed security data from a VM's filesystem, it is quite likely that the data will still in fact exist in the unallocated sectors of VM's block devices and can be fairly easily recovered from them. The libguestfs project provide tools to perform offline cloning of VM disk images. Its virt-sysprep knows how to delete alot (but by no means all possible) sensitive file data for common Linux & Windows OS. It still has to be combined with use of the virt-sparsify tool though, to ensure the deleted data is actually purged from the VM disk image as well as the filesystem, by releasing all unused VM disk sectors back to the host storage (and not all storage supports that). > Conclusion > ========== > > There are obviously problems to be solved with the whole idea of live cloning, but > I think it enables some important new ways of deploying applications. Imagine for > example a PaaS built on fast-cloning vms instead of containers. This is clearly a > long term project but if we block it now it may never get the support it needs to > become a real option. > > Proposal > ======== > > I propose we allow the patch in and we leave the live-snapshot extension disabled > by default. Deployers can turn on the extension to experiment with the feature. > This will allow other hypervisors do do an implementation, and the community in > general to improve the security and usefulness of live-cloned virtual machines. > > I'm very interested in your thoughts and feedback. Thank you to everyone who made > it this far. I don't think it is a good idea to add a feature which is considered to be unsupportable by the developers of the virt platform. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

2 1

[libvirt] [test-API][PATCH] Fix the repeating display of testcase name issue when generating log.xml
by Hongming Zhang 16 Aug '13

16 Aug '13

modified: src/log_generator.py --- src/log_generator.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/src/log_generator.py b/src/log_generator.py index de18654..7685da7 100644 --- a/src/log_generator.py +++ b/src/log_generator.py @@ -70,7 +70,8 @@ class LogGenerator(object): valuedict = test_procedure[casename] test_casename = self.doc.createElement('action') - test_casename.setAttribute('name', casename) + casenamexml = casename[:casename.rfind(":")] + test_casename.setAttribute('name', casenamexml) for arg in valuedict.keys(): test_arg = self.doc.createElement('arg') -- 1.7.1

3 2

[libvirt] [PATCH] virsh-domain: Fix memleak in cmdCPUBaseline
by Peter Krempa 16 Aug '13

16 Aug '13

https://bugzilla.redhat.com/show_bug.cgi?id=997765 ==1349431== 8 bytes in 1 blocks are definitely lost in loss record 11 of 760 ==1349431== at 0x4C2A554: calloc (vg_replace_malloc.c:593) ==1349431== by 0x4E9AA3E: virAllocN (in /usr/lib64/libvirt.so.0.1001.1) ==1349431== by 0x4EF28C4: virXPathNodeSet (in /usr/lib64/libvirt.so.0.1001.1) ==1349431== by 0x130B83: cmdCPUBaseline (in /usr/bin/virsh) ==1349431== by 0x12C608: vshCommandRun (in /usr/bin/virsh) ==1349431== by 0x12889A: main (in /usr/bin/virsh) --- Pushed as trivial. tools/virsh-domain.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index ab3804a..18fcb3e 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -6238,6 +6238,7 @@ cleanup: } VIR_FREE(list); VIR_FREE(buffer); + VIR_FREE(node_list); return ret; -- 1.8.3.2

2 1

[libvirt] tests: add qemu startupPolicy testcase
by Guannan Ren 16 Aug '13

16 Aug '13

Add a group of testcase of startupPolicy features for CDROM and Harddisk. It compares the output xml string after checking the presence of disk and applying startupPolicy with expected xml string. New file: tests/qemustartuppolicytest.c TestData: tests/qemustartuppolicytestdata/ Guannan Ren(4) tests: add qemustartuppolicy testcase tests: add startuppolicy testcases for cdrom tests: add startuppolicy testcases for single harddisk tests: add startuppolicy testcase for multiple mixed harddisks tests/Makefile.am | 10 ++++- tests/qemustartuppolicytest.c | 230 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/qemustartuppolicytestdata/domain-input-cdrom-absent-policy-mandatory.xml | 28 ++++++++++++++ tests/qemustartuppolicytestdata/domain-input-cdrom-absent-policy-optional.xml | 28 ++++++++++++++ tests/qemustartuppolicytestdata/domain-input-cdrom-absent-policy-requisite.xml | 28 ++++++++++++++ tests/qemustartuppolicytestdata/domain-input-cdrom-present-policy-optional.xml | 28 ++++++++++++++ tests/qemustartuppolicytestdata/domain-input-disk-absent-policy-mandatory.xml | 27 +++++++++++++ tests/qemustartuppolicytestdata/domain-input-disk-absent-policy-optional.xml | 27 +++++++++++++ tests/qemustartuppolicytestdata/domain-input-disk-present-policy-none.xml | 27 +++++++++++++ tests/qemustartuppolicytestdata/domain-input-mixed-multiple-disks-absent-policy-mandatory.xml | 36 +++++++++++++++++ tests/qemustartuppolicytestdata/domain-input-mixed-multiple-disks-absent-policy-optional.xml | 36 +++++++++++++++++ tests/qemustartuppolicytestdata/domain-input-multiple-disks-absent-policy-optional.xml | 36 +++++++++++++++++ tests/qemustartuppolicytestdata/domain-output-cdrom-absent-policy-optional.xml | 29 ++++++++++++++ tests/qemustartuppolicytestdata/domain-output-cdrom-absent-policy-requisite.xml | 29 ++++++++++++++ tests/qemustartuppolicytestdata/domain-output-cdrom-present-policy-optional.xml | 29 ++++++++++++++ tests/qemustartuppolicytestdata/domain-output-disk-absent-policy-optional.xml | 22 +++++++++++ tests/qemustartuppolicytestdata/domain-output-disk-present-policy-none.xml | 28 ++++++++++++++ tests/qemustartuppolicytestdata/domain-output-mixed-multiple-disks-absent-policy-mandatory.xml | 36 +++++++++++++++++ tests/qemustartuppolicytestdata/domain-output-mixed-multiple-disks-absent-policy-optional.xml | 27 +++++++++++++ tests/qemustartuppolicytestdata/domain-output-multiple-disks-absent-policy-optional.xml | 22 +++++++++++ 20 files changed, 761 insertions(+), 2 deletions(-)

1 4

[libvirt] [PATCHv3 0/4] write separate module for hostdev passthrough
by cyliu＠suse.com 16 Aug '13

16 Aug '13

From: Chunyan Liu <cyliu(a)suse.com> These patches implements a separate module for hostdev passthrough so that it could be shared by different drivers and can maintain a global state of a host device. Plus, add passthrough to libxl driver, and change qemu driver and lxc driver to use hostdev common library instead of their own hostdev APIs. --- Changes to v2: * add patches for qemu driver and lxc driver, use common library APIs instead of their own hostdev APIs. * add APIs for nodedev-detach and nodedev-reattach calling. * rename functions to use unified prefix 'virHostdev' * use VIR_ONCE_GLOBAL_INIT() as others instead of previous Init and Cleanup. * use VIR_STRDUP instead of strdup * rebase to latest code v2 is here: https://www.redhat.com/archives/libvir-list/2013-June/msg00263.html Chunyan Liu (4): add hostdev passthrough common library add pci passthrough to libxl driver change qemu driver to use hostdev common library change lxc driver to use hostdev common library po/POTFILES.in | 3 +- src/Makefile.am | 3 +- src/libvirt_private.syms | 22 + src/libxl/libxl_conf.c | 65 ++ src/libxl/libxl_conf.h | 5 +- src/libxl/libxl_driver.c | 250 +++++++- src/lxc/lxc_conf.h | 4 - src/lxc/lxc_driver.c | 45 +- src/lxc/lxc_hostdev.c | 413 ------------ src/lxc/lxc_hostdev.h | 43 -- src/lxc/lxc_process.c | 21 +- src/qemu/qemu_command.c | 1 - src/qemu/qemu_conf.h | 9 +- src/qemu/qemu_driver.c | 72 +-- src/qemu/qemu_hostdev.c | 1289 ----------------------------------- src/qemu/qemu_hostdev.h | 72 -- src/qemu/qemu_hotplug.c | 127 ++-- src/qemu/qemu_process.c | 34 +- src/util/virhostdev.c | 1683 ++++++++++++++++++++++++++++++++++++++++++++++ src/util/virhostdev.h | 123 ++++ src/util/virpci.c | 21 +- src/util/virpci.h | 7 +- src/util/virscsi.c | 22 +- src/util/virscsi.h | 8 +- src/util/virusb.c | 23 +- src/util/virusb.h | 8 +- 26 files changed, 2341 insertions(+), 2032 deletions(-) delete mode 100644 src/lxc/lxc_hostdev.c delete mode 100644 src/lxc/lxc_hostdev.h delete mode 100644 src/qemu/qemu_hostdev.c delete mode 100644 src/qemu/qemu_hostdev.h create mode 100644 src/util/virhostdev.c create mode 100644 src/util/virhostdev.h

3 10

[libvirt] [PATCH] maint: fix typo for 'switch'
by Eric Blake 15 Aug '13

15 Aug '13

* src/util/virnetdevvportprofile.c: Fix typo. * src/conf/domain_conf.c: Likewise. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- Pushing under the trivial rule. src/conf/domain_conf.c | 2 +- src/util/virnetdevvportprofile.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 7309877..12b68ea 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2603,7 +2603,7 @@ virDomainDeviceInfoIterateInternal(virDomainDefPtr def, } /* This switch statement is here to trigger compiler warning when adding - * a new device type. When you are adding a new field to the swtich you + * a new device type. When you are adding a new field to the switch you * also have to add a iteration statement above. Otherwise the switch * statement has no real function here and should be optimized out by the * compiler. */ diff --git a/src/util/virnetdevvportprofile.c b/src/util/virnetdevvportprofile.c index 01ba399..07155b9 100644 --- a/src/util/virnetdevvportprofile.c +++ b/src/util/virnetdevvportprofile.c @@ -1,5 +1,5 @@ /* - * Copyright (C) 2009-2012 Red Hat, Inc. + * Copyright (C) 2009-2013 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -1114,7 +1114,7 @@ cleanup: * @vmOp : The VM operation (i.e., create, no-op) * @setlink_only : Only set the link - dont wait for the link to come up * - * Associate a port on a swtich with a profile. This function + * Associate a port on a switch with a profile. This function * may notify a kernel driver or an external daemon to run * the setup protocol. If profile parameters were not supplied * by the user, then this function returns without doing -- 1.8.3.1

1 0