- Devel - libvirt List Archives

[libvirt] [PATCH] Fix virPipeReadUntilEOF on more than 1024 bytes of data
by Daniel Veillard 03 Nov '10

03 Nov '10

This is a bug I found while testing the smbios support, it uses virPipeReadUntilEOF to read the stdout of the exec'ed dmidecode process and this failed to capture the full output. It read only 1024 bytes. The problem is that this is based on a poll loop, and in the loop we read at most 1024 bytes per filedescriptor, but we also note in the loop if poll indicates that the process won't output more than that on that fd by setting finished[i] = 1. The simplest way is that if we read a full buffer make sure finished[i] is still 0 because we will need another pass in the loop. Daniel diff --git a/src/util/util.c b/src/util/util.c index f7c7d32..a059675 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -984,6 +984,9 @@ virPipeReadUntilEOF(int outfd, int errfd, got = read(fds[i].fd, data, sizeof(data)); + if (got == sizeof(data)) + finished[i] = 0; + if (got == 0) { finished[i] = 1; continue; -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

2 2

[libvirt] Changing the default for qcow2 creation
by Matthew Booth 03 Nov '10

03 Nov '10

I've recently been investigating a severe performance issue I noticed when writing to a qcow2-backed image. When virt-v2v is doing a format conversion from raw to qcow2, it does the following: 1. Create a new qcow2 image 2. Launch a libguestfs appliance (kvm) using the new image 3. Write the source raw data to the appliance's block device I noticed that the same process writing to a raw image rather than a qcow2 image was adequately fast, and decided to do some testing. I've attached my simple test program.[1] It does the following: 1. Start an appliance with test.img as a disk. 2. Format test.img with ext2. 3. Create a file /test 4. Write 256M of data to /test in 2M chunks Only step 4 is timed. I ran the program against test.img prepared in 4 different ways: 1. A sparse raw file: 15.3 seconds truncate --size 300M test.img 2. A preallocated raw file: 14.8 seconds fallocate -l 300M test.img 3. A sparse qcow2 file: 223.0 seconds qemu-img create -f qcow2 test.img 300M 4. A metadata preallocated qcow2 file: 14.5 seconds qemu-img create -f qcow2 -o preallocated=metadata test.img 300M With the exception of (3), I ran the test 3 times and took the middle time rounded to 1DP. I saw about 5-10% variation. I only ran the test against (3) once. The precise ordering of 1, 2 and 4 is surprising, but given the variation probably not that interesting: they're all about the same. The interesting thing is that the overhead of qcow2 metadata creation during the test seems to account for a 15x performance penalty. I had a cursory look at metadata preallocation, which I hadn't been aware of before today. Creating a qcow2 image of any size with no preallocation results in a 136k file. If you preallocate the metadata, a sparse file is created large enough to accomodate the entire image, with >136k actually used. In the above 300M case this is 204k. On a slightly more practical 20G image, 3.3M is preallocated. It's also worth noting that the image takes considerably longer to create. On my laptop, creation without preallocation is 'instantaneous' at any size. With preallocation, a 20G image takes 6 seconds to create, and a 100G image takes 26 seconds. libvirt's qemu driver doesn't currently preallocate qcow2 metadata when creating a new image. Given the tiny disk space overhead of the metadata (0.02%) and the small processing overhead of pre-creation relative to subsequent creation on-the-fly, I suggest that the libvirt qemu driver is updated to pre-allocate metadata by default. Thoughts? Matt [1] Note that I'm running this against libguestfs from git, which uses virtio-serial rather than usermode networking for appliance<->host communication. This change alone improved the performance of this test by about 10x. If your numbers don't match mine, that's probably why. I don't know off the top of my head if this change has made it into F14 yet. It's definitely not in F13. -- Matthew Booth, RHCA, RHCSS Red Hat Engineering, Virtualisation Team GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

5 6

[libvirt] New PHP bindings
by Arnar Mar Sig 03 Nov '10

03 Nov '10

Hi There are few PHP libvirt bindings out there, but all I found are procedural based and incomplete. To address this I created another binding that is OO based and in large part generated from libvirt-api.xml. The extension already supports most of the API, split into 6 classes: virConnect, virDomain, virInterface, virNetwork, virStoragePool, and virStorageVol. Snapshot of the current code can be found at: http://libvirt-debian.vm.antab.is/files/php-libvirt.tar.bz2 Or using svn from: http://svn.antab.is/php-libvirt/trunk Build instructions are in the README file. Documents can be found at http://libvirt-debian.vm.antab.is/ NOTE: Most of the docs are copied from libvirt. It has mostly been tested with libvirt test backend and PHP unit tests (phpt) are used to test, covering about 40% of basic usage. It is by no means complete and missing some functionality like classmaps (So virConnect::domainLookupBy* can return user defined class extended from virDomain) and probably leaks memory. I would appreciate any input and comments. Thanks Arnar Mar Sig

3 7

[libvirt] [PATCH 00/10] Allow access to text console with virStream APIs (v2)
by Daniel P. Berrange 03 Nov '10

03 Nov '10

2 29

[libvirt] [PATCH] xen: work with ia64 MAX_VIRT_CPUS of 64
by Eric Blake 03 Nov '10

03 Nov '10

* src/xen/xen_hypervisor.c (MAX_VIRT_CPUS): Move... * src/xen/xen_driver.h (MAX_VIRT_CPUS): ...so all xen code can see same value. * src/xen/xend_internal.c (sexpr_to_xend_domain_info) (xenDaemonDomainGetVcpusFlags, xenDaemonParseSxpr) (xenDaemonFormatSxpr): Work if MAX_VIRT_CPUS is 64 on a platform where long is 64-bits. * src/xen/xm_internal.c (xenXMDomainConfigParse) (xenXMDomainConfigFormat): Likewise. --- On at least the xen/stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git we have the case where MAX_VIRT_CPUS is 32 for x86 but 64 for ia64. That branch does not have XEN_LEGACY_MAX_VCPUS. But according to our comments, other branches had only XEN_LEGACY_MAX_VCPUS, which means that xm_internal.c would fail to compile without this patch, when targetting xen headers that lack MAX_VIRT_CPUS. src/xen/xen_driver.h | 8 ++++++++ src/xen/xen_hypervisor.c | 8 -------- src/xen/xend_internal.c | 17 +++++++++-------- src/xen/xm_internal.c | 9 +++++---- 4 files changed, 22 insertions(+), 20 deletions(-) diff --git a/src/xen/xen_driver.h b/src/xen/xen_driver.h index 53f97d4..16d22f1 100644 --- a/src/xen/xen_driver.h +++ b/src/xen/xen_driver.h @@ -29,6 +29,14 @@ # include <winsock2.h> # endif +/* xen-unstable changeset 19788 removed MAX_VIRT_CPUS from public + * headers. Its semantic was retained with XEN_LEGACY_MAX_VCPUS. + * Ensure MAX_VIRT_CPUS is defined accordingly. + */ +# if !defined(MAX_VIRT_CPUS) && defined(XEN_LEGACY_MAX_VCPUS) +# define MAX_VIRT_CPUS XEN_LEGACY_MAX_VCPUS +# endif + extern int xenRegister (void); # define XEN_UNIFIED_HYPERVISOR_OFFSET 0 diff --git a/src/xen/xen_hypervisor.c b/src/xen/xen_hypervisor.c index 3797865..ec726fe 100644 --- a/src/xen/xen_hypervisor.c +++ b/src/xen/xen_hypervisor.c @@ -109,14 +109,6 @@ typedef privcmd_hypercall_t hypercall_t; # define SYS_IFACE_MIN_VERS_NUMA 4 #endif -/* xen-unstable changeset 19788 removed MAX_VIRT_CPUS from public - * headers. Its semanitc was retained with XEN_LEGACY_MAX_VCPUS. - * Ensure MAX_VIRT_CPUS is defined accordingly. - */ -#if !defined(MAX_VIRT_CPUS) && defined(XEN_LEGACY_MAX_VCPUS) -# define MAX_VIRT_CPUS XEN_LEGACY_MAX_VCPUS -#endif - static int xen_ioctl_hypercall_cmd = 0; static int initialized = 0; static int in_init = 0; diff --git a/src/xen/xend_internal.c b/src/xen/xend_internal.c index 974e96b..614c036 100644 --- a/src/xen/xend_internal.c +++ b/src/xen/xend_internal.c @@ -2192,7 +2192,7 @@ xenDaemonParseSxpr(virConnectPtr conn, } def->maxvcpus = sexpr_int(root, "domain/vcpus"); - def->vcpus = count_one_bits(sexpr_int(root, "domain/vcpu_avail")); + def->vcpus = count_one_bits_l(sexpr_u64(root, "domain/vcpu_avail")); if (!def->vcpus || def->maxvcpus < def->vcpus) def->vcpus = def->maxvcpus; @@ -2468,7 +2468,7 @@ sexpr_to_xend_domain_info(virDomainPtr domain, const struct sexpr *root, } info->cpuTime = sexpr_float(root, "domain/cpu_time") * 1000000000; vcpus = sexpr_int(root, "domain/vcpus"); - info->nrVirtCpu = count_one_bits(sexpr_int(root, "domain/vcpu_avail")); + info->nrVirtCpu = count_one_bits_l(sexpr_u64(root, "domain/vcpu_avail")); if (!info->nrVirtCpu || vcpus < info->nrVirtCpu) info->nrVirtCpu = vcpus; @@ -3706,7 +3706,7 @@ xenDaemonDomainGetVcpusFlags(virDomainPtr domain, unsigned int flags) ret = sexpr_int(root, "domain/vcpus"); if (!(flags & VIR_DOMAIN_VCPU_MAXIMUM)) { - int vcpus = count_one_bits(sexpr_int(root, "domain/vcpu_avail")); + int vcpus = count_one_bits_l(sexpr_u64(root, "domain/vcpu_avail")); if (vcpus) ret = MIN(vcpus, ret); } @@ -5770,10 +5770,11 @@ xenDaemonFormatSxpr(virConnectPtr conn, virBufferVSprintf(&buf, "(memory %lu)(maxmem %lu)", def->mem.cur_balloon/1024, def->mem.max_balloon/1024); virBufferVSprintf(&buf, "(vcpus %u)", def->maxvcpus); - /* Computing the vcpu_avail bitmask works because MAX_VIRT_CPUS is 32. */ - verify(MAX_VIRT_CPUS <= sizeof(1U) * CHAR_BIT); + /* Computing the vcpu_avail bitmask works because MAX_VIRT_CPUS is + either 32, or 64 on a platform where long is big enough. */ + verify(MAX_VIRT_CPUS <= sizeof(1UL) * CHAR_BIT); if (def->vcpus < def->maxvcpus) - virBufferVSprintf(&buf, "(vcpu_avail %u)", (1U << def->vcpus) - 1); + virBufferVSprintf(&buf, "(vcpu_avail %lu)", (1UL << def->vcpus) - 1); if (def->cpumask) { char *ranges = virDomainCpuSetFormat(def->cpumask, def->cpumasklen); @@ -5870,8 +5871,8 @@ xenDaemonFormatSxpr(virConnectPtr conn, virBufferVSprintf(&buf, "(vcpus %u)", def->maxvcpus); if (def->vcpus < def->maxvcpus) - virBufferVSprintf(&buf, "(vcpu_avail %u)", - (1U << def->vcpus) - 1); + virBufferVSprintf(&buf, "(vcpu_avail %lu)", + (1UL << def->vcpus) - 1); for (i = 0 ; i < def->os.nBootDevs ; i++) { switch (def->os.bootDevs[i]) { diff --git a/src/xen/xm_internal.c b/src/xen/xm_internal.c index f80e252..6c5df0f 100644 --- a/src/xen/xm_internal.c +++ b/src/xen/xm_internal.c @@ -776,7 +776,7 @@ xenXMDomainConfigParse(virConnectPtr conn, virConfPtr conf) { def->maxvcpus = count; if (xenXMConfigGetULong(conf, "vcpu_avail", &count, -1) < 0) goto cleanup; - def->vcpus = MIN(count_one_bits(count), def->maxvcpus); + def->vcpus = MIN(count_one_bits_l(count), def->maxvcpus); if (xenXMConfigGetString(conf, "cpus", &str, NULL) < 0) goto cleanup; @@ -2336,10 +2336,11 @@ virConfPtr xenXMDomainConfigFormat(virConnectPtr conn, if (xenXMConfigSetInt(conf, "vcpus", def->maxvcpus) < 0) goto no_memory; - /* Computing the vcpu_avail bitmask works because MAX_VIRT_CPUS is 32. */ - verify(MAX_VIRT_CPUS <= sizeof(1U) * CHAR_BIT); + /* Computing the vcpu_avail bitmask works because MAX_VIRT_CPUS is + either 32, or 64 on a platform where long is big enough. */ + verify(MAX_VIRT_CPUS <= sizeof(1UL) * CHAR_BIT); if (def->vcpus < def->maxvcpus && - xenXMConfigSetInt(conf, "vcpu_avail", (1U << def->vcpus) - 1) < 0) + xenXMConfigSetInt(conf, "vcpu_avail", (1UL << def->vcpus) - 1) < 0) goto no_memory; if ((def->cpumask != NULL) && -- 1.7.2.3

5 12

[libvirt] could no query memory balloon allocation
by Marc Haber 03 Nov '10

03 Nov '10

Hi, I am playing around with libvirt and KVM on a Debian sid system (with a self-built kernel 2.6.36, 64bit kernel and 32bit userland). I have pulled libvirt 0.8.4 from experimental. When I try staring a VM using virtu-manager, I get the error message "Error starting domain: operation failed: could not query memory balloon allocation". This is probably some weird timing issue - when I have an strace attached to the libvirtd, everything is fine. In the #kvm IRC channel, people have told me that there may be an issue between some kernel versions and some libvirt versions, but neither the list archives nor the git changelog of 0.8.5 indicate such an issue (and #virt on oftc is quiet as usual). Can anybody explain what is going on on my system and what I could do to work around the issue? Greetings Marc -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834 Nordisch by Nature | How to make an American Quilt | Fax: *49 3221 2323190

1 0

[libvirt] [PATCH] Fix build with polkit 0
by Jim Fehlig 03 Nov '10

03 Nov '10

Commit e8066d53 broke the build with polkit0: remote.c: In function 'remoteDispatchAuthPolkit': remote.c:4177: error: 'rv' undeclared (first use in this function) Add missing identifier. --- daemon/remote.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/daemon/remote.c b/daemon/remote.c index 886d53d..3cf3886 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -4152,6 +4152,7 @@ remoteDispatchAuthPolkit (struct qemud_server *server, DBusError err; const char *action; char ident[100]; + int rv; memset(ident, 0, sizeof ident); -- 1.6.0.2

2 2

[libvirt] how to attach PCI/USB device
by Zdenek Styblik 02 Nov '10

02 Nov '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list and its users, as subject suggests, I'm trying to attach PCI/USB devices to Guest(VM). A-and, it doesn't work. But I swear I had it before(somewhat in the beginnings). I'm using virt-managers to add/attach devices. I've failed to attach PCI device, but it "does not" matter right now. I can attach USB devices, but they are not in VM. I've tried to attach USB HUB as well and what not - nothing. So, could somebody please tell me, how to debug it? Could it be somehow connected with qemu-kvm running 'root-less'? libvirt-0.8.4 and 0.8.5. Thank you all and nice whatever-timezone-you-are-in, Zdenek - -- Zdenek Styblik Net/Linux admin OS TurnovFree.net email: stybla(a)turnovfree.net jabber: stybla(a)jabber.turnovfree.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkzPCuQACgkQ8MreUbSH7imBMACcCW+DQ4lwNmcPiM9kXNjbHIWD o68An3RwjHN98zlQumJsCGbnG5Xie0WB =4ZBd -----END PGP SIGNATURE-----

3 6

[libvirt] libvirtd: how to react on qemu watchdog events
by Hu Tao 02 Nov '10

02 Nov '10

Hi, I'm currently seeking a way to monitor qemu watchdog events and do somthing based on these events. Although there is a good example at libvirt-source/examples/domain-events/events-c/event-test.c to start with, and I did achieve my goal by little modification of the example, but it ran as a libvirt client. How can I make libvirtd monitor the events and react on them? Thanks a lot!

2 1

[libvirt] [PATCH] Fix several warnings about a non-literal format string
by Jiri Denemark 02 Nov '10

02 Nov '10

They only popped up during --disable-nls build. Without this configure option, gcc wasn't able to detect them. --- I already pushed this since it's both trivial and build-breaking. src/esx/esx_vmx.c | 2 +- src/util/iptables.c | 2 +- src/util/network.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/esx/esx_vmx.c b/src/esx/esx_vmx.c index 0a26614..36818df 100644 --- a/src/esx/esx_vmx.c +++ b/src/esx/esx_vmx.c @@ -2611,7 +2611,7 @@ esxVMX_FormatConfig(esxVMX_Context *ctx, virCapsPtr caps, virDomainDefPtr def, /* def:maxvcpus -> vmx:numvcpus */ if (def->vcpus != def->maxvcpus) { - ESX_ERROR(VIR_ERR_CONFIG_UNSUPPORTED, + ESX_ERROR(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("No support for domain XML entry 'vcpu' attribute " "'current'")); goto cleanup; diff --git a/src/util/iptables.c b/src/util/iptables.c index fc656b9..fe78d1c 100644 --- a/src/util/iptables.c +++ b/src/util/iptables.c @@ -334,7 +334,7 @@ static char *iptablesFormatNetwork(virSocketAddr *netaddr, if (!VIR_SOCKET_IS_FAMILY(netaddr, AF_INET) || !VIR_SOCKET_IS_FAMILY(netmask, AF_INET)) { - iptablesError(VIR_ERR_CONFIG_UNSUPPORTED, + iptablesError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("Only IPv4 addresses can be used with iptables")); return NULL; } diff --git a/src/util/network.c b/src/util/network.c index 81ac228..1abe78b 100644 --- a/src/util/network.c +++ b/src/util/network.c @@ -80,7 +80,7 @@ virSocketParseAddr(const char *val, virSocketAddrPtr addr, int family) { int err; if (val == NULL) { - virSocketError(VIR_ERR_INVALID_ARG, _("Missing address")); + virSocketError(VIR_ERR_INVALID_ARG, "%s", _("Missing address")); return -1; } @@ -173,7 +173,7 @@ virSocketFormatAddrFull(virSocketAddrPtr addr, int err; if (addr == NULL) { - virSocketError(VIR_ERR_INVALID_ARG, _("Missing address")); + virSocketError(VIR_ERR_INVALID_ARG, "%s", _("Missing address")); return NULL; } -- 1.7.3.2

1 0