- Devel - libvirt List Archives

[libvirt] Fix slow storage file allocation with O_DSYNC
by Jiri Denemark 16 Mar '10

16 Mar '10

Hi. By opening storage file with O_DSYNC before allocating disk blocks for it we made the whole operation terribly slow when the allocation is done using posix_fallocate() on a filesystem which does not support fallocate(), i.e. anything but ext4. And by terribly slow I mean 45 minutes (RHEL5) to 10+ hours (Fedora Rawhide) for a 12GB storage. To fix this issue, we have two options. Either avoid using fallocate() emulation inside posix_fallocate() or stop using O_DSYNC. I prepared a patch for each of these options and they will follow as replies to this introduction. Some numbers for the same 12GB file: - libvirt's internal fallocate emulation using - mmap ~3.5 minutes (will be used whenever mmap() is present) - write ~5 minutes - fsync() instead of O_DSYNC - ~3.5 minutes So basically the two options are equivalent wrt to time consumption. The second one seems to be less hacky at the expense of non-lienar behavior which makes progress reporting less useful. Jirka

3 11

[libvirt] 'build' on FS pool now unconditionally formats?
by Cole Robinson 16 Mar '10

16 Mar '10

Hi guys, Looking at the new FS pool build options and talking with Dave, I see that calling PoolBuild on an FS pool now unconditionally calls mkfs. This is really bad when mixed with virt-manager: previously, we assumed the FS build command was always non destructive (at most it created a directory), so we called it every time, and didn't even allow users to opt out, since there wasn't a use case that called for it. This new formatting behavior really needs to be opt in, otherwise all virt-manager versions creating an FS pool can destroy data. Just FYI, for disk pools (and certain LVM configurations) where this operation has always been destructive, we default to build=off, and loudly warn the user if they choose otherwise. We can do that with this new option as well, but the previous behavior really needs to be reinstated IMO (and before the new release). I fully accept that this could be a bug in virt-manager's assumptions of the build command, but even consider a virsh user: previously build just created a directory, now it formats a partition, without any XML change. Thanks, Cole

4 11

[libvirt] XenAPI remote storage - target path
by Sharadha Prabhakar (3P) 16 Mar '10

16 Mar '10

Hi, I'm trying to write a Remote Storage driver for XenAPI. I see that target-path is used for both storage pools and volumes. In the case of XenAPI remote storage, the storage is not mounted on the local host where libvirt is running. Storage is maintained in a remote location only. In this case how do I specify target-path and how do I go about creating a VM with storage using libvirt APIs with virsh and virt-manager. Virt-manager expects me to give an absolute path for target-path. But In my case I don't have an absolute path. Can I have a target path like this for a particular storage volume "/storage pool uuid/storage-vol uuid"? This will help me identify which storage pool is Libvirt talking about and which volume in it. Using this information I can fetch data from The remote location and give it back to libvirt. Is this approach ok or does libvirt support my specific remote storage case in some way. Could someone clarify please? Regards, Sharadha

2 2

[libvirt] RFC: Add further domain event callbacks
by Daniel P. Berrange 16 Mar '10

16 Mar '10

In the current domain APIs, we currently have support for getting notified of domain lifecycle transition events. THis is done using two methods int virConnectDomainEventRegister(virConnectPtr conn, virConnectDomainEventCallback cb, void *opaque, virFreeCallback freecb); int virConnectDomainEventDeregister(virConnectPtr conn, virConnectDomainEventCallback cb); This allows an app to register a callback that looks like this typedef int (*virConnectDomainEventCallback)(virConnectPtr conn, virDomainPtr dom, int event, int detail, void *opaque); Where 'event' is the lifecycle transition (suspended, stopped, started, etc) and 'detail' is the cause of he transition (pause, migration, shutdown, etc) I have outstanding feature requests to add a lot more event notifications to the libvirt API. In particular - IO Errors. Parameter 'alias' the name of the block device with the error - Reboot. No parameters You can argue whether this should be part of the lifecycle events. On the one hand it is not guest visible, because the guest does a internal machine reset without the host ever seeing a change. This is different from other cases where QEMU itself is stopping/starting. - Watchdog. Parameter 'action', saying what is going to happen to the guest due to this watchdog firing (ignored, shutdown, paused, etc) - VNC client. In fact three events, connect, authenticated and disconnect. Parameters, TCP address & port number of client, and also of the server. Optionally a SASL username and TLS certificate name of the authenticated user - Guest user. Logon/logoff events. This requires co-operation from a guest agent, so it may not actually be of scope of libvirt. - Disk 'high watermark'. Emitted when a QCow volume grows beyond a certain physical allocation. THis allows an app to enlarge the underlying storage holding the qcow volume before an 'out of space' occurs. Parameter is the disk alias name. So we can see there are events with a wide variety of parameters and we need to figure out how to represent this in the API. Option 1 -------- Follow the existing lifecycle event model. For each new event, add a virConnectXXXXXEventRegister & virConnectEventDeregister method, and typedef a new callback for them. eg typedef int (*virConnectDomainBlockIOEventCallback)(virConnectPtr conn, virDomainPtr dom, const char *diskname, const char *alias, void *opaque); int virConnectDomainBlockIOEventRegister(virConnectPtr conn, virConnectDomainBlockIOEventCallback cb, void *opaque, virFreeCallback freecb); int virConnectDomainBlockIOEventDeregister(virConnectPtr conn, virConnectDomainEventCallback cb); So we'll have 2 extra APIs for every event + a new typedef. We'll also need to add new APIs in src/conf/domain_event.h to cope with dispatch Option 2 -------- GLib/GObject take a very loosely typed approach to registering/unregistering events. The have a single pair of methods that work for any event & a generic callback signature, requiring application casts. typedef int (*virConnectEventCallback)(void *opaque); int virConnectEventRegister(virConnectPtr conn, const char *eventname, virConnectEventCallback cb, void *opaque, virFreeCallback freecb); int virCOnnectEventUnregister(virConnectPtr conn, int eventID); In this model, the register method returns a unique integer ID for the callback which can be used to unregister it. Application's using this will still need a strongly typed callback for receiving the event, but when calling virConnectEventRegister(), the would do an explicit 'bad' cast to 'virConnectEventCallback' Option 3 -------- A hybrid of both approaches. Have a new 'register' method for each type of event that takes a strongly typed callback, but have a generic 'unregister' method that just uses the 'int eventID' int virConnectDomainBlockIOEventRegister(virConnectPtr conn, virConnectDomainBlockIOEventCallback cb, void *opaque, virFreeCallback freecb); int virCOnnectEventUnregister(virConnectPtr conn, int eventID); Option 4 -------- Have one pair of register/unregister events, but instead of passing diffeerent parameters to each callback, have a generic callback that takes a single parameter. This parameter would be declared as a union. So depending on the type of event being received, you'd access different parts of the union typedef union { virConnectDomainBlockIOEvent blockio; virConnectDomainWatchdogEvent watchdog; ...other events... } virConnectEvent; Either we could include a dummy member in the union with padding to 1024 bytes in size for future expansion, or we could simply declare that apps must never allocate this data type themselves, thus allowing us to enlarge it at will. typedef int (*virConnectEventCallback)(int eventType, virConnectEvent, void *opaque); int virConnectEventRegister(virConnectPtr conn, const char *eventname, virConnectEventCallback cb, void *opaque, virFreeCallback freecb); int virConnectEventUnregister(virConnectPtr conn, int eventID); There is one final question unrelated to these 4 options. For the lifecycle events we always registered against the 'virConnectPtr' since that is needed to capture 'domain created' events where there's no virDomainPtr to register a callback against yet. Do we want to always register all events aganist the virConnectPtr, and then pass a 'virDomainPtr' as a parameter to the callbacks as needed. Or should we allow registering events against the virDomainPtr directly. The latter might make it simpler to map libvirt into GLib/GObjects event system in the future. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

2 2

[libvirt] emulation failed (pagetable) rip
by Adam Mooz 15 Mar '10

15 Mar '10

Good morning list, I run a few VM's using libvirt on a C2Q 6600, have been steadily and happily since November, most of the VM's have been happily chugging away since then as well. I woke on Thursday to find one of the VM's had crashed, couldn't login and wasn't responding to the "virsh shutdown" commands. Now when I try to start it; it comes up as 'running' according to virsh but doesn't even request an IP from my DHCP server. I've enabled info logging and the log output looks like this: > Mar 14 09:52:55 Godfather kernel: [24816.021257] __ratelimit: 932597 callbacks suppressed > Mar 14 09:52:55 Godfather kernel: [24816.021262] emulation failed (pagetable) rip 7f2019b7aafb 66 0f 7f 07 > Mar 14 09:52:55 Godfather kernel: [24816.021297] emulation failed (pagetable) rip 7f2019b7aafb 66 0f 7f 07 > Mar 14 09:52:55 Godfather kernel: [24816.021321] emulation failed (pagetable) rip 7f2019b7aafb 66 0f 7f 07 > Mar 14 09:52:55 Godfather kernel: [24816.021344] emulation failed (pagetable) rip 7f2019b7aafb 66 0f 7f 07 Any ideas on fixing this? ----------------------------------------------------------------- Adam Mooz Adam.Mooz(a)gmail.com http://www.AdamMooz.com

2 2

[libvirt] [PATCH] security: Set permissions for kernel/initrd
by Cole Robinson 15 Mar '10

15 Mar '10

Fixes URL installs when running virt-install as root on Fedora. Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- src/qemu/qemu_security_dac.c | 21 +++++++++++++++++++++ src/security/security_selinux.c | 16 ++++++++++++++++ 2 files changed, 37 insertions(+), 0 deletions(-) diff --git a/src/qemu/qemu_security_dac.c b/src/qemu/qemu_security_dac.c index 6911f48..1883fbe 100644 --- a/src/qemu/qemu_security_dac.c +++ b/src/qemu/qemu_security_dac.c @@ -332,6 +332,15 @@ qemuSecurityDACRestoreSecurityAllLabel(virDomainObjPtr vm) vm->def->disks[i]) < 0) rc = -1; } + + if (vm->def->os.kernel && + qemuSecurityDACRestoreSecurityFileLabel(vm->def->os.kernel) < 0) + rc = -1; + + if (vm->def->os.initrd && + qemuSecurityDACRestoreSecurityFileLabel(vm->def->os.initrd) < 0) + rc = -1; + return rc; } @@ -356,6 +365,18 @@ qemuSecurityDACSetSecurityAllLabel(virDomainObjPtr vm) return -1; } + if (vm->def->os.kernel && + qemuSecurityDACSetOwnership(vm->def->os.kernel, + driver->user, + driver->group) < 0) + return -1; + + if (vm->def->os.initrd && + qemuSecurityDACSetOwnership(vm->def->os.initrd, + driver->user, + driver->group) < 0) + return -1; + return 0; } diff --git a/src/security/security_selinux.c b/src/security/security_selinux.c index b2c8581..975b315 100644 --- a/src/security/security_selinux.c +++ b/src/security/security_selinux.c @@ -616,6 +616,14 @@ SELinuxRestoreSecurityAllLabel(virDomainObjPtr vm) rc = -1; } + if (vm->def->os.kernel && + SELinuxRestoreSecurityFileLabel(vm->def->os.kernel) < 0) + rc = -1; + + if (vm->def->os.initrd && + SELinuxRestoreSecurityFileLabel(vm->def->os.initrd) < 0) + rc = -1; + return rc; } @@ -736,6 +744,14 @@ SELinuxSetSecurityAllLabel(virDomainObjPtr vm) return -1; } + if (vm->def->os.kernel && + SELinuxSetFilecon(vm->def->os.kernel, default_content_context) < 0) + return -1; + + if (vm->def->os.initrd && + SELinuxSetFilecon(vm->def->os.initrd, default_content_context) < 0) + return -1; + return 0; } -- 1.6.6.1

2 2

[libvirt] [PATCH] qemu: Fix USB by product with security enabled
by Cole Robinson 15 Mar '10

15 Mar '10

We need to call PrepareHostdevs to determine the USB device path before any security calls. PrepareHostUSBDevices was also incorrectly skipping all USB devices. Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- src/qemu/qemu_driver.c | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 26b5600..0fa9b2f 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2360,7 +2360,7 @@ qemuPrepareHostUSBDevices(struct qemud_driver *driver ATTRIBUTE_UNUSED, if (hostdev->mode != VIR_DOMAIN_HOSTDEV_MODE_SUBSYS) continue; - if (hostdev->source.subsys.type != VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI) + if (hostdev->source.subsys.type != VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_USB) continue; /* Resolve a vendor/product to bus/device */ @@ -2703,6 +2703,11 @@ static int qemudStartVMDaemon(virConnectPtr conn, return -1; } + DEBUG0("Preparing host devices"); + /* Must be run before security labelling */ + if (qemuPrepareHostDevices(driver, vm->def) < 0) + goto cleanup; + DEBUG0("Generating domain security label (if required)"); /* If you are using a SecurityDriver with dynamic labelling, then generate a security label for isolation */ @@ -2769,10 +2774,6 @@ static int qemudStartVMDaemon(virConnectPtr conn, if (qemuSetupCgroup(driver, vm) < 0) goto cleanup; - DEBUG0("Preparing host devices"); - if (qemuPrepareHostDevices(driver, vm->def) < 0) - goto cleanup; - if (VIR_ALLOC(priv->monConfig) < 0) { virReportOOMError(); goto cleanup; -- 1.6.6

3 3

[libvirt] compile libvirt in windows
by Luis Costa 15 Mar '10

15 Mar '10

Dear friends It would like to know as I I make to compile libvirt in the Windows? I already installed the MingGW, you I have some cake prescription? Regards, -- Luís Carlos Moreira da Costa Eclipse RAP, RCP, GMF, OSGI, Spring-DM and Pentaho Developer Regional Communities/Brazil http://wiki.eclipse.org/Regional_Communities/Brazil Java, write once, run everywhere™

2 1

[libvirt] libvirt modifying iptables rules
by Varrun Ramani 15 Mar '10

15 Mar '10

Hi I am right now undertaking a project which deals with verification of firewall rules. I wish to know which applications/libraries modify/query firewall rules. I came to know that libvirt modifies iptables rules. Can anyone let me know for what purposes/how libvirt modifies the rules? Thanks & Regards Varrun Ramani

4 3

[libvirt] [PATCH] qemu: pass the information when disks are read-only
by Daniel Veillard 15 Mar '10

15 Mar '10

The problem is that even when the user indicates that a drive has to be opened as read-only (for example a cd-rom) qemu didn't get the information and tried to open read-write. This could get really nasty in case SELinux is monitoring that no read-write access is attempted: https://bugzilla.redhat.com/show_bug.cgi?id=536760 Patch is relatively simple and based on teh assumption that if the -device qemu options are allowed then qemu also knows about the readonly option (this was added to the help string quite later so we should not rely on that in this case). * src/qemu/qemu_conf.c: add the ",readonly=on" for read-only disks and also parse it back in qemuParseCommandLineDisk() * tests/qemuxml2argvtest.c tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.args tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.xml: add a specific regression test diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 40ca221..fb23c52 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -2398,6 +2398,9 @@ qemuBuildDriveStr(virDomainDiskDefPtr disk, if (bootable && disk->device == VIR_DOMAIN_DISK_DEVICE_DISK) virBufferAddLit(&opt, ",boot=on"); + if (disk->readonly && + qemuCmdFlags & QEMUD_CMD_FLAG_DEVICE) + virBufferAddLit(&opt, ",readonly=on"); if (disk->driverType && disk->type != VIR_DOMAIN_DISK_TYPE_DIR && qemuCmdFlags & QEMUD_CMD_FLAG_DRIVE_FORMAT) @@ -4694,6 +4697,9 @@ qemuParseCommandLineDisk(const char *val, _("cannot parse drive unit '%s'"), val); goto cleanup; } + } else if (STREQ(keywords[i], "readonly")) { + if ((values[i] == NULL) || STREQ(values[i], "on")) + def->readonly = 1; } } diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.args b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.args new file mode 100644 index 0000000..fbc3226 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.args @@ -0,0 +1 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test /usr/bin/qemu -S -M pc -m 214 -smp 1 -nographic -nodefaults -monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c -drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-ide0-0-0 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=/dev/sr0,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -usb -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.xml b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.xml new file mode 100644 index 0000000..39c3a1c --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-readonly-disk.xml @@ -0,0 +1,30 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219200</memory> + <currentMemory>219200</currentMemory> + <vcpu>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' unit='0'/> + </disk> + <disk type='block' device='cdrom'> + <source dev='/dev/sr0'/> + <target dev='hdc' bus='ide'/> + <readonly/> + <address type='drive' controller='0' bus='1' unit='0'/> + </disk> + <controller type='ide' index='0'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 4ca946a..e3762c9 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -249,6 +249,8 @@ mymain(int argc, char **argv) QEMUD_CMD_FLAG_DRIVE_BOOT | QEMUD_CMD_FLAG_DRIVE_FORMAT); DO_TEST("disk-drive-fat", QEMUD_CMD_FLAG_DRIVE | QEMUD_CMD_FLAG_DRIVE_BOOT | QEMUD_CMD_FLAG_DRIVE_FORMAT); + DO_TEST("disk-drive-readonly-disk", QEMUD_CMD_FLAG_DRIVE | + QEMUD_CMD_FLAG_DEVICE); DO_TEST("disk-drive-fmt-qcow", QEMUD_CMD_FLAG_DRIVE | QEMUD_CMD_FLAG_DRIVE_BOOT | QEMUD_CMD_FLAG_DRIVE_FORMAT); DO_TEST("disk-drive-shared", QEMUD_CMD_FLAG_DRIVE | -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

2 2