August 2011 - Devel - libvirt List Archives

[libvirt] [PATCH] Fix python error reporting for some storage operations
by Cole Robinson 07 Mar '25

07 Mar '25

In the python bindings, all vir* classes expect to be passed a virConnect object when instantiated. Before the storage stuff, these classes were only instantiated in virConnect methods, so the generator is hardcoded to pass 'self' as the connection instance to these classes. Problem is there are some methods that return pool or vol instances which aren't called from virConnect: you can lookup a storage volume's associated pool, and can lookup volumes from a pool. In these cases passing 'self' doesn't give the vir* instance a connection, so when it comes time to raise an exception crap hits the fan. Rather than rework the generator to accomodate this edge case, I just fixed the init functions for virStorage* to pull the associated connection out of the passed value if it's not a virConnect instance. Thanks, Cole diff --git a/python/generator.py b/python/generator.py index 01a17da..c706b19 100755 --- a/python/generator.py +++ b/python/generator.py @@ -962,8 +962,12 @@ def buildWrappers(): list = reference_keepers[classname] for ref in list: classes.write(" self.%s = None\n" % ref[1]) - if classname in [ "virDomain", "virNetwork", "virStoragePool", "virStorageVol" ]: + if classname in [ "virDomain", "virNetwork" ]: classes.write(" self._conn = conn\n") + elif classname in [ "virStorageVol", "virStoragePool" ]: + classes.write(" self._conn = conn\n" + \ + " if not isinstance(conn, virConnect):\n" + \ + " self._conn = conn._conn\n") classes.write(" if _obj != None:self._o = _obj;return\n") classes.write(" self._o = None\n\n"); destruct=None

4 3

[libvirt] [PATCH] web docs -- macvtap mode explanation
by Stefan Berger 27 Oct '23

27 Oct '23

This adds more information about the different macvtap device modes. Regards, Stefan

3 2

[libvirt] Supporting vhost-net and macvtap in libvirt for QEMU
by Anthony Liguori 27 Oct '23

27 Oct '23

Disclaimer: I am neither an SR-IOV nor a vhost-net expert, but I've CC'd people that are who can throw tomatoes at me for getting bits wrong :-) I wanted to start a discussion about supporting vhost-net in libvirt. vhost-net has not yet been merged into qemu but I expect it will be soon so it's a good time to start this discussion. There are two modes worth supporting for vhost-net in libvirt. The first mode is where vhost-net backs to a tun/tap device. This is behaves in very much the same way that -net tap behaves in qemu today. Basically, the difference is that the virtio backend is in the kernel instead of in qemu so there should be some performance improvement. Current, libvirt invokes qemu with -net tap,fd=X where X is an already open fd to a tun/tap device. I suspect that after we merge vhost-net, libvirt could support vhost-net in this mode by just doing -net vhost,fd=X. I think the only real question for libvirt is whether to provide a user visible switch to use vhost or to just always use vhost when it's available and it makes sense. Personally, I think the later makes sense. The more interesting invocation of vhost-net though is one where the vhost-net device backs directly to a physical network card. In this mode, vhost should get considerably better performance than the current implementation. I don't know the syntax yet, but I think it's reasonable to assume that it will look something like -net tap,dev=eth0. The effect will be that eth0 is dedicated to the guest. On most modern systems, there is a small number of network devices so this model is not all that useful except when dealing with SR-IOV adapters. In that case, each physical device can be exposed as many virtual devices (VFs). There are a few restrictions here though. The biggest is that currently, you can only change the number of VFs by reloading a kernel module so it's really a parameter that must be set at startup time. I think there are a few ways libvirt could support vhost-net in this second mode. The simplest would be to introduce a new tag similar to <source network='br0'>. In fact, if you probed the device type for the network parameter, you could probably do something like <source network='eth0'> and have it Just Work. Another model would be to have libvirt see an SR-IOV adapter as a network pool whereas it handled all of the VF management. Considering how inflexible SR-IOV is today, I'm not sure whether this is the best model. Has anyone put any more thought into this problem or how this should be modeled in libvirt? Michael, could you share your current thinking for -net syntax? -- Regards, Anthony Liguori

8 26

[libvirt] [PATCH 0/4] Multiple problems with saving to block devices
by Daniel P. Berrange 24 Mar '13

24 Mar '13

This patch series makes it possible to save to a block device, instead of a plain file. There were multiple problems - WHen save failed, we might de-reference a NULL pointer - When save failed, we unlinked the device node !! - The approach of using >> to append, doesn't work with block devices - CGroups was blocking QEMU access to the block device when enabled One remaining problem is not in libvirt, but rather QEMU. The QEMU exec: based migration often fails to detect failure of the command and will thus hang forever attempting a migration that'll never succeed! Fortunately you can now work around this in libvirt using the virsh domjobabort command

4 21

[libvirt] I need help for use virDomainInterfaceStats
by Фёдор Пранович 03 Apr '12

03 Apr '12

Hi I want use the virDomainInterfaceStats for get statistic of domain, but I don't now how get path parametr. Please help me! Thanks.

4 3

[libvirt] [PATCH 0/2] Introduce two new virsh commands
by Osier Yang 20 Dec '11

20 Dec '11

These two patches is to introduce two new virsh commands, one is eject-media, which is to eject media from CD or floppy drive, the other is insert-media, which is to insert media into CD or floppy drive. There are commands existed can be used to eject/insert media, such as "update-device", but it's not quite easy to use. That's the original intention of these patches. Both of the two commands only allow to operate on CDROM or floppy disk. [PATCH 1/2] virsh: Introduce two new commands to insert or eject media [PATCH 2/2] doc: Add docs for two new introduced commands Regards Osier

3 6

[libvirt] [PATCH 00/14] Add a virtlockd lock manager daemon
by Daniel P. Berrange 06 Dec '11

06 Dec '11

The lock manager infrastructure we recently added to QEMU only has two possible drivers at this time, 'nop' and 'sanlock'. The former does absolutely nothing, while the latter requires a 3rd party package installed and is a little heavy on disk I/O and storage requirements. This series adds a new daemon 'virtlockd' which is intended to be enabled by default on all hosts running 'libvirtd'. This daemon provides a service for disk locking based on the traditional fcntl() lock primitives. There is a new libvirt manager plugin which talks to this daemon over RPC. The reason for doing the locks in a separate process is that we want the locks to remain active, even if libvirtd crashes, or is restarted. The virtlockd daemon has this one single job so should be pretty reliable and selfcontained. This patch series really benefits from the new RPC APIs, requiring minimal code for the new daemon / client At this time, virtlockd does not lock the actual disk files, but instead creates a lockspace & leases under /var/lib/libvirt/lockd. The lockspace we use for disks is named org.libvirt.lockd.files, and lease names are based on a SHA256 checksum of the fully qualified disk name. eg /var/lib/libvirt/lockd/org.libvirt.lockd.files/adf94fc33a24da1abff7dd7374a9919bb51efee646da8c3ac464c10cd59750bd These leases are all zero-bytes long and no I/O is ever performed on them, only fcntl() is used. So there is material overhead. Whenever creating or deleting leases, we first acquire a lock on /var/lib/libvirt/lockd/org.libvirt.lockd.files/org.libvirt.lockd.index A non-root virtlockd will instead use $HOME/.libvirt/lockd By default we gain protection out of the box against - Starting two guests on the same host with the same disk image not marked with <shareable/> - libvirtd getting confused and forgetting a guest, allowing it to be started for a 2nd time If the admin mounts a shared filesytem (eg NFS) on /var/lib/libvirt/lockd then this protection is extended across all hosts sharing that mount volume. As part of this series, I also introduce support for systemd services for libvirtd and libvir-guests.

4 27

[libvirt] [RFC PATCH 0/5] Support online resizing of block devices.
by Osier Yang 29 Nov '11

29 Nov '11

This patch series introduce new API "virDomainBlockResize" to expose qemu monitor command "block_size", which is for resizing the a block device while the domain is running. The prototype for the new API is: int virDomainBlockResize (virDomainPtr dom, const char *path, unsigned long long size, unsigned int flags) * "@path" is the absolute path of the block device, which can be extraced from domain xml. * The units for "@size" is kilobytes, which might be not quite properly. (qemu HMP uses Megabytes as the default units, QMP uses Bytes as the default units, so it means we need to divice "@size" by 1024 for HMP, and multiply "@size" by 1024 for QMP. On the other hand, we need to check the overflowing). Any ideas on this is welcomed. * "@flags" is unused currently. [PATCH 1/5] block_resize: Define the new API [PATCH 2/5] block_resize: Wire up the remote protocol [PATCH 3/5] block_resize: Implement qemu monitor functions [PATCH 4/5] block_resize: Implement qemu driver method [PATCH 5/5] block_resize: Expose the new API to virsh

3 8

[libvirt] [PATCH 1/2] Fix order of disks and controllers
by Jiri Denemark 04 Nov '11

04 Nov '11

Commit 2d6adabd53c8f1858191d521dc9b4948d1205955 replaced qsorting disk and controller devices with inserting them at the right position. That was to fix unnecessary reordering of devices. However, when parsing domain XML devices are just taken in the order in which they appear in the XML since. Use the correct insertion algorithm to honor device target. --- src/conf/domain_conf.c | 4 +- .../qemuxml2argvdata/qemuxml2argv-boot-order.args | 35 ++++++++++++------- tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml | 14 ++++---- tests/xmconfigdata/test-escape-paths.cfg | 2 +- tests/xmconfigdata/test-escape-paths.xml | 10 +++--- 5 files changed, 37 insertions(+), 28 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index b406202..48ed7eb 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -5883,7 +5883,7 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps, if (!disk) goto error; - def->disks[def->ndisks++] = disk; + virDomainDiskInsertPreAlloced(def, disk); } VIR_FREE(nodes); @@ -5899,7 +5899,7 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps, if (!controller) goto error; - def->controllers[def->ncontrollers++] = controller; + virDomainControllerInsertPreAlloced(def, controller); } VIR_FREE(nodes); diff --git a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args index 23249f3..14367b1 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args @@ -1,13 +1,22 @@ -LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test /usr/bin/qemu -S -M \ -pc -m 214 -smp 1 -nographic -nodefaults -monitor unix:/tmp/test-monitor,server,\ -nowait -no-acpi -drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-ide0-0-0 \ --device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive \ -file=sheepdog:example.org:6000:image,if=none,id=drive-virtio-disk0 -device \ -virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,\ -bootindex=3 -drive file=/root/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0 \ --device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,\ -bootindex=1 -drive file=/dev/null,if=none,id=drive-fdc0-0-1 -global \ -isa-fdc.driveB=drive-fdc0-0-1 -global isa-fdc.bootindexB=4 -device \ -virtio-net-pci,vlan=0,id=net0,mac=00:11:22:33:44:55,bus=pci.0,addr=0x3,\ -bootindex=2 -net user,vlan=0,name=hostnet0 -usb -device virtio-balloon-pci,\ -id=balloon0,bus=pci.0,addr=0x5 +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test /usr/bin/qemu \ +-S \ +-M pc \ +-m 214 \ +-smp 1 \ +-nographic \ +-nodefaults \ +-monitor unix:/tmp/test-monitor,server,nowait \ +-no-acpi \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-ide0-0-0 \ +-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \ +-drive file=/root/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0 \ +-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 \ +-drive file=sheepdog:example.org:6000:image,if=none,id=drive-virtio-disk0 \ +-device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=3 \ +-drive file=/dev/null,if=none,id=drive-fdc0-0-1 \ +-global isa-fdc.driveB=drive-fdc0-0-1 \ +-global isa-fdc.bootindexB=4 \ +-device virtio-net-pci,vlan=0,id=net0,mac=00:11:22:33:44:55,bus=pci.0,addr=0x3,bootindex=2 \ +-net user,vlan=0,name=hostnet0 \ +-usb \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml index ba8a9b2..0022c92 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml @@ -18,6 +18,13 @@ <target dev='hda' bus='ide'/> <address type='drive' controller='0' bus='0' unit='0'/> </disk> + <disk type='file' device='cdrom'> + <source file='/root/boot.iso'/> + <target dev='hdc' bus='ide'/> + <boot order='1'/> + <readonly/> + <address type='drive' controller='0' bus='1' unit='0'/> + </disk> <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <source protocol='sheepdog' name='image'> @@ -26,13 +33,6 @@ <target dev='vda' bus='virtio'/> <boot order='3'/> </disk> - <disk type='file' device='cdrom'> - <source file='/root/boot.iso'/> - <target dev='hdc' bus='ide'/> - <boot order='1'/> - <readonly/> - <address type='drive' controller='0' bus='1' unit='0'/> - </disk> <disk type='file' device='floppy'> <driver name='qemu' type='raw'/> <source file='/dev/null'/> diff --git a/tests/xmconfigdata/test-escape-paths.cfg b/tests/xmconfigdata/test-escape-paths.cfg index e3e6db9..13be2a0 100644 --- a/tests/xmconfigdata/test-escape-paths.cfg +++ b/tests/xmconfigdata/test-escape-paths.cfg @@ -19,7 +19,7 @@ vnc = 1 vncunused = 1 vnclisten = "127.0.0.1" vncpasswd = "123poi" -disk = [ "phy:/dev/HostVG/XenGuest2,hda,w", "file:/root/boot.iso&test,hdc:cdrom,r", """phy:/dev/HostVG/XenGuest'",hdb,w""" ] +disk = [ "phy:/dev/HostVG/XenGuest2,hda,w", """phy:/dev/HostVG/XenGuest'",hdb,w""", "file:/root/boot.iso&test,hdc:cdrom,r" ] vif = [ "mac=00:16:3e:66:92:9c,bridge=xenbr1,script=vif-bridge,model=e1000,type=ioemu" ] parallel = "none" serial = "none" diff --git a/tests/xmconfigdata/test-escape-paths.xml b/tests/xmconfigdata/test-escape-paths.xml index 13e6e29..9eaf90c 100644 --- a/tests/xmconfigdata/test-escape-paths.xml +++ b/tests/xmconfigdata/test-escape-paths.xml @@ -25,17 +25,17 @@ <source dev='/dev/HostVG/XenGuest2'/> <target dev='hda' bus='ide'/> </disk> + <disk type='block' device='disk'> + <driver name='phy'/> + <source dev='/dev/HostVG/XenGuest'"'/> + <target dev='hdb' bus='ide'/> + </disk> <disk type='file' device='cdrom'> <driver name='file'/> <source file='/root/boot.iso&test'/> <target dev='hdc' bus='ide'/> <readonly/> </disk> - <disk type='block' device='disk'> - <driver name='phy'/> - <source dev='/dev/HostVG/XenGuest'"'/> - <target dev='hdb' bus='ide'/> - </disk> <interface type='bridge'> <mac address='00:16:3e:66:92:9c'/> <source bridge='xenbr1'/> -- 1.7.5.3

4 21

[libvirt] Notes from the KVM Forum relevant to libvirt
by Daniel P. Berrange 03 Nov '11

03 Nov '11

I was at the KVM Forum / LinuxCon last week and there were many interesting things discussed which are relevant to ongoing libvirt development. Here was the list that caught my attention. If I have missed any, fill in the gaps.... - Sandbox/container KVM. The Solaris port of KVM puts QEMU inside a zone so that an exploit of QEMU can't escape into the full OS. Containers are Linux's parallel of Zones, and while not nearly as secure yet, it would still be worth using more containers support to confine QEMU. - Events for object changes. We already have async events for virDomainPtr. We need the same for virInterfacePtr, virStoragePoolPtr, virStorageVolPtr and virNodeDevPtr, so that at the very least applications can be notified when objects are created or removed. For virNodeDevPtr we also want to be notified when properties change (ie CDROM media change). - CGroups passthrough. There is alot of experimentation with cgroups. We don't want to expose cgroups as a direct concept in the libvirt API, but we should consider putting a generic cgroups get/set in the libvirt-qemu.so library, or create a libvirt-linux.so library. Also likely add a <linux:cgroups> XML element to store arbitrary tunables in the XML. Same (low) level of support as with qemu:XXX of course - CPUSet for changing CPU + Memory NUMA pinning. The CPUset cgroups controller is able to actually move a guest's memory between NUMA nodes. We can already change VCPU pinning, but we need a new API to do node pinning of the whole VM, so we can ensure the I/O threads are also moved. We also need an API to move the memory pinning to new nodes. - Guest NUMA topology. If we have guests with RAM size > node size, we need to expose a NUMA topology into the guest. The CPU/memory pinning APIs will also need to be able to pin individual guest NUMA nodes to individual host NUMA nodes. - AHCI controller. IDE is going the way of the dodo. We need to add support for QEMU's new AHCI controller. This is quite simple, we already have a 'sata' disk type we can wire up to QEMU - VFIO PCI passthru. The current PCI assignment code may well be changed to use something called 'VFIO'. This will need some work in libvirt to support new CLI arg syntax, and probably some SELinux work - QCow3. There will soon be a QCow3 format. We need to add code to detect it and extract backing stores, etc. Trivial since the primary header format will still be the same as QCow2. - QMP completion. Given anthony's plan for a complete replacement of the current CLI + monitor syntax in QEMU 2.0 (long way out), he has dropped objections to adding new commands to QMP in the near future. So all existing HMP commands will immediately be made available in QMP with no attempt to re-design them now. So the need for the HMP passthrough command will soon go away. - Migration + VEPA/VNLink failures. As raised previously on this list, Cisco really wants libvirt to have the ability to do migration, and optionally *not* fail, even if the VEPA/VNLink setup fails. This will require an event notification to the app if a failure of a device backend occurs, and an API to let the admin app fix the device backend (virDomainUpdateDevice) and some way to tell migration what bits are allowed to fail. - Virtio SCSI. We need to support this new stuff in QEMU when it is eventually implemented. It will mean we avoid the PCI slot usage problems inherant in virtio-blk, and get other things like multipath and decent SCSI passthrough support. - USB 2.0. We need to support this in libvirt asap. It is very important for desktop experiance and to support better integration with SPICE This also gets us proper USB port addressing. Fun footnote, QEMU USB has *never* supported migration. The USB tablet only works by sheer luck, as OS' see the device disappear on migration & come back with different device ID/port addr and so does a re-initialize ! - Native KVM tool. The problem statement was that the QEMU code is too big/complex & and command line args are too complex, so lets rewrite from scratch to make the code small & CLI simple. They achieve this, but of course primarily because they lack so many features compared to QEMU. They had libvirt support as a bullet point on their preso, but I'm not expecting it to replace the current QEMU KVM support in the forseeable future, given its current level of features and the size of its dev team compared to QEMU/KVM. They did have some fun demos of booting using the host OS filesystem though. We can actually do the same with regular KVM/libvirt but there's no nice demo tool to show it off. I'm hoping to create one.... - Shared memory devices. Some people doing high performance work are using the QEMU shared memory device. We don't support this (ivhshm device) in libvirt yet. Fairly niche use cases but might be nice to have this. - SDK / Docs. Request for a more SDK like approach to KVM development tools and documentation. Also want to simplify libvirt operations. The exposure of the virt-install internal API as official GObjects would have significantly helped the project Ricardo (from IBM) described in his presentation. Of course no one can argue that we need more documentation in every area. - USB managed mode. As we do with PCI passthrough, we should be able to detach USB device from host OS, and perform a reset before attaching to the guest, and most importantly track which USB devices have been given to which guest, so we don't duplicate assign. We have all neccessary APIs, just need to wire them up. - PCI passthrough. We need to support setting of MAC addr, VLAN and VEPA/VNLink properties against VFs from SRIOV NICs that are assigned to a guest. For those who were not at the KVM Forum, the presentations are already available online at: http://www.linux-kvm.org/page/KVM_Forum_2011 All the session were also video recorded, so sometime in the next week or two, there should be OGG videos of the talks being uploaded to the same site. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

6 13