[libvirt] [PATCH] Fix python error reporting for some storage operations
by Cole Robinson
In the python bindings, all vir* classes expect to be
passed a virConnect object when instantiated. Before
the storage stuff, these classes were only instantiated
in virConnect methods, so the generator is hardcoded to
pass 'self' as the connection instance to these classes.
Problem is there are some methods that return pool or vol
instances which aren't called from virConnect: you can
lookup a storage volume's associated pool, and can lookup
volumes from a pool. In these cases passing 'self' doesn't
give the vir* instance a connection, so when it comes time
to raise an exception crap hits the fan.
Rather than rework the generator to accomodate this edge
case, I just fixed the init functions for virStorage* to
pull the associated connection out of the passed value
if it's not a virConnect instance.
Thanks,
Cole
diff --git a/python/generator.py b/python/generator.py
index 01a17da..c706b19 100755
--- a/python/generator.py
+++ b/python/generator.py
@@ -962,8 +962,12 @@ def buildWrappers():
list = reference_keepers[classname]
for ref in list:
classes.write(" self.%s = None\n" % ref[1])
- if classname in [ "virDomain", "virNetwork", "virStoragePool", "virStorageVol" ]:
+ if classname in [ "virDomain", "virNetwork" ]:
classes.write(" self._conn = conn\n")
+ elif classname in [ "virStorageVol", "virStoragePool" ]:
+ classes.write(" self._conn = conn\n" + \
+ " if not isinstance(conn, virConnect):\n" + \
+ " self._conn = conn._conn\n")
classes.write(" if _obj != None:self._o = _obj;return\n")
classes.write(" self._o = None\n\n");
destruct=None
1 month, 3 weeks
[libvirt] Supporting vhost-net and macvtap in libvirt for QEMU
by Anthony Liguori
Disclaimer: I am neither an SR-IOV nor a vhost-net expert, but I've CC'd
people that are who can throw tomatoes at me for getting bits wrong :-)
I wanted to start a discussion about supporting vhost-net in libvirt.
vhost-net has not yet been merged into qemu but I expect it will be soon
so it's a good time to start this discussion.
There are two modes worth supporting for vhost-net in libvirt. The
first mode is where vhost-net backs to a tun/tap device. This is
behaves in very much the same way that -net tap behaves in qemu today.
Basically, the difference is that the virtio backend is in the kernel
instead of in qemu so there should be some performance improvement.
Current, libvirt invokes qemu with -net tap,fd=X where X is an already
open fd to a tun/tap device. I suspect that after we merge vhost-net,
libvirt could support vhost-net in this mode by just doing -net
vhost,fd=X. I think the only real question for libvirt is whether to
provide a user visible switch to use vhost or to just always use vhost
when it's available and it makes sense. Personally, I think the later
makes sense.
The more interesting invocation of vhost-net though is one where the
vhost-net device backs directly to a physical network card. In this
mode, vhost should get considerably better performance than the current
implementation. I don't know the syntax yet, but I think it's
reasonable to assume that it will look something like -net
tap,dev=eth0. The effect will be that eth0 is dedicated to the guest.
On most modern systems, there is a small number of network devices so
this model is not all that useful except when dealing with SR-IOV
adapters. In that case, each physical device can be exposed as many
virtual devices (VFs). There are a few restrictions here though. The
biggest is that currently, you can only change the number of VFs by
reloading a kernel module so it's really a parameter that must be set at
startup time.
I think there are a few ways libvirt could support vhost-net in this
second mode. The simplest would be to introduce a new tag similar to
<source network='br0'>. In fact, if you probed the device type for the
network parameter, you could probably do something like <source
network='eth0'> and have it Just Work.
Another model would be to have libvirt see an SR-IOV adapter as a
network pool whereas it handled all of the VF management. Considering
how inflexible SR-IOV is today, I'm not sure whether this is the best model.
Has anyone put any more thought into this problem or how this should be
modeled in libvirt? Michael, could you share your current thinking for
-net syntax?
--
Regards,
Anthony Liguori
1 year, 6 months
[libvirt] [PATCH 0/4] Multiple problems with saving to block devices
by Daniel P. Berrange
This patch series makes it possible to save to a block device,
instead of a plain file. There were multiple problems
- WHen save failed, we might de-reference a NULL pointer
- When save failed, we unlinked the device node !!
- The approach of using >> to append, doesn't work with block devices
- CGroups was blocking QEMU access to the block device when enabled
One remaining problem is not in libvirt, but rather QEMU. The QEMU
exec: based migration often fails to detect failure of the command
and will thus hang forever attempting a migration that'll never
succeed! Fortunately you can now work around this in libvirt using
the virsh domjobabort command
12 years, 1 month
[libvirt] [PATCH 0/2] Introduce two new virsh commands
by Osier Yang
These two patches is to introduce two new virsh commands, one is
eject-media, which is to eject media from CD or floppy drive, the other
is insert-media, which is to insert media into CD or floppy drive.
There are commands existed can be used to eject/insert media, such as
"update-device", but it's not quite easy to use. That's the original
intention of these patches.
Both of the two commands only allow to operate on CDROM or floppy disk.
[PATCH 1/2] virsh: Introduce two new commands to insert or eject media
[PATCH 2/2] doc: Add docs for two new introduced commands
Regards
Osier
13 years, 4 months
[libvirt] [PATCH 00/14] Add a virtlockd lock manager daemon
by Daniel P. Berrange
The lock manager infrastructure we recently added to QEMU only has
two possible drivers at this time, 'nop' and 'sanlock'. The former
does absolutely nothing, while the latter requires a 3rd party
package installed and is a little heavy on disk I/O and storage
requirements.
This series adds a new daemon 'virtlockd' which is intended to be
enabled by default on all hosts running 'libvirtd'. This daemon
provides a service for disk locking based on the traditional
fcntl() lock primitives. There is a new libvirt manager plugin
which talks to this daemon over RPC. The reason for doing the
locks in a separate process is that we want the locks to remain
active, even if libvirtd crashes, or is restarted. The virtlockd
daemon has this one single job so should be pretty reliable and
selfcontained. This patch series really benefits from the new RPC
APIs, requiring minimal code for the new daemon / client
At this time, virtlockd does not lock the actual disk files, but
instead creates a lockspace & leases under /var/lib/libvirt/lockd.
The lockspace we use for disks is named org.libvirt.lockd.files,
and lease names are based on a SHA256 checksum of the fully
qualified disk name. eg
/var/lib/libvirt/lockd/org.libvirt.lockd.files/adf94fc33a24da1abff7dd7374a9919bb51efee646da8c3ac464c10cd59750bd
These leases are all zero-bytes long and no I/O is ever performed
on them, only fcntl() is used. So there is material overhead.
Whenever creating or deleting leases, we first acquire a lock on
/var/lib/libvirt/lockd/org.libvirt.lockd.files/org.libvirt.lockd.index
A non-root virtlockd will instead use $HOME/.libvirt/lockd
By default we gain protection out of the box against
- Starting two guests on the same host with the same disk image
not marked with <shareable/>
- libvirtd getting confused and forgetting a guest, allowing it
to be started for a 2nd time
If the admin mounts a shared filesytem (eg NFS) on /var/lib/libvirt/lockd
then this protection is extended across all hosts sharing that
mount volume.
As part of this series, I also introduce support for systemd
services for libvirtd and libvir-guests.
13 years, 4 months
[libvirt] [RFC PATCH 0/5] Support online resizing of block devices.
by Osier Yang
This patch series introduce new API "virDomainBlockResize" to expose
qemu monitor command "block_size", which is for resizing the a block
device while the domain is running.
The prototype for the new API is:
int
virDomainBlockResize (virDomainPtr dom,
const char *path,
unsigned long long size,
unsigned int flags)
* "@path" is the absolute path of the block device, which can be
extraced from domain xml.
* The units for "@size" is kilobytes, which might be not quite properly.
(qemu HMP uses Megabytes as the default units, QMP uses Bytes as the
default units, so it means we need to divice "@size" by 1024 for HMP,
and multiply "@size" by 1024 for QMP. On the other hand, we need to
check the overflowing). Any ideas on this is welcomed.
* "@flags" is unused currently.
[PATCH 1/5] block_resize: Define the new API
[PATCH 2/5] block_resize: Wire up the remote protocol
[PATCH 3/5] block_resize: Implement qemu monitor functions
[PATCH 4/5] block_resize: Implement qemu driver method
[PATCH 5/5] block_resize: Expose the new API to virsh
13 years, 5 months
[libvirt] [PATCH 1/2] Fix order of disks and controllers
by Jiri Denemark
Commit 2d6adabd53c8f1858191d521dc9b4948d1205955 replaced qsorting disk
and controller devices with inserting them at the right position. That
was to fix unnecessary reordering of devices. However, when parsing
domain XML devices are just taken in the order in which they appear in
the XML since. Use the correct insertion algorithm to honor device
target.
---
src/conf/domain_conf.c | 4 +-
.../qemuxml2argvdata/qemuxml2argv-boot-order.args | 35 ++++++++++++-------
tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml | 14 ++++----
tests/xmconfigdata/test-escape-paths.cfg | 2 +-
tests/xmconfigdata/test-escape-paths.xml | 10 +++---
5 files changed, 37 insertions(+), 28 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index b406202..48ed7eb 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -5883,7 +5883,7 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps,
if (!disk)
goto error;
- def->disks[def->ndisks++] = disk;
+ virDomainDiskInsertPreAlloced(def, disk);
}
VIR_FREE(nodes);
@@ -5899,7 +5899,7 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps,
if (!controller)
goto error;
- def->controllers[def->ncontrollers++] = controller;
+ virDomainControllerInsertPreAlloced(def, controller);
}
VIR_FREE(nodes);
diff --git a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args
index 23249f3..14367b1 100644
--- a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args
+++ b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.args
@@ -1,13 +1,22 @@
-LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test /usr/bin/qemu -S -M \
-pc -m 214 -smp 1 -nographic -nodefaults -monitor unix:/tmp/test-monitor,server,\
-nowait -no-acpi -drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-ide0-0-0 \
--device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive \
-file=sheepdog:example.org:6000:image,if=none,id=drive-virtio-disk0 -device \
-virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,\
-bootindex=3 -drive file=/root/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0 \
--device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,\
-bootindex=1 -drive file=/dev/null,if=none,id=drive-fdc0-0-1 -global \
-isa-fdc.driveB=drive-fdc0-0-1 -global isa-fdc.bootindexB=4 -device \
-virtio-net-pci,vlan=0,id=net0,mac=00:11:22:33:44:55,bus=pci.0,addr=0x3,\
-bootindex=2 -net user,vlan=0,name=hostnet0 -usb -device virtio-balloon-pci,\
-id=balloon0,bus=pci.0,addr=0x5
+LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test /usr/bin/qemu \
+-S \
+-M pc \
+-m 214 \
+-smp 1 \
+-nographic \
+-nodefaults \
+-monitor unix:/tmp/test-monitor,server,nowait \
+-no-acpi \
+-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-ide0-0-0 \
+-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
+-drive file=/root/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0 \
+-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 \
+-drive file=sheepdog:example.org:6000:image,if=none,id=drive-virtio-disk0 \
+-device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=3 \
+-drive file=/dev/null,if=none,id=drive-fdc0-0-1 \
+-global isa-fdc.driveB=drive-fdc0-0-1 \
+-global isa-fdc.bootindexB=4 \
+-device virtio-net-pci,vlan=0,id=net0,mac=00:11:22:33:44:55,bus=pci.0,addr=0x3,bootindex=2 \
+-net user,vlan=0,name=hostnet0 \
+-usb \
+-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
diff --git a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml
index ba8a9b2..0022c92 100644
--- a/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml
+++ b/tests/qemuxml2argvdata/qemuxml2argv-boot-order.xml
@@ -18,6 +18,13 @@
<target dev='hda' bus='ide'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
+ <disk type='file' device='cdrom'>
+ <source file='/root/boot.iso'/>
+ <target dev='hdc' bus='ide'/>
+ <boot order='1'/>
+ <readonly/>
+ <address type='drive' controller='0' bus='1' unit='0'/>
+ </disk>
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<source protocol='sheepdog' name='image'>
@@ -26,13 +33,6 @@
<target dev='vda' bus='virtio'/>
<boot order='3'/>
</disk>
- <disk type='file' device='cdrom'>
- <source file='/root/boot.iso'/>
- <target dev='hdc' bus='ide'/>
- <boot order='1'/>
- <readonly/>
- <address type='drive' controller='0' bus='1' unit='0'/>
- </disk>
<disk type='file' device='floppy'>
<driver name='qemu' type='raw'/>
<source file='/dev/null'/>
diff --git a/tests/xmconfigdata/test-escape-paths.cfg b/tests/xmconfigdata/test-escape-paths.cfg
index e3e6db9..13be2a0 100644
--- a/tests/xmconfigdata/test-escape-paths.cfg
+++ b/tests/xmconfigdata/test-escape-paths.cfg
@@ -19,7 +19,7 @@ vnc = 1
vncunused = 1
vnclisten = "127.0.0.1"
vncpasswd = "123poi"
-disk = [ "phy:/dev/HostVG/XenGuest2,hda,w", "file:/root/boot.iso&test,hdc:cdrom,r", """phy:/dev/HostVG/XenGuest'",hdb,w""" ]
+disk = [ "phy:/dev/HostVG/XenGuest2,hda,w", """phy:/dev/HostVG/XenGuest'",hdb,w""", "file:/root/boot.iso&test,hdc:cdrom,r" ]
vif = [ "mac=00:16:3e:66:92:9c,bridge=xenbr1,script=vif-bridge,model=e1000,type=ioemu" ]
parallel = "none"
serial = "none"
diff --git a/tests/xmconfigdata/test-escape-paths.xml b/tests/xmconfigdata/test-escape-paths.xml
index 13e6e29..9eaf90c 100644
--- a/tests/xmconfigdata/test-escape-paths.xml
+++ b/tests/xmconfigdata/test-escape-paths.xml
@@ -25,17 +25,17 @@
<source dev='/dev/HostVG/XenGuest2'/>
<target dev='hda' bus='ide'/>
</disk>
+ <disk type='block' device='disk'>
+ <driver name='phy'/>
+ <source dev='/dev/HostVG/XenGuest'"'/>
+ <target dev='hdb' bus='ide'/>
+ </disk>
<disk type='file' device='cdrom'>
<driver name='file'/>
<source file='/root/boot.iso&test'/>
<target dev='hdc' bus='ide'/>
<readonly/>
</disk>
- <disk type='block' device='disk'>
- <driver name='phy'/>
- <source dev='/dev/HostVG/XenGuest'"'/>
- <target dev='hdb' bus='ide'/>
- </disk>
<interface type='bridge'>
<mac address='00:16:3e:66:92:9c'/>
<source bridge='xenbr1'/>
--
1.7.5.3
13 years, 5 months
[libvirt] Notes from the KVM Forum relevant to libvirt
by Daniel P. Berrange
I was at the KVM Forum / LinuxCon last week and there were many
interesting things discussed which are relevant to ongoing libvirt
development. Here was the list that caught my attention. If I have
missed any, fill in the gaps....
- Sandbox/container KVM. The Solaris port of KVM puts QEMU inside
a zone so that an exploit of QEMU can't escape into the full OS.
Containers are Linux's parallel of Zones, and while not nearly as
secure yet, it would still be worth using more containers support
to confine QEMU.
- Events for object changes. We already have async events for virDomainPtr.
We need the same for virInterfacePtr, virStoragePoolPtr, virStorageVolPtr
and virNodeDevPtr, so that at the very least applications can be notified
when objects are created or removed. For virNodeDevPtr we also want to
be notified when properties change (ie CDROM media change).
- CGroups passthrough. There is alot of experimentation with cgroups. We
don't want to expose cgroups as a direct concept in the libvirt API,
but we should consider putting a generic cgroups get/set in the
libvirt-qemu.so library, or create a libvirt-linux.so library.
Also likely add a <linux:cgroups> XML element to store arbitrary
tunables in the XML. Same (low) level of support as with qemu:XXX
of course
- CPUSet for changing CPU + Memory NUMA pinning. The CPUset cgroups
controller is able to actually move a guest's memory between NUMA
nodes. We can already change VCPU pinning, but we need a new API
to do node pinning of the whole VM, so we can ensure the I/O threads
are also moved. We also need an API to move the memory pinning to
new nodes.
- Guest NUMA topology. If we have guests with RAM size > node size,
we need to expose a NUMA topology into the guest. The CPU/memory
pinning APIs will also need to be able to pin individual guest
NUMA nodes to individual host NUMA nodes.
- AHCI controller. IDE is going the way of the dodo. We need to add
support for QEMU's new AHCI controller. This is quite simple, we
already have a 'sata' disk type we can wire up to QEMU
- VFIO PCI passthru. The current PCI assignment code may well be
changed to use something called 'VFIO'. This will need some
work in libvirt to support new CLI arg syntax, and probably
some SELinux work
- QCow3. There will soon be a QCow3 format. We need to add code to
detect it and extract backing stores, etc. Trivial since the primary
header format will still be the same as QCow2.
- QMP completion. Given anthony's plan for a complete replacement of
the current CLI + monitor syntax in QEMU 2.0 (long way out), he has
dropped objections to adding new commands to QMP in the near future.
So all existing HMP commands will immediately be made available in
QMP with no attempt to re-design them now. So the need for the HMP
passthrough command will soon go away.
- Migration + VEPA/VNLink failures. As raised previously on this list,
Cisco really wants libvirt to have the ability to do migration, and
optionally *not* fail, even if the VEPA/VNLink setup fails. This will
require an event notification to the app if a failure of a device
backend occurs, and an API to let the admin app fix the device backend
(virDomainUpdateDevice) and some way to tell migration what bits are
allowed to fail.
- Virtio SCSI. We need to support this new stuff in QEMU when it is
eventually implemented. It will mean we avoid the PCI slot usage
problems inherant in virtio-blk, and get other things like multipath
and decent SCSI passthrough support.
- USB 2.0. We need to support this in libvirt asap. It is very important
for desktop experiance and to support better integration with SPICE
This also gets us proper USB port addressing. Fun footnote, QEMU USB
has *never* supported migration. The USB tablet only works by sheer
luck, as OS' see the device disappear on migration & come back with
different device ID/port addr and so does a re-initialize !
- Native KVM tool. The problem statement was that the QEMU code is too
big/complex & and command line args are too complex, so lets rewrite
from scratch to make the code small & CLI simple. They achieve this,
but of course primarily because they lack so many features compared
to QEMU. They had libvirt support as a bullet point on their preso,
but I'm not expecting it to replace the current QEMU KVM support in
the forseeable future, given its current level of features and the
size of its dev team compared to QEMU/KVM. They did have some fun
demos of booting using the host OS filesystem though. We can
actually do the same with regular KVM/libvirt but there's no nice
demo tool to show it off. I'm hoping to create one....
- Shared memory devices. Some people doing high performance work are
using the QEMU shared memory device. We don't support this (ivhshm
device) in libvirt yet. Fairly niche use cases but might be nice to
have this.
- SDK / Docs. Request for a more SDK like approach to KVM development
tools and documentation. Also want to simplify libvirt operations.
The exposure of the virt-install internal API as official GObjects
would have significantly helped the project Ricardo (from IBM)
described in his presentation. Of course no one can argue that we
need more documentation in every area.
- USB managed mode. As we do with PCI passthrough, we should be able
to detach USB device from host OS, and perform a reset before
attaching to the guest, and most importantly track which USB devices
have been given to which guest, so we don't duplicate assign. We have
all neccessary APIs, just need to wire them up.
- PCI passthrough. We need to support setting of MAC addr, VLAN and
VEPA/VNLink properties against VFs from SRIOV NICs that are assigned
to a guest.
For those who were not at the KVM Forum, the presentations are already
available online at:
http://www.linux-kvm.org/page/KVM_Forum_2011
All the session were also video recorded, so sometime in the next week
or two, there should be OGG videos of the talks being uploaded to the
same site.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
13 years, 6 months
