1:29 p.m.
On 04/23/2010 09:24 AM, Avi Kivity wrote:
On 04/23/2010 04:48 PM, Anthony Liguori wrote:
> On 04/23/2010 07:48 AM, Avi Kivity wrote:
>> On 04/22/2010 09:49 PM, Anthony Liguori wrote:
>>>> real API. Say, adding a device libvirt doesn't know about or
>>>> stopping the VM
>>>> while libvirt thinks it's still running or anything like that.
>>> Another problem is issuing Monitor commands that could confuse
>>> libvirt's
>>>
>>> We need to make libvirt and qemu smarter.
>>>
>>> We already face this problem today with multiple libvirt users.
>>> This is why sophisticated management mechanisms (like LDAP) have
>>> mechanisms to do transactions or at least a series of atomic
>>> operations.
>>
>> And people said qmp/json was overengineered...
>>
>> But seriously, transactions won't help anything. qemu maintains
>> state, and when you have two updaters touching a shared variable not
>> excepting each other to, things break, no matter how much locking
>> there is.
>
> Let's consider some concrete examples. I'm using libvirt and QMP and
> in QMP, I want to hot unplug a device.
>
> Today, I do this by listing the pci devices, and issuing a pci_del
> that takes a PCI address. This is intrinsically racy though because
> in the worst case scenario, in between when I enumerate pci devices
> and do the pci_del in QMP, in libvirt, I've done a pci_del and then a
> pci_add within libvirt of a completely different device.
Obviously you should do the pci_del through libvirt. Once libvirt
supports an API, use it.
It was just an example...
>
> There are a few ways to solve this, the simplest being that we give
> devices unique ids that are never reused and instead of pci_del
> taking a pci bus address, it takes a device id. That would address
> this race.
>
> You can get very far by just being clever about unique ids and
> notifications. There are some cases where a true RMW may be required
> but I can't really think of one off hand. The way LDAP addresses
> this is that it has a batched operation and a simple set of boolean
> comparison operations. This lets you execute a batched operation
> that will do a RMW.
I'm sure we can be very clever, but I'd rather direct this cleverness
to qemu core issues, not to the QMP (which in turn requires that users
be clever to use it correctly). QMP is a low bandwidth protocol, so
races will never show up in testing. We're laying mines here for
users to step on that we will never encounter ourselves.
>
>> The only way that separate monitors could work is if they touch
>> completely separate state, which is difficult to ensure if you
>> upgrade your libvirt.
>>
>
> I don't think this is as difficult of a problem as you think it is.
> If you look at Active Directory and the whole set of management tools
> based on it, they certainly allow concurrent management
> applications. You can certainly get into trouble still but with just
> some careful considerations, you can make two management applications
> work together 90% of the time without much fuss on the applications
> part.
Maybe. We'll still have issues. For example, sVirt: if a QMP command
names a labeled resource, the non-libvirt user will have no way of
knowing how to label it.
This is orthogonal to QMP and has to do strictly with how libvirt
prepares a resource for qemu.
Much better to exact a commitment from libvirt to track all QMP (and
command line) capabilities. Instead of adding cleverness to QMP, add
APIs to libvirt.
Let's step back for a minute because I think we're missing the forest
through the trees.
We're trying to address a few distinct problems:
1) Allow libvirt users to access features of qemu that are not exposed
through libvirt
2) Provide a means for non-libvirt users to interact with qemu
3) Provide a unified and interoperable view of the world for non-libvirt
and libvirt users
For (1), we all agree that the best case scenario would be for libvirt
to support every qemu feature. I think we can also all agree though
that this is not really practical and certainly not practical for
developers since there is a development cost associated with libvirt
support (to model an API appropriately).
The new API proposed addresses (1) by allowing a user to drill down to
the QMP context. It's a good solution IMHO and I think we all agree
that there's an inherent risk to this that users will have to evaluate
on a case-by-case basis. It's a good stop-gap though.
(2) is largely addressed by QMP and a config file. I'd like to see a
nice C library, but I think a lot of other folks are happy with JSON
support in higher level languages.
(3) is the place where there are still potential challenges. I think at
the very least, our goal should be to enable conversion from (2) and (1)
to be as easy as possible. That's why I have proposed implementing a C
library for the JSON transport because we could plumb that through the
new libvirt API. This would allow a user to very quickly port an
application from QMP to libvirt. In order to do this, we need the
libvirt API to expose a dedicated monitor because we'll need to be able
to manipulate events and negotiate features.
Beyond simple porting, there's a secondary question of having
non-libvirt apps co-exist with libvirt apps. I think it's a good long
term goal, but I don't think we should worry too much about it now.
Regards,
Anthony Liguori