Hi, Currently libvirt supports metadata in the domain XML. This is very convenient for data related to the VM, but it is a little awkward for devices. Let's pretend I want to have extradata (say, a specific port for a virtual switch) related to a device (say, a NIC). Nowadays I can store that data in the metadata section, but I need some kind of mapping to correlate this piece of information to the specific device. I can use the device alias, but this is not available when the device is created. This is also more complex when doing hotplug/hotunplug, because I need to do update device and update metadata; if either fails, the entire operation must be considered failed. It would be nice to be able to attach metadata to the device, and this is what I'm asking for/proposing in this mail. Would it be possible in a future libvirt release? If this is not possible, what's the best way to do the aforementioned mapping? If it's alias (or device address), how can I be sure than I'm addressing (no pun intended) the right device when I don't have them? (e.g. hotplug new device, or just first time VM created)? Thanks,

For techincal discussions on design it's better to post to the development list. On Mon, Feb 27, 2017 at 16:41:28 +0100, Francesco Romani wrote: > Hi, Hi, > > > Currently libvirt supports metadata in the domain XML. This is very > convenient for data related to the VM, but it is a little awkward for > devices. Let's pretend I want to have extradata (say, a specific port > for a virtual switch) related to a device (say, a NIC). Nowadays I can > store that data in the metadata section, but I need some kind of mapping > to correlate this piece of information to the specific device. > > I can use the device alias, but this is not available when the device is > created. This is also more complex when doing hotplug/hotunplug, because > I need to do update device and update metadata; if either fails, the > entire operation must be considered failed. > > It would be nice to be able to attach metadata to the device, and this > is what I'm asking for/proposing in this mail. > > Would it be possible in a future libvirt release? In general I don't have a problem with this and if it would make the life of mgmt tools easier I don't see a reason why not. There's one problem though. For setter (or getter) APIs for the metadata you need a way how to uniquely identify a device on which you want to operate. There's no such way currently in use in libvirt so you need to come up with something to do it.

On Thu, Mar 16, 2017 at 14:42:30 +0000, Daniel Berrange wrote: > On Thu, Mar 16, 2017 at 01:46:38PM +0100, Peter Krempa wrote: > > On Mon, Feb 27, 2017 at 16:41:28 +0100, Francesco Romani wrote: [...] > > In general I don't have a problem with this and if it would make the > > life of mgmt tools easier I don't see a reason why not. > > > > There's one problem though. For setter (or getter) APIs for the metadata > > you need a way how to uniquely identify a device on which you want to > > operate. There's no such way currently in use in libvirt so you need to > > come up with something to do it. > > Once you have a reliable identifier for a device, it is not compelling > to store metadata against each device as you can reliably identify the > device from data in the existing metadata block. I don't find the > atomicity argument very compelling, because even if hidden behind a > single libvirt API, it is still fundamentally multiple operations in > libvirt which are just as likely to fail as if the app did multiple You get the atomicity for free if you store the metadata in the device object and parse it along with the device definition. We parse the XML at first (which would include the metadata). If the device attach fails during hotplug we will free the device definition (including the metadata). The same applies for unplug. Once unplug finishes we will free the device data structure and thus the metadata with it.

On Thu, Mar 16, 2017 at 14:52:47 +0000, Daniel Berrange wrote: > On Thu, Mar 16, 2017 at 03:50:51PM +0100, Peter Krempa wrote: > > On Thu, Mar 16, 2017 at 14:42:30 +0000, Daniel Berrange wrote: > > > On Thu, Mar 16, 2017 at 01:46:38PM +0100, Peter Krempa wrote: > > > > On Mon, Feb 27, 2017 at 16:41:28 +0100, Francesco Romani wrote: [...] > The scenario where device attach fails is not the problem - you can > get the same level of reliabilty to that by simply updating the > global metadata before & after hotplug in the same way. What is > difficult is when libvirt fails to persist the XML config on disk > or when libvirt crashes part way through the operation, and other > akward failure scenarios unrelated to QEMU itself. In that case you lose the device definition too, since saving the XML is the integral part of the hotplug operation.

On Thu, Mar 16, 2017 at 01:46:38PM +0100, Peter Krempa wrote: > For techincal discussions on design it's better to post to the > development list. > > On Mon, Feb 27, 2017 at 16:41:28 +0100, Francesco Romani wrote: >> Hi, > Hi, > >> >> Currently libvirt supports metadata in the domain XML. This is very >> convenient for data related to the VM, but it is a little awkward for >> devices. Let's pretend I want to have extradata (say, a specific port >> for a virtual switch) related to a device (say, a NIC). Nowadays I can >> store that data in the metadata section, but I need some kind of mapping >> to correlate this piece of information to the specific device. >> >> I can use the device alias, but this is not available when the device is >> created. This is also more complex when doing hotplug/hotunplug, because >> I need to do update device and update metadata; if either fails, the >> entire operation must be considered failed. >> >> It would be nice to be able to attach metadata to the device, and this >> is what I'm asking for/proposing in this mail. >> >> Would it be possible in a future libvirt release? > In general I don't have a problem with this and if it would make the > life of mgmt tools easier I don't see a reason why not. > > There's one problem though. For setter (or getter) APIs for the metadata > you need a way how to uniquely identify a device on which you want to > operate. There's no such way currently in use in libvirt so you need to > come up with something to do it. Once you have a reliable identifier for a device, it is not compelling to store metadata against each device as you can reliably identify the device from data in the existing metadata block. I don't find the atomicity argument very compelling, because even if hidden behind a single libvirt API, it is still fundamentally multiple operations in libvirt which are just as likely to fail as if the app did multiple libvirt API calls as we're not able to provide any kind of transactional guarantees here.

IOW, we could effectively solve this by assigning the device alias names at XML define time, instead of VM launch time. We historically didn't do this since we wanted to cope with old QEMU which didn't allow custom device aliases. We now mandate a new QEMU that supports -device, so that's no longer a problem. So I'd prefer to see us just define device aliases persistently and let apps use those to track metadata in the global metadata section.