Re: [libvirt] mdevctl: A shoestring mediated device management and persistence utility

On Wed, Jun 19, 2019 at 12:27 AM Alex Williamson <alex.williamson(a)redhat.com&gt; wrote: > On Tue, 18 Jun 2019 14:48:11 +0200 > Sylvain Bauza <sbauza(a)redhat.com&gt; wrote: > > > On Tue, Jun 18, 2019 at 1:01 PM Cornelia Huck <cohuck(a)redhat.com&gt; wrote: > > > > > On Mon, 17 Jun 2019 11:05:17 -0600 > > > Alex Williamson <alex.williamson(a)redhat.com&gt; wrote: > > > > > > > On Mon, 17 Jun 2019 16:10:30 +0100 > > > > Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > > > > > > > > On Mon, Jun 17, 2019 at 08:54:38AM -0600, Alex Williamson wrote: > > > > > > On Mon, 17 Jun 2019 15:00:00 +0100 > > > > > > Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > > > > > > > > > > > > On Thu, May 23, 2019 at 05:20:01PM -0600, Alex Williamson > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > Currently mediated device management, much like SR-IOV VF > > > management, > > > > > > > > is largely left as an exercise for the user. This is an > attempt > > > to > > > > > > > > provide something and see where it goes. I doubt we'll solve > > > > > > > > everyone's needs on the first pass, but maybe we'll solve > enough > > > and > > > > > > > > provide helpers for the rest. Without further ado, I'll > point > > > to what > > > > > > > > I have so far: > > > > > > > > > > > > > > > > https://github.com/awilliam/mdevctl > > > > > > > > > > > > > > > > This is inspired by driverctl, which is also a bash > utility. > > > mdevctl > > > > > > > > uses udev and systemd to record and recreate mdev devices for > > > > > > > > persistence and provides a command line utility for > querying, > > > listing, > > > > > > > > starting, stopping, adding, and removing mdev devices. > > > Currently, for > > > > > > > > better or worse, it considers anything created to be > > > persistent. I can > > > > > > > > imagine a global configuration option that might disable > this and > > > > > > > > perhaps an autostart flag per mdev device, such that > mdevctl > > > might > > > > > > > > simply "know" about some mdevs but not attempt to create them > > > > > > > > automatically. Clearly command line usage help, man pages, > and > > > > > > > > packaging are lacking as well, release early, release > often, > > > plus this > > > > > > > > is a discussion starter to see if perhaps this is sufficient > to > > > meet > > > > > > > > some needs. > > > > > > > > > > > > > > I think from libvirt's POV, we would *not* want devices to be > made > > > > > > > unconditionally persistent. We usually wish to expose a choice > to > > > > > > > applications whether to have resources be transient or > persistent. > > > > > > > > > > > > > > So from that POV, a global config option to turn off > persistence > > > > > > > is not workable either. We would want control per-device, with > > > > > > > autostart control per device too. > > > > > > > > > > > > The code has progressed somewhat in the past 3+ weeks, we still > > > persist > > > > > > all devices, but the start-up mode can be selected per device > or > > > with a > > > > > > global default mode. Devices configured with 'auto' start-up > > > > > > automatically while 'manual' devices are simply known and > available > > > to > > > > > > be started. I imagine we could add a 'transient' mode where we > purge > > > > > > the information about the device when it is removed or the next > time > > > > > > the parent device is added. > > > > > > > > > > Having a pesistent config written out & then purged later is still > > > > > problematic. If the host crashes, nothing will purge the config > file, > > > > > so it will become a persistent device. Also when listing devices we > > > > > want to be able to report whether it is persistent or transient. > The > > > > > obvious way todo that is to simply look if a config file exists or > > > > > not. > > > > > > > > I was thinking that the config file would identify the device as > > > > transient, therefore if the system crashed we'd have the opportunity > to > > > > purge those entries on the next boot as we're processing the entries > > > > for that parent device. Clearly it has yet to be implemented, but I > > > > expect there are some advantages to tracking devices via a transient > > > > config entry or else we're constantly re-discovering foreign mdevs. > > > > > > I think we need to reach consensus about the actual scope of the > > > mdevctl tool. > > > > > > > > Thanks Cornelia, my thoughts: > > > > - Is it supposed to be responsible for managing *all* mdev devices in > > > the system, or is it more supposed to be a convenience helper for > > > users/software wanting to manage mdevs? > > > > > > > The latter. If an operator (or some software) wants to create mdevs by > not > > using mdevctl (and rather directly calling the sysfs), I think it's OK. > > That said, mdevs created by mdevctl would be supported by systemctl, > while > > the others not but I think it's okay. > > I agree (sort of), and I'm hearing that we should drop any sort of > automatic persistence of mdevs created outside of mdevctl. The problem > comes when we try to draw the line between unmanaged and manged > devices. For instance, if we have a command to list mdevs it would > feel incomplete if it didn't list all mdevs both those managed by > mdevctl and those created elsewhere. For managed devices, I expect > we'll also have commands that allow the mode of the device to be > switched between transient, saved, and persistent. Should a user then > be allowed to promote an unmanaged device to one of these modes via the > same command? Should they be allowed to stop an unmanaged device > through driverctl? Through systemctl? These all seem like reasonable > things to do, so what then is the difference between transient and > unmanaged mdev and is mdevctl therefore managing all mdevs, not just > those it has created? > > Well, IMHO, mdevs created by mdevctl could all be persisted or transient just by adding an option when calling mdevctl, like : "mdevctl create-mdev [--transient] <uuid> <pci_id> <type>" where default would be persisting the mdev. For mdevs *not* created by mdevctl, then a usecase could be "I'd like to ask mdevctl to manage mdevs I already created" and if so, a mdevctl command like : "mdevctl manage-mdev [--transient] <mdev_uuid>" Of course, that would mean that when you list mdevs by "mdev list-all" you wouldn't get mdevs managed by mdevctl. Thoughts ?