Re: [libvirt] [PATCH] Changes to support Veritas HyperScale (VxHS) block device protocol with qemu-kvm

On Wed, Jan 4, 2017 at 7:00 AM, John Ferlan <jferlan(a)redhat.com&gt; wrote: > > [...] > >>>> We don't anticipate a need for this. >>>> >>>>> 4. There's no VxHS Storage Pool support in this patch (OK actually an >>>>> additional set of patches in order to support). That would be expected >>>>> especially for a networked storage environment. You'll note there are >>>>> src/storage/storage_backend_{iscsi|gluster|rbd}.{c|h} files that manage >>>>> iSCSI, Gluster, and RBD protocol specific things. For starters, create, >>>>> delete, and refresh - especially things that a stat() wouldn't >>>>> necessarily return (capacity, allocation, type a/k/a target.format). >>>>> Perhaps even the ability to upload/download and wipe volumes in the >>>>> pool. Having a pool is a bit more work, but look at the genesis of >>>>> existing storage_backend_*.c files to get a sense for what needs to change. >>>>> >>>> >>>> VxHS does not need the Storage Pool functionality. Do we still need to >>>> implement this? >>>> >>> >>> It's something that's expected. See my reasoning above. >>> >> >> Some explanation is in order - >> >> HyperScale is not designed to be used as a stand-alone independent >> storage. It is designed only to be used in the OpenStack environment >> with all the related Cinder/Nova changes in place. Therefore, we do >> not have a need for most of the above-mentioned functions from >> libvirt/qemu. >> >> Even in the OpenStack environment, we do not support creating storage >> volumes independent of a guest VM. A VxHS storage volume can only be >> created for a particular guest VM. With this scheme, a user does not >> have to manage storage pools separately. VxHS automatically configures >> and consumes the direct attached SSD/HDD on the OpenStack compute when >> enabled. After this, all the requests to add storage to guest VMs are >> forwarded by OpenStack directly to the hyperscale daemon on the >> compute, and it takes care of creating the underlying storage etc. > > And how does that volume creation occur? It would seem that there's some > command that does that. That's "independent" of libvirt, so in order for > a guest to use that storage - it'd need to be created first anyway. I > then assume as part of guest vm destruction you must somehow destroy the > volume too. Synchronizing with creation and ensuring deletion would seem > to be fairly important tasks and would seemingly be something you'd like > to have more integrated with the environment. > We have hooks in nova that trap volume creation/deletion request. Then we send messages to our service running on every compute to carry out the necessary steps to create/delete the volume. > So what about hotplug? Can someone add in VxHS storage to a guest after > it's started? > Yes, we do support hot-plugging. We use OpenStack nova framework to generate a config for the new volume and attach_device it to a running guest. > And migration? Can the guest be migrated? I haven't crawled through > that code recently, but I know there'd be changes to either allow or not > based on the storage type. > We do support storage migration. We simulate shared storage on top of direct-attached storage, therefore libvirt/qemu can assume shared storage for the purpose of migration. >> >> The role of libvirt is limited to opening the volume specified in the >> guest XML file. Volume creation, deletion etc is done by the VxHS >> daemon in response to messages from the OpenStack controller. Our >> OpenStack orchestration code (Nova) is responsible for updating the >> guest XML with correct volume information for libvirt to use. A >> regular user (libvirt) is not expected to know about what volume IDs >> exist on any given host. A regular user also does not have a volume >> device node on the local compute to query. The only way to get to a >> volume is via the network using the server IP-port address. >> >> > > For me having the storage pool is a more complete solution. You don't > have to support all the storage backend volume functions (build, create, > upload, download, wipe, resize, etc.), but knowing what volumes exist > and can be used for a domain is nice. It's also nice to know how much > storage exists (allocation, capacity, physical). It also allows a single > point of authentication - the pool authenticates at startup (see the > iSCSI and RBD code) and then the domains can use storage from the pool. > > From a guest view point, rather than having to provide: > > <source protocol='vxhs' name='eb90327c-8302-4725-9e1b-4e85ed4dc251'> > <host name='192.168.0.1' port='9999'/> > </source> > <auth username='user'> > <secret type='vxfs' usage='somestring'/> > </auth> > > you'd have: > > <source pool='pool-name' volume='vol-name'/> > > The "host" and "auth" would be part of the <pool> definition. Having to > discover the available 'names' ends up being the difference it seems for > vxhs since your model would seem to be create storage, create/start > guest, destroy guest, destroy storage. I think there's value in being > able to use the libvirt storage API's especially w/r/t integrating the > volume and guest management. > Would it be OK to consider adding storage pool functionality to vxhs after Nova starts using it?

From the libvirt POV there's no requirement to have storage pool support