Re: [libvirt-users] Ang: Ang: Re: Ang: Re: attaching storage pool error

On 08/24/2016 06:31 AM, Johan Kragsterman wrote: I saw this last week while I was at KVM Forum, but just haven't had the time until now to start thinking about this stuff again ... as you point out with your questions and replies - NPIV/vHBA is tricky and complicated... I always have try to "clear the decks" of anything else before trying to page how this all works back into the frontal cortex. Once done, I quickly do a page flush. It was also a bit confusing with respect to how the responses have been threaded - so I just took the most recent one and started there. The Red Hat documentation is most up-to-date - it was sourced (more or less) from: http://wiki.libvirt.org/page/NPIV_in_libvirt There's some old stuff in there and probably needs a cleanse to provide all the "supported" options. This is an 'iscsi' pool format, but something similar can be crafted for the 'scsi' pool used for fc_host devices. So for the purposes of taking the right steps, I assume you used 'virsh nodedev-list --cap vports' in order to find FC capable scsi_host#'s. Then you created your vHBA based on the FC capable fc_host, using XML such as: <device> <parent>scsi_hostN</parent> <capability type='scsi_host'> <capability type='fc_host'> </capability> </capability> </device> where scsi_hostN and 'N' in particular is the FC capable fc_host Then creation of the node device : #virsh nodedev-create vhba.xml Node device scsi_hostM created from vhba.xml where 'M' is whatever the next available scsi_host# is on your host. If you 'virsh nodedev-dumpxml scsi_hostM' you'll see the wwnn/wwpn details. You can then create a vHBA scsi pool from that in order to ensure the persistence of the vHBA. Although it's not required - the vHBA scsi pool just allows you to provide a source pool and volume by unit # for your guest rather than having to edit guests between host reboots or other such conditions which cause The "volume" and "disk" or "volume" and "lun" syntax can be used somewhat interchangeably. As your point out the features for disk and lun are slightly different. Usage of the 'lun' syntax allows addition of the attribute "sgio='unfiltered'" Let's forget about iscsi Works on my test guest (ok not the order from dumpxml): ... <controller type='scsi' index='0' model='virtio-scsi'> <alias name='scsi0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </controller> ... <disk type='volume' device='lun'> <driver name='qemu' type='raw'/> <source pool='vhbapool_host3' volume='unit:0:4:0'/> <backingStore/> <target dev='sda' bus='scsi'/> <shareable/> <alias name='scsi0-0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> ... See above... The controller has a "type", "index", and "model"... Then when adding the disk, use the type='drive' controller='#', where # is the index number from your virtio-scsi controller. You can use 'virsh pool-refresh $poolname' - it will find new luns... Err, it *should* find new luns ;-) Existing 'unit:#:#:#' values shouldn't change - they should be "tied to" the same wwnn. Use "virsh vol-list $poolname" to see the Path. So when new ones are added they are given new unit number's. Reboots should find the same order. Fair warning, use of sgio='unfiltered' does require some specific kernels... There were many "issues" with this - mostly related to kernel support. If not supported by the kernel, you are advised : error: Requested operation is not valid: unpriv_sgio is not supported by this kernel Uhh... multipath... Not my strong suit... I'm taking an example from a bz that you won't be able to read because it's marked private. Once you have your vHBA and scsi_hostM for that vHBA on the host you can use 'lsscsi' (you may have to yum/dnf install it - it's a very useful tool)... # lsscsi ... //assume scsi_host6 is the new vHBA created as follow [6:0:0:0] disk IBM 1726-4xx FAStT 0617 - [6:0:1:0] disk IBM 1726-4xx FAStT 0617 - [6:0:2:0] disk IBM 1726-4xx FAStT 0617 /dev/sdf [6:0:3:0] disk IBM 1726-4xx FAStT 0617 /dev/sdg You'll need an mpath pool: # virsh pool-dumpxml mpath <pool type='mpath'> <name>mpath</name> <source> </source> <target> <path>/dev/mapper</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool> # virsh pool-define mpath # virsh pool-start mpath # virsh vol-list mpath Name Path ----------------------------------------- dm-0 /dev/mapper/3600a0b80005adb0b0000ab2d4cae9254 dm-5 /dev/mapper/3600a0b80005ad1d700002dde4fa32ca8 <=== this one is from vhba scsi_host6 Then using something like: <disk type='block' device='lun' sgio='unfiltered'> <driver name='qemu' type='raw'/> <source dev='/dev/mapper/3600a0b80005ad1d700002dde4fa32ca8'/> <target dev='sda' bus='scsi'/> <alias name='scsi0-0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> HTH, John (FWIW: I'm not sure how the leap of faith was taken that dm-5 is the vHBA for scsi_host6... Although I think it's from the wwnn for a volume in the vHBA as seen when using a virsh vol-list from a pool created using the vHBA within the bz).