[libvirt-users] which vols added to storage pool during refresh?
(opps, actually meant to post this to libvirt-users instead of libvirt-cim...) We're looking at a problem when we have a libvirt pool defined, below [root@xnl2192 ~]# virsh pool-dumpxml ScsiPool-host2 <pool type='scsi'> <name>ScsiPool-host2</name> <uuid>f2ea1882-881f-9f49-838d-39e070f68002</uuid> <capacity>136583315456</capacity> <allocation>136583315456</allocation> <available>0</available> <source> <adapter name='host2'/> </source> <target> <path>/dev/disk/by-id</path> <permissions> <mode>0700</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool> As you can see, this pool maps to /dev/disk/by-id. There may be a large number of actual volume entries in /dev/disk/by-id, however it appears only a distinct subset of them are actually added into the libvirt pool when you do a pool-refresh. What does libvirt check when determining which, and whether or not, to add one of these volume into the vol-list? In particular, we have a case when one specific volume in /dev/disk/by-id is *not* added into the pool, and no amount of pool-refresh makes a difference. We did notice that if you remove all the sym links in /dev/disk/by-id and do a pool-refresh, that libvirt re-scans the bus, recreates the links in /dev/disk/by-id, and adds (some of) the volumes back into the vol-list. But not this one problematic vol, so clearly there is something about this vol that libvirt doesnt like and why it is (deliberately?) not added to the pool. What checks, etc does libvirt make against vols when determining whether to add them to a pool? - G Dr. Gareth S. Bestor IBM Senior Software Engineer Systems & Technology Group - Systems Management Standards 971-285-6375 (mobile) bestor@us.ibm.com_______________________________________________ Libvirt-cim mailing list Libvirt-cim@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-cim
On Wed, Aug 17, 2011 at 10:36:26AM -0700, Gareth S Bestor wrote:
As you can see, this pool maps to /dev/disk/by-id. There may be a large number of actual volume entries in /dev/disk/by-id, however it appears only a distinct subset of them are actually added into the libvirt pool when you do a pool-refresh. What does libvirt check when determining which, and whether or not, to add one of these volume into the vol-list?
In particular, we have a case when one specific volume in /dev/disk/by-id is *not* added into the pool, and no amount of pool-refresh makes a difference. We did notice that if you remove all the sym links in /dev/disk/by-id and do a pool-refresh, that libvirt re-scans the bus, recreates the links in /dev/disk/by-id, and adds (some of) the volumes back into the vol-list. But not this one problematic vol, so clearly there is something about this vol that libvirt doesnt like and why it is (deliberately?) not added to the pool.
What checks, etc does libvirt make against vols when determining whether to add them to a pool?
If we have an error processing any volume, that will be skipped. There is one non-error related check though, that may cause a volume to be skipped: /* We don't create volumes for devices other than disk and cdrom * devices, but finding a device that isn't one of those types * isn't an error, either. */ if (!(device_type == VIR_STORAGE_DEVICE_TYPE_DISK || device_type == VIR_STORAGE_DEVICE_TYPE_ROM)) { retval = 0; goto out; } What value is in /sys/bus/scsi/devices/XXXXX/type for the LUN in question ? And is that value different for all the other LUNs in that HBA ? Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Thanks for the quick response Daniel.
What value is in
/sys/bus/scsi/devices/XXXXX/type
for the LUN in question ? And is that value different for all the other LUNs in that HBA ?
I'll check. In the mean time, what sort of 'processing' does libvirt do that might result in an error ("If we have an error processing any volume, that will be skipped.")? eg if we see a vol that is not getting added, is there a command-line we can do to confirm there's something 'wrong' with this vol? thnx again, - G Dr. Gareth S. Bestor IBM Senior Software Engineer Systems & Technology Group - Systems Management Standards 971-285-6375 (mobile) bestor@us.ibm.com Re: [libvirt-users] which vols added to storage pool during refresh? Daniel P. Berrange to: Gareth S Bestor 08/17/11 10:44 AM Cc: libvirt-users Please respond to "Daniel P. Berrange" On Wed, Aug 17, 2011 at 10:36:26AM -0700, Gareth S Bestor wrote:
As you can see, this pool maps to /dev/disk/by-id. There may be a large number of actual volume entries in /dev/disk/by-id, however it appears only a distinct subset of them are actually added into the libvirt pool when you do a pool-refresh. What does libvirt check when determining which, and whether or not, to add one of these volume into the vol-list?
In particular, we have a case when one specific volume in
is *not* added into the pool, and no amount of pool-refresh makes a difference. We did notice that if you remove all the sym links in /dev/disk/by-id and do a pool-refresh, that libvirt re-scans the bus, recreates the links in /dev/disk/by-id, and adds (some of) the volumes back into the vol-list. But not this one problematic vol, so clearly
/dev/disk/by-id there
is something about this vol that libvirt doesnt like and why it is (deliberately?) not added to the pool.
What checks, etc does libvirt make against vols when determining whether
to add them to a pool?
If we have an error processing any volume, that will be skipped. There is one non-error related check though, that may cause a volume to be skipped: /* We don't create volumes for devices other than disk and cdrom * devices, but finding a device that isn't one of those types * isn't an error, either. */ if (!(device_type == VIR_STORAGE_DEVICE_TYPE_DISK || device_type == VIR_STORAGE_DEVICE_TYPE_ROM)) { retval = 0; goto out; } What value is in /sys/bus/scsi/devices/XXXXX/type for the LUN in question ? And is that value different for all the other LUNs in that HBA ? Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Wed, Aug 17, 2011 at 10:59:50AM -0700, Gareth S Bestor wrote:
Thanks for the quick response Daniel.
What value is in
/sys/bus/scsi/devices/XXXXX/type
for the LUN in question ? And is that value different for all the other LUNs in that HBA ?
I'll check.
In the mean time, what sort of 'processing' does libvirt do that might result in an error ("If we have an error processing any volume, that will be skipped.")? eg if we see a vol that is not getting added, is there a command-line we can do to confirm there's something 'wrong' with this vol?
There's no real list of things. It is just stuff like 'open' failing or symlink resolving failing, etc. None of this should ever happen in the real world, but ... Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (2)
-
Daniel P. Berrange -
Gareth S Bestor