12:36 p.m.
(opps, actually meant to post this to libvirt-users instead of
libvirt-cim...)
We're looking at a problem when we have a libvirt pool defined, below
[root@xnl2192 ~]# virsh pool-dumpxml ScsiPool-host2
<pool type='scsi'>
<name>ScsiPool-host2</name>
<uuid>f2ea1882-881f-9f49-838d-39e070f68002</uuid>
<capacity>136583315456</capacity>
<allocation>136583315456</allocation>
<available>0</available>
<source>
<adapter name='host2'/>
</source>
<target>
<path>/dev/disk/by-id</path>
<permissions>
<mode>0700</mode>
<owner>-1</owner>
<group>-1</group>
</permissions>
</target>
</pool>
As you can see, this pool maps to /dev/disk/by-id. There may be a large
number of actual volume entries in /dev/disk/by-id, however it appears
only a distinct subset of them are actually added into the libvirt pool
when you do a pool-refresh. What does libvirt check when determining
which, and whether or not, to add one of these volume into the vol-list?
In particular, we have a case when one specific volume in /dev/disk/by-id
is *not* added into the pool, and no amount of pool-refresh makes a
difference. We did notice that if you remove all the sym links in
/dev/disk/by-id and do a pool-refresh, that libvirt re-scans the bus,
recreates the links in /dev/disk/by-id, and adds (some of) the volumes
back into the vol-list. But not this one problematic vol, so clearly there
is something about this vol that libvirt doesnt like and why it is
(deliberately?) not added to the pool.
What checks, etc does libvirt make against vols when determining whether
to add them to a pool?
- G
Dr. Gareth S. Bestor
IBM Senior Software Engineer
Systems & Technology Group - Systems Management Standards
971-285-6375 (mobile)
bestor(a)us.ibm.com_______________________________________________
Libvirt-cim mailing list
Libvirt-cim(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvirt-cim
12:44 p.m.
On Wed, Aug 17, 2011 at 10:36:26AM -0700, Gareth S Bestor wrote:
As you can see, this pool maps to /dev/disk/by-id. There may be a
large
number of actual volume entries in /dev/disk/by-id, however it appears
only a distinct subset of them are actually added into the libvirt pool
when you do a pool-refresh. What does libvirt check when determining
which, and whether or not, to add one of these volume into the vol-list?
In particular, we have a case when one specific volume in /dev/disk/by-id
is *not* added into the pool, and no amount of pool-refresh makes a
difference. We did notice that if you remove all the sym links in
/dev/disk/by-id and do a pool-refresh, that libvirt re-scans the bus,
recreates the links in /dev/disk/by-id, and adds (some of) the volumes
back into the vol-list. But not this one problematic vol, so clearly there
is something about this vol that libvirt doesnt like and why it is
(deliberately?) not added to the pool.
What checks, etc does libvirt make against vols when determining whether
to add them to a pool?
If we have an error processing any volume, that will be skipped. There
is one non-error related check though, that may cause a volume to be
skipped:
/* We don't create volumes for devices other than disk and cdrom
* devices, but finding a device that isn't one of those types
* isn't an error, either. */
if (!(device_type == VIR_STORAGE_DEVICE_TYPE_DISK ||
device_type == VIR_STORAGE_DEVICE_TYPE_ROM))
{
retval = 0;
goto out;
}
What value is in
/sys/bus/scsi/devices/XXXXX/type
for the LUN in question ? And is that value different for all the
other LUNs in that HBA ?
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
12:59 p.m.
Thanks for the quick response Daniel.
What value is in
/sys/bus/scsi/devices/XXXXX/type
for the LUN in question ? And is that value different for all the
other LUNs in that HBA ?
I'll check.
In the mean time, what sort of 'processing' does libvirt do that might
result in an error ("If we have an error processing any volume, that will
be skipped.")? eg if we see a vol that is not getting added, is there a
command-line we can do to confirm there's something 'wrong' with this vol?
thnx again,
- G
Dr. Gareth S. Bestor
IBM Senior Software Engineer
Systems & Technology Group - Systems Management Standards
971-285-6375 (mobile)
bestor(a)us.ibm.com
Re: [libvirt-users] which vols added to storage pool during refresh?
Daniel P. Berrange
to:
Gareth S Bestor
08/17/11 10:44 AM
Cc:
libvirt-users
Please respond to "Daniel P. Berrange"
On Wed, Aug 17, 2011 at 10:36:26AM -0700, Gareth S Bestor wrote:
As you can see, this pool maps to /dev/disk/by-id. There may be a
large
number of actual volume entries in /dev/disk/by-id, however it appears
only a distinct subset of them are actually added into the libvirt pool
when you do a pool-refresh. What does libvirt check when determining
which, and whether or not, to add one of these volume into the vol-list?
In particular, we have a case when one specific volume in
/dev/disk/by-id
is *not* added into the pool, and no amount of pool-refresh makes a
difference. We did notice that if you remove all the sym links in
/dev/disk/by-id and do a pool-refresh, that libvirt re-scans the bus,
recreates the links in /dev/disk/by-id, and adds (some of) the volumes
back into the vol-list. But not this one problematic vol, so clearly
there
is something about this vol that libvirt doesnt like and why it is
(deliberately?) not added to the pool.
What checks, etc does libvirt make against vols when determining whether
to add them to a pool?
If we have an error processing any volume, that will be skipped. There
is one non-error related check though, that may cause a volume to be
skipped:
/* We don't create volumes for devices other than disk and cdrom
* devices, but finding a device that isn't one of those types
* isn't an error, either. */
if (!(device_type == VIR_STORAGE_DEVICE_TYPE_DISK ||
device_type == VIR_STORAGE_DEVICE_TYPE_ROM))
{
retval = 0;
goto out;
}
What value is in
/sys/bus/scsi/devices/XXXXX/type
for the LUN in question ? And is that value different for all the
other LUNs in that HBA ?
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/
:|
|: http://libvirt.org -o- http://virt-manager.org
:|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/
:|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc
:|
4:29 p.m.
On Wed, Aug 17, 2011 at 10:59:50AM -0700, Gareth S Bestor wrote:
Thanks for the quick response Daniel.
>What value is in
>
> /sys/bus/scsi/devices/XXXXX/type
>
>for the LUN in question ? And is that value different for all the
>other LUNs in that HBA ?
I'll check.
In the mean time, what sort of 'processing' does libvirt do that might
result in an error ("If we have an error processing any volume, that will
be skipped.")? eg if we see a vol that is not getting added, is there a
command-line we can do to confirm there's something 'wrong' with this vol?
There's no real list of things. It is just stuff like 'open' failing or
symlink resolving failing, etc. None of this should ever happen in the
real world, but ...
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
4785
days inactive
4785
days old
3 comments
2 participants
participants (2)
-
Daniel P. Berrange -
Gareth S Bestor