[libvirt] LVM storage pool with existing mirrored volumes
Hi, I tried to setup a lvm-backed storage pool, reusing an existing volume group which already contains a handful of mirrored logical volumes. For reference, I am using lvm 2.02.56-r3, qemu-kvm 0.12.1.2 and libvirt 0.7.5 on an up-to-date ~amd64 gentoo system. The first issue I encountered is when trying to activate the storage pool, the process fails with "internal error lvs command failed". This appears to be because the output of lvs is slightly different when using mirrored lvs, and is breaking the parser: /sbin/lvs --separator , --noheadings --units b --unbuffered --nosuffix --options lv_name,origin,uuid,devices,seg_size,vg_extent_size bigpool vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304 backups,,Snk2NL-oJe3-GpTL-Qihs-ZjjG-xPyj-ufmwCp,backups_mimage_0(0),backups_mimage_1(0),214748364800,4194304 The device column of output would normally contain something like "/dev/sda(1234)", but here contains "vms_mimage_0(0),vms_mimage_1(0). This is causing the parser to misinterpret the output, and it believes the lv name to be suffixed with a trailing comma, eg "vms,". When trying to access "/dev/bigpool/vms,", the command errors with File or Directory not Found. When adding the --all parameter to lvs, the hidden mirroring lvs are visible, and these have the expected structure: vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304 [vms_mlog],,vnOzg8-ynKc-kM8F-RKdM-xXC0-PJlG-qcgmFh,/dev/sda2(0),4194304,4194304 [vms_mimage_0],,2pXjHt-aARa-FOO7-ZL6N-ahpu-ebdr-P6Qxj3,/dev/sdb1(0),107374182400,4194304 [vms_mimage_1],,ITd8CT-JpKi-Y2a6-a4jc-2yDa-9VmG-gWgRd8,/dev/sdc1(51200),107374182400,4194304 Presumably, without the --all parameter and listing the internal mirroring volumes, libvirt will miscalculate sizes of LVs in the VG. This bug appears to prevent the use of any volume group which contains one or more mirrored logical volumes as a backend storage pool. -- Ben Roberts Computer Science MEng, Part IV Electronics and Computer Science, University of Southampton
On Sun, Jan 17, 2010 at 03:44:28PM +0000, Ben Roberts wrote:
Hi,
I tried to setup a lvm-backed storage pool, reusing an existing volume group which already contains a handful of mirrored logical volumes.
For reference, I am using lvm 2.02.56-r3, qemu-kvm 0.12.1.2 and libvirt 0.7.5 on an up-to-date ~amd64 gentoo system.
The first issue I encountered is when trying to activate the storage pool, the process fails with "internal error lvs command failed". This appears to be because the output of lvs is slightly different when using mirrored lvs, and is breaking the parser:
/sbin/lvs --separator , --noheadings --units b --unbuffered --nosuffix --options lv_name,origin,uuid,devices,seg_size,vg_extent_size bigpool
vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304
backups,,Snk2NL-oJe3-GpTL-Qihs-ZjjG-xPyj-ufmwCp,backups_mimage_0(0),backups_mimage_1(0),214748364800,4194304
The device column of output would normally contain something like "/dev/sda(1234)", but here contains "vms_mimage_0(0),vms_mimage_1(0).
This is causing the parser to misinterpret the output, and it believes the lv name to be suffixed with a trailing comma, eg "vms,". When trying to access "/dev/bigpool/vms,", the command errors with File or Directory not Found.
When adding the --all parameter to lvs, the hidden mirroring lvs are visible, and these have the expected structure:
vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304 [vms_mlog],,vnOzg8-ynKc-kM8F-RKdM-xXC0-PJlG-qcgmFh,/dev/sda2(0),4194304,4194304 [vms_mimage_0],,2pXjHt-aARa-FOO7-ZL6N-ahpu-ebdr-P6Qxj3,/dev/sdb1(0),107374182400,4194304 [vms_mimage_1],,ITd8CT-JpKi-Y2a6-a4jc-2yDa-9VmG-gWgRd8,/dev/sdc1(51200),107374182400,4194304
Presumably, without the --all parameter and listing the internal mirroring volumes, libvirt will miscalculate sizes of LVs in the VG.
Yes, the output of mirrored devices is definitely going to cause problems for our parsing. This doesn't look all that easy to fix to me, but we'll clearly have to figure out some way to cope with this. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
2010/1/17 Daniel P. Berrange <berrange@redhat.com>:
On Sun, Jan 17, 2010 at 03:44:28PM +0000, Ben Roberts wrote:
Hi,
I tried to setup a lvm-backed storage pool, reusing an existing volume group which already contains a handful of mirrored logical volumes.
For reference, I am using lvm 2.02.56-r3, qemu-kvm 0.12.1.2 and libvirt 0.7.5 on an up-to-date ~amd64 gentoo system.
The first issue I encountered is when trying to activate the storage pool, the process fails with "internal error lvs command failed". This appears to be because the output of lvs is slightly different when using mirrored lvs, and is breaking the parser:
/sbin/lvs --separator , --noheadings --units b --unbuffered --nosuffix --options lv_name,origin,uuid,devices,seg_size,vg_extent_size bigpool
vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304
backups,,Snk2NL-oJe3-GpTL-Qihs-ZjjG-xPyj-ufmwCp,backups_mimage_0(0),backups_mimage_1(0),214748364800,4194304
The device column of output would normally contain something like "/dev/sda(1234)", but here contains "vms_mimage_0(0),vms_mimage_1(0).
This is causing the parser to misinterpret the output, and it believes the lv name to be suffixed with a trailing comma, eg "vms,". When trying to access "/dev/bigpool/vms,", the command errors with File or Directory not Found.
When adding the --all parameter to lvs, the hidden mirroring lvs are visible, and these have the expected structure:
vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304 [vms_mlog],,vnOzg8-ynKc-kM8F-RKdM-xXC0-PJlG-qcgmFh,/dev/sda2(0),4194304,4194304 [vms_mimage_0],,2pXjHt-aARa-FOO7-ZL6N-ahpu-ebdr-P6Qxj3,/dev/sdb1(0),107374182400,4194304 [vms_mimage_1],,ITd8CT-JpKi-Y2a6-a4jc-2yDa-9VmG-gWgRd8,/dev/sdc1(51200),107374182400,4194304
Presumably, without the --all parameter and listing the internal mirroring volumes, libvirt will miscalculate sizes of LVs in the VG.
Yes, the output of mirrored devices is definitely going to cause problems for our parsing. This doesn't look all that easy to fix to me, but we'll clearly have to figure out some way to cope with this.
Daniel
We discussed this on IRC. One suggestion was to change the separator from comma to # for example. This would allow to identify the expected structure of sections again, because the devices list will still be separated by a comma. Then the devices section could be parsed in a second phase. vms##0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu#vms_mimage_0(0),vms_mimage_1(0)#107374182400#4194304 But the more complicated question (at least to me) is how to handle mirrored LVs correctly, because with vms_mimage_* the offset seems to be always 0, but the underlying LVs may have different offsets also different from 0, and libvirt uses the offset for size calculations. Well, I'm not really familiar with LVM, so I can't judge if that's a real/hard problem. Maybe fixing the parser is sufficient to fix the problem. Matthias
participants (3)
-
Ben Roberts -
Daniel P. Berrange -
Matthias Bolte