6:23 a.m.
DV suggested that we document some libvirt setups using shared storage. I'm
not a fan of NFS, so I wrote some blog posts on how to use iSCSI in the
context of libvirt + KVM.
There is of course more than one way todo things, so I've outlined a couple
of different options. One completely manual command line approach using
tgtadm on the iSCSI server:
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
..and virsh/virt-install on the KVM host
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
Then one completely GUI based approach, using a QNAP NAS as the iSCSI server
http://berrange.com/posts/2010/05/04/provisioning-kvm-virtual-machines-on...
..and a virt-manager on the KVM host:
http://berrange.com/posts/2010/05/04/provisioning-kvm-virtual-machines-on...
You can of course mix+match, doing the iSCSI server the manual way and the
libvirt bits the GUI way and vica-verca. Any other commercial iSCSI server
will have the same concepts so the libvirt instructions apply in all likely
scenarios.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:09 a.m.
On 05/05/2010 07:23 AM, Daniel P. Berrange wrote:
DV suggested that we document some libvirt setups using shared
storage. I'm
not a fan of NFS, so I wrote some blog posts on how to use iSCSI in the
context of libvirt + KVM.
Thank you for creating and sharing these.
I have a question about this example, specifically the shareable
permission option and tag: (perm=sh, <shareable/>)
What does that do, exactly? Is it simply the required r/w option to
prevent corruption of VMs when using shared storage, or is it needed
only if the disk will be actively by more than one VM? (That 2nd idea
doesn't seem quite right to me, I would expect to have to use some kind
of locking technology to prevent it two or more VMs trying to write to
the same disk at the same time).
Thanks,
Tom
7:34 a.m.
On Wed, May 05, 2010 at 08:09:50AM -0400, Tom Georgoulias wrote:
On 05/05/2010 07:23 AM, Daniel P. Berrange wrote:
>DV suggested that we document some libvirt setups using shared storage. I'm
>not a fan of NFS, so I wrote some blog posts on how to use iSCSI in the
>context of libvirt + KVM.
Thank you for creating and sharing these.
>
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
I have a question about this example, specifically the shareable
permission option and tag: (perm=sh, <shareable/>)
What does that do, exactly? Is it simply the required r/w option to
prevent corruption of VMs when using shared storage, or is it needed
only if the disk will be actively by more than one VM? (That 2nd idea
doesn't seem quite right to me, I would expect to have to use some kind
of locking technology to prevent it two or more VMs trying to write to
the same disk at the same time).
The <shareable/> flag tells libvirt that the disk will be *concurrently*
accessed by multiple VMs. This means that the hypervisor will not try to
take an exclusive lock on the disk, that all I/O caching is disabled,
and any SELinux labelling allows use by all VMs.
NB: if you are marking a disk as <sharable/> you need to know what you
are doing in the guest OS. You would typically use a cluster filesystem
on the disk like GFS or OCFS. Attempts to use a normal FS like ext3
from multiple VMs at once is not healthy for your data :-)
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:28 a.m.
Daniel,
Thank you Very much for you efforts. I'll be studying these with interest.
I also plan on replying to the previous response to my request for help on
shard storage later today. I'm still trying to run some tests to see if I
understand how the Pools vs Volumes work with libvirt and iSCSI.
Thanks again!
David.
On Wed, 5 May 2010, Daniel P. Berrange wrote:
DV suggested that we document some libvirt setups using shared
storage. I'm
not a fan of NFS, so I wrote some blog posts on how to use iSCSI in the
context of libvirt + KVM.
There is of course more than one way todo things, so I've outlined a couple
of different options. One completely manual command line approach using
tgtadm on the iSCSI server:
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
..and virsh/virt-install on the KVM host
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
Then one completely GUI based approach, using a QNAP NAS as the iSCSI server
http://berrange.com/posts/2010/05/04/provisioning-kvm-virtual-machines-on...
..and a virt-manager on the KVM host:
http://berrange.com/posts/2010/05/04/provisioning-kvm-virtual-machines-on...
You can of course mix+match, doing the iSCSI server the manual way and the
libvirt bits the GUI way and vica-verca. Any other commercial iSCSI server
will have the same concepts so the libvirt instructions apply in all likely
scenarios.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
_______________________________________________
libvirt-users mailing list
libvirt-users(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvirt-users
8:43 a.m.
On Wed, May 05, 2010 at 08:28:15AM -0500, David Ehle wrote:
Daniel,
Thank you Very much for you efforts. I'll be studying these with interest.
I also plan on replying to the previous response to my request for help on
shard storage later today. I'm still trying to run some tests to see if I
understand how the Pools vs Volumes work with libvirt and iSCSI.
For iSCSI
- Pool == A single target
- Volume == A LUN within a target
So, if your iSCSI server exports many iSCSI targets, you'd setup a separate
pool for each target you want to access from libvirt. This post covers iSCSI
pool setup at a low level:
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:50 a.m.
Daniel,
OK so I think I get it now. While for other storage backends one pool can
hold multiple volumes, when using iSCIS there is alwasy a 1 to 1
relationship for pool to volume? Each pool contains exactly one volume?
David.
On Wed, 5 May 2010, Daniel P. Berrange wrote:
On Wed, May 05, 2010 at 08:28:15AM -0500, David Ehle wrote:
>
> Daniel,
>
>
> Thank you Very much for you efforts. I'll be studying these with interest.
>
> I also plan on replying to the previous response to my request for help on
> shard storage later today. I'm still trying to run some tests to see if I
> understand how the Pools vs Volumes work with libvirt and iSCSI.
For iSCSI
- Pool == A single target
- Volume == A LUN within a target
So, if your iSCSI server exports many iSCSI targets, you'd setup a separate
pool for each target you want to access from libvirt. This post covers iSCSI
pool setup at a low level:
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:01 a.m.
On Wed, May 05, 2010 at 08:50:57AM -0500, David Ehle wrote:
Daniel,
OK so I think I get it now. While for other storage backends one pool can
hold multiple volumes, when using iSCIS there is alwasy a 1 to 1
relationship for pool to volume? Each pool contains exactly one volume?
No, each pool can contain multiple volumes. If you have created a iSCSI
target and given it multiple LUNs, then you'll see multiple volumes in
libvirt.
See the virsh vol-list output in this example:
>http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on-iscsi-the-hard-way-part-2-of-2/
for an iSCSI target with 2 LUNs
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:51 p.m.
Daniel,
Thank you. I was misunderstanding how iscsi iSCSI Enterprise Target
Daemon /ietd.conf work. I was setting up multiple targets with only one
LUN each, and thinking I was creating multiple LUNs on the same target.
So right way to set up multiple LUNS for one target under Ubuntu Karmic
using iSCSI Enterprise Target via ietd.conf:
Target iqn.2010-04.gov.blah.blah.blah.keeper:vmstorage
<SNIP>
# Logical Unit definition
# You must define one logical unit at least.
# Block devices, regular files, LVM, and RAID can be offered
# to the initiators as a block device.
Lun 0 Path=/dev/vg0/iscsi_pool,Type=fileio
Lun 1 Path=/dev/vg0/test_pool,Type=fileio
# Alias name for this target
Alias VMStoragePool
<SNIP>
<NOT>
Target iqn.2010-04.gov.blah.blah.blah.keeper:vmstorage.lun0
Lun 0 Path=/dev/vg0/iscsi_pool,Type=fileio
ALIAS LUN0
Target iqn.2010-04.gov.blah.blah.blah.keeper:vmstorage.lun1
Lun 1 Path=/dev/vg0/test_pool,Type=fileio
ALIAS LUN1
</NOT>
To add additional volumes, additional LUNs will need to be created on the
TARGET side, and then exported via iSCSI
After correcting my Target side configuration, I was able to create an
iSCSI pool through virt-manager, and see the two LUNs as available
volumes.
On Wed, 5 May 2010, Daniel P. Berrange wrote:
On Wed, May 05, 2010 at 08:50:57AM -0500, David Ehle wrote:
>
> Daniel,
>
> OK so I think I get it now. While for other storage backends one pool can
> hold multiple volumes, when using iSCIS there is alwasy a 1 to 1
> relationship for pool to volume? Each pool contains exactly one volume?
No, each pool can contain multiple volumes. If you have created a iSCSI
target and given it multiple LUNs, then you'll see multiple volumes in
libvirt.
See the virsh vol-list output in this example:
>>
http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on...
for an iSCSI target with 2 LUNs
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5196
days inactive
5196
days old
7 comments
3 participants
participants (3)
-
Daniel P. Berrange -
David Ehle -
Tom Georgoulias