[libvirt-users] Connect libvirt to iSCSI target
Hi! I'm new to libvirt and face problems connecting to an iSCSI target. What I intend to do is to connect libvirt (I tried virt-manager and virsh) to an iSCSI target and then boot from the LUNs which contain the VMs. I followed the documentation¹ but got stuck at section 12.1.5.4.3. 1) virsh pool-define-as \ --name foo \ --type iscsi \ --source-host vmhost \ --source-dev iqn.2013-11.vmhost:vmpool \ --target /dev/disk/by-path 2) virsh pool-start foo error: Failed to start pool foo error: internal error: cannot find session virsh -c qemu:///system pool-start foo error: Failed to start pool foo error: internal error: cannot find session Why can't libvirt find the session, and which session? Running iscsiadm --mode discovery --type sendtargets --portal vmhost:3260,1 succeeds and lists the targets. How to connect libvirt to the iSCSI target? System info: Debian sid, Linux 3.11-2-amd64 libvirtd (libvirt) 1.1.4 virsh version: 1.1.4 iSCSI target: FreeNAS-9.1.1 Best regards Marco ¹ https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/...
On 2013–12–18 Marco wrote:
virsh pool-start foo error: Failed to start pool foo error: internal error: cannot find session
virsh -c qemu:///system pool-start foo error: Failed to start pool foo error: internal error: cannot find session
Why can't libvirt find the session, and which session?
Any ideas? Shall I provide more information? Best regards Marco
On 12/17/2013 07:13 PM, Marco wrote:
Hi!
I'm new to libvirt and face problems connecting to an iSCSI target. What I intend to do is to connect libvirt (I tried virt-manager and virsh) to an iSCSI target and then boot from the LUNs which contain the VMs.
I followed the documentation¹ but got stuck at section 12.1.5.4.3.
1) virsh pool-define-as \ --name foo \ --type iscsi \ --source-host vmhost \ --source-dev iqn.2013-11.vmhost:vmpool \ --target /dev/disk/by-path
2) virsh pool-start foo error: Failed to start pool foo error: internal error: cannot find session
virsh -c qemu:///system pool-start foo error: Failed to start pool foo error: internal error: cannot find session
I'm assuming 'vmhost' is actually some valid IP Address/Name that you're just using here to hide from the world your actual iSCSI tgt server. Don't forget that the 'source-host' may be listed as 'host1.example.com'; however, the iqn has 'com.example.host1'. IOW - reversed... Have you looked at http://libvirt.org/formatstorage.html ? There's an example there of output that works for me. Use virsh pool-dumpxml <pool> to view the XML... Also, do you need to authenticate to the iSCSI target? The iscsiadm will use '/etc/iscsi/iscsid.conf'. See the node.session.auth.{authmethod | username | password} fields, so while iscsiadm may work, when libvirt goes to connect it fails. Looking at the output in the system messages log may help (for my f19 /var/log/messages). You may also find if you have selinux enabled as Enforcing there could be more configuration that you need (if the iSCSI tgtd is local). Setting up a label on the file - something I'm not an expert at... Another tool to try is 'iscsi-ls' from package 'libiscsi-utils' iscsi-ls -s -i iqn.2013-11.vmhost:vmpool \ iscsi://<username>%<password>@<ipAddr>:3260 where the <username>/<password> allow the authentication to the ipAddr iSCSI tgt server. If you need to authenticate, then set up a secret to the iSCSI server, see http://libvirt.org/formatsecret.html for an example. HTH, John
Why can't libvirt find the session, and which session? Running
iscsiadm --mode discovery --type sendtargets --portal vmhost:3260,1
succeeds and lists the targets. How to connect libvirt to the iSCSI target?
System info:
Debian sid, Linux 3.11-2-amd64 libvirtd (libvirt) 1.1.4 virsh version: 1.1.4
What's your QEMU version?
iSCSI target: FreeNAS-9.1.1
Best regards Marco
¹ https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/...
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On 2013–12–21 John Ferlan wrote:
On 12/17/2013 07:13 PM, Marco wrote:
Hi!
I'm new to libvirt and face problems connecting to an iSCSI target. What I intend to do is to connect libvirt (I tried virt-manager and virsh) to an iSCSI target and then boot from the LUNs which contain the VMs.
I followed the documentation¹ but got stuck at section 12.1.5.4.3.
1) virsh pool-define-as \ --name foo \ --type iscsi \ --source-host vmhost \ --source-dev iqn.2013-11.vmhost:vmpool \ --target /dev/disk/by-path
2) virsh pool-start foo error: Failed to start pool foo error: internal error: cannot find session
virsh -c qemu:///system pool-start foo error: Failed to start pool foo error: internal error: cannot find session
I'm assuming 'vmhost' is actually some valid IP Address/Name that you're just using here to hide from the world your actual iSCSI tgt server. Don't forget that the 'source-host' may be listed as 'host1.example.com'; however, the iqn has 'com.example.host1'. IOW - reversed...
It's a test setup and I don't use a FQDN.
Have you looked at http://libvirt.org/formatstorage.html ?
I did, indeed.
There's an example there of output that works for me.
For me it does not.
Use virsh pool-dumpxml <pool> to view the XML...
Here's my XML which is directly taken from the manual. I only changed the host name to “freenas” and the device path. <pool type='iscsi'> <name>virtimages</name> <uuid>347feb68-7e11-4f2b-ba82-ada027296fb0</uuid> <capacity unit='bytes'>0</capacity> <allocation unit='bytes'>0</allocation> <available unit='bytes'>0</available> <source> <host name='freenas'/> <device path='iqn.2013-11.freenas:vmpool'/> </source> <target> <path>/dev/disk/by-path</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool>
Also, do you need to authenticate to the iSCSI target?
Not yet. To rule out authentication issues for testing I turned off authentication.
The iscsiadm will use '/etc/iscsi/iscsid.conf'. See the node.session.auth.{authmethod | username | password} fields, so while iscsiadm may work, when libvirt goes to connect it fails.
Thanks for pointing that out, but I don't think it's the case here.
Looking at the output in the system messages log may help (for my f19 /var/log/messages).
The log of a failed virsh pool-start virtimages can be found here http://dpaste.com/1518444/ Most notable is I assume: libvirtd[8028]: internal error: Child process (/usr/bin/iscsiadm --mode session) unexpected exit status 21
From “man iscsiadm”:
21 ISCSI_ERR_NO_OBJS_FOUND - no records/targets/sessions/portals found to execute operation on. If I run “iscsiadm --mode session” in the console, I indeed get and error: iscsiadm: No active sessions. Do I need to create a session or setup libvert to do that before iSCSI is supposed to work? The log shows that libvirt is able to connect to the target: iscsid: Connection85:0 to [target: iqn.2013-11.freenas:vmpool, portal: 10.0.10.23,3260] through [iface: libvirt-iface-04d9ea67] is operational now
You may also find if you have selinux enabled as Enforcing there could be more configuration that you need (if the iSCSI tgtd is local). Setting up a label on the file - something I'm not an expert at...
selinux is not used.
Another tool to try is 'iscsi-ls' from package 'libiscsi-utils'
iscsi-ls -s -i iqn.2013-11.vmhost:vmpool \ iscsi://<username>%<password>@<ipAddr>:3260
This succeeds: iscsi-ls -s -i iqn.2013-11.freenas:vmpool iscsi://freenas Target:iqn.2013-11.freenas:vmpool Portal:10.0.10.23:3260,1 Lun:0 Type:DIRECT_ACCESS (Size:9G) […]
If you need to authenticate, then set up a secret to the iSCSI server, see http://libvirt.org/formatsecret.html for an example.
If the simple setup is working I'll add authentication but for the moment I try to keep is as simple as possible. To rule out issues with the iSCSI target I manually tested using “iscsiadm” and VirtualBox which both succeeded without any session errors. Thanks for your quick response. Best regards Marco
On 12/22/2013 10:09 AM, Marco wrote:
On 2013–12–21 John Ferlan wrote:
On 12/17/2013 07:13 PM, Marco wrote:
Hi!
I'm new to libvirt and face problems connecting to an iSCSI target. What I intend to do is to connect libvirt (I tried virt-manager and virsh) to an iSCSI target and then boot from the LUNs which contain the VMs.
I followed the documentation¹ but got stuck at section 12.1.5.4.3.
1) virsh pool-define-as \ --name foo \ --type iscsi \ --source-host vmhost \ --source-dev iqn.2013-11.vmhost:vmpool \ --target /dev/disk/by-path
2) virsh pool-start foo error: Failed to start pool foo error: internal error: cannot find session
virsh -c qemu:///system pool-start foo error: Failed to start pool foo error: internal error: cannot find session
I'm assuming 'vmhost' is actually some valid IP Address/Name that you're just using here to hide from the world your actual iSCSI tgt server. Don't forget that the 'source-host' may be listed as 'host1.example.com'; however, the iqn has 'com.example.host1'. IOW - reversed...
It's a test setup and I don't use a FQDN.
Have you looked at http://libvirt.org/formatstorage.html ?
I did, indeed.
There's an example there of output that works for me.
For me it does not.
Use virsh pool-dumpxml <pool> to view the XML...
Here's my XML which is directly taken from the manual. I only changed the host name to “freenas” and the device path.
<pool type='iscsi'> <name>virtimages</name> <uuid>347feb68-7e11-4f2b-ba82-ada027296fb0</uuid> <capacity unit='bytes'>0</capacity> <allocation unit='bytes'>0</allocation> <available unit='bytes'>0</available> <source> <host name='freenas'/> <device path='iqn.2013-11.freenas:vmpool'/> </source> <target> <path>/dev/disk/by-path</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool>
Also, do you need to authenticate to the iSCSI target?
Not yet. To rule out authentication issues for testing I turned off authentication.
The iscsiadm will use '/etc/iscsi/iscsid.conf'. See the node.session.auth.{authmethod | username | password} fields, so while iscsiadm may work, when libvirt goes to connect it fails.
Thanks for pointing that out, but I don't think it's the case here.
Looking at the output in the system messages log may help (for my f19 /var/log/messages).
The log of a failed virsh pool-start virtimages can be found here http://dpaste.com/1518444/
Most notable is I assume:
libvirtd[8028]: internal error: Child process (/usr/bin/iscsiadm --mode session) unexpected exit status 21
Yeah - there just wasn't enough information there for me to go on. During my work on this I did run into one other issue that's discussed in a bz (https://bugzilla.redhat.com/show_bug.cgi?id=957295) where libiscsi had a bug where it wasn't handling the name string built up by libvirt properly. I don't remember all the details of what was happening, but you'd see messages from libvirtqemu indicating something like: " could not open disk image iscsi://freenas:3260/iqn.2013-11.freenas%3Avmpool:" This was for the case for when the iSCSI target was directly added to the guest though, but that's just a subset of the pool changes. Still the whole issue occurred in late October which I is within the same time frame of 1.1.4 which you are using. So what is your libiscsi version. Not sure how much time I'll have over the next week to look at anything as I'm busy with non work related things... John
From “man iscsiadm”:
21 ISCSI_ERR_NO_OBJS_FOUND - no records/targets/sessions/portals found to execute operation on.
If I run “iscsiadm --mode session” in the console, I indeed get and error:
iscsiadm: No active sessions.
Do I need to create a session or setup libvert to do that before iSCSI is supposed to work?
The log shows that libvirt is able to connect to the target:
iscsid: Connection85:0 to [target: iqn.2013-11.freenas:vmpool, portal: 10.0.10.23,3260] through [iface: libvirt-iface-04d9ea67] is operational now
You may also find if you have selinux enabled as Enforcing there could be more configuration that you need (if the iSCSI tgtd is local). Setting up a label on the file - something I'm not an expert at...
selinux is not used.
Another tool to try is 'iscsi-ls' from package 'libiscsi-utils'
iscsi-ls -s -i iqn.2013-11.vmhost:vmpool \ iscsi://<username>%<password>@<ipAddr>:3260
This succeeds:
iscsi-ls -s -i iqn.2013-11.freenas:vmpool iscsi://freenas
Target:iqn.2013-11.freenas:vmpool Portal:10.0.10.23:3260,1 Lun:0 Type:DIRECT_ACCESS (Size:9G) […]
If you need to authenticate, then set up a secret to the iSCSI server, see http://libvirt.org/formatsecret.html for an example.
If the simple setup is working I'll add authentication but for the moment I try to keep is as simple as possible.
To rule out issues with the iSCSI target I manually tested using “iscsiadm” and VirtualBox which both succeeded without any session errors.
Thanks for your quick response.
Best regards Marco
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On 2013–12–23 John Ferlan wrote:
On 12/22/2013 10:09 AM, Marco wrote:
On 2013–12–21 John Ferlan wrote:
On 12/17/2013 07:13 PM, Marco wrote:
Hi!
I'm new to libvirt and face problems connecting to an iSCSI target. What I intend to do is to connect libvirt (I tried virt-manager and virsh) to an iSCSI target and then boot from the LUNs which contain the VMs.
I followed the documentation¹ but got stuck at section 12.1.5.4.3.
1) virsh pool-define-as \ --name foo \ --type iscsi \ --source-host vmhost \ --source-dev iqn.2013-11.vmhost:vmpool \ --target /dev/disk/by-path
2) virsh pool-start foo error: Failed to start pool foo error: internal error: cannot find session
virsh -c qemu:///system pool-start foo error: Failed to start pool foo error: internal error: cannot find session
I'm assuming 'vmhost' is actually some valid IP Address/Name that you're just using here to hide from the world your actual iSCSI tgt server. Don't forget that the 'source-host' may be listed as 'host1.example.com'; however, the iqn has 'com.example.host1'. IOW - reversed...
It's a test setup and I don't use a FQDN.
Have you looked at http://libvirt.org/formatstorage.html ?
I did, indeed.
There's an example there of output that works for me.
For me it does not.
Use virsh pool-dumpxml <pool> to view the XML...
Here's my XML which is directly taken from the manual. I only changed the host name to “freenas” and the device path.
<pool type='iscsi'> <name>virtimages</name> <uuid>347feb68-7e11-4f2b-ba82-ada027296fb0</uuid> <capacity unit='bytes'>0</capacity> <allocation unit='bytes'>0</allocation> <available unit='bytes'>0</available> <source> <host name='freenas'/> <device path='iqn.2013-11.freenas:vmpool'/> </source> <target> <path>/dev/disk/by-path</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool>
Also, do you need to authenticate to the iSCSI target?
Not yet. To rule out authentication issues for testing I turned off authentication.
The iscsiadm will use '/etc/iscsi/iscsid.conf'. See the node.session.auth.{authmethod | username | password} fields, so while iscsiadm may work, when libvirt goes to connect it fails.
Thanks for pointing that out, but I don't think it's the case here.
Looking at the output in the system messages log may help (for my f19 /var/log/messages).
The log of a failed virsh pool-start virtimages can be found here http://dpaste.com/1518444/
Most notable is I assume:
libvirtd[8028]: internal error: Child process (/usr/bin/iscsiadm --mode session) unexpected exit status 21
Yeah - there just wasn't enough information there for me to go on. During my work on this I did run into one other issue that's discussed in a bz (https://bugzilla.redhat.com/show_bug.cgi?id=957295) where
Thanks for the link to the bug report, it looks interesting. I will read it the next days when I have some time.
libiscsi had a bug where it wasn't handling the name string built up by libvirt properly. I don't remember all the details of what was happening, but you'd see messages from libvirtqemu indicating something like:
" could not open disk image iscsi://freenas:3260/iqn.2013-11.freenas%3Avmpool:"
This was for the case for when the iSCSI target was directly added to the guest though, but that's just a subset of the pool changes.
The disk image can be opened and the size is reported correctly. I may have a different issue here. Something session related, but I couldn't yet figure out what.
Still the whole issue occurred in late October which I is within the same time frame of 1.1.4 which you are using. So what is your libiscsi version.
My libiscsi version is 1.4.0
Not sure how much time I'll have over the next week to look at anything as I'm busy with non work related things...
Same here. Take it easy :) Warm wishes… Marco
On 12/23/2013 08:37 AM, Marco wrote: <...snip...>
Still the whole issue occurred in late October which I is within the same time frame of 1.1.4 which you are using. So what is your libiscsi version.
My libiscsi version is 1.4.0
That could be the issue... From the upstream github libiscsi repo, the following commit could be what's causing your issue: https://github.com/sahlberg/libiscsi/commit/e061cba1b91650ab08ae8fa50e8cadb1... According to how I read the commit - the fix is in 1.10.0. Without using some sort of packet/network sniffer or getting a more clear error message regarding that the failure was because there's a "%3A" in the libvirt created URI, I cannot say for sure. But I would certainly be inclined to believe that could be the problem. The libvirt API 'virURIFormat()' will call 'xmlSaveUri()' to generate a URI from whatever is found in the pool or domain XML. I know that the generated URI has a '%3A' in it from my experience... John
On 2013–12–27 John Ferlan wrote:
On 12/23/2013 08:37 AM, Marco wrote:
<...snip...>
Still the whole issue occurred in late October which I is within the same time frame of 1.1.4 which you are using. So what is your libiscsi version.
My libiscsi version is 1.4.0
That could be the issue... From the upstream github libiscsi repo, the following commit could be what's causing your issue:
https://github.com/sahlberg/libiscsi/commit/e061cba1b91650ab08ae8fa50e8cadb1...
Thanks for digging into the source.
According to how I read the commit - the fix is in 1.10.0.
Without using some sort of packet/network sniffer or getting a more clear error message regarding that the failure was because there's a "%3A" in the libvirt created URI, I cannot say for sure. But I would certainly be inclined to believe that could be the problem.
I'll do some more testing with an updated libiscsi next year. I hope that will fix it. Marco
participants (2)
-
John Ferlan -
Marco