[libvirt] [PATCH 0/2] qemu: add vhost-scsi-pci support
From: Nicholas Bellinger <nab@linux-iscsi.org> Hi Daniel & Co, The following is the patch series to add support for vhost-scsi-pci from QEMU >= v1.5.x into libvirt. It includes proper support for passing a pre-opened vhostfd into vhost-scsi, which is required in order to properly support nova performing a device_add for adding a individual vhost-scsi-pci WWPN endpoint -> SCSI controller. The logic closely follows what vhost-net already does in order to allow a child qemu process to interact with the vhost-scsi character device for registering vhost memory, setting vhost-scsi endpoints, etc. This code has been tested using openstack nova volume-attach, using a Juno v2 development head from 07192014. Thank you, --nab Mike Perez (1): qemu: add vhost-scsi-pci definitions Nicholas Bellinger (1): qemu: add support vhost-scsi-pci for device_add hotplug Makefile.am | 2 +- docs/formatdomain.html.in | 20 ++-- docs/schemas/domaincommon.rng | 5 + src/conf/domain_conf.c | 20 +++- src/conf/domain_conf.h | 2 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 115 ++++++++++++++++++-- src/qemu/qemu_command.h | 8 +- src/qemu/qemu_hotplug.c | 47 +++++++- src/qemu/qemu_monitor.c | 24 ++++ src/qemu/qemu_monitor.h | 4 + src/vmx/vmx.c | 1 + tests/qemucapabilitiesdata/caps_1.5.3-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.6.0-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.6.50-1.caps | 1 + .../qemuxml2argv-disk-vhost-scsi-cmd_per_lun.args | 9 ++ .../qemuxml2argv-disk-vhost-scsi-cmd_per_lun.xml | 29 +++++ .../qemuxml2argv-disk-vhost-scsi-max_sectors.args | 9 ++ .../qemuxml2argv-disk-vhost-scsi-max_sectors.xml | 29 +++++ .../qemuxml2argv-disk-vhost-scsi-num_queues.args | 9 ++ .../qemuxml2argv-disk-vhost-scsi-num_queues.xml | 29 +++++ .../qemuxml2argv-disk-vhost-scsi-wwpn.args | 9 ++ .../qemuxml2argv-disk-vhost-scsi-wwpn.xml | 29 +++++ tests/qemuxml2argvtest.c | 12 ++ tests/qemuxml2xmltest.c | 4 + 26 files changed, 399 insertions(+), 23 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.xml -- 1.7.9.5
From: Mike Perez <thingee@gmail.com> This patch adds the necessary definitions to support vhost-scsi-pci and VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI in QEMU >= v1.5.x code. This includes: - Add QEMU_CAPS_VHOST_SCSI - Add virDomainControllerDef->wwpn pointer for the vhost-scsi wwpn. - Setting the new model type in qemuSetSCSIControllerModel() - Invoking virDomainPCIAddressReserveAddr() for vhost-scsi-pci in qemuAssignDevicePCISlots() - Add the 'wwpn=' device parameter in qemuBuildControllerDevStr() - Add vhost-scsi-pci to >= v1.5.x qemucapabilitiesdata - Add xml2argv and xml2xml tests for cmd_per_lun + max_sectors + num_queues + wwpn. Signed-off-by: Mike Perez <thingee@gmail.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org> --- Makefile.am | 2 +- docs/formatdomain.html.in | 20 +++---- docs/schemas/domaincommon.rng | 5 ++ src/conf/domain_conf.c | 20 ++++++- src/conf/domain_conf.h | 2 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 59 ++++++++++++++++++-- src/vmx/vmx.c | 1 + tests/qemucapabilitiesdata/caps_1.5.3-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.6.0-1.caps | 1 + tests/qemucapabilitiesdata/caps_1.6.50-1.caps | 1 + .../qemuxml2argv-disk-vhost-scsi-cmd_per_lun.args | 9 +++ .../qemuxml2argv-disk-vhost-scsi-cmd_per_lun.xml | 29 ++++++++++ .../qemuxml2argv-disk-vhost-scsi-max_sectors.args | 9 +++ .../qemuxml2argv-disk-vhost-scsi-max_sectors.xml | 29 ++++++++++ .../qemuxml2argv-disk-vhost-scsi-num_queues.args | 9 +++ .../qemuxml2argv-disk-vhost-scsi-num_queues.xml | 29 ++++++++++ .../qemuxml2argv-disk-vhost-scsi-wwpn.args | 9 +++ .../qemuxml2argv-disk-vhost-scsi-wwpn.xml | 29 ++++++++++ tests/qemuxml2argvtest.c | 12 ++++ tests/qemuxml2xmltest.c | 4 ++ 22 files changed, 266 insertions(+), 17 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.xml diff --git a/Makefile.am b/Makefile.am index a374e1a..cb8d606 100644 --- a/Makefile.am +++ b/Makefile.am @@ -19,7 +19,7 @@ LCOV = lcov GENHTML = genhtml -SUBDIRS = . gnulib/lib include src daemon tools docs gnulib/tests \ +SUBDIRS = . gnulib/lib include src daemon tools gnulib/tests \ tests po examples/object-events examples/hellolibvirt \ examples/dominfo examples/domsuspend examples/apparmor \ examples/xml/nwfilter examples/openauth examples/systemtap \ diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index b69da4c..32b1864 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -2550,16 +2550,16 @@ control how many devices can be connected through the controller. A "scsi" controller has an optional attribute <code>model</code>, which is one of "auto", "buslogic", - "ibmvscsi", "lsilogic", "lsisas1068", "lsisas1078", "virtio-scsi" or - "vmpvscsi". A "usb" controller has an optional attribute - <code>model</code>, which is one of "piix3-uhci", "piix4-uhci", "ehci", - "ich9-ehci1", "ich9-uhci1", "ich9-uhci2", "ich9-uhci3", "vt82c686b-uhci", - "pci-ohci" or "nec-xhci". Additionally, - <span class="since">since 0.10.0</span>, if the USB bus needs to be - explicitly disabled for the guest, <code>model='none'</code> may be - used. <span class="since">Since 1.0.5</span>, no default USB controller - will be built on s390. The PowerPC64 "spapr-vio" addresses do not have an - associated controller. + "ibmvscsi", "lsilogic", "lsisas1068", "lsisas1078", "virtio-scsi", + "vhost-scsi", or "vmpvscsi". A "usb" controller has an optional + attribute <code>model</code>, which is one of "piix3-uhci", "piix4-uhci", + "ehci", "ich9-ehci1", "ich9-uhci1", "ich9-uhci2", "ich9-uhci3", + "vt82c686b-uhci", "pci-ohci" or "nec-xhci". Additionally, <span + class="since">since 0.10.0</span>, if the USB bus needs to be explicitly + disabled for the guest, <code>model='none'</code> may be used. <span + class="since">Since 1.0.5</span>, no default USB controller will be built + on s390. The PowerPC64 "spapr-vio" addresses do not have an associated + controller. </p> <p> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 7be028d..7f2622e 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -1650,6 +1650,7 @@ <value>vmpvscsi</value> <value>ibmvscsi</value> <value>virtio-scsi</value> + <value>vhost-scsi</value> <value>lsisas1078</value> </choice> </attribute> @@ -1745,6 +1746,10 @@ <ref name="unsignedInt"/> </attribute> </optional> + <optional> + <attribute name="wwpn"> + </attribute> + </optional> </element> </optional> </interleave> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index fa76eb4..edb0d9d 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -306,6 +306,7 @@ VIR_ENUM_IMPL(virDomainControllerModelSCSI, VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LAS "vmpvscsi", "ibmvscsi", "virtio-scsi", + "vhost-scsi", "lsisas1078"); VIR_ENUM_IMPL(virDomainControllerModelUSB, VIR_DOMAIN_CONTROLLER_MODEL_USB_LAST, @@ -6049,6 +6050,7 @@ virDomainControllerDefParseXML(xmlNodePtr node, char *queues = NULL; char *cmd_per_lun = NULL; char *max_sectors = NULL; + char *wwpn = NULL; xmlNodePtr saved = ctxt->node; int rc; @@ -6094,6 +6096,7 @@ virDomainControllerDefParseXML(xmlNodePtr node, queues = virXMLPropString(cur, "queues"); cmd_per_lun = virXMLPropString(cur, "cmd_per_lun"); max_sectors = virXMLPropString(cur, "max_sectors"); + wwpn = virXMLPropString(cur, "wwpn"); } cur = cur->next; } @@ -6115,6 +6118,17 @@ virDomainControllerDefParseXML(xmlNodePtr node, _("Malformed 'max_sectors' value %s'"), max_sectors); } + if (wwpn) { + if (VIR_ALLOC(def->wwpn) < 0) + return NULL; + if (!STRPREFIX(wwpn, "naa") || + (virStrcpy(def->wwpn, wwpn, strlen(wwpn) + 1) == NULL)) { + virReportError(VIR_ERR_XML_ERROR, _("Malformed 'wwpn' value " + "'%s'"), def->wwpn); + goto error; + } + } + if (virDomainDeviceInfoParseXML(node, NULL, &def->info, flags) < 0) goto error; @@ -6224,6 +6238,7 @@ virDomainControllerDefParseXML(xmlNodePtr node, VIR_FREE(queues); VIR_FREE(cmd_per_lun); VIR_FREE(max_sectors); + VIR_FREE(wwpn); return def; @@ -15327,7 +15342,7 @@ virDomainControllerDefFormat(virBufferPtr buf, break; } - if (def->queues || def->cmd_per_lun || def->max_sectors || + if (def->queues || def->cmd_per_lun || def->max_sectors || def->wwpn || virDomainDeviceInfoIsSet(&def->info, flags) || pcihole64) { virBufferAddLit(buf, ">\n"); virBufferAdjustIndent(buf, 2); @@ -15340,6 +15355,9 @@ virDomainControllerDefFormat(virBufferPtr buf, if (def->max_sectors) virBufferAsprintf(buf, "<driver max_sectors='%u'/>\n", def->max_sectors); + if (def->wwpn) + virBufferAsprintf(buf, "<driver wwpn='%s'/>\n", def->wwpn); + if (virDomainDeviceInfoIsSet(&def->info, flags) && virDomainDeviceInfoFormat(buf, &def->info, flags) < 0) return -1; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 32674e0..28ad6c7 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -676,6 +676,7 @@ typedef enum { VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VMPVSCSI, VIR_DOMAIN_CONTROLLER_MODEL_SCSI_IBMVSCSI, VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VIRTIO_SCSI, + VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI, VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LSISAS1078, VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LAST @@ -719,6 +720,7 @@ struct _virDomainControllerDef { unsigned int queues; unsigned int cmd_per_lun; unsigned int max_sectors; + char *wwpn; union { virDomainVirtioSerialOpts vioserial; virDomainPCIControllerOpts pciopts; diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index c665e2b..d9f23ec 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -259,6 +259,7 @@ VIR_ENUM_IMPL(virQEMUCaps, QEMU_CAPS_LAST, "msg-timestamp", "active-commit", "change-backing-file", + "vhost-scsi-pci", ); @@ -1475,6 +1476,7 @@ struct virQEMUCapsStringFlags virQEMUCapsObjectTypes[] = { { "ich9-intel-hda", QEMU_CAPS_DEVICE_ICH9_INTEL_HDA }, { "pvpanic", QEMU_CAPS_DEVICE_PANIC }, { "usb-kbd", QEMU_CAPS_DEVICE_USB_KBD }, + { "vhost-scsi-pci", QEMU_CAPS_VHOST_SCSI }, }; static struct virQEMUCapsStringFlags virQEMUCapsObjectPropsVirtioBlk[] = { diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index 99cf9ed..d90fa04 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -209,6 +209,7 @@ typedef enum { QEMU_CAPS_MSG_TIMESTAMP = 167, /* -msg timestamp */ QEMU_CAPS_ACTIVE_COMMIT = 168, /* block-commit works without 'top' */ QEMU_CAPS_CHANGE_BACKING_FILE = 169, /* change name of backing file in metadata */ + QEMU_CAPS_VHOST_SCSI = 170, QEMU_CAPS_LAST, /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 2185ef4..43c0e1c 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -712,6 +712,15 @@ qemuSetSCSIControllerModel(virDomainDefPtr def, return -1; } break; + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI: + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_VHOST_SCSI)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("This QEMU doesn't support " + "vhost scsi controller")); + return -1; + } + break; + default: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Unsupported controller model: %s"), @@ -724,6 +733,8 @@ qemuSetSCSIControllerModel(virDomainDefPtr def, *model = VIR_DOMAIN_CONTROLLER_MODEL_SCSI_IBMVSCSI; } else if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_SCSI_LSI)) { *model = VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LSILOGIC; + } else if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_VHOST_SCSI)) { + *model = VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI; } else if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_SCSI)) { *model = VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VIRTIO_SCSI; } else { @@ -2172,10 +2183,28 @@ qemuAssignDevicePCISlots(virDomainDefPtr def, def->controllers[i]->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; def->controllers[i]->info.addr.pci = addr; } else { + if (def->controllers[i]->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI) { + virDevicePCIAddress addr = { 0, 0, 0, 0, false }; + memset(&tmp_addr, 0, sizeof(tmp_addr)); + if (virDomainPCIAddressGetNextSlot(addrs, &tmp_addr, flags) < 0) + goto error; + + addr.bus = tmp_addr.bus; + addr.slot = tmp_addr.slot; + + addrs->lastaddr = addr; + addrs->lastaddr.function = 0; + addrs->lastaddr.multi = 0; + + if (virDomainPCIAddressReserveAddr(addrs, &addr, flags, + false, false) < 0) + goto error; + } else { if (virDomainPCIAddressReserveNextSlot(addrs, &def->controllers[i]->info, flags) < 0) - goto error; + goto error; + } } } @@ -4130,20 +4159,34 @@ qemuBuildControllerDevStr(virDomainDefPtr domainDef, int model; if (!(def->type == VIR_DOMAIN_CONTROLLER_TYPE_SCSI && - def->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VIRTIO_SCSI)) { + (def->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VIRTIO_SCSI || + def->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI))) { if (def->queues) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("'queues' is only supported by virtio-scsi controller")); + _("'queues' is only supported by virtio-scsi and " + "vhost-scsi controller")); return NULL; } if (def->cmd_per_lun) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("'cmd_per_lun' is only supported by virtio-scsi controller")); + _("'cmd_per_lun' is only supported by virtio-scsi " + "and vhost-scsi controller")); return NULL; } if (def->max_sectors) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("'max_sectors' is only supported by virtio-scsi controller")); + _("'max_sectors' is only supported by virtio-scsi " + "and vhost-scsi controller")); + return NULL; + } + } + + if (!(def->type == VIR_DOMAIN_CONTROLLER_TYPE_SCSI && + def->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI)) { + if (def->wwpn) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("'wwpn' is only supported by vhost-scsi " + "controller")); return NULL; } } @@ -4167,6 +4210,9 @@ qemuBuildControllerDevStr(virDomainDefPtr domainDef, else virBufferAddLit(&buf, "virtio-scsi-pci"); break; + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI: + virBufferAddLit(&buf, "vhost-scsi-pci"); + break; case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LSILOGIC: virBufferAddLit(&buf, "lsi"); break; @@ -4279,6 +4325,9 @@ qemuBuildControllerDevStr(virDomainDefPtr domainDef, if (def->max_sectors) virBufferAsprintf(&buf, ",max_sectors=%u", def->max_sectors); + if (def->wwpn) + virBufferAsprintf(&buf, ",wwpn=%s", def->wwpn); + if (qemuBuildDeviceAddressStr(&buf, domainDef, &def->info, qemuCaps) < 0) goto error; diff --git a/src/vmx/vmx.c b/src/vmx/vmx.c index cd6c51e..903f0da 100644 --- a/src/vmx/vmx.c +++ b/src/vmx/vmx.c @@ -517,6 +517,7 @@ VIR_ENUM_IMPL(virVMXControllerModelSCSI, VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LAST, "pvscsi", "UNUSED ibmvscsi", "UNUSED virtio-scsi", + "UNUSED vhost-scsi", "UNUSED lsisas1078"); diff --git a/tests/qemucapabilitiesdata/caps_1.5.3-1.caps b/tests/qemucapabilitiesdata/caps_1.5.3-1.caps index 36758c8..7d53e6e 100644 --- a/tests/qemucapabilitiesdata/caps_1.5.3-1.caps +++ b/tests/qemucapabilitiesdata/caps_1.5.3-1.caps @@ -137,4 +137,5 @@ <flag name='spiceport'/> <flag name='usb-kbd'/> <flag name='host-pci-multidomain'/> + <flag name='vhost-scsi-pci'/> </qemuCaps> diff --git a/tests/qemucapabilitiesdata/caps_1.6.0-1.caps b/tests/qemucapabilitiesdata/caps_1.6.0-1.caps index ca2c236..a67c212 100644 --- a/tests/qemucapabilitiesdata/caps_1.6.0-1.caps +++ b/tests/qemucapabilitiesdata/caps_1.6.0-1.caps @@ -144,4 +144,5 @@ <flag name='usb-kbd'/> <flag name='host-pci-multidomain'/> <flag name='msg-timestamp'/> + <flag name='vhost-scsi-pci'/> </qemuCaps> diff --git a/tests/qemucapabilitiesdata/caps_1.6.50-1.caps b/tests/qemucapabilitiesdata/caps_1.6.50-1.caps index 32bccdb..9b527cc 100644 --- a/tests/qemucapabilitiesdata/caps_1.6.50-1.caps +++ b/tests/qemucapabilitiesdata/caps_1.6.50-1.caps @@ -142,4 +142,5 @@ <flag name='usb-kbd'/> <flag name='host-pci-multidomain'/> <flag name='msg-timestamp'/> + <flag name='vhost-scsi-pci'/> </qemuCaps> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.args b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.args new file mode 100644 index 0000000..33f74fe --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.args @@ -0,0 +1,9 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \ +/usr/bin/qemu -S -M pc -m 214 -smp 8 -nographic -nodefconfig -nodefaults \ +-monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c \ +-device vhost-scsi-pci,id=scsi0,cmd_per_lun=50 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.xml b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.xml new file mode 100644 index 0000000..70be0cc --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-cmd_per_lun.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>8</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver cmd_per_lun='50'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.args b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.args new file mode 100644 index 0000000..c46bc58 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.args @@ -0,0 +1,9 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \ +/usr/bin/qemu -S -M pc -m 214 -smp 8 -nographic -nodefconfig -nodefaults \ +-monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c \ +-device vhost-scsi-pci,id=scsi0,max_sectors=512 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.xml b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.xml new file mode 100644 index 0000000..41e51d5 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-max_sectors.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>8</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver max_sectors='512'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.args b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.args new file mode 100644 index 0000000..fa6ed85 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.args @@ -0,0 +1,9 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \ +/usr/bin/qemu -S -M pc -m 214 -smp 8 -nographic -nodefconfig -nodefaults \ +-monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c \ +-device vhost-scsi-pci,id=scsi0,num_queues=8 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.xml b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.xml new file mode 100644 index 0000000..9e24b47 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-num_queues.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>8</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver queues='8'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.args b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.args new file mode 100644 index 0000000..8815487 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.args @@ -0,0 +1,9 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \ +/usr/bin/qemu -S -M pc -m 214 -smp 8 -nographic -nodefconfig -nodefaults \ +-monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c \ +-device vhost-scsi-pci,id=scsi0,wwpn=naa.60014050a13df4f2 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.xml b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.xml new file mode 100644 index 0000000..b89410a --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-vhost-scsi-wwpn.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>8</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver wwpn='naa.60014050a13df4f2'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index bbc0fb7..397cca7 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -815,6 +815,18 @@ mymain(void) DO_TEST("disk-virtio-scsi-max_sectors", QEMU_CAPS_DRIVE, QEMU_CAPS_DEVICE, QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_VIRTIO_SCSI); + DO_TEST("disk-vhost-scsi-num_queues", + QEMU_CAPS_DRIVE, QEMU_CAPS_DEVICE, QEMU_CAPS_NODEFCONFIG, + QEMU_CAPS_VHOST_SCSI); + DO_TEST("disk-vhost-scsi-cmd_per_lun", + QEMU_CAPS_DRIVE, QEMU_CAPS_DEVICE, QEMU_CAPS_NODEFCONFIG, + QEMU_CAPS_VHOST_SCSI); + DO_TEST("disk-vhost-scsi-max_sectors", + QEMU_CAPS_DRIVE, QEMU_CAPS_DEVICE, QEMU_CAPS_NODEFCONFIG, + QEMU_CAPS_VHOST_SCSI); + DO_TEST("disk-vhost-scsi-wwpn", + QEMU_CAPS_DRIVE, QEMU_CAPS_DEVICE, QEMU_CAPS_NODEFCONFIG, + QEMU_CAPS_VHOST_SCSI); DO_TEST("disk-scsi-megasas", QEMU_CAPS_DRIVE, QEMU_CAPS_DEVICE, QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_SCSI_MEGASAS); diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 26e3cad..fffb254 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -225,6 +225,10 @@ mymain(void) DO_TEST("disk-virtio-scsi-num_queues"); DO_TEST("disk-virtio-scsi-cmd_per_lun"); DO_TEST("disk-virtio-scsi-max_sectors"); + DO_TEST("disk-vhost-scsi-num_queues"); + DO_TEST("disk-vhost-scsi-cmd_per_lun"); + DO_TEST("disk-vhost-scsi-max_sectors"); + DO_TEST("disk-vhost-scsi-wwpn"); DO_TEST("disk-scsi-megasas"); DO_TEST_DIFFERENT("disk-mirror-old"); DO_TEST_FULL("disk-mirror", false, WHEN_ACTIVE); -- 1.7.9.5
Il 24/07/2014 05:24, Nicholas A. Bellinger ha scritto:
From: Mike Perez <thingee@gmail.com>
This patch adds the necessary definitions to support vhost-scsi-pci and VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI in QEMU >= v1.5.x code.
This includes:
- Add QEMU_CAPS_VHOST_SCSI - Add virDomainControllerDef->wwpn pointer for the vhost-scsi wwpn. - Setting the new model type in qemuSetSCSIControllerModel() - Invoking virDomainPCIAddressReserveAddr() for vhost-scsi-pci in qemuAssignDevicePCISlots() - Add the 'wwpn=' device parameter in qemuBuildControllerDevStr() - Add vhost-scsi-pci to >= v1.5.x qemucapabilitiesdata - Add xml2argv and xml2xml tests for cmd_per_lun + max_sectors + num_queues + wwpn.
Signed-off-by: Mike Perez <thingee@gmail.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Configuration still has to go through configfs; I'm not sure how using <disk> and vhost-scsi together makes sense: + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver wwpn='naa.60014050a13df4f2'/> + </controller> ... +-device vhost-scsi-pci,id=scsi0,wwpn=naa.60014050a13df4f2 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \ and I'm not sure even how it can work since vhost-scsi-pci doesn't expose a QEMU SCSI bus. Paolo
On Thu, Jul 24, 2014 at 08:25:59AM +0200, Paolo Bonzini wrote:
Il 24/07/2014 05:24, Nicholas A. Bellinger ha scritto:
From: Mike Perez <thingee@gmail.com>
This patch adds the necessary definitions to support vhost-scsi-pci and VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI in QEMU >= v1.5.x code.
This includes:
- Add QEMU_CAPS_VHOST_SCSI - Add virDomainControllerDef->wwpn pointer for the vhost-scsi wwpn. - Setting the new model type in qemuSetSCSIControllerModel() - Invoking virDomainPCIAddressReserveAddr() for vhost-scsi-pci in qemuAssignDevicePCISlots() - Add the 'wwpn=' device parameter in qemuBuildControllerDevStr() - Add vhost-scsi-pci to >= v1.5.x qemucapabilitiesdata - Add xml2argv and xml2xml tests for cmd_per_lun + max_sectors + num_queues + wwpn.
Signed-off-by: Mike Perez <thingee@gmail.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Configuration still has to go through configfs; I'm not sure how using <disk> and vhost-scsi together makes sense:
+ <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver wwpn='naa.60014050a13df4f2'/> + </controller>
...
+-device vhost-scsi-pci,id=scsi0,wwpn=naa.60014050a13df4f2 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \
and I'm not sure even how it can work since vhost-scsi-pci doesn't expose a QEMU SCSI bus.
Indeed, it doesn't seem to actually work. I have a vm where I'm using virtio-scsi, so I changed the model to vhost-scsi and added a random wwpn string (unclear if there's any rules for generating these wwpn strings?). <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data1.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data2.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data3.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <controller type='scsi' index='0' model='vhost-scsi'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> <driver wwpn='naa.60014050a13df4f2'/> </controller> When I try to start the guest it fails to find the SCSI controller qemu-system-x86_64: -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,\ lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0: Bus 'scsi0.0' not found A further problem is that the QEMU process itself wants access to the /dev/vhost-scsi device, which it cannot have unless running as root, which is not something people will do in production. It also requires us to update the cgroups device ACL to allow access to the device and update the SELinux policy to allow access. It is unclear to me if it is safe to allow untrusted processes access to /dev/vhost-scsi device. With vhost-net, libvirt itself opens the /dev/vhost-net device and passes the pre-opened file descriptors down to QEMU, so the untrusted processes never access the device directly, which I think is a better model that we use quite alot in QEMU. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Thu, 2014-07-24 at 10:53 +0100, Daniel P. Berrange wrote:
On Thu, Jul 24, 2014 at 08:25:59AM +0200, Paolo Bonzini wrote:
Il 24/07/2014 05:24, Nicholas A. Bellinger ha scritto:
From: Mike Perez <thingee@gmail.com>
This patch adds the necessary definitions to support vhost-scsi-pci and VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI in QEMU >= v1.5.x code.
This includes:
- Add QEMU_CAPS_VHOST_SCSI - Add virDomainControllerDef->wwpn pointer for the vhost-scsi wwpn. - Setting the new model type in qemuSetSCSIControllerModel() - Invoking virDomainPCIAddressReserveAddr() for vhost-scsi-pci in qemuAssignDevicePCISlots() - Add the 'wwpn=' device parameter in qemuBuildControllerDevStr() - Add vhost-scsi-pci to >= v1.5.x qemucapabilitiesdata - Add xml2argv and xml2xml tests for cmd_per_lun + max_sectors + num_queues + wwpn.
Signed-off-by: Mike Perez <thingee@gmail.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Configuration still has to go through configfs; I'm not sure how using <disk> and vhost-scsi together makes sense:
+ <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver wwpn='naa.60014050a13df4f2'/> + </controller>
...
+-device vhost-scsi-pci,id=scsi0,wwpn=naa.60014050a13df4f2 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \
and I'm not sure even how it can work since vhost-scsi-pci doesn't expose a QEMU SCSI bus.
Indeed, it doesn't seem to actually work. I have a vm where I'm using virtio-scsi, so I changed the model to vhost-scsi and added a random wwpn string (unclear if there's any rules for generating these wwpn strings?).
<disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data1.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data2.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data3.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <controller type='scsi' index='0' model='vhost-scsi'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> <driver wwpn='naa.60014050a13df4f2'/> </controller>
When I try to start the guest it fails to find the SCSI controller
qemu-system-x86_64: -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,\ lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0: Bus 'scsi0.0' not found
Thanks for the comments. Dropping the disk virtio-scsi disk sections now.
A further problem is that the QEMU process itself wants access to the /dev/vhost-scsi device, which it cannot have unless running as root, which is not something people will do in production. It also requires us to update the cgroups device ACL to allow access to the device and update the SELinux policy to allow access. It is unclear to me if it is safe to allow untrusted processes access to /dev/vhost-scsi device. With vhost-net, libvirt itself opens the /dev/vhost-net device and passes the pre-opened file descriptors down to QEMU, so the untrusted processes never access the device directly, which I think is a better model that we use quite alot in QEMU.
Yes. Patch #2 attempts to follow what vhost-net currently does wrt opening and passing FDs into QEMU monitor, and utilizes existing vhostfd= parameter logic for passing active FDs into QEMU for this case. --nab
On Thu, Jul 24, 2014 at 11:18:14PM -0700, Nicholas A. Bellinger wrote:
On Thu, 2014-07-24 at 10:53 +0100, Daniel P. Berrange wrote:
On Thu, Jul 24, 2014 at 08:25:59AM +0200, Paolo Bonzini wrote:
Il 24/07/2014 05:24, Nicholas A. Bellinger ha scritto:
From: Mike Perez <thingee@gmail.com>
This patch adds the necessary definitions to support vhost-scsi-pci and VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI in QEMU >= v1.5.x code.
This includes:
- Add QEMU_CAPS_VHOST_SCSI - Add virDomainControllerDef->wwpn pointer for the vhost-scsi wwpn. - Setting the new model type in qemuSetSCSIControllerModel() - Invoking virDomainPCIAddressReserveAddr() for vhost-scsi-pci in qemuAssignDevicePCISlots() - Add the 'wwpn=' device parameter in qemuBuildControllerDevStr() - Add vhost-scsi-pci to >= v1.5.x qemucapabilitiesdata - Add xml2argv and xml2xml tests for cmd_per_lun + max_sectors + num_queues + wwpn.
Signed-off-by: Mike Perez <thingee@gmail.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Configuration still has to go through configfs; I'm not sure how using <disk> and vhost-scsi together makes sense:
+ <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver wwpn='naa.60014050a13df4f2'/> + </controller>
...
+-device vhost-scsi-pci,id=scsi0,wwpn=naa.60014050a13df4f2 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \
and I'm not sure even how it can work since vhost-scsi-pci doesn't expose a QEMU SCSI bus.
Indeed, it doesn't seem to actually work. I have a vm where I'm using virtio-scsi, so I changed the model to vhost-scsi and added a random wwpn string (unclear if there's any rules for generating these wwpn strings?).
<disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data1.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data2.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data3.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <controller type='scsi' index='0' model='vhost-scsi'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> <driver wwpn='naa.60014050a13df4f2'/> </controller>
When I try to start the guest it fails to find the SCSI controller
qemu-system-x86_64: -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,\ lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0: Bus 'scsi0.0' not found
Thanks for the comments.
Dropping the disk virtio-scsi disk sections now.
Huh ? If you don't have the virtio-scsi disk args how are you supposed to attach the disks to the VM. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
(Resending, as the last email did not make it to the list) On Fri, 2014-07-25 at 09:53 +0100, Daniel P. Berrange wrote:
On Thu, Jul 24, 2014 at 11:18:14PM -0700, Nicholas A. Bellinger wrote:
On Thu, 2014-07-24 at 10:53 +0100, Daniel P. Berrange wrote:
On Thu, Jul 24, 2014 at 08:25:59AM +0200, Paolo Bonzini wrote:
Il 24/07/2014 05:24, Nicholas A. Bellinger ha scritto:
From: Mike Perez <thingee@gmail.com>
This patch adds the necessary definitions to support vhost-scsi-pci and VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI in QEMU >= v1.5.x code.
This includes:
- Add QEMU_CAPS_VHOST_SCSI - Add virDomainControllerDef->wwpn pointer for the vhost-scsi wwpn. - Setting the new model type in qemuSetSCSIControllerModel() - Invoking virDomainPCIAddressReserveAddr() for vhost-scsi-pci in qemuAssignDevicePCISlots() - Add the 'wwpn=' device parameter in qemuBuildControllerDevStr() - Add vhost-scsi-pci to >= v1.5.x qemucapabilitiesdata - Add xml2argv and xml2xml tests for cmd_per_lun + max_sectors + num_queues + wwpn.
Signed-off-by: Mike Perez <thingee@gmail.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Configuration still has to go through configfs; I'm not sure how using <disk> and vhost-scsi together makes sense:
+ <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='scsi' index='0' model='vhost-scsi'> + <driver wwpn='naa.60014050a13df4f2'/> + </controller>
...
+-device vhost-scsi-pci,id=scsi0,wwpn=naa.60014050a13df4f2 \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,if=none,id=drive-scsi0-0-0-0 \ +-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \
and I'm not sure even how it can work since vhost-scsi-pci doesn't expose a QEMU SCSI bus.
Indeed, it doesn't seem to actually work. I have a vm where I'm using virtio-scsi, so I changed the model to vhost-scsi and added a random wwpn string (unclear if there's any rules for generating these wwpn strings?).
<disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data1.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data2.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/data3.img'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <controller type='scsi' index='0' model='vhost-scsi'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> <driver wwpn='naa.60014050a13df4f2'/> </controller>
When I try to start the guest it fails to find the SCSI controller
qemu-system-x86_64: -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,\ lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0: Bus 'scsi0.0' not found
Thanks for the comments.
Dropping the disk virtio-scsi disk sections now.
Huh ? If you don't have the virtio-scsi disk args how are you supposed to attach the disks to the VM.
A vhost-scsi controller instance doesn't require the extra virtio-scsi disk args, at least not in order to boot QEMU proper. The configuration of vhost-scsi WWPNs and their associated LUNs is done using a configfs based control plane provided by the in-kernel target, for which all in-kernel drivers share common code within drivers/target/target_core_fabric_configfs.c. Configfs provides reference counting for data structures within vhost-scsi itself, and also inter-module reference counting between LIO backend devices under /sys/kernel/config/target/core/$HBA/$DEV/, and vhost-scsi LUN export. The virtio-scsi LLD Host:Channel:Target:LUN disk locations in the guest are based upon what is populated using configfs groups + symlinks under /sys/kernel/config/target/vhost/$WWPN/$TPGT/lun/$LUN. The rtslib library + targetcli shell are the preferred (and friendliest) way for driving the creation of vhost-scsi controllers + LUN exports. Thanks, --nab
On Fri, Jul 25, 2014 at 02:58:58PM -0700, Nicholas A. Bellinger wrote:
A vhost-scsi controller instance doesn't require the extra virtio-scsi disk args, at least not in order to boot QEMU proper.
The configuration of vhost-scsi WWPNs and their associated LUNs is done using a configfs based control plane provided by the in-kernel target, for which all in-kernel drivers share common code within drivers/target/target_core_fabric_configfs.c.
Configfs provides reference counting for data structures within vhost-scsi itself, and also inter-module reference counting between LIO backend devices under /sys/kernel/config/target/core/$HBA/$DEV/, and vhost-scsi LUN export.
The virtio-scsi LLD Host:Channel:Target:LUN disk locations in the guest are based upon what is populated using configfs groups + symlinks under /sys/kernel/config/target/vhost/$WWPN/$TPGT/lun/$LUN.
The rtslib library + targetcli shell are the preferred (and friendliest) way for driving the creation of vhost-scsi controllers + LUN exports.
Hmm, now I see why vhost-scsi is so crippled in terms of features compared to virtio-scsi. Almost everything about it is completely opaque to QEMU. Portraying it as equivalent to virtio-scsi, only faster, is really rather misleading / confusing :-( Since QEMU doesn't get to configure the attached LUNs we loose any ability to do block I/O measurement or throttling, all the drive-mirror block job functionality, LUN hotplug/unplug, control of what happens on LUN read or write errors, disk image formats and more besides. I don't see any viable way to address any of that via QEMU even if we wanted to fix it. If it is also opaque to libvirt, then it makes it impossible to actually use this feature via the libvirt API unless you also have a side-channel giving you root access to the host to configure configfs :-( W can also no longer apply disk locking / lease aquisition per LUN to prevent the same disk image being used by two VMs at the same time, and loose the SELinux/sVirt isolation of guest from disks. It would be possible to partially address much of this by making libvirt itself responsible for all of the configuration HBA in configfs, but that is a major amount of work to undertake. I also wonder if QEMU is placed in a cgroup with the blkio controller attached, will I/O to vhost-scsi be correctly attributed and controlled by the blkio controller. As long as everything about the LUN configuration is completely opaque to libvirt & QEMU, I don't think that representing vhost-scsi as a <controller> really makes any sense. The <controller> stuff in libvirt only exists in the first place in order to have somewhere to hang the <disk> configs off. Using <controller> would only make sense if the patch were to properly support the corresponding <disk> attachments and take full ownership of configuring things via configfs. I'm not sure that is worth the effort or ongoing maintainence cost though given that we're expecting virtio-scsi to be able to match vhost-scsi for performance, so there won't be much compelling reason to sacrifice so many QEMU features and use vhost-scsi. As proposed, this patch is really doing something more akin to SCSI HBA passthrough from host to guest, which would be something that's more appropriate for the <hostdev> configuration data. That's making it clear that the device is completely opaque to libvirt/QEMU from a functional configuration POV. We currently have a <hostdev> 'scsi' feature, but that is about passthrough of individual LUNs, so we'd have to invent new configuration schema for 'scsi_host' <hostdev> type. This seems like the most viable approach to supporting this feature right now. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
From: Nicholas Bellinger <nab@linux-iscsi.org> This patch adds support for device_add hotplug for vhost-scsi-pci against QEMU >= v1.5.x code. This includes: - Add qemuOpenVhostSCSI() to open fds to /dev/vhost-scsi character device. - Changes to qemuBuildControllerDevStr() for adding 'vhostfd=' parameters to device_add. - Add qemuMonitorAddControllerVhost() that is specific to VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI - Update qemuDomainAttachControllerDevice() to setup vhostfd fd + name arrays, and invoke qemuOpenVhostSCSI(). - Update qemuDomainAttachControllerDevice() to invoke qemuMonitorAddControllerVhost() -> qemuMonitorSendFileHandle() -> qemuMonitorAddDevice() in order to pass the pre-opened vhostfd. This code has been tested using openstack nova volume-attach, using a Juno v2 development head from 07192014. Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org> Signed-off-by: Mike Perez <thingee@gmail.com> --- src/qemu/qemu_command.c | 56 ++++++++++++++++++++++++++++++++++++++++++++--- src/qemu/qemu_command.h | 8 ++++++- src/qemu/qemu_hotplug.c | 47 +++++++++++++++++++++++++++++++++++++-- src/qemu/qemu_monitor.c | 24 ++++++++++++++++++++ src/qemu/qemu_monitor.h | 4 ++++ 5 files changed, 133 insertions(+), 6 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 43c0e1c..1e987f9 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -678,6 +678,38 @@ static int qemuAssignDeviceDiskAliasFixed(virDomainDiskDefPtr disk) return 0; } +int +qemuOpenVhostSCSI(virDomainControllerDefPtr controller, + int *vhostfd, + int *vhostfdSize) +{ + size_t i; + + if (controller->model != VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI) { + *vhostfdSize = 0; + return 0; + } + + for (i = 0; i < *vhostfdSize; i++) { + vhostfd[i] = open("/dev/vhost-scsi", O_RDWR); + + if (vhostfd[i] < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + "%s", _("vhost-scsi was requested for an interface, " + "but is unavailable")); + goto error; + } + } + + return 0; + + error: + while (i--) + VIR_FORCE_CLOSE(vhostfd[i]); + + return -1; +} + static int qemuSetSCSIControllerModel(virDomainDefPtr def, virQEMUCapsPtr qemuCaps, @@ -4153,10 +4185,13 @@ char * qemuBuildControllerDevStr(virDomainDefPtr domainDef, virDomainControllerDefPtr def, virQEMUCapsPtr qemuCaps, - int *nusbcontroller) + int *nusbcontroller, + char **vhostfd, + int vhostfdSize) { virBuffer buf = VIR_BUFFER_INITIALIZER; int model; + size_t i; if (!(def->type == VIR_DOMAIN_CONTROLLER_TYPE_SCSI && (def->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VIRTIO_SCSI || @@ -4328,6 +4363,19 @@ qemuBuildControllerDevStr(virDomainDefPtr domainDef, if (def->wwpn) virBufferAsprintf(&buf, ",wwpn=%s", def->wwpn); + if (vhostfdSize) { + if (vhostfdSize == 1) { + virBufferAsprintf(&buf, ",vhostfd=%s", vhostfd[0]); + } else { + virBufferAddLit(&buf, ",vhostfds="); + for (i = 0; i < vhostfdSize; i++) { + if (i) + virBufferAddChar(&buf, ':'); + virBufferAdd(&buf, vhostfd[i], -1); + } + } + } + if (qemuBuildDeviceAddressStr(&buf, domainDef, &def->info, qemuCaps) < 0) goto error; @@ -7843,7 +7891,8 @@ qemuBuildCommandLine(virConnectPtr conn, virCommandAddArg(cmd, "-device"); if (!(devstr = qemuBuildControllerDevStr(def, cont, - qemuCaps, NULL))) + qemuCaps, NULL, + NULL, 0))) goto error; virCommandAddArg(cmd, devstr); @@ -7867,7 +7916,8 @@ qemuBuildCommandLine(virConnectPtr conn, char *devstr; if (!(devstr = qemuBuildControllerDevStr(def, cont, qemuCaps, - &usbcontroller))) + &usbcontroller, NULL, + 0))) goto error; virCommandAddArg(cmd, devstr); diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index cf51056..b0651ff 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -114,6 +114,10 @@ char * qemuBuildNicDevStr(virDomainDefPtr def, char *qemuDeviceDriveHostAlias(virDomainDiskDefPtr disk, virQEMUCapsPtr qemuCaps); +int qemuOpenVhostSCSI(virDomainControllerDefPtr controller, + int *vhostfd, + int *vhostfdSize); + /* Both legacy & current support */ char *qemuBuildDriveStr(virConnectPtr conn, virDomainDiskDefPtr disk, @@ -134,7 +138,9 @@ char * qemuBuildFSDevStr(virDomainDefPtr domainDef, char * qemuBuildControllerDevStr(virDomainDefPtr domainDef, virDomainControllerDefPtr def, virQEMUCapsPtr qemuCaps, - int *nusbcontroller); + int *nusbcontroller, + char **vhostfd, + int vhostfdSize); char * qemuBuildWatchdogDevStr(virDomainDefPtr domainDef, virDomainWatchdogDefPtr dev, diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 1fc28b8..4cdac56 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -360,9 +360,13 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, virDomainObjPtr vm, virDomainControllerDefPtr controller) { + size_t i; int ret = -1; const char* type = virDomainControllerTypeToString(controller->type); char *devstr = NULL; + char **vhostfdName = NULL; + int *vhostfd = NULL; + int vhostfdSize = 0; qemuDomainObjPrivatePtr priv = vm->privateData; bool releaseaddr = false; @@ -403,7 +407,30 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, goto cleanup; } - if (!(devstr = qemuBuildControllerDevStr(vm->def, controller, priv->qemuCaps, NULL))) { + if (controller->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI) { + /* FIXME: Get total number of virtio-scsi queues */ + vhostfdSize = 1; + + if (VIR_ALLOC_N(vhostfd, vhostfdSize) < 0) + goto cleanup; + + memset(vhostfd, -1, sizeof(*vhostfd) * vhostfdSize); + + if (VIR_ALLOC_N(vhostfdName, vhostfdSize) < 0) + goto cleanup; + + if (qemuOpenVhostSCSI(controller, vhostfd, &vhostfdSize) < 0) + goto cleanup; + + for (i = 0; i < vhostfdSize; i++) { + if (virAsprintf(&vhostfdName[i], "vhostfd-%s%zu", + controller->info.alias, i) < 0) + goto cleanup; + } + } + + if (!(devstr = qemuBuildControllerDevStr(vm->def, controller, priv->qemuCaps, + NULL, vhostfdName, vhostfdSize))) { goto cleanup; } } @@ -413,7 +440,12 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, qemuDomainObjEnterMonitor(driver, vm); if (virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_DEVICE)) { - ret = qemuMonitorAddDevice(priv->mon, devstr); + if (controller->model == VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VHOST_SCSI) { + ret = qemuMonitorAddControllerVhost(priv->mon, devstr, + vhostfd, vhostfdName, vhostfdSize); + } else { + ret = qemuMonitorAddDevice(priv->mon, devstr); + } } else { ret = qemuMonitorAttachPCIDiskController(priv->mon, type, @@ -421,6 +453,9 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, } qemuDomainObjExitMonitor(driver, vm); + for (i = 0; i < vhostfdSize; i++) + VIR_FORCE_CLOSE(vhostfd[i]); + if (ret == 0) { if (controller->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) controller->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; @@ -431,6 +466,14 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, if (ret != 0 && releaseaddr) qemuDomainReleaseDeviceAddress(vm, &controller->info, NULL); + for (i = 0; vhostfd && i < vhostfdSize; i++) { + VIR_FORCE_CLOSE(vhostfd[i]); + if (vhostfdName) + VIR_FREE(vhostfdName[i]); + } + VIR_FREE(vhostfd); + VIR_FREE(vhostfdName); + VIR_FREE(devstr); return ret; } diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index db3dd73..3e67c85 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -3052,6 +3052,30 @@ int qemuMonitorAddDevice(qemuMonitorPtr mon, return qemuMonitorAddDeviceWithFd(mon, devicestr, -1, NULL); } +int qemuMonitorAddControllerVhost(qemuMonitorPtr mon, + const char *devicestr, + int *vhostfd, char **vhostfdName, int vhostfdSize) +{ + size_t i = 0; + + for (i = 0; i < vhostfdSize; i++) { + if (qemuMonitorSendFileHandle(mon, vhostfdName[i], vhostfd[i]) < 0) + goto cleanup; + } + + if (qemuMonitorAddDevice(mon, devicestr) < 0) + goto cleanup; + + return 0; + + cleanup: + while (i--) { + if (qemuMonitorCloseFileHandle(mon, vhostfdName[i]) < 0) + VIR_WARN("failed to close device handle '%s'", vhostfdName[i]); + } + return -1; +} + int qemuMonitorAddDrive(qemuMonitorPtr mon, const char *drivestr) { diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 8a23267..8fe21a0 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -620,6 +620,10 @@ int qemuMonitorAddDeviceWithFd(qemuMonitorPtr mon, int fd, const char *fdname); +int qemuMonitorAddControllerVhost(qemuMonitorPtr mon, + const char *devicestr, + int *vhostfd, char **vhostfdName, int vhostfdSize); + int qemuMonitorDelDevice(qemuMonitorPtr mon, const char *devalias); -- 1.7.9.5
On Thu, Jul 24, 2014 at 03:24:12AM +0000, Nicholas A. Bellinger wrote:
From: Nicholas Bellinger <nab@linux-iscsi.org>
Hi Daniel & Co,
The following is the patch series to add support for vhost-scsi-pci from QEMU >= v1.5.x into libvirt. It includes proper support for passing a pre-opened vhostfd into vhost-scsi, which is required in order to properly support nova performing a device_add for adding a individual vhost-scsi-pci WWPN endpoint -> SCSI controller.
It does not appear to actually be passing any vhost-scsi fd when I start QEMU, so something is missing here. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Thu, 2014-07-24 at 10:58 +0100, Daniel P. Berrange wrote:
On Thu, Jul 24, 2014 at 03:24:12AM +0000, Nicholas A. Bellinger wrote:
From: Nicholas Bellinger <nab@linux-iscsi.org>
Hi Daniel & Co,
The following is the patch series to add support for vhost-scsi-pci from QEMU >= v1.5.x into libvirt. It includes proper support for passing a pre-opened vhostfd into vhost-scsi, which is required in order to properly support nova performing a device_add for adding a individual vhost-scsi-pci WWPN endpoint -> SCSI controller.
It does not appear to actually be passing any vhost-scsi fd when I start QEMU, so something is missing here.
Thanks for your comments. The patch adds vhost-scsi WWPN controller support using device_add hotplug, and does not include QEMU initial command line support. We will get this item addressed for -v2. Thanks, --nab
participants (3)
-
Daniel P. Berrange -
Nicholas A. Bellinger -
Paolo Bonzini