[PATCH 0/2] hyperv: storage volume XML changes

Here's a GitLab MR if you'd prefer to review it there: https://gitlab.com/iammattcoleman/libvirt/-/merge_requests/12/commits Matt Coleman (2): hyperv: XML parsing of storage volumes schema: add support for Windows file paths and device names docs/schemas/basictypes.rng | 2 +- docs/schemas/domaincommon.rng | 5 +- src/hyperv/hyperv_driver.c | 408 +++++++++++++++++- src/hyperv/hyperv_driver.h | 3 + src/hyperv/hyperv_wmi.c | 45 ++ src/hyperv/hyperv_wmi.h | 8 + src/hyperv/hyperv_wmi_classes.h | 19 + src/hyperv/hyperv_wmi_generator.input | 134 ++++++ .../disk-hyperv-physical.xml | 17 + .../disk-hyperv-virtual.xml | 17 + .../disk-hyperv-physical.xml | 23 + .../disk-hyperv-virtual.xml | 23 + tests/genericxml2xmltest.c | 2 + 13 files changed, 703 insertions(+), 3 deletions(-) create mode 100644 tests/genericxml2xmlindata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmlindata/disk-hyperv-virtual.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-virtual.xml -- 2.27.0

dumpxml can now serialize: * floppy drives * file-backed and device-backed disk drives * images mounted to virtual CD/DVD drives * IDE and SCSI controllers Co-authored-by: Sri Ramanujam <sramanujam@datto.com> Signed-off-by: Matt Coleman <matt@datto.com> --- src/hyperv/hyperv_driver.c | 408 +++++++++++++++++++++++++- src/hyperv/hyperv_driver.h | 3 + src/hyperv/hyperv_wmi.c | 45 +++ src/hyperv/hyperv_wmi.h | 8 + src/hyperv/hyperv_wmi_classes.h | 19 ++ src/hyperv/hyperv_wmi_generator.input | 134 +++++++++ 6 files changed, 616 insertions(+), 1 deletion(-) diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c index 40739595ac..ce9ba2a02a 100644 --- a/src/hyperv/hyperv_driver.c +++ b/src/hyperv/hyperv_driver.c @@ -293,6 +293,385 @@ hypervCapsInit(hypervPrivate *priv) return NULL; } +/* + * Virtual device functions + */ +static int +hypervGetDeviceParentRasdFromDeviceId(const char *parentDeviceId, + Msvm_ResourceAllocationSettingData *list, + Msvm_ResourceAllocationSettingData **out) +{ + Msvm_ResourceAllocationSettingData *entry = list; + g_autofree char *escapedDeviceId = NULL; + *out = NULL; + + while (entry) { + escapedDeviceId = g_strdup_printf("%s\"", entry->data->InstanceID); + escapedDeviceId = virStringReplace(escapedDeviceId, "\\", "\\\\"); + + if (g_str_has_suffix(parentDeviceId, escapedDeviceId)) { + *out = entry; + break; + } + + entry = entry->next; + } + + if (*out) + return 0; + + return -1; +} + + + +/* + * Functions for deserializing device entries + */ +static int +hypervDomainDefAppendController(virDomainDefPtr def, + int idx, + virDomainControllerType controllerType) +{ + virDomainControllerDefPtr controller = NULL; + + if (!(controller = virDomainControllerDefNew(controllerType))) + return -1; + + controller->idx = idx; + + if (VIR_APPEND_ELEMENT(def->controllers, def->ncontrollers, controller) < 0) + return -1; + + return 0; +} + + +static int +hypervDomainDefParseIDEController(virDomainDefPtr def, int idx) +{ + return hypervDomainDefAppendController(def, idx, VIR_DOMAIN_CONTROLLER_TYPE_IDE); +} + + +static int +hypervDomainDefParseSCSIController(virDomainDefPtr def, int idx) +{ + return hypervDomainDefAppendController(def, idx, VIR_DOMAIN_CONTROLLER_TYPE_SCSI); +} + + +static int +hypervDomainDefAppendDisk(virDomainDefPtr def, + virDomainDiskDefPtr disk, + virDomainDiskBus busType, + int diskNameOffset, + const char *diskNamePrefix, + int maxControllers, + Msvm_ResourceAllocationSettingData **controllers, + Msvm_ResourceAllocationSettingData *diskParent, + Msvm_ResourceAllocationSettingData *diskController) +{ + size_t i = 0; + int ctrlr_idx = -1; + int addr = -1; + + if (virStrToLong_i(diskParent->data->AddressOnParent, NULL, 10, &addr) < 0) + return -1; + + if (addr < 0) + return -1; + + /* Find controller index */ + for (i = 0; i < maxControllers; i++) { + if (diskController == controllers[i]) { + ctrlr_idx = i; + break; + } + } + + if (ctrlr_idx < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not find controller for disk!")); + return -1; + } + + disk->bus = busType; + disk->dst = virIndexToDiskName(ctrlr_idx * diskNameOffset + addr, diskNamePrefix); + disk->info.addr.drive.controller = ctrlr_idx; + disk->info.addr.drive.bus = 0; + disk->info.addr.drive.target = 0; + disk->info.addr.drive.unit = addr; + + if (VIR_APPEND_ELEMENT(def->disks, def->ndisks, disk) < 0) + return -1; + + return 0; +} + + +static int +hypervDomainDefParseFloppyStorageExtent(virDomainDefPtr def, virDomainDiskDefPtr disk) +{ + disk->bus = VIR_DOMAIN_DISK_BUS_FDC; + disk->dst = g_strdup("fda"); + + if (VIR_APPEND_ELEMENT(def->disks, def->ndisks, disk) < 0) + return -1; + + return 0; +} + + +static int +hypervDomainDefParseVirtualExtent(hypervPrivate *priv, + virDomainDefPtr def, + Msvm_StorageAllocationSettingData *disk_entry, + Msvm_ResourceAllocationSettingData *rasd, + Msvm_ResourceAllocationSettingData **ideControllers, + Msvm_ResourceAllocationSettingData **scsiControllers) +{ + Msvm_ResourceAllocationSettingData *diskParent = NULL; + Msvm_ResourceAllocationSettingData *controller = NULL; + virDomainDiskDefPtr disk = NULL; + int result = -1; + + if (disk_entry->data->HostResource.count < 1) + goto cleanup; + + if (!(disk = virDomainDiskDefNew(priv->xmlopt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not allocate disk definition")); + goto cleanup; + } + + /* get disk associated with storage extent */ + if (hypervGetDeviceParentRasdFromDeviceId(disk_entry->data->Parent, rasd, &diskParent) < 0) + goto cleanup; + + /* get associated controller */ + if (hypervGetDeviceParentRasdFromDeviceId(diskParent->data->Parent, rasd, &controller) < 0) + goto cleanup; + + /* common fields first */ + disk->src->type = VIR_STORAGE_TYPE_FILE; + disk->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DRIVE; + + /* note if it's a CDROM disk */ + if (STREQ(disk_entry->data->ResourceSubType, "Microsoft:Hyper-V:Virtual CD/DVD Disk")) + disk->device = VIR_DOMAIN_DISK_DEVICE_CDROM; + else + disk->device = VIR_DOMAIN_DISK_DEVICE_DISK; + + /* copy in the source path */ + virDomainDiskSetSource(disk, *(char **)disk_entry->data->HostResource.data); + + /* controller-specific fields */ + if (controller->data->ResourceType == MSVM_RASD_RESOURCETYPE_PARALLEL_SCSI_HBA) { + if (hypervDomainDefAppendDisk(def, disk, VIR_DOMAIN_DISK_BUS_SCSI, + 64, "sd", HYPERV_MAX_SCSI_CONTROLLERS, + scsiControllers, diskParent, controller) < 0) { + goto cleanup; + } + } else if (controller->data->ResourceType == MSVM_RASD_RESOURCETYPE_IDE_CONTROLLER) { + if (hypervDomainDefAppendDisk(def, disk, VIR_DOMAIN_DISK_BUS_IDE, + 4, "hd", HYPERV_MAX_IDE_CONTROLLERS, + ideControllers, diskParent, controller) < 0) { + goto cleanup; + } + } else if (controller->data->ResourceType == MSVM_RASD_RESOURCETYPE_OTHER && + diskParent->data->ResourceType == MSVM_RASD_RESOURCETYPE_DISKETTE_DRIVE) { + if (hypervDomainDefParseFloppyStorageExtent(def, disk) < 0) + goto cleanup; + disk->device = VIR_DOMAIN_DISK_DEVICE_FLOPPY; + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unrecognized controller type %d"), + controller->data->ResourceType); + goto cleanup; + } + + result = 0; + + cleanup: + if (result != 0 && disk) + virDomainDiskDefFree(disk); + + return result; +} + + +static int +hypervDomainDefParsePhysicalDisk(hypervPrivate *priv, + virDomainDefPtr def, + Msvm_ResourceAllocationSettingData *entry, + Msvm_ResourceAllocationSettingData *rasd, + Msvm_ResourceAllocationSettingData **ideControllers, + Msvm_ResourceAllocationSettingData **scsiControllers) +{ + int result = -1; + Msvm_ResourceAllocationSettingData *controller = NULL; + Msvm_DiskDrive *diskdrive = NULL; + virDomainDiskDefPtr disk = NULL; + char **hostResource = entry->data->HostResource.data; + g_autofree char *hostEscaped = NULL; + g_autofree char *driveNumberStr = NULL; + g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER; + int addr = -1, ctrlr_idx = -1; + size_t i = 0; + + if (virStrToLong_i(entry->data->AddressOnParent, NULL, 10, &addr) < 0) + return -1; + + if (addr < 0) + return -1; + + if (hypervGetDeviceParentRasdFromDeviceId(entry->data->Parent, rasd, &controller) < 0) + goto cleanup; + + /* create disk definition */ + if (!(disk = virDomainDiskDefNew(priv->xmlopt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not allocate disk def")); + goto cleanup; + } + + /* Query Msvm_DiskDrive for the DriveNumber */ + hostEscaped = virStringReplace(*hostResource, "\\\"", "\""); + hostEscaped = virStringReplace(hostEscaped, "\\", "\\\\"); + + /* quotes must be preserved, so virBufferEscapeSQL can't be used */ + virBufferAsprintf(&query, + MSVM_DISKDRIVE_WQL_SELECT "WHERE __PATH='%s'", + hostEscaped); + + if (hypervGetWmiClass(Msvm_DiskDrive, &diskdrive) < 0) + goto cleanup; + + if (!diskdrive) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not find Msvm_DiskDrive object")); + goto cleanup; + } + + driveNumberStr = g_strdup_printf("%u", diskdrive->data->DriveNumber); + virDomainDiskSetSource(disk, driveNumberStr); + + if (addr < 0) + goto cleanup; + + if (controller->data->ResourceType == MSVM_RASD_RESOURCETYPE_PARALLEL_SCSI_HBA) { + for (i = 0; i < HYPERV_MAX_SCSI_CONTROLLERS; i++) { + if (controller == scsiControllers[i]) { + ctrlr_idx = i; + break; + } + } + disk->bus = VIR_DOMAIN_DISK_BUS_SCSI; + disk->dst = virIndexToDiskName(ctrlr_idx * 64 + addr, "sd"); + disk->info.addr.drive.unit = addr; + } else if (controller->data->ResourceType == MSVM_RASD_RESOURCETYPE_IDE_CONTROLLER) { + for (i = 0; i < HYPERV_MAX_IDE_CONTROLLERS; i++) { + if (controller == ideControllers[i]) { + ctrlr_idx = i; + break; + } + } + disk->bus = VIR_DOMAIN_DISK_BUS_IDE; + disk->dst = virIndexToDiskName(ctrlr_idx * 4 + addr, "hd"); + disk->info.addr.drive.unit = addr; + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Invalid controller type for LUN")); + goto cleanup; + } + + disk->info.addr.drive.controller = ctrlr_idx; + disk->info.addr.drive.bus = 0; + disk->info.addr.drive.target = 0; + virDomainDiskSetType(disk, VIR_STORAGE_TYPE_BLOCK); + disk->device = VIR_DOMAIN_DISK_DEVICE_DISK; + + disk->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DRIVE; + + if (VIR_APPEND_ELEMENT(def->disks, def->ndisks, disk) < 0) + goto cleanup; + + result = 0; + + cleanup: + if (result != 0 && disk) + virDomainDiskDefFree(disk); + hypervFreeObject(priv, (hypervObject *)diskdrive); + + return result; +} + + +static int +hypervDomainDefParseStorage(hypervPrivate *priv, + virDomainDefPtr def, + Msvm_ResourceAllocationSettingData *rasd, + Msvm_StorageAllocationSettingData *sasd) +{ + Msvm_ResourceAllocationSettingData *entry = rasd; + Msvm_StorageAllocationSettingData *disk_entry = sasd; + Msvm_ResourceAllocationSettingData *ideControllers[HYPERV_MAX_IDE_CONTROLLERS]; + Msvm_ResourceAllocationSettingData *scsiControllers[HYPERV_MAX_SCSI_CONTROLLERS]; + int ide_idx = -1; + int scsi_idx = 0; + + /* first pass: populate storage controllers */ + while (entry) { + if (entry->data->ResourceType == MSVM_RASD_RESOURCETYPE_IDE_CONTROLLER) { + ide_idx = entry->data->Address[0] - '0'; + ideControllers[ide_idx] = entry; + if (hypervDomainDefParseIDEController(def, ide_idx) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not parse IDE controller")); + return -1; + } + } else if (entry->data->ResourceType == MSVM_RASD_RESOURCETYPE_PARALLEL_SCSI_HBA) { + scsiControllers[scsi_idx++] = entry; + if (hypervDomainDefParseSCSIController(def, scsi_idx - 1) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not parse SCSI controller")); + return -1; + } + } + + entry = entry->next; + } + + /* second pass: populate physical disks */ + entry = rasd; + while (entry) { + if (entry->data->ResourceType == MSVM_RASD_RESOURCETYPE_DISK_DRIVE && + entry->data->HostResource.count > 0) { + char **hostResource = entry->data->HostResource.data; + + if (strstr(*hostResource, "NODRIVE")) { + /* Hyper-V doesn't let you define LUNs with no connection */ + VIR_DEBUG("Skipping empty LUN '%s'", *hostResource); + entry = entry->next; + continue; + } + + if (hypervDomainDefParsePhysicalDisk(priv, def, entry, rasd, + ideControllers, scsiControllers) < 0) + return -1; + } + + entry = entry->next; + } + + /* third pass: populate virtual disks */ + while (disk_entry) { + if (hypervDomainDefParseVirtualExtent(priv, def, disk_entry, rasd, + ideControllers, scsiControllers) < 0) + return -1; + + disk_entry = disk_entry->next; + } + + return 0; +} + + + /* * Driver functions */ @@ -1249,6 +1628,8 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int flags) Msvm_VirtualSystemSettingData *virtualSystemSettingData = NULL; Msvm_ProcessorSettingData *processorSettingData = NULL; Msvm_MemorySettingData *memorySettingData = NULL; + Msvm_ResourceAllocationSettingData *rasd = NULL; + Msvm_StorageAllocationSettingData *sasd = NULL; virCheckFlags(VIR_DOMAIN_XML_COMMON_FLAGS, NULL); @@ -1275,6 +1656,18 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int flags) &memorySettingData) < 0) goto cleanup; + if (hypervGetResourceAllocationSD(priv, + virtualSystemSettingData->data->InstanceID, + &rasd) < 0) { + goto cleanup; + } + + if (hypervGetStorageAllocationSD(priv, + virtualSystemSettingData->data->InstanceID, + &sasd) < 0) { + goto cleanup; + } + /* Fill struct */ def->virtType = VIR_DOMAIN_VIRT_HYPERV; @@ -1324,7 +1717,18 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int flags) def->os.type = VIR_DOMAIN_OSTYPE_HVM; - /* FIXME: devices section is totally missing */ + /* Allocate space for all potential devices */ + + /* 256 scsi drives + 8 ide drives */ + def->disks = g_new0(virDomainDiskDefPtr, 264); + def->ndisks = 0; + + /* 2 ide & 4 scsi controllers */ + def->controllers = g_new0(virDomainControllerDefPtr, 6); + def->ncontrollers = 0; + + if (hypervDomainDefParseStorage(priv, def, rasd, sasd) < 0) + goto cleanup; /* XXX xmlopts must be non-NULL */ xml = virDomainDefFormat(def, NULL, @@ -1336,6 +1740,8 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int flags) hypervFreeObject(priv, (hypervObject *)virtualSystemSettingData); hypervFreeObject(priv, (hypervObject *)processorSettingData); hypervFreeObject(priv, (hypervObject *)memorySettingData); + hypervFreeObject(priv, (hypervObject *)rasd); + hypervFreeObject(priv, (hypervObject *)sasd); return xml; } diff --git a/src/hyperv/hyperv_driver.h b/src/hyperv/hyperv_driver.h index 8099b5714b..e49550a2c8 100644 --- a/src/hyperv/hyperv_driver.h +++ b/src/hyperv/hyperv_driver.h @@ -22,4 +22,7 @@ #pragma once +#define HYPERV_MAX_SCSI_CONTROLLERS 4 +#define HYPERV_MAX_IDE_CONTROLLERS 2 + int hypervRegister(void); diff --git a/src/hyperv/hyperv_wmi.c b/src/hyperv/hyperv_wmi.c index efd0659051..466296fe2a 100644 --- a/src/hyperv/hyperv_wmi.c +++ b/src/hyperv/hyperv_wmi.c @@ -1490,6 +1490,32 @@ hypervGetMsvmVirtualSystemSettingDataFromUUID(hypervPrivate *priv, } +int +hypervGetResourceAllocationSD(hypervPrivate *priv, + const char *id, + Msvm_ResourceAllocationSettingData **data) +{ + g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER; + virBufferEscapeSQL(&query, + "ASSOCIATORS OF {Msvm_VirtualSystemSettingData.InstanceID='%s'} " + "WHERE AssocClass = Msvm_VirtualSystemSettingDataComponent " + "ResultClass = Msvm_ResourceAllocationSettingData", + id); + + if (hypervGetWmiClass(Msvm_ResourceAllocationSettingData, data) < 0) + return -1; + + if (!*data) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not look up resource allocation setting data with virtual system instance ID '%s'"), + id); + return -1; + } + + return 0; +} + + int hypervGetProcessorSD(hypervPrivate *priv, const char *id, @@ -1536,6 +1562,25 @@ hypervGetMemorySD(hypervPrivate *priv, } +int +hypervGetStorageAllocationSD(hypervPrivate *priv, + const char *id, + Msvm_StorageAllocationSettingData **data) +{ + g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER; + virBufferEscapeSQL(&query, + "ASSOCIATORS OF {Msvm_VirtualSystemSettingData.InstanceID='%s'} " + "WHERE AssocClass = Msvm_VirtualSystemSettingDataComponent " + "ResultClass = Msvm_StorageAllocationSettingData", + id); + + if (hypervGetWmiClass(Msvm_StorageAllocationSettingData, data) < 0) + return -1; + + return 0; +} + + /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Msvm_VirtualSystemManagementService */ diff --git a/src/hyperv/hyperv_wmi.h b/src/hyperv/hyperv_wmi.h index 34334a0153..31f7e2e3ba 100644 --- a/src/hyperv/hyperv_wmi.h +++ b/src/hyperv/hyperv_wmi.h @@ -236,6 +236,10 @@ int hypervGetMsvmVirtualSystemSettingDataFromUUID(hypervPrivate *priv, const char *uuid_string, Msvm_VirtualSystemSettingData **list); +int hypervGetResourceAllocationSD(hypervPrivate *priv, + const char *id, + Msvm_ResourceAllocationSettingData **data); + int hypervGetProcessorSD(hypervPrivate *priv, const char *id, Msvm_ProcessorSettingData **data); @@ -244,6 +248,10 @@ int hypervGetMemorySD(hypervPrivate *priv, const char *vssd_instanceid, Msvm_MemorySettingData **list); +int hypervGetStorageAllocationSD(hypervPrivate *priv, + const char *id, + Msvm_StorageAllocationSettingData **data); + /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Msvm_VirtualSystemManagementService */ diff --git a/src/hyperv/hyperv_wmi_classes.h b/src/hyperv/hyperv_wmi_classes.h index 161e9be131..36c0e60c2a 100644 --- a/src/hyperv/hyperv_wmi_classes.h +++ b/src/hyperv/hyperv_wmi_classes.h @@ -98,6 +98,25 @@ enum _Msvm_ConcreteJob_JobState { }; + +/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * + * Msvm_ResourceAllocationSettingData + */ + +/* https://docs.microsoft.com/en-us/windows/win32/hyperv_v2/msvm-resourcealloca... */ +enum _Msvm_ResourceAllocationSettingData_ResourceType { + MSVM_RASD_RESOURCETYPE_OTHER = 1, + MSVM_RASD_RESOURCETYPE_IDE_CONTROLLER = 5, + MSVM_RASD_RESOURCETYPE_PARALLEL_SCSI_HBA = 6, + MSVM_RASD_RESOURCETYPE_DISKETTE_DRIVE = 14, + MSVM_RASD_RESOURCETYPE_CD_DRIVE = 15, + MSVM_RASD_RESOURCETYPE_DVD_DRIVE = 16, + MSVM_RASD_RESOURCETYPE_DISK_DRIVE = 17, + MSVM_RASD_RESOURCETYPE_STORAGE_EXTENT = 19, +}; + + + /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * WMI */ diff --git a/src/hyperv/hyperv_wmi_generator.input b/src/hyperv/hyperv_wmi_generator.input index bc0e781676..e1b4fd3f9f 100644 --- a/src/hyperv/hyperv_wmi_generator.input +++ b/src/hyperv/hyperv_wmi_generator.input @@ -601,6 +601,34 @@ class Msvm_VirtualSystemManagementService end +class Msvm_ResourceAllocationSettingData + string InstanceID + string Caption + string Description + string ElementName + uint16 ResourceType + string OtherResourceType + string ResourceSubType + string PoolID + uint16 ConsumerVisibility + string HostResource[] + string AllocationUnits + uint64 VirtualQuantity + uint64 Reservation + uint64 Limit + uint32 Weight + boolean AutomaticAllocation + boolean AutomaticDeallocation + string Parent + string Connection[] + string Address + uint16 MappingBehavior + string AddressOnParent + string VirtualQuantityUnits + string VirtualSystemIdentifiers[] +end + + class Msvm_Keyboard string InstanceID string Caption @@ -688,3 +716,109 @@ class Msvm_ShutdownComponent uint16 AdditionalAvailability[] uint64 MaxQuiesceTime end + + +class Msvm_DiskDrive + string InstanceID + string Caption + string Description + string ElementName + datetime InstallDate + string Name + uint16 OperationalStatus[] + string StatusDescriptions[] + string Status + uint16 HealthState + uint16 CommunicationStatus + uint16 DetailedStatus + uint16 OperatingStatus + uint16 PrimaryStatus + uint16 EnabledState + string OtherEnabledState + uint16 RequestedState + uint16 EnabledDefault + datetime TimeOfLastStateChange + uint16 AvailableRequestedStates[] + uint16 TransitioningToState + string SystemCreationClassName + string SystemName + string CreationClassName + string DeviceID + boolean PowerManagementSupported + uint16 PowerManagementCapabilities[] + uint16 Availability + uint16 StatusInfo + uint32 LastErrorCode + string ErrorDescription + boolean ErrorCleared + string OtherIdentifyingInfo[] + uint64 PowerOnHours + uint64 TotalPowerOnHours + string IdentifyingDescriptions[] + uint16 AdditionalAvailability[] + uint64 MaxQuiesceTime + uint16 Capabilities[] + string CapabilityDescriptions[] + string ErrorMethodology + string CompressionMethod + uint32 NumberOfMediaSupported + uint64 MaxMediaSize + uint64 DefaultBlockSize + uint64 MaxBlockSize + uint64 MinBlockSize + boolean NeedsCleaning + boolean MediaIsLocked + uint16 Security + datetime LastCleaned + uint64 MaxAccessTime + uint32 UncompressedDataRate + uint64 LoadTime + uint64 UnloadTime + uint64 MountCount + datetime TimeOfLastMount + uint64 TotalMountTime + string UnitsDescription + uint64 MaxUnitsBeforeCleaning + uint64 UnitsUsed + uint32 DriveNumber +end + + +class Msvm_StorageAllocationSettingData + string InstanceID + string Caption + string Description + string ElementName + uint16 ResourceType + string OtherResourceType + string ResourceSubType + string PoolID + uint16 ConsumerVisibility + string HostResource[] + string AllocationUnits + uint64 VirtualQuantity + uint64 Limit + uint32 Weight + boolean AutomaticAllocation + boolean AutomaticDeallocation + string Parent + string Connection[] + string Address + uint16 MappingBehavior + string AddressOnParent + uint64 VirtualResourceBlockSize + string VirtualQuantityUnits + uint16 Access + uint64 HostResourceBlockSize + uint64 Reservation + uint64 HostExtentStartingAddress + string HostExtentName + uint16 HostExtentNameFormat + string OtherHostExtentNameFormat + uint16 HostExtentNameNamespace + string OtherHostExtentNameNamespace + uint64 IOPSLimit + uint64 IOPSReservation + string IOPSAllocationUnits + boolean PersistentReservationsSupported +end -- 2.27.0

On Nov 18, 2020, at 1:48 PM, Matt Coleman <mcoleman@datto.com> wrote:
+ if (!(disk = virDomainDiskDefNew(priv->xmlopt))) {
+ if (!(disk = virDomainDiskDefNew(priv->xmlopt))) {
I forgot to include changes to hyperv_private.h in this commit. Please squash the following additional change with this patch... Signed-off-by: Matt Coleman <matt@datto.com> diff --git a/src/hyperv/hyperv_private.h b/src/hyperv/hyperv_private.h index f400f58c3a..7a2a1d59ee 100644 --- a/src/hyperv/hyperv_private.h +++ b/src/hyperv/hyperv_private.h @@ -28,10 +28,12 @@ #include "virerror.h" #include "hyperv_util.h" #include "capabilities.h" +#include "domain_conf.h" typedef struct _hypervPrivate hypervPrivate; struct _hypervPrivate { hypervParsedUri *parsedUri; WsManClient *client; virCapsPtr caps; + virDomainXMLOptionPtr xmlopt; };

On Wed, Nov 18, 2020 at 01:48:24PM -0500, Matt Coleman wrote:
dumpxml can now serialize: * floppy drives * file-backed and device-backed disk drives * images mounted to virtual CD/DVD drives * IDE and SCSI controllers
Co-authored-by: Sri Ramanujam <sramanujam@datto.com> Signed-off-by: Matt Coleman <matt@datto.com> --- src/hyperv/hyperv_driver.c | 408 +++++++++++++++++++++++++- src/hyperv/hyperv_driver.h | 3 + src/hyperv/hyperv_wmi.c | 45 +++ src/hyperv/hyperv_wmi.h | 8 + src/hyperv/hyperv_wmi_classes.h | 19 ++ src/hyperv/hyperv_wmi_generator.input | 134 +++++++++ 6 files changed, 616 insertions(+), 1 deletion(-)
diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c index 40739595ac..ce9ba2a02a 100644 --- a/src/hyperv/hyperv_driver.c +++ b/src/hyperv/hyperv_driver.c @@ -293,6 +293,385 @@ hypervCapsInit(hypervPrivate *priv) return NULL; }
+/* + * Virtual device functions + */ +static int +hypervGetDeviceParentRasdFromDeviceId(const char *parentDeviceId, + Msvm_ResourceAllocationSettingData *list, + Msvm_ResourceAllocationSettingData **out) +{ + Msvm_ResourceAllocationSettingData *entry = list; + g_autofree char *escapedDeviceId = NULL; + *out = NULL; + + while (entry) { + escapedDeviceId = g_strdup_printf("%s\"", entry->data->InstanceID); + escapedDeviceId = virStringReplace(escapedDeviceId, "\\", "\\\\");
This leaks the original escapedDeviceId pointer. The autofree only runs when variables go out of scope, not when you overwrite pointers. It'll also leak on every loop iteration.
+ + if (g_str_has_suffix(parentDeviceId, escapedDeviceId)) { + *out = entry; + break; + } + + entry = entry->next; + } + + if (*out) + return 0; + + return -1; +}
diff --git a/src/hyperv/hyperv_driver.h b/src/hyperv/hyperv_driver.h index 8099b5714b..e49550a2c8 100644 --- a/src/hyperv/hyperv_driver.h +++ b/src/hyperv/hyperv_driver.h @@ -22,4 +22,7 @@
#pragma once
+#define HYPERV_MAX_SCSI_CONTROLLERS 4 +#define HYPERV_MAX_IDE_CONTROLLERS 2
Is it really possible to have 2 IDE controllers ? Most cases there is a single IDE controller, with two channels, and each channel has two devices, giving a total of 4 disks. I'm just wondering if the code is mistakenly creating separate libvirt controllers for the 2 IDE channels Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 11/19/20 8:38 AM, Daniel P. Berrangé wrote:
On Wed, Nov 18, 2020 at 01:48:24PM -0500, Matt Coleman wrote:
dumpxml can now serialize: * floppy drives * file-backed and device-backed disk drives * images mounted to virtual CD/DVD drives * IDE and SCSI controllers
Co-authored-by: Sri Ramanujam <sramanujam@datto.com> Signed-off-by: Matt Coleman <matt@datto.com> --- src/hyperv/hyperv_driver.c | 408 +++++++++++++++++++++++++- src/hyperv/hyperv_driver.h | 3 + src/hyperv/hyperv_wmi.c | 45 +++ src/hyperv/hyperv_wmi.h | 8 + src/hyperv/hyperv_wmi_classes.h | 19 ++ src/hyperv/hyperv_wmi_generator.input | 134 +++++++++ 6 files changed, 616 insertions(+), 1 deletion(-)
diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c index 40739595ac..ce9ba2a02a 100644 --- a/src/hyperv/hyperv_driver.c +++ b/src/hyperv/hyperv_driver.c @@ -293,6 +293,385 @@ hypervCapsInit(hypervPrivate *priv) return NULL; }
+/* + * Virtual device functions + */ +static int +hypervGetDeviceParentRasdFromDeviceId(const char *parentDeviceId, + Msvm_ResourceAllocationSettingData *list, + Msvm_ResourceAllocationSettingData **out) +{ + Msvm_ResourceAllocationSettingData *entry = list; + g_autofree char *escapedDeviceId = NULL; + *out = NULL; + + while (entry) { + escapedDeviceId = g_strdup_printf("%s\"", entry->data->InstanceID); + escapedDeviceId = virStringReplace(escapedDeviceId, "\\", "\\\\"); This leaks the original escapedDeviceId pointer. The autofree only runs when variables go out of scope, not when you overwrite pointers.
It'll also leak on every loop iteration.
+ + if (g_str_has_suffix(parentDeviceId, escapedDeviceId)) { + *out = entry; + break; + } + + entry = entry->next; + } + + if (*out) + return 0; + + return -1; +}
diff --git a/src/hyperv/hyperv_driver.h b/src/hyperv/hyperv_driver.h index 8099b5714b..e49550a2c8 100644 --- a/src/hyperv/hyperv_driver.h +++ b/src/hyperv/hyperv_driver.h @@ -22,4 +22,7 @@
#pragma once
+#define HYPERV_MAX_SCSI_CONTROLLERS 4 +#define HYPERV_MAX_IDE_CONTROLLERS 2 Is it really possible to have 2 IDE controllers ?
I know nothing about hyperv, but qemu does allow multiple IDE controllers, and I even once wrote patches to support it, but then we (or maybe it was just "me"?) decided that allowing lots of IDE disks would just be encouraging people to use the oldest, slowest type of disk in large configurations that would most benefit from going to the trouble of getting the proper virtio drivers installed in the guest. So in the end, the qemu driver in libvirt only supports a single IDE controller, and only if it is built into the basic machinetype. That won't prevent any other driver from supporting multiple IDE controllers, but I would think twice about doing it.
Most cases there is a single IDE controller, with two channels, and each channel has two devices, giving a total of 4 disks.
I'm just wondering if the code is mistakenly creating separate libvirt controllers for the 2 IDE channels

On Nov 19, 2020, at 12:01 PM, Laine Stump <laine@redhat.com> wrote:
On 11/19/20 8:38 AM, Daniel P. Berrangé wrote:
On Wed, Nov 18, 2020 at 01:48:24PM -0500, Matt Coleman wrote:
+#define HYPERV_MAX_IDE_CONTROLLERS 2 Is it really possible to have 2 IDE controllers ?
I know nothing about hyperv, but qemu does allow multiple IDE controllers, and I even once wrote patches to support it, but then we (or maybe it was just "me"?) decided that allowing lots of IDE disks would just be encouraging people to use the oldest, slowest type of disk in large configurations that would most benefit from going to the trouble of getting the proper virtio drivers installed in the guest.
So in the end, the qemu driver in libvirt only supports a single IDE controller, and only if it is built into the basic machinetype. That won't prevent any other driver from supporting multiple IDE controllers, but I would think twice about doing it.
Hyper-V's UI presents it as two separate controllers that support two devices each, but I checked from within the guest and the drives are all attached to a single PIIX4 controller. I'll rework this commit. Thanks for the feedback, Laine and Daniel! -- Matt

On Nov 19, 2020, at 8:38 AM, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Wed, Nov 18, 2020 at 01:48:24PM -0500, Matt Coleman wrote:
+ escapedDeviceId = g_strdup_printf("%s\"", entry->data->InstanceID); + escapedDeviceId = virStringReplace(escapedDeviceId, "\\", "\\\\");
This leaks the original escapedDeviceId pointer. The autofree only runs when variables go out of scope, not when you overwrite pointers.
It'll also leak on every loop iteration.
Thanks! This will be fixed in V2. -- Matt

Signed-off-by: Matt Coleman <matt@datto.com> --- docs/schemas/basictypes.rng | 2 +- docs/schemas/domaincommon.rng | 5 +++- .../disk-hyperv-physical.xml | 17 ++++++++++++++ .../disk-hyperv-virtual.xml | 17 ++++++++++++++ .../disk-hyperv-physical.xml | 23 +++++++++++++++++++ .../disk-hyperv-virtual.xml | 23 +++++++++++++++++++ tests/genericxml2xmltest.c | 2 ++ 7 files changed, 87 insertions(+), 2 deletions(-) create mode 100644 tests/genericxml2xmlindata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmlindata/disk-hyperv-virtual.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-virtual.xml diff --git a/docs/schemas/basictypes.rng b/docs/schemas/basictypes.rng index ea18b2d2fb..fc52799466 100644 --- a/docs/schemas/basictypes.rng +++ b/docs/schemas/basictypes.rng @@ -299,7 +299,7 @@ <define name="absFilePath"> <data type="string"> - <param name="pattern">/.+</param> + <param name="pattern">(/|[a-zA-Z]:\\).+</param> </data> </define> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index f86a854863..da9d1da187 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -1691,7 +1691,10 @@ <interleave> <optional> <attribute name="dev"> - <ref name="absFilePath"/> + <choice> + <ref name="absFilePath"/> + <ref name="deviceName"/> + </choice> </attribute> </optional> <ref name="diskSourceCommon"/> diff --git a/tests/genericxml2xmlindata/disk-hyperv-physical.xml b/tests/genericxml2xmlindata/disk-hyperv-physical.xml new file mode 100644 index 0000000000..a7f34c18f4 --- /dev/null +++ b/tests/genericxml2xmlindata/disk-hyperv-physical.xml @@ -0,0 +1,17 @@ +<domain type='hyperv'> + <name>test_disk-hyperv</name> + <uuid>deadbeef-dead-beef-dead-beefdeadbeef</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>3</vcpu> + <os> + <type>hvm</type> + </os> + <devices> + <disk type='block' device='disk'> + <source dev='2'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='scsi' index='0'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/disk-hyperv-virtual.xml b/tests/genericxml2xmlindata/disk-hyperv-virtual.xml new file mode 100644 index 0000000000..bbc56c9f89 --- /dev/null +++ b/tests/genericxml2xmlindata/disk-hyperv-virtual.xml @@ -0,0 +1,17 @@ +<domain type='hyperv'> + <name>test_disk-hyperv</name> + <uuid>deadbeef-dead-beef-dead-beefdeadbeef</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>3</vcpu> + <os> + <type>hvm</type> + </os> + <devices> + <disk type='file' device='disk'> + <source file='X:\test_disk-hyperv.vhdx'/> + <target dev='sda' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='scsi' index='0'/> + </devices> +</domain> diff --git a/tests/genericxml2xmloutdata/disk-hyperv-physical.xml b/tests/genericxml2xmloutdata/disk-hyperv-physical.xml new file mode 100644 index 0000000000..112a5081cd --- /dev/null +++ b/tests/genericxml2xmloutdata/disk-hyperv-physical.xml @@ -0,0 +1,23 @@ +<domain type='hyperv'> + <name>test_disk-hyperv</name> + <uuid>deadbeef-dead-beef-dead-beefdeadbeef</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>3</vcpu> + <os> + <type arch='x86_64'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <disk type='block' device='disk'> + <source dev='2'/> + <target dev='sdb' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='scsi' index='0'/> + </devices> +</domain> diff --git a/tests/genericxml2xmloutdata/disk-hyperv-virtual.xml b/tests/genericxml2xmloutdata/disk-hyperv-virtual.xml new file mode 100644 index 0000000000..dadd3318ab --- /dev/null +++ b/tests/genericxml2xmloutdata/disk-hyperv-virtual.xml @@ -0,0 +1,23 @@ +<domain type='hyperv'> + <name>test_disk-hyperv</name> + <uuid>deadbeef-dead-beef-dead-beefdeadbeef</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>3</vcpu> + <os> + <type arch='x86_64'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <disk type='file' device='disk'> + <source file='X:\test_disk-hyperv.vhdx'/> + <target dev='sda' bus='scsi'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='scsi' index='0'/> + </devices> +</domain> diff --git a/tests/genericxml2xmltest.c b/tests/genericxml2xmltest.c index 5110bfba86..5e8f58b4a2 100644 --- a/tests/genericxml2xmltest.c +++ b/tests/genericxml2xmltest.c @@ -154,6 +154,8 @@ mymain(void) DO_TEST_FULL(name, 1, false, TEST_COMPARE_DOM_XML2XML_RESULT_SUCCESS) DO_TEST_DIFFERENT("disk-virtio"); + DO_TEST_DIFFERENT("disk-hyperv-physical"); + DO_TEST_DIFFERENT("disk-hyperv-virtual"); DO_TEST_DIFFERENT("graphics-vnc-minimal"); DO_TEST_DIFFERENT("graphics-vnc-manual-port"); -- 2.27.0

On Wed, Nov 18, 2020 at 01:48:25PM -0500, Matt Coleman wrote:
Signed-off-by: Matt Coleman <matt@datto.com> --- docs/schemas/basictypes.rng | 2 +- docs/schemas/domaincommon.rng | 5 +++- .../disk-hyperv-physical.xml | 17 ++++++++++++++ .../disk-hyperv-virtual.xml | 17 ++++++++++++++ .../disk-hyperv-physical.xml | 23 +++++++++++++++++++ .../disk-hyperv-virtual.xml | 23 +++++++++++++++++++ tests/genericxml2xmltest.c | 2 ++ 7 files changed, 87 insertions(+), 2 deletions(-) create mode 100644 tests/genericxml2xmlindata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmlindata/disk-hyperv-virtual.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-virtual.xml
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Wed, Nov 18, 2020 at 01:48:23PM -0500, Matt Coleman wrote:
Here's a GitLab MR if you'd prefer to review it there: https://gitlab.com/iammattcoleman/libvirt/-/merge_requests/12/commits
Matt Coleman (2): hyperv: XML parsing of storage volumes schema: add support for Windows file paths and device names
Side-point.... Looking at these patches makes me quite aware of the lack of unit test coverage. The native <-> XML conversions are one of the places we find most benefit from unit testing, as it catches many regrssions we would otherwise introduce. Traditionally we would have 1 text file containing the hypervisor native representation, and one containing the XML representation. The unit tests would load the former, generate the XML and then compare to the expected XML. This is potentially complicated in the hyperv driver as IIUC there's no real config file containing the native hyperv repr. Is there any convenient way we can serialize the Msvm_ComputerSystem Msvm_VirtualSystemSettingData Msvm_ProcessorSettingData Msvm_MemorySettingData classes to a text file format, suc that we can later load them from a unit test ? Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Nov 23, 2020, at 10:13 AM, Daniel P. Berrangé <berrange@redhat.com> wrote:
Side-point....
Looking at these patches makes me quite aware of the lack of unit test coverage. The native <-> XML conversions are one of the places we find most benefit from unit testing, as it catches many regrssions we would otherwise introduce.
Traditionally we would have 1 text file containing the hypervisor native representation, and one containing the XML representation. The unit tests would load the former, generate the XML and then compare to the expected XML.
This is potentially complicated in the hyperv driver as IIUC there's no real config file containing the native hyperv repr.
Is there any convenient way we can serialize the
Msvm_ComputerSystem Msvm_VirtualSystemSettingData Msvm_ProcessorSettingData Msvm_MemorySettingData
classes to a text file format, suc that we can later load them from a unit test ?
As you suggested in the accidental off-the-mailing-list tangent that I created, I'll look into using openwsman response data for the tests, since that's effectively the "hypervisor native" format for remote access to a Hyper-V server. Project deadlines require me to keep working on implementing new functionality, so I'll have to consider how to approach unit tests at a later point. Once the base functionality (defining/undefining VMs, device attachment, basic networking, and screenshots) is complete, I'll be focusing on security. I'll make adding unit tests part of that work. -- Matt
participants (3)
-
Daniel P. Berrangé
-
Laine Stump
-
Matt Coleman