[PATCH 0/1] Support Windows 10 Arm64 ISO installations
This patch was originally submitted via https://gitlab.com/libvirt/libvirt/-/merge_requests/19 That merge request contains additional information, if you find the commit message of the patch lacking in some way. It did pass CI and the test suite four weeks ago, seems small enough, and the rebase had no conflicts, so I didn't run the tests again on current master. And based on the comment from Peter Krempa in that merge request, it probably needs further discussion with QEMU developers. FWIW: while the installer generally works, I'm experiencing massive timeouts from the "Out of Box Experience" (OOBE) part of the Windows configuration after the install. And the latest ISOs don't even start in QEMU (5.1 and older) for me on Debian, so I used 1909 instead of the 2004 ISO "releases". I'm running the aarch64 system QEMU on x86_64 Debian Buster with my current QEMU build. Jan-Marek Glogowski (1): QEMU: support USB cdrom devices src/qemu/qemu_command.c | 21 ++++++++++++++++++- src/qemu/qemu_command.h | 1 + src/qemu/qemu_domain.c | 2 +- .../disk-cdrom-bus-other.x86_64-latest.args | 12 +++++------ 4 files changed, 28 insertions(+), 8 deletions(-) -- 2.20.1
If a USB cdrom is configured, the media type is silently ignored, when generating the QEMU command, so libvirt actually generates USB disks. The main problem is, that -blockdev mechanism relies on the device type to handle the media type, like ide-cd and ide-hd. But there is just usb-storage. As a result the Windows 10 Arm64 installer won't find the virtio ISO or even its own install media. You can actually turn them into USB sticks, by setting them removable, so they show up in the installer, but then it doesn't expect joliet or UDF filesystems, so can't access these either. So this generates the old driver+device commandline arguments in the case of USB cdrom devices to make the installer happy. Signed-off-by: Jan-Marek Glogowski <glogow@fbihome.de> --- src/qemu/qemu_command.c | 21 ++++++++++++++++++- src/qemu/qemu_command.h | 1 + src/qemu/qemu_domain.c | 2 +- .../disk-cdrom-bus-other.x86_64-latest.args | 12 +++++------ 4 files changed, 28 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index bd98b0a97c..f98af612de 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1121,6 +1121,22 @@ qemuDiskBusIsSD(int bus) } +/** + * qemuDiskIsUSBCD + * @disk: the disk + * + * Returns true, if the disk is an USB cdrom, which can't be currently + * represented by using -blockdev entries (other frontends have extra + * '-hd' and '-cd' devices to distinguish the media). + */ +bool +qemuDiskIsUSBCD(virDomainDiskDefPtr disk) +{ + return ((virDomainDiskBus)disk->bus) == VIR_DOMAIN_DISK_BUS_USB + && disk->device == VIR_DOMAIN_DISK_DEVICE_CDROM; +} + + /** * qemuDiskSourceNeedsProps: * @src: disk source @@ -1425,6 +1441,9 @@ qemuBuildDriveStr(virDomainDiskDefPtr disk, if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_STORAGE_WERROR)) qemuBuildDiskFrontendAttributeErrorPolicy(disk, &opt); + if (qemuDiskIsUSBCD(disk)) + virBufferAddLit(&opt, ",media=cdrom"); + if (disk->src->readonly) virBufferAddLit(&opt, ",readonly=on"); @@ -2075,7 +2094,7 @@ qemuBuildDiskSourceCommandLine(virCommandPtr cmd, size_t i; if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKDEV) && - !qemuDiskBusIsSD(disk->bus)) { + !qemuDiskBusIsSD(disk->bus) && !qemuDiskIsUSBCD(disk)) { if (virStorageSourceIsEmpty(disk->src)) return 0; diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index 89d99b111f..54093b388e 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -110,6 +110,7 @@ char *qemuBuildNicDevStr(virDomainDefPtr def, char *qemuDeviceDriveHostAlias(virDomainDiskDefPtr disk); bool qemuDiskBusIsSD(int bus); +bool qemuDiskIsUSBCD(virDomainDiskDefPtr disk); qemuBlockStorageSourceAttachDataPtr qemuBuildStorageSourceAttachPrepareDrive(virDomainDiskDefPtr disk, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index b1884b6c84..27d8bd6cc2 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -7282,7 +7282,7 @@ qemuDomainDiskGetBackendAlias(virDomainDiskDefPtr disk, *backendAlias = NULL; if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKDEV) || - qemuDiskBusIsSD(disk->bus)) { + qemuDiskBusIsSD(disk->bus) || qemuDiskIsUSBCD(disk)) { if (!(*backendAlias = qemuAliasDiskDriveFromDisk(disk))) return -1; diff --git a/tests/qemuxml2argvdata/disk-cdrom-bus-other.x86_64-latest.args b/tests/qemuxml2argvdata/disk-cdrom-bus-other.x86_64-latest.args index be091f150f..64e6fe1f42 100644 --- a/tests/qemuxml2argvdata/disk-cdrom-bus-other.x86_64-latest.args +++ b/tests/qemuxml2argvdata/disk-cdrom-bus-other.x86_64-latest.args @@ -28,13 +28,13 @@ file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \ -no-acpi \ -boot strict=on \ -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \ --blockdev '{"driver":"file","filename":"/root/boot.iso",\ -"node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ --blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw",\ -"file":"libvirt-2-storage"}' \ --device usb-storage,bus=usb.0,port=1,drive=libvirt-2-format,id=usb-disk0,\ +-drive file=/root/boot.iso,format=raw,if=none,id=drive-usb-disk0,media=cdrom,\ +readonly=on \ +-device usb-storage,bus=usb.0,port=1,drive=drive-usb-disk0,id=usb-disk0,\ +removable=off \ +-drive if=none,id=drive-usb-disk1,media=cdrom,readonly=on \ +-device usb-storage,bus=usb.0,port=2,drive=drive-usb-disk1,id=usb-disk1,\ removable=off \ --device usb-storage,bus=usb.0,port=2,id=usb-disk1,removable=off \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,\ resourcecontrol=deny \ -msg timestamp=on -- 2.20.1
USB disks. The main problem is, that -blockdev mechanism relies on the device type to handle the media type, like ide-cd and ide-hd. But there is just usb-storage.
You can use -device usb-bot + -device scsi-cd ... take care, Gerd
Am 07.09.20 um 08:04 schrieb Gerd Hoffmann:
USB disks. The main problem is, that -blockdev mechanism relies on the device type to handle the media type, like ide-cd and ide-hd. But there is just usb-storage.
You can use -device usb-bot + -device scsi-cd ... So I tried
-blockdev '{"driver":"file","filename":"/tmp/win10_arm64_wim_18363.959_en-us.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/virtio-win-0.1.189.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ with -device nec-usb-xhci,id=xhci \ -device usb-bot,id=scsi0,bus=xhci.0 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-1-format,id=scsi0-0-0-0,bootindex=1 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=libvirt-1-format,id=scsi0-0-0-1 \ which starts, but gives me a "Synchronous Exception" from the TianoCore firmware, which I often saw with PCI enumeration problems. Also "info usb" just shows a single device. I just copied these lines from a normal libvirt SCSI setup with two cdroms. Switching to -device nec-usb-xhci,id=xhci \ -device usb-bot,id=scsi0,bus=xhci.0 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-1-format,id=scsi0-0-0-0,bootindex=1 \ -device usb-bot,id=scsi1,bus=xhci.0 \ -device scsi-cd,bus=scsi1.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi1-0-0-0,drive=libvirt-2-format,id=scsi1-0-0-0 \ boots here. Don't know if this is a error in my edk2 2020.05-2 Debian package or my QEMU 5.1 build. But compared to my small patch with the usb-storage based USB CDROM, this looks like a much larger change and a lot of work implement. Current configuration is: <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/virtio-win-0.1.189.iso'/> <target dev='sdb' bus='usb' removable='off'/> <readonly/> <address type='usb' bus='0' port='2'/> </disk> <controller type='usb' index='0' model='nec-xhci'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> Now you would have to insert an usb-bot SCSI controller. I guess it could be done by either 1. doing this transparently with a controller per device, just like the usb-storage controller does internally (ok - it's just a scsi-cd there), even if the usb-bot supports multiple LUNs. Guess multiple LUNs would act like an USB hub? or 2. invalidating the cdrom with USB addresses configurations and exposing the controller in the XML. This seems easier from the libvirt code POV, like: <controller type='scsi' model='usb-bot'> <address type='usb' port='1'/> </controller> But that isn't easy or obvious to setup. So I still would like to see my much simpler solution merged. Maybe I'm missing something, and the implementation of this special case is easier then my "quick" look at the code suggested, but for me it looks like at least a week of work, notwithstanding the other problems with the installer timeouts in QEMU (where I just saw that https://bugs.launchpad.net/qemu/+bug/1717708 is now closed as invaid...). Jan-Marek
Hi,
-device nec-usb-xhci,id=xhci \ -device usb-bot,id=scsi0,bus=xhci.0 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-1-format,id=scsi0-0-0-0,bootindex=1 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=libvirt-1-format,id=scsi0-0-0-1 \
which starts, but gives me a "Synchronous Exception" from the TianoCore firmware, which I often saw with PCI enumeration problems.
Hmm, firmware bug? Does seabios work?
Also "info usb" just shows a single device. I just copied these lines from a normal libvirt SCSI setup with two cdroms.
That is normal, it actually is only one usb device after all ;)
But compared to my small patch with the usb-storage based USB CDROM, this looks like a much larger change and a lot of work implement.
Yes. As far I know libvirt wants move away from the old syntax and shift to use -blockdev more, not the other way around.
1. doing this transparently with a controller per device, just like the usb-storage controller does internally (ok - it's just a scsi-cd there), even if the usb-bot supports multiple LUNs. Guess multiple LUNs would act like an USB hub?
usb-storage simply doesn't support multiple LUNs, so when going that route you can completely ignore the multiple LUN case.
2. invalidating the cdrom with USB addresses configurations and exposing the controller in the XML. This seems easier from the libvirt code POV, like:
<controller type='scsi' model='usb-bot'> <address type='usb' port='1'/> </controller>
Yep, that is the other obvious approach.
So I still would like to see my much simpler solution merged.
See above, but I'm not a libvirt maintainer so that's not for me to judge. I'm just pointing out that this can be fixed without switching back to the old -drive syntax. take care, Gerd
On Mon, Sep 07, 2020 at 01:45:09PM +0200, Gerd Hoffmann wrote:
Hi,
-device nec-usb-xhci,id=xhci \ -device usb-bot,id=scsi0,bus=xhci.0 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-1-format,id=scsi0-0-0-0,bootindex=1 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=libvirt-1-format,id=scsi0-0-0-1 \
which starts, but gives me a "Synchronous Exception" from the TianoCore firmware, which I often saw with PCI enumeration problems.
Hmm, firmware bug? Does seabios work?
Also "info usb" just shows a single device. I just copied these lines from a normal libvirt SCSI setup with two cdroms.
That is normal, it actually is only one usb device after all ;)
But compared to my small patch with the usb-storage based USB CDROM, this looks like a much larger change and a lot of work implement.
Yes.
As far I know libvirt wants move away from the old syntax and shift to use -blockdev more, not the other way around.
1. doing this transparently with a controller per device, just like the usb-storage controller does internally (ok - it's just a scsi-cd there), even if the usb-bot supports multiple LUNs. Guess multiple LUNs would act like an USB hub?
usb-storage simply doesn't support multiple LUNs, so when going that route you can completely ignore the multiple LUN case.
2. invalidating the cdrom with USB addresses configurations and exposing the controller in the XML. This seems easier from the libvirt code POV, like:
<controller type='scsi' model='usb-bot'> <address type='usb' port='1'/> </controller>
Yep, that is the other obvious approach.
So I still would like to see my much simpler solution merged.
See above, but I'm not a libvirt maintainer so that's not for me to judge. I'm just pointing out that this can be fixed without switching back to the old -drive syntax.
Switching back to -drive is out of the question. We've worked very hard to eliminate its usage and get to an exclusively -blockdev based solution because supporting both approaches in parallel complicates too much. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Sep 07, 2020 at 12:48:21 +0100, Daniel Berrange wrote:
On Mon, Sep 07, 2020 at 01:45:09PM +0200, Gerd Hoffmann wrote:
Hi,
[...]
1. doing this transparently with a controller per device, just like the usb-storage controller does internally (ok - it's just a scsi-cd there), even if the usb-bot supports multiple LUNs. Guess multiple LUNs would act like an USB hub?
usb-storage simply doesn't support multiple LUNs, so when going that route you can completely ignore the multiple LUN case.
2. invalidating the cdrom with USB addresses configurations and exposing the controller in the XML. This seems easier from the libvirt code POV, like:
<controller type='scsi' model='usb-bot'> <address type='usb' port='1'/> </controller>
Yep, that is the other obvious approach.
So I still would like to see my much simpler solution merged.
See above, but I'm not a libvirt maintainer so that's not for me to judge. I'm just pointing out that this can be fixed without switching back to the old -drive syntax.
Switching back to -drive is out of the question. We've worked very hard to eliminate its usage and get to an exclusively -blockdev based solution because supporting both approaches in parallel complicates too much.
Yes, indeed. We theoretically can emulate a complex device consisting of a USB-bot device including the cdrom for this including the hotplug operations in a single step. I will need to investigate this, but I'm backed up on my email as I was on vacation.
Am 07.09.20 um 13:48 schrieb Daniel P. Berrangé:
On Mon, Sep 07, 2020 at 01:45:09PM +0200, Gerd Hoffmann wrote:
...
Switching back to -drive is out of the question. We've worked very hard to eliminate its usage and get to an exclusively -blockdev based solution because supporting both approaches in parallel complicates too much.
The USB CDROM functionality is currently broken and the patch handles just that, like the SD card workaround (qemuDiskBusIsSD). The whole patch is src/qemu/qemu_command.c src/qemu/qemu_command.h src/qemu/qemu_domain.c tests/qemuxml2argvdata/disk-cdrom-bus-other.x86_64-latest.arg 4 files changed, 28 insertions(+), 8 deletions(-) I offered my simple fix and don't have the time to implement the proposed -blockdev based solution, after investigating Gerd's alternative suggestion for the alternate fix. That's it, no harm intended. Jan-Marek P.S. also, because currently the additional (probably QEMU / edk2) bugs won't let me fix my LibreOffice build at this state anyway, so it seems I'm far away from a working environment to fix my real problem.
Hi Am 07.09.20 um 13:45 schrieb Gerd Hoffmann:
-device nec-usb-xhci,id=xhci \ -device usb-bot,id=scsi0,bus=xhci.0 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-1-format,id=scsi0-0-0-0,bootindex=1 \ -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=libvirt-1-format,id=scsi0-0-0-1 \
which starts, but gives me a "Synchronous Exception" from the TianoCore firmware, which I often saw with PCI enumeration problems.
Hmm, firmware bug? Does seabios work?
I don't know, but AFAIK Windows Arm64 requires EFI, so that isn't an option. But I'll check later, just for curiosity. Maybe in the case of usb-bot with multiple devices, you get an USB EP with multiple devices?
Also "info usb" just shows a single device. I just copied these lines from a normal libvirt SCSI setup with two cdroms.
That is normal, it actually is only one usb device after all ;)
I was referring to the posted usb-bot example, where "your" docs (<qemu>/docs/usb-storage.txt) says it supports multiple LUNs, so I was wondering, how these would be exposed on the USB bus.
But compared to my small patch with the usb-storage based USB CDROM, this looks like a much larger change and a lot of work implement.
Yes.
As far I know libvirt wants move away from the old syntax and shift to use -blockdev more, not the other way around.
I'm aware of it. Didn't realize I could use usb-bot instead of usb-storage, when I looked into this back then. The following variants are about a solution using usb-bot together with blockdev. My simple usb-storage already works.
1. doing this transparently with a controller per device, just like the usb-storage controller does internally (ok - it's just a scsi-cd there), even if the usb-bot supports multiple LUNs. Guess multiple LUNs would act like an USB hub?
usb-storage simply doesn't support multiple LUNs, so when going that route you can completely ignore the multiple LUN case.
2. invalidating the cdrom with USB addresses configurations and exposing the controller in the XML. This seems easier from the libvirt code POV, like:
<controller type='scsi' model='usb-bot'> <address type='usb' port='1'/> </controller>
Yep, that is the other obvious approach.
So I still would like to see my much simpler solution merged.
See above, but I'm not a libvirt maintainer so that's not for me to judge. I'm just pointing out that this can be fixed without switching back to the old -drive syntax.
All fine. I've spent my time on this. I just want to fix my LibreOffice Windows Arm64 cross-build and need an install to debug my obviously existing errors, but the current Windows installer ISOs even break somewhere in the initial ramdisk setup in QEMU and the older versions have timeouts, if current uup-based images are even correctly generated on Linux, which I have no way to verify. I don't know if there will be fixes, after the QEMU upstream bug was closed as invalid (https://bugs.launchpad.net/qemu/+bug/1717708), but currently fixing the LibreOffice Windows Arm64 cross build needs more work, just to get it compiled and the MSI generated. So I'll probably test this again later and hope future versions of the QEMU + edk2 will work better :-) This all is just some personal, private side project, and I simply didn't expect this many problems, after reading about Windows Arm64 in QEMU in year old blog posts. Jan-Marek
Hi,
Also "info usb" just shows a single device. I just copied these lines from a normal libvirt SCSI setup with two cdroms.
That is normal, it actually is only one usb device after all ;)
I was referring to the posted usb-bot example, where "your" docs (<qemu>/docs/usb-storage.txt) says it supports multiple LUNs, so I was wondering, how these would be exposed on the USB bus.
You can't see that on the USB bus. usb-bot/usb-storage is a scsi host adapter, with some limitations (only one target, 15 luns max and luns can't be sparse). You just have to set the LUN in the scsi commands to address the device you want.
All fine. I've spent my time on this. I just want to fix my LibreOffice Windows Arm64 cross-build and need an install to debug my obviously existing errors, but the current Windows installer ISOs even break somewhere in the initial ramdisk setup in QEMU and the older versions have timeouts, if current uup-based images are even correctly generated on Linux, which I have no way to verify.
FYI; You can simply copy all the files from the iso to a (virtual) usb stick and use that to boot the windows installer. take care, Gerd
Hi Am 08.09.20 um 08:07 schrieb Gerd Hoffmann:
Also "info usb" just shows a single device. I just copied these lines from a normal libvirt SCSI setup with two cdroms.
That is normal, it actually is only one usb device after all ;)
I was referring to the posted usb-bot example, where "your" docs (<qemu>/docs/usb-storage.txt) says it supports multiple LUNs, so I was wondering, how these would be exposed on the USB bus.
You can't see that on the USB bus. usb-bot/usb-storage is a scsi host adapter, with some limitations (only one target, 15 luns max and luns can't be sparse). You just have to set the LUN in the scsi commands to address the device you want.
All fine. I've spent my time on this. I just want to fix my LibreOffice Windows Arm64 cross-build and need an install to debug my obviously existing errors, but the current Windows installer ISOs even break somewhere in the initial ramdisk setup in QEMU and the older versions have timeouts, if current uup-based images are even correctly generated on Linux, which I have no way to verify.
FYI; You can simply copy all the files from the iso to a (virtual) usb stick and use that to boot the windows installer.
The problem is not using an USB stick or an USB CDROM. The old ISO from 1909 (18363.959) boots for me, but 2004 (19041.450) doesn't, generated with the linux tools from uupdump.ml. For debugging I once modified the BCD to get a menu with longer timeout and that shows up just fine. Something breaks later, when the installer prepares the ramdisk ... would be my guess. And I have no real knowledge how to proceed any further with this. And there are still all the OOBE timeouts / errors (like OOBEKEYBOARD), which I get when the setup process runs at first start, after the installer has finished. Very seldomly it works, most times it doesn't. The only solution the Internet provides, is to try again, until it doesn't happen; that is not practical with my x86_64 notebook hardware. It's too slow and happens almost always for unknown reasons.
From my POV the best seems to be to get some AArch64 HW, like a RasPi 4. That seems to be able to run Windows Arm64 via KVM, from all I can tell reading about it.
After getting a LibreOffice bug report for an Windows Arm64 build, reading about Windows Arm64 in QEMU in years old blog posts, I just thought I would fix the Windows cross build fallout and in the end do a "quick" Windows install, check and fix my LibreOffice test program for LO's UNO bridge and be done with it... then reality happened. ATB and thanks for the comments, Jan-Marek
participants (4)
-
Daniel P. Berrangé -
Gerd Hoffmann -
Jan-Marek Glogowski -
Peter Krempa