On Mon, Mar 13, 2023 at 02:25:01PM -0600, Jim Fehlig wrote: > If an explicit machine type is not specified in the VM config, the qemu > driver will select the first machine type in the list of machine types for > the specified accelerator. See virQEMUCapsGetPreferredMachine > > https://gitlab.com/libvirt/libvirt/-/blob/master/src/qemu/qemu_capabiliti... > > On my test machines, this works reasonably well for x86_64 where the first > machine type is pc-i440fx-7.1. But for aarch64, the first machine is > integratorcp, which is not very useful with maxCpus=1 and other limitations. What you're describing is the historical behavior, but these days it's a bit more complicated than that: we actually go out of our way to make sure that the default machine for an architecture is the one that QEMU *originally* listed first. See [1] for the implementation as well as the rationale.

> Would it be possible to select a "better" default machine type? E.g. 'pc' > for x86_64, 'virt' for aarch64, etc. I'm happy to work on this if folks deem > it's a reasonable improvement. It would benefit tools like the libvirt > terraform provider, which prefer to use something more generic like > <type>hvm</type>. As much as I would *love* for virt to be the default aarch64 (and riscv64) machine type, unfortunately our strong backwards compatibility guarantees prevent us from changing this :(

On the plus side, at least implementing more reasonable defaults at the application level should be quite straightforward. See [2] for virt-manager's take.

[1] https://gitlab.com/libvirt/libvirt/-/commit/26cfb1a3cd39d731099ee7d5d1c47... [2] https://github.com/virt-manager/virt-manager/blob/33ff193ee9fcfdb74f95d94...

On a Monday in 2023, Jim Fehlig wrote: > If an explicit machine type is not specified in the VM config, the qemu > driver will select the first machine type in the list of machine types > for the specified accelerator. See virQEMUCapsGetPreferredMachine > > https://gitlab.com/libvirt/libvirt/-/blob/master/src/qemu/qemu_capabiliti... > > On my test machines, this works reasonably well for x86_64 where the > first machine type is pc-i440fx-7.1. But for aarch64, the first machine > is integratorcp, which is not very useful with maxCpus=1 and other > limitations. Is it possible to run such machine with libvirt? If not, we could say our default is faulty and pick a new one.

On 3/15/23 08:40, Ján Tomko wrote: > On a Monday in 2023, Jim Fehlig wrote: >> If an explicit machine type is not specified in the VM config, the qemu >> driver will select the first machine type in the list of machine types for >> the specified accelerator. See virQEMUCapsGetPreferredMachine >> >> https://gitlab.com/libvirt/libvirt/-/blob/master/src/qemu/qemu_capabiliti... >> >> On my test machines, this works reasonably well for x86_64 where the first >> machine type is pc-i440fx-7.1. But for aarch64, the first machine is >> integratorcp, which is not very useful with maxCpus=1 and other limitations. > > Is it possible to run such machine with libvirt? I just did a quick check with libvirt 9.1.0 (qemu is a bit older, at 7.1.0): # cat test.xml <domain type='kvm'>   <name>test</name>   <memory unit='KiB'>2097152</memory>   <vcpu placement='static'>1</vcpu>   <os>     <type>hvm</type>     <loader readonly='yes' type='pflash'>/usr/share/qemu/aavmf-aarch64-code.bin</loader>     <nvram template='/usr/share/qemu/aavmf-aarch64-vars.bin'/>     <boot dev='hd'/>   </os>   <clock offset='utc'/>   <on_poweroff>destroy</on_poweroff>   <on_reboot>restart</on_reboot>   <on_crash>destroy</on_crash>   <devices>     <emulator>/usr/bin/qemu-system-aarch64</emulator>     <disk type='file' device='disk'>       <driver name='qemu' type='qcow2' discard='unmap'/>       <source file='/var/lib/libvirt/images/test.qcow2'/>       <target dev='vda' bus='virtio'/>     </disk>   </devices> </domain>

# virsh create test.xml error: Failed to create domain from test.xml error: internal error: Unexpected enum value 0 for virDomainDeviceAddressType

On 3/16/23 11:56, Jim Fehlig wrote: > On 3/15/23 08:40, Ján Tomko wrote: > > On a Monday in 2023, Jim Fehlig wrote: > > > If an explicit machine type is not specified in the VM config, > > > the qemu driver will select the first machine type in the list > > > of machine types for the specified accelerator. See > > > virQEMUCapsGetPreferredMachine > > > > > > https://gitlab.com/libvirt/libvirt/-/blob/master/src/qemu/qemu_capabiliti... > > > > > > On my test machines, this works reasonably well for x86_64 where > > > the first machine type is pc-i440fx-7.1. But for aarch64, the > > > first machine is integratorcp, which is not very useful with > > > maxCpus=1 and other limitations. > > > > Is it possible to run such machine with libvirt? > > I just did a quick check with libvirt 9.1.0 (qemu is a bit older, at 7.1.0): > > # cat test.xml > <domain type='kvm'> >   <name>test</name> >   <memory unit='KiB'>2097152</memory> >   <vcpu placement='static'>1</vcpu> >   <os> >     <type>hvm</type> >     <loader readonly='yes' > type='pflash'>/usr/share/qemu/aavmf-aarch64-code.bin</loader> >     <nvram template='/usr/share/qemu/aavmf-aarch64-vars.bin'/> >     <boot dev='hd'/> >   </os> >   <clock offset='utc'/> >   <on_poweroff>destroy</on_poweroff> >   <on_reboot>restart</on_reboot> >   <on_crash>destroy</on_crash> >   <devices> >     <emulator>/usr/bin/qemu-system-aarch64</emulator> >     <disk type='file' device='disk'> >       <driver name='qemu' type='qcow2' discard='unmap'/> >       <source file='/var/lib/libvirt/images/test.qcow2'/> >       <target dev='vda' bus='virtio'/> >     </disk> >   </devices> > </domain> > # virsh create test.xml > error: Failed to create domain from test.xml > error: internal error: Unexpected enum value 0 for virDomainDeviceAddressType > > I don't _think_ it's a downstream bug, nor fixed in git in the meantime. > It appears running the old integratorcp machine with libivrt is not > possible. I trimmed the config to remove things like virtio devices that are not supported by the default machine type, but still it does not work with libvirt: # cat test.xml <domain type='kvm'> <name>test</name> <memory unit='KiB'>2097152</memory> <vcpu placement='static'>1</vcpu> <os> <type>hvm</type> </os> <devices> <emulator>/usr/bin/qemu-system-aarch64</emulator> </devices> </domain> # virsh create test.xml error: Failed to create domain from test.xml error: internal error: process exited while connecting to monitor: 2023-04-05T20:36:19.564896Z qemu-system-aarch64: Property 'integratorcp-machine.acpi' not found

On Wed, Apr 05, 2023 at 15:19:07 -0600, Jim Fehlig wrote: > # virsh create test.xml > error: Failed to create domain from test.xml > error: internal error: process exited while connecting to monitor: > 2023-04-05T20:36:19.564896Z qemu-system-aarch64: Property > 'integratorcp-machine.acpi' not found This is a known issue which should be already fixed if you use the upcoming qemu version. Historically ACPI was controled via '-no-acpi' which is issued when you don't use the '<acpi/>' feature element in the definition. Also historically it was only used on x86. With development of the 'virt' machine type for arm which does use acpi this was un-carefully extended to aarch64 as well where it didn't work with non-virt machines. Now with the latest qemu which reports which machine type actually support ACPI we avoid use of '-no-acpi' for those which don't support it. Unfortunately the workaround of adding <acpi/> element to definition for a machine which doesn't have it doesn't work either, because a further validation check forces you to configure uefi if you want acpi.

On 3/16/23 11:56, Jim Fehlig wrote: > I just did a quick check with libvirt 9.1.0 (qemu is a bit older, at 7.1.0): > > # cat test.xml > <domain type='kvm'> >   <name>test</name> >   <memory unit='KiB'>2097152</memory> >   <vcpu placement='static'>1</vcpu> >   <os> >     <type>hvm</type> >     <loader readonly='yes' type='pflash'>/usr/share/qemu/aavmf-aarch64-code.bin</loader> >     <nvram template='/usr/share/qemu/aavmf-aarch64-vars.bin'/> >     <boot dev='hd'/> >   </os> >   <clock offset='utc'/> >   <on_poweroff>destroy</on_poweroff> >   <on_reboot>restart</on_reboot> >   <on_crash>destroy</on_crash> >   <devices> >     <emulator>/usr/bin/qemu-system-aarch64</emulator> >     <disk type='file' device='disk'> >       <driver name='qemu' type='qcow2' discard='unmap'/> >       <source file='/var/lib/libvirt/images/test.qcow2'/> >       <target dev='vda' bus='virtio'/> >     </disk> >   </devices> > </domain> > # virsh create test.xml > error: Failed to create domain from test.xml > error: internal error: Unexpected enum value 0 for virDomainDeviceAddressType > > I don't _think_ it's a downstream bug, nor fixed in git in the meantime. > It appears running the old integratorcp machine with libivrt is not > possible. I trimmed the config to remove things like virtio devices that are not supported by the default machine type, but still it does not work with libvirt: # cat test.xml <domain type='kvm'> <name>test</name> <memory unit='KiB'>2097152</memory> <vcpu placement='static'>1</vcpu> <os> <type>hvm</type> </os> <devices> <emulator>/usr/bin/qemu-system-aarch64</emulator> </devices> </domain> # virsh create test.xml error: Failed to create domain from test.xml error: internal error: process exited while connecting to monitor: 2023-04-05T20:36:19.564896Z qemu-system-aarch64: Property 'integratorcp-machine.acpi' not found

For the second, do any board types other than virt support kvm acceleration? It appears to be the only one https://www.qemu.org/docs/master/system/target-arm.html I guess it's a slippery slope. The virt board also requires specifying a cpu model, with the only reasonable values being host and max https://www.qemu.org/docs/master/system/arm/cpu-features.html#a-note-abou... In fact, that doc implies host is the only choice: "but mostly if KVM is enabled the host CPU type must be used". The status quo may by fine for domain type qemu, but it seems there's room for improvement for kvm domains.

On Thu, Apr 06, 2023 at 06:10:11AM -0700, Andrea Bolognani wrote: > In conclusion, there currently doesn't seem to exist a way to define > a useful integratorcp-based VM in libvirt, which IMO means we can > safely change the default machine type for Arm architectures without > any concerns about breaking existing VMs. > > I will look into whether the same can be said for RISC-V > architectures. Hopefully that's the case. Yeah, the spike machine on RISC-V is unusable in basically the same ways that the integratorcp machine is on Arm. Let's just change the default to virt on all of those then :)

On Thu, Apr 13, 2023 at 08:23:25AM -0600, Jim Fehlig wrote: > On 4/6/23 08:27, Andrea Bolognani wrote: > > On Thu, Apr 06, 2023 at 06:10:11AM -0700, Andrea Bolognani wrote: > > > In conclusion, there currently doesn't seem to exist a way to define > > > a useful integratorcp-based VM in libvirt, which IMO means we can > > > safely change the default machine type for Arm architectures without > > > any concerns about breaking existing VMs. > > > > > > I will look into whether the same can be said for RISC-V > > > architectures. Hopefully that's the case. > > > > Yeah, the spike machine on RISC-V is unusable in basically the same > > ways that the integratorcp machine is on Arm. Let's just change the > > default to virt on all of those then :) > > Thanks for the confirmation. I should have time to work on this over the > next days. I haven't looked in detail yet, but I get the feeling there is > more to it than changing a few variables :-). Beyond code, do you have a > mental checklist of the items that will need adjusted? E.g. docs, examples, > tests, etc? I might be excessively optimistic, but I truly believe it could be as simple as changing a couple of lines in the QEMU driver :) I don't think we rely on default machine types anywhere in the test suite, and even if we were only minor touch-ups would probably be necessary.

On Fri, Apr 14, 2023 at 03:33:21PM +0200, Peter Krempa wrote: > On Fri, Apr 14, 2023 at 06:26:02 -0700, Andrea Bolognani wrote: >> I don't think we rely on default machine types anywhere in the test >> suite, and even if we were only minor touch-ups would probably be >> necessary. > > The test suite does use/validate/assign machine types, but I've already > removed the all of the fake-machine type infrastructure for RISC-V so > you only get to use the qemuCapabilities object populated from what we > have in tests/qemucapabilitiesdata/. > > There doesn't seem to be any qemuxml2argv test case referencing 'spike' > so obviously the default machine is not covered by tests. That also > means that the test suite will not break :) Yeah, exactly: for all existing test cases we pick the machine type explicitly, so changing the default shouldn't affect them at all. And the fact that we don't have test cases covering integratorcp or spike is not unexpected, since for both Arm and RISC-V we've only ever really considered the virt machine type.