4:04 p.m.
Hi,
For some reason didn't notice this was broken until recently. We're
missing an API in the RISC-V CPU driver to allow QEMU TCG risc-v domains
to run in non-native hosts.
Daniel Henrique Barboza (1):
cpu_riscv64.c: add update() implementation
src/cpu/cpu_riscv64.c | 28 +++++++++++++++++++++++++++-
1 file changed, 27 insertions(+), 1 deletion(-)
--
2.40.0
4:04 p.m.
New subject: [PATCH 1/1] cpu_riscv64.c: add update() implementation
At this moment it is not possible to launch a 'riscv64' domain of type
'qemu' (i.e. TCG) and machine 'virt' in a x86 host:
$ sudo ./run tools/virsh start riscv-virt1
error: Failed to start domain 'riscv-virt1'
error: this function is not supported by the connection driver:
cannot update guest CPU for riscv64 architecture
A guest CPU definition will be added if absent by
qemuDomainSetDefaultCPU():
<cpu mode='custom' match='exact' check='none'>
<model fallback='forbid'>rv64</model>
</cpu>
This def will then be updated in qemuProcessUpdateGuestCPU(), which
calls virCPUUpdate(). The error above is being thrown by this function
because the current RISC-V driver does not have an 'update' API
implemented.
There is no particular reason to not support TCG RISC-V guests in x86 or
any other host arch, so let's add an 'update' API to the RISC-V driver.
The code was copied from virCPUarmUpdate() from the ARM driver since
it's a good enough implementation to get us started.
Signed-off-by: Daniel Henrique Barboza <dbarboza(a)ventanamicro.com>
---
src/cpu/cpu_riscv64.c | 28 +++++++++++++++++++++++++++-
1 file changed, 27 insertions(+), 1 deletion(-)
diff --git a/src/cpu/cpu_riscv64.c b/src/cpu/cpu_riscv64.c
index c44bdeb291..9796f8c967 100644
--- a/src/cpu/cpu_riscv64.c
+++ b/src/cpu/cpu_riscv64.c
@@ -46,6 +46,32 @@ virCPURiscv64ValidateFeatures(virCPUDef *cpu G_GNUC_UNUSED)
}
+static int
+virCPUriscvUpdate(virCPUDef *guest,
+ const virCPUDef *host,
+ bool relative)
+{
+ g_autoptr(virCPUDef) updated = virCPUDefCopyWithoutModel(guest);
+
+ if (!relative || guest->mode != VIR_CPU_MODE_HOST_MODEL)
+ return 0;
+
+ if (!host) {
+ virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+ _("unknown host CPU model"));
+ return -1;
+ }
+
+ updated->mode = VIR_CPU_MODE_CUSTOM;
+ virCPUDefCopyModel(updated, host, true);
+
+ virCPUDefStealModel(guest, updated, false);
+ guest->mode = VIR_CPU_MODE_CUSTOM;
+ guest->match = VIR_CPU_MATCH_EXACT;
+
+ return 0;
+}
+
struct cpuArchDriver cpuDriverRiscv64 = {
.name = "riscv64",
.arch = archs,
@@ -54,6 +80,6 @@ struct cpuArchDriver cpuDriverRiscv64 = {
.decode = NULL,
.encode = NULL,
.baseline = NULL,
- .update = NULL,
+ .update = virCPUriscvUpdate,
.validateFeatures = virCPURiscv64ValidateFeatures,
};
--
2.40.0
10:40 a.m.
New subject: [PATCH 1/1] cpu_riscv64.c: add update() implementation
On Thu, Apr 27, 2023 at 06:04:10PM -0300, Daniel Henrique Barboza wrote:
At this moment it is not possible to launch a 'riscv64'
domain of type
'qemu' (i.e. TCG) and machine 'virt' in a x86 host:
$ sudo ./run tools/virsh start riscv-virt1
error: Failed to start domain 'riscv-virt1'
error: this function is not supported by the connection driver:
cannot update guest CPU for riscv64 architecture
I get a different error message, specifically
unsupported configuration: CPU mode 'custom' for riscv64 qemu
domain on x86_64 host is not supported by hypervisor
A guest CPU definition will be added if absent by
qemuDomainSetDefaultCPU():
No such function exists, did you mean qemuDomainDefSetDefaultCPU()?
<cpu mode='custom' match='exact'
check='none'>
<model fallback='forbid'>rv64</model>
</cpu>
Note that I only started seeing the error message mentioned above
after I manually added this snippet to my domain XML.
If I don't do that, the <cpu> element doesn't show up at all and the
VM can be started just fine.
struct cpuArchDriver cpuDriverRiscv64 = {
.name = "riscv64",
.arch = archs,
@@ -54,6 +80,6 @@ struct cpuArchDriver cpuDriverRiscv64 = {
.decode = NULL,
.encode = NULL,
.baseline = NULL,
- .update = NULL,
+ .update = virCPUriscvUpdate,
.validateFeatures = virCPURiscv64ValidateFeatures,
You should follow the naming convention already established for the
file.
I have not actually looked at the code yet. In theory the changes
sound fine, but the current situation as you describe it differs
enough from what I'm seeing here that I want to make sure we're on
the same page before spending any time on that. Are you working off
master? Do you have additional patches applied locally?
The full XML for the VM that I'm using for testing is below.
----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< -----
<domain type='qemu'>
<name>ubuntu-riscv</name>
<uuid>586faf27-2bf8-49cb-9601-9a62fe3de28d</uuid>
<memory unit='KiB'>4194304</memory>
<currentMemory unit='KiB'>4194304</currentMemory>
<vcpu placement='static'>8</vcpu>
<os>
<type arch='riscv64' machine='virt'>hvm</type>
<kernel>/var/lib/libvirt/images/riscv.u-boot.bin</kernel>
<boot dev='hd'/>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-riscv64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/ubuntu-riscv.qcow2'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x04'
slot='0x00'
function='0x0'/>
</disk>
<controller type='usb' index='0' model='qemu-xhci'>
<address type='pci' domain='0x0000' bus='0x02'
slot='0x00'
function='0x0'/>
</controller>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x8'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='2'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x9'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x1'/>
</controller>
<controller type='pci' index='3'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0xa'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x2'/>
</controller>
<controller type='pci' index='4'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0xb'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x3'/>
</controller>
<controller type='pci' index='5'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0xc'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x4'/>
</controller>
<controller type='pci' index='6'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0xd'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x5'/>
</controller>
<controller type='pci' index='7'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='7' port='0xe'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x6'/>
</controller>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x03'
slot='0x00'
function='0x0'/>
</controller>
<interface type='network'>
<mac address='52:54:00:30:b4:bd'/>
<source network='default'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x01'
slot='0x00'
function='0x0'/>
</interface>
<serial type='pty'>
<target type='system-serial' port='0'>
<model name='16550a'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='unix'>
<target type='virtio' name='org.qemu.guest_agent.0'/>
<address type='virtio-serial' controller='0' bus='0'
port='1'/>
</channel>
<audio id='1' type='none'/>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x05'
slot='0x00'
function='0x0'/>
</memballoon>
<rng model='virtio'>
<backend model='random'>/dev/urandom</backend>
<address type='pci' domain='0x0000' bus='0x06'
slot='0x00'
function='0x0'/>
</rng>
</devices>
</domain>
--
Andrea Bolognani / Red Hat / Virtualization
12:15 p.m.
New subject: [PATCH 1/1] cpu_riscv64.c: add update() implementation
On 4/28/23 12:40, Andrea Bolognani wrote:
On Thu, Apr 27, 2023 at 06:04:10PM -0300, Daniel Henrique Barboza
wrote:
> At this moment it is not possible to launch a 'riscv64' domain of type
> 'qemu' (i.e. TCG) and machine 'virt' in a x86 host:
>
> $ sudo ./run tools/virsh start riscv-virt1
> error: Failed to start domain 'riscv-virt1'
> error: this function is not supported by the connection driver:
> cannot update guest CPU for riscv64 architecture
I get a different error message, specifically
unsupported configuration: CPU mode 'custom' for riscv64 qemu
domain on x86_64 host is not supported by hypervisor
Ha, I believe the patch is still relevant but I did a serious mess in the commit
message :(
The difference between what we're seeing is that I used a QEMU from
riscv-to-apply.next
from the maintainer tree [1] instead of master. That tree has the following patch:
commit c13dfff2ded9cf2098a07e2426ffcad674602030
Author: Daniel Henrique Barboza <dbarboza(a)ventanamicro.com>
Date: Tue Apr 11 15:35:10 2023 -0300
target/riscv: add query-cpy-definitions support
With this QEMU, two things you're not seeing will start to happen:
- virQEMUCapsGetMachineDefaultCPU() will return the 'rv64' CPU, and
qemuDomainDefSetDefaultCPU() will add the <cpu> element during PostParse()
time;
- the error you're seeing from virQEMUCapsIsCPUModeSupported() won't happen
because this:
cpus = virQEMUCapsGetAccel(qemuCaps, type)->cpuModels;
return cpus && cpus->ncpus > 0;
won't return NULL anymore. You'll error out in the same point I am right now,
in virCPUUpdate(), because there's no driver->update() for the RISC-V driver.
I apologize for the confusion. I'll re-send the patch with a proper description:
we need the update() implementation to support the <cpu> definition. I'll also
make sure to test with QEMU master this time.
Thanks,
Daniel
[1] https://github.com/alistair23/qemu/tree/riscv-to-apply.next
> A guest CPU definition will be added if absent by
> qemuDomainSetDefaultCPU():
No such function exists, did you mean qemuDomainDefSetDefaultCPU()?
> <cpu mode='custom' match='exact' check='none'>
> <model fallback='forbid'>rv64</model>
> </cpu>
Note that I only started seeing the error message mentioned above
after I manually added this snippet to my domain XML.
If I don't do that, the <cpu> element doesn't show up at all and the
VM can be started just fine.
> struct cpuArchDriver cpuDriverRiscv64 = {
> .name = "riscv64",
> .arch = archs,
> @@ -54,6 +80,6 @@ struct cpuArchDriver cpuDriverRiscv64 = {
> .decode = NULL,
> .encode = NULL,
> .baseline = NULL,
> - .update = NULL,
> + .update = virCPUriscvUpdate,
> .validateFeatures = virCPURiscv64ValidateFeatures,
You should follow the naming convention already established for the
file.
I have not actually looked at the code yet. In theory the changes
sound fine, but the current situation as you describe it differs
enough from what I'm seeing here that I want to make sure we're on
the same page before spending any time on that. Are you working off
master? Do you have additional patches applied locally?
The full XML for the VM that I'm using for testing is below.
----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8<
-----
<domain type='qemu'>
<name>ubuntu-riscv</name>
<uuid>586faf27-2bf8-49cb-9601-9a62fe3de28d</uuid>
<memory unit='KiB'>4194304</memory>
<currentMemory unit='KiB'>4194304</currentMemory>
<vcpu placement='static'>8</vcpu>
<os>
<type arch='riscv64' machine='virt'>hvm</type>
<kernel>/var/lib/libvirt/images/riscv.u-boot.bin</kernel>
<boot dev='hd'/>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-riscv64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/ubuntu-riscv.qcow2'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x04'
slot='0x00'
function='0x0'/>
</disk>
<controller type='usb' index='0' model='qemu-xhci'>
<address type='pci' domain='0x0000' bus='0x02'
slot='0x00'
function='0x0'/>
</controller>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x8'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='2'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x9'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x1'/>
</controller>
<controller type='pci' index='3'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0xa'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x2'/>
</controller>
<controller type='pci' index='4'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0xb'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x3'/>
</controller>
<controller type='pci' index='5'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0xc'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x4'/>
</controller>
<controller type='pci' index='6'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0xd'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x5'/>
</controller>
<controller type='pci' index='7'
model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='7' port='0xe'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x6'/>
</controller>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x03'
slot='0x00'
function='0x0'/>
</controller>
<interface type='network'>
<mac address='52:54:00:30:b4:bd'/>
<source network='default'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x01'
slot='0x00'
function='0x0'/>
</interface>
<serial type='pty'>
<target type='system-serial' port='0'>
<model name='16550a'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='unix'>
<target type='virtio' name='org.qemu.guest_agent.0'/>
<address type='virtio-serial' controller='0' bus='0'
port='1'/>
</channel>
<audio id='1' type='none'/>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x05'
slot='0x00'
function='0x0'/>
</memballoon>
<rng model='virtio'>
<backend model='random'>/dev/urandom</backend>
<address type='pci' domain='0x0000' bus='0x06'
slot='0x00'
function='0x0'/>
</rng>
</devices>
</domain>
4:07 a.m.
New subject: [PATCH 1/1] cpu_riscv64.c: add update() implementation
On Fri, Apr 28, 2023 at 02:15:04PM -0300, Daniel Henrique Barboza wrote:
On 4/28/23 12:40, Andrea Bolognani wrote:
> On Thu, Apr 27, 2023 at 06:04:10PM -0300, Daniel Henrique Barboza wrote:
> > At this moment it is not possible to launch a 'riscv64' domain of type
> > 'qemu' (i.e. TCG) and machine 'virt' in a x86 host:
> >
> > $ sudo ./run tools/virsh start riscv-virt1
> > error: Failed to start domain 'riscv-virt1'
> > error: this function is not supported by the connection driver:
> > cannot update guest CPU for riscv64 architecture
>
> I get a different error message, specifically
>
> unsupported configuration: CPU mode 'custom' for riscv64 qemu
> domain on x86_64 host is not supported by hypervisor
Ha, I believe the patch is still relevant but I did a serious mess in the commit
message :(
The difference between what we're seeing is that I used a QEMU from
riscv-to-apply.next
from the maintainer tree [1] instead of master.
I was thinking about libvirt's master branch rather than QEMU's, but
that makes sense. I'll check out v2 after building QEMU from the
appropriate branch.
commit c13dfff2ded9cf2098a07e2426ffcad674602030
Author: Daniel Henrique Barboza <dbarboza(a)ventanamicro.com>
Date: Tue Apr 11 15:35:10 2023 -0300
target/riscv: add query-cpy-definitions support
Since this hasn't made its way into master yet, maybe you can still
get the maintainer to s/cpy/cpu/ for you? :)
--
Andrea Bolognani / Red Hat / Virtualization
4:48 a.m.
New subject: [PATCH 1/1] cpu_riscv64.c: add update() implementation
On 5/3/23 06:07, Andrea Bolognani wrote:
On Fri, Apr 28, 2023 at 02:15:04PM -0300, Daniel Henrique Barboza
wrote:
> On 4/28/23 12:40, Andrea Bolognani wrote:
>> On Thu, Apr 27, 2023 at 06:04:10PM -0300, Daniel Henrique Barboza wrote:
>>> At this moment it is not possible to launch a 'riscv64' domain of
type
>>> 'qemu' (i.e. TCG) and machine 'virt' in a x86 host:
>>>
>>> $ sudo ./run tools/virsh start riscv-virt1
>>> error: Failed to start domain 'riscv-virt1'
>>> error: this function is not supported by the connection driver:
>>> cannot update guest CPU for riscv64 architecture
>>
>> I get a different error message, specifically
>>
>> unsupported configuration: CPU mode 'custom' for riscv64 qemu
>> domain on x86_64 host is not supported by hypervisor
>
> Ha, I believe the patch is still relevant but I did a serious mess in the commit
> message :(
>
> The difference between what we're seeing is that I used a QEMU from
riscv-to-apply.next
> from the maintainer tree [1] instead of master.
I was thinking about libvirt's master branch rather than QEMU's, but
that makes sense. I'll check out v2 after building QEMU from the
appropriate branch.
> commit c13dfff2ded9cf2098a07e2426ffcad674602030
> Author: Daniel Henrique Barboza <dbarboza(a)ventanamicro.com>
> Date: Tue Apr 11 15:35:10 2023 -0300
>
> target/riscv: add query-cpy-definitions support
Since this hasn't made its way into master yet, maybe you can still
get the maintainer to s/cpy/cpu/ for you? :)
Nice catch! I'll let him know. Thanks!
Daniel
572
days inactive
578
days old
5 comments
2 participants
participants (2)
-
Andrea Bolognani -
Daniel Henrique Barboza