[PATCH v4 00/12] Add riscv kvm accel support
This series adds both riscv32 and riscv64 kvm support, and implements migration based on riscv. Because of RISC-V KVM has been merged into the Linux master, so this series are changed from RFC to patch. Several steps to use this: 1. Build emulation $ ./configure --target-list=riscv64-softmmu $ make -j$(nproc) 2. Build kernel 3. Build QEMU VM Cross built in riscv toolchain. $ PKG_CONFIG_LIBDIR=<toolchain pkgconfig path> $ export PKG_CONFIG_SYSROOT_DIR=<toolchain sysroot path> $ ./configure --target-list=riscv64-softmmu --enable-kvm \ --cross-prefix=riscv64-linux-gnu- --disable-libiscsi --disable-glusterfs \ --disable-libusb --disable-usb-redir --audio-drv-list= --disable-opengl \ --disable-libxml2 $ make -j$(nproc) 4. Start emulation $ ./qemu-system-riscv64 -M virt -m 4096M -cpu rv64,x-h=true -nographic \ -name guest=riscv-hyp,debug-threads=on \ -smp 4 \ -bios ./fw_jump.bin \ -kernel ./Image \ -drive file=./hyp.img,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -append "root=/dev/vda rw console=ttyS0 earlycon=sbi" 5. Start kvm-acceled QEMU VM in emulation $ ./qemu-system-riscv64 -M virt,accel=kvm -m 1024M -cpu host -nographic \ -name guest=riscv-guset \ -smp 2 \ -bios none \ -kernel ./Image \ -drive file=./guest.img,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -append "root=/dev/vda rw console=ttyS0 earlycon=sbi" Changes since patch v3 - Re-write the for-loop in sifive_plic_create(). - Drop unnecessary change in hw/riscv/virt.c. - Use serial to handle console sbi call. Changes since patch v2 - Create a macro for get and put timer csr. - Remove M-mode PLIC contexts when kvm is enabled. - Add get timer frequency. - Move cpu_host_load to vmstate_kvmtimer. Changes since patch v1 - Rebase on recent commit a216e7cf119c91ffdf5931834a1a030ebea40d70 - Sync-up headers with Linux-5.16-rc4. - Fixbug in kvm_arch_init_vcpu. - Create a macro for get and put regs csr. - Start kernel directly when kvm_enabled. - Use riscv_cpu_set_irq to inject KVM interrupts. - Use the Semihosting Console API for RISC-V kvm handle sbi. - Update vmstate_riscv_cpu version id. Placing kvm_timer into a subsection. Changes since RFC v6 - Rebase on recent commit 8627edfb3f1fca24a96a0954148885c3241c10f8 - Sync-up headers with Linux-5.16-rc1 Changes since RFC v5 - Rebase on QEMU v6.1.0-rc1 and kvm-riscv linux v19. - Move kvm interrupt setting to riscv_cpu_update_mip(). - Replace __u64 with uint64_t. Changes since RFC v4 - Rebase on QEMU v6.0.0-rc2 and kvm-riscv linux v17. - Remove time scaling support as software solution is incomplete. Because it will cause unacceptable performance degradation. and We will post a better solution. - Revise according to Alistair's review comments. - Remove compile time XLEN checks in kvm_riscv_reg_id - Surround TYPE_RISCV_CPU_HOST definition by CONFIG_KVM and share it between RV32 and RV64. - Add kvm-stub.c for reduce unnecessary compilation checks. - Add riscv_setup_direct_kernel() to direct boot kernel for KVM. Changes since RFC v3 - Rebase on QEMU v5.2.0-rc2 and kvm-riscv linux v15. - Add time scaling support(New patches 13, 14 and 15). - Fix the bug that guest vm can't reboot. Changes since RFC v2 - Fix checkpatch error at target/riscv/sbi_ecall_interface.h. - Add riscv migration support. Changes since RFC v1 - Add separate SBI ecall interface header. - Add riscv32 kvm accel support. Yifei Jiang (12): update-linux-headers: Add asm-riscv/kvm.h target/riscv: Add target/riscv/kvm.c to place the public kvm interface target/riscv: Implement function kvm_arch_init_vcpu target/riscv: Implement kvm_arch_get_registers target/riscv: Implement kvm_arch_put_registers target/riscv: Support start kernel directly by KVM target/riscv: Support setting external interrupt by KVM target/riscv: Handle KVM_EXIT_RISCV_SBI exit target/riscv: Add host cpu type target/riscv: Add kvm_riscv_get/put_regs_timer target/riscv: Implement virtual time adjusting with vm state changing target/riscv: Support virtual time context synchronization hw/intc/sifive_plic.c | 21 +- hw/riscv/boot.c | 16 +- hw/riscv/virt.c | 83 +++-- include/hw/riscv/boot.h | 1 + linux-headers/asm-riscv/kvm.h | 128 +++++++ meson.build | 2 + target/riscv/cpu.c | 29 +- target/riscv/cpu.h | 11 + target/riscv/kvm-stub.c | 30 ++ target/riscv/kvm.c | 532 +++++++++++++++++++++++++++++ target/riscv/kvm_riscv.h | 25 ++ target/riscv/machine.c | 30 ++ target/riscv/meson.build | 1 + target/riscv/sbi_ecall_interface.h | 72 ++++ 14 files changed, 948 insertions(+), 33 deletions(-) create mode 100644 linux-headers/asm-riscv/kvm.h create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm.c create mode 100644 target/riscv/kvm_riscv.h create mode 100644 target/riscv/sbi_ecall_interface.h -- 2.19.1
Add asm-riscv/kvm.h for RISC-V KVM. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- linux-headers/asm-riscv/kvm.h | 128 ++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 linux-headers/asm-riscv/kvm.h diff --git a/linux-headers/asm-riscv/kvm.h b/linux-headers/asm-riscv/kvm.h new file mode 100644 index 0000000000..f808ad1ce5 --- /dev/null +++ b/linux-headers/asm-riscv/kvm.h @@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (C) 2019 Western Digital Corporation or its affiliates. + * + * Authors: + * Anup Patel <anup.patel@wdc.com> + */ + +#ifndef __LINUX_KVM_RISCV_H +#define __LINUX_KVM_RISCV_H + +#ifndef __ASSEMBLY__ + +#include <linux/types.h> +#include <asm/ptrace.h> + +#define __KVM_HAVE_READONLY_MEM + +#define KVM_COALESCED_MMIO_PAGE_OFFSET 1 + +#define KVM_INTERRUPT_SET -1U +#define KVM_INTERRUPT_UNSET -2U + +/* for KVM_GET_REGS and KVM_SET_REGS */ +struct kvm_regs { +}; + +/* for KVM_GET_FPU and KVM_SET_FPU */ +struct kvm_fpu { +}; + +/* KVM Debug exit structure */ +struct kvm_debug_exit_arch { +}; + +/* for KVM_SET_GUEST_DEBUG */ +struct kvm_guest_debug_arch { +}; + +/* definition of registers in kvm_run */ +struct kvm_sync_regs { +}; + +/* for KVM_GET_SREGS and KVM_SET_SREGS */ +struct kvm_sregs { +}; + +/* CONFIG registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_config { + unsigned long isa; +}; + +/* CORE registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_core { + struct user_regs_struct regs; + unsigned long mode; +}; + +/* Possible privilege modes for kvm_riscv_core */ +#define KVM_RISCV_MODE_S 1 +#define KVM_RISCV_MODE_U 0 + +/* CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_csr { + unsigned long sstatus; + unsigned long sie; + unsigned long stvec; + unsigned long sscratch; + unsigned long sepc; + unsigned long scause; + unsigned long stval; + unsigned long sip; + unsigned long satp; + unsigned long scounteren; +}; + +/* TIMER registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_timer { + __u64 frequency; + __u64 time; + __u64 compare; + __u64 state; +}; + +/* Possible states for kvm_riscv_timer */ +#define KVM_RISCV_TIMER_STATE_OFF 0 +#define KVM_RISCV_TIMER_STATE_ON 1 + +#define KVM_REG_SIZE(id) \ + (1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT)) + +/* If you need to interpret the index values, here is the key: */ +#define KVM_REG_RISCV_TYPE_MASK 0x00000000FF000000 +#define KVM_REG_RISCV_TYPE_SHIFT 24 + +/* Config registers are mapped as type 1 */ +#define KVM_REG_RISCV_CONFIG (0x01 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_CONFIG_REG(name) \ + (offsetof(struct kvm_riscv_config, name) / sizeof(unsigned long)) + +/* Core registers are mapped as type 2 */ +#define KVM_REG_RISCV_CORE (0x02 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_CORE_REG(name) \ + (offsetof(struct kvm_riscv_core, name) / sizeof(unsigned long)) + +/* Control and status registers are mapped as type 3 */ +#define KVM_REG_RISCV_CSR (0x03 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_CSR_REG(name) \ + (offsetof(struct kvm_riscv_csr, name) / sizeof(unsigned long)) + +/* Timer registers are mapped as type 4 */ +#define KVM_REG_RISCV_TIMER (0x04 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_TIMER_REG(name) \ + (offsetof(struct kvm_riscv_timer, name) / sizeof(__u64)) + +/* F extension registers are mapped as type 5 */ +#define KVM_REG_RISCV_FP_F (0x05 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_FP_F_REG(name) \ + (offsetof(struct __riscv_f_ext_state, name) / sizeof(__u32)) + +/* D extension registers are mapped as type 6 */ +#define KVM_REG_RISCV_FP_D (0x06 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_FP_D_REG(name) \ + (offsetof(struct __riscv_d_ext_state, name) / sizeof(__u64)) + +#endif + +#endif /* __LINUX_KVM_RISCV_H */ -- 2.19.1
Add target/riscv/kvm.c to place kvm_arch_* function needed by kvm/kvm-all.c. Meanwhile, add kvm support in meson.build file. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- meson.build | 2 + target/riscv/kvm.c | 133 +++++++++++++++++++++++++++++++++++++++ target/riscv/meson.build | 1 + 3 files changed, 136 insertions(+) create mode 100644 target/riscv/kvm.c diff --git a/meson.build b/meson.build index 53065e96ec..7eaec31a3a 100644 --- a/meson.build +++ b/meson.build @@ -90,6 +90,8 @@ elif cpu in ['ppc', 'ppc64'] kvm_targets = ['ppc-softmmu', 'ppc64-softmmu'] elif cpu in ['mips', 'mips64'] kvm_targets = ['mips-softmmu', 'mipsel-softmmu', 'mips64-softmmu', 'mips64el-softmmu'] +elif cpu in ['riscv'] + kvm_targets = ['riscv32-softmmu', 'riscv64-softmmu'] else kvm_targets = [] endif diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c new file mode 100644 index 0000000000..687dd4b621 --- /dev/null +++ b/target/riscv/kvm.c @@ -0,0 +1,133 @@ +/* + * RISC-V implementation of KVM hooks + * + * Copyright (c) 2020 Huawei Technologies Co., Ltd + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#include "qemu/osdep.h" +#include <sys/ioctl.h> + +#include <linux/kvm.h> + +#include "qemu-common.h" +#include "qemu/timer.h" +#include "qemu/error-report.h" +#include "qemu/main-loop.h" +#include "sysemu/sysemu.h" +#include "sysemu/kvm.h" +#include "sysemu/kvm_int.h" +#include "cpu.h" +#include "trace.h" +#include "hw/pci/pci.h" +#include "exec/memattrs.h" +#include "exec/address-spaces.h" +#include "hw/boards.h" +#include "hw/irq.h" +#include "qemu/log.h" +#include "hw/loader.h" + +const KVMCapabilityInfo kvm_arch_required_capabilities[] = { + KVM_CAP_LAST_INFO +}; + +int kvm_arch_get_registers(CPUState *cs) +{ + return 0; +} + +int kvm_arch_put_registers(CPUState *cs, int level) +{ + return 0; +} + +int kvm_arch_release_virq_post(int virq) +{ + return 0; +} + +int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route, + uint64_t address, uint32_t data, PCIDevice *dev) +{ + return 0; +} + +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + return 0; +} + +unsigned long kvm_arch_vcpu_id(CPUState *cpu) +{ + return cpu->cpu_index; +} + +void kvm_arch_init_irq_routing(KVMState *s) +{ +} + +int kvm_arch_init_vcpu(CPUState *cs) +{ + return 0; +} + +int kvm_arch_msi_data_to_gsi(uint32_t data) +{ + abort(); +} + +int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route, + int vector, PCIDevice *dev) +{ + return 0; +} + +int kvm_arch_init(MachineState *ms, KVMState *s) +{ + return 0; +} + +int kvm_arch_irqchip_create(KVMState *s) +{ + return 0; +} + +int kvm_arch_process_async_events(CPUState *cs) +{ + return 0; +} + +void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run) +{ +} + +MemTxAttrs kvm_arch_post_run(CPUState *cs, struct kvm_run *run) +{ + return MEMTXATTRS_UNSPECIFIED; +} + +bool kvm_arch_stop_on_emulation_error(CPUState *cs) +{ + return true; +} + +int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) +{ + return 0; +} + +bool kvm_arch_cpu_check_are_resettable(void) +{ + return true; +} diff --git a/target/riscv/meson.build b/target/riscv/meson.build index d5e0bc93ea..2faf08a941 100644 --- a/target/riscv/meson.build +++ b/target/riscv/meson.build @@ -19,6 +19,7 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', )) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files( -- 2.19.1
On Mon, Jan 10, 2022 at 11:48 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Add target/riscv/kvm.c to place kvm_arch_* function needed by kvm/kvm-all.c. Meanwhile, add kvm support in meson.build file.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- meson.build | 2 + target/riscv/kvm.c | 133 +++++++++++++++++++++++++++++++++++++++ target/riscv/meson.build | 1 + 3 files changed, 136 insertions(+) create mode 100644 target/riscv/kvm.c
diff --git a/meson.build b/meson.build index 53065e96ec..7eaec31a3a 100644 --- a/meson.build +++ b/meson.build @@ -90,6 +90,8 @@ elif cpu in ['ppc', 'ppc64'] kvm_targets = ['ppc-softmmu', 'ppc64-softmmu'] elif cpu in ['mips', 'mips64'] kvm_targets = ['mips-softmmu', 'mipsel-softmmu', 'mips64-softmmu', 'mips64el-softmmu'] +elif cpu in ['riscv'] + kvm_targets = ['riscv32-softmmu', 'riscv64-softmmu'] else kvm_targets = [] endif
Can you add this as a separate commit at the end of the series? That way we have implemented KVM support before we enable it for users. Alistair
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Tuesday, January 11, 2022 7:10 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; kvm-riscv@lists.infradead.org; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Anup Patel <anup@brainfault.org>; Palmer Dabbelt <palmer@dabbelt.com>; Alistair Francis <Alistair.Francis@wdc.com>; Bin Meng <bin.meng@windriver.com>; Fanliang (EulerOS) <fanliang@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Wanghaibin (D) <wanghaibin.wang@huawei.com>; wanbo (G) <wanbo13@huawei.com>; limingwang (A) <limingwang@huawei.com>; Anup Patel <anup.patel@wdc.com> Subject: Re: [PATCH v4 02/12] target/riscv: Add target/riscv/kvm.c to place the public kvm interface
On Mon, Jan 10, 2022 at 11:48 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Add target/riscv/kvm.c to place kvm_arch_* function needed by kvm/kvm-all.c. Meanwhile, add kvm support in meson.build file.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- meson.build | 2 + target/riscv/kvm.c | 133
+++++++++++++++++++++++++++++++++++++++
target/riscv/meson.build | 1 + 3 files changed, 136 insertions(+) create mode 100644 target/riscv/kvm.c
diff --git a/meson.build b/meson.build index 53065e96ec..7eaec31a3a 100644 --- a/meson.build +++ b/meson.build @@ -90,6 +90,8 @@ elif cpu in ['ppc', 'ppc64'] kvm_targets = ['ppc-softmmu', 'ppc64-softmmu'] elif cpu in ['mips', 'mips64'] kvm_targets = ['mips-softmmu', 'mipsel-softmmu', 'mips64-softmmu', 'mips64el-softmmu'] +elif cpu in ['riscv'] + kvm_targets = ['riscv32-softmmu', 'riscv64-softmmu'] else kvm_targets = [] endif
Can you add this as a separate commit at the end of the series?
That way we have implemented KVM support before we enable it for users.
Alistair
Yes, that makes sense. Yifei
Get isa info from kvm while kvm init. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/kvm.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 687dd4b621..ccf3753048 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -38,6 +38,23 @@ #include "qemu/log.h" #include "hw/loader.h" +static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) +{ + uint64_t id = KVM_REG_RISCV | type | idx; + + switch (riscv_cpu_mxl(env)) { + case MXL_RV32: + id |= KVM_REG_SIZE_U32; + break; + case MXL_RV64: + id |= KVM_REG_SIZE_U64; + break; + default: + g_assert_not_reached(); + } + return id; +} + const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO }; @@ -79,7 +96,20 @@ void kvm_arch_init_irq_routing(KVMState *s) int kvm_arch_init_vcpu(CPUState *cs) { - return 0; + int ret = 0; + target_ulong isa; + RISCVCPU *cpu = RISCV_CPU(cs); + CPURISCVState *env = &cpu->env; + uint64_t id; + + id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa)); + ret = kvm_get_one_reg(cs, id, &isa); + if (ret) { + return ret; + } + env->misa_ext = isa; + + return ret; } int kvm_arch_msi_data_to_gsi(uint32_t data) -- 2.19.1
Get GPR CSR and FP registers from kvm by KVM_GET_ONE_REG ioctl. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/kvm.c | 112 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 111 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index ccf3753048..6d4df0ef6d 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -55,13 +55,123 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx return id; } +#define RISCV_CORE_REG(env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_CORE, \ + KVM_REG_RISCV_CORE_REG(name)) + +#define RISCV_CSR_REG(env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_CSR, \ + KVM_REG_RISCV_CSR_REG(name)) + +#define RISCV_FP_F_REG(env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_F, idx) + +#define RISCV_FP_D_REG(env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_D, idx) + +#define KVM_RISCV_GET_CSR(cs, env, csr, reg) \ + do { \ + int ret = kvm_get_one_reg(cs, RISCV_CSR_REG(env, csr), ®); \ + if (ret) { \ + return ret; \ + } \ + } while(0) + +static int kvm_riscv_get_regs_core(CPUState *cs) +{ + int ret = 0; + int i; + target_ulong reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + ret = kvm_get_one_reg(cs, RISCV_CORE_REG(env, regs.pc), ®); + if (ret) { + return ret; + } + env->pc = reg; + + for (i = 1; i < 32; i++) { + uint64_t id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CORE, i); + ret = kvm_get_one_reg(cs, id, ®); + if (ret) { + return ret; + } + env->gpr[i] = reg; + } + + return ret; +} + +static int kvm_riscv_get_regs_csr(CPUState *cs) +{ + int ret = 0; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + KVM_RISCV_GET_CSR(cs, env, sstatus, env->mstatus); + KVM_RISCV_GET_CSR(cs, env, sie, env->mie); + KVM_RISCV_GET_CSR(cs, env, stvec, env->stvec); + KVM_RISCV_GET_CSR(cs, env, sscratch, env->sscratch); + KVM_RISCV_GET_CSR(cs, env, sepc, env->sepc); + KVM_RISCV_GET_CSR(cs, env, scause, env->scause); + KVM_RISCV_GET_CSR(cs, env, stval, env->stval); + KVM_RISCV_GET_CSR(cs, env, sip, env->mip); + KVM_RISCV_GET_CSR(cs, env, satp, env->satp); + return ret; +} + +static int kvm_riscv_get_regs_fp(CPUState *cs) +{ + int ret = 0; + int i; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (riscv_has_ext(env, RVD)) { + uint64_t reg; + for (i = 0; i < 32; i++) { + ret = kvm_get_one_reg(cs, RISCV_FP_D_REG(env, i), ®); + if (ret) { + return ret; + } + env->fpr[i] = reg; + } + return ret; + } + + if (riscv_has_ext(env, RVF)) { + uint32_t reg; + for (i = 0; i < 32; i++) { + ret = kvm_get_one_reg(cs, RISCV_FP_F_REG(env, i), ®); + if (ret) { + return ret; + } + env->fpr[i] = reg; + } + return ret; + } + + return ret; +} + const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO }; int kvm_arch_get_registers(CPUState *cs) { - return 0; + int ret = 0; + + ret = kvm_riscv_get_regs_core(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_get_regs_csr(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_get_regs_fp(cs); + if (ret) { + return ret; + } + + return ret; } int kvm_arch_put_registers(CPUState *cs, int level) -- 2.19.1
Put GPR CSR and FP registers to kvm by KVM_SET_ONE_REG ioctl Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/kvm.c | 104 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 6d4df0ef6d..e695b91dc7 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -73,6 +73,14 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx } \ } while(0) +#define KVM_RISCV_SET_CSR(cs, env, csr, reg) \ + do { \ + int ret = kvm_set_one_reg(cs, RISCV_CSR_REG(env, csr), ®); \ + if (ret) { \ + return ret; \ + } \ + } while(0) + static int kvm_riscv_get_regs_core(CPUState *cs) { int ret = 0; @@ -98,6 +106,31 @@ static int kvm_riscv_get_regs_core(CPUState *cs) return ret; } +static int kvm_riscv_put_regs_core(CPUState *cs) +{ + int ret = 0; + int i; + target_ulong reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + reg = env->pc; + ret = kvm_set_one_reg(cs, RISCV_CORE_REG(env, regs.pc), ®); + if (ret) { + return ret; + } + + for (i = 1; i < 32; i++) { + uint64_t id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CORE, i); + reg = env->gpr[i]; + ret = kvm_set_one_reg(cs, id, ®); + if (ret) { + return ret; + } + } + + return ret; +} + static int kvm_riscv_get_regs_csr(CPUState *cs) { int ret = 0; @@ -115,6 +148,24 @@ static int kvm_riscv_get_regs_csr(CPUState *cs) return ret; } +static int kvm_riscv_put_regs_csr(CPUState *cs) +{ + int ret = 0; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + KVM_RISCV_SET_CSR(cs, env, sstatus, env->mstatus); + KVM_RISCV_SET_CSR(cs, env, sie, env->mie); + KVM_RISCV_SET_CSR(cs, env, stvec, env->stvec); + KVM_RISCV_SET_CSR(cs, env, sscratch, env->sscratch); + KVM_RISCV_SET_CSR(cs, env, sepc, env->sepc); + KVM_RISCV_SET_CSR(cs, env, scause, env->scause); + KVM_RISCV_SET_CSR(cs, env, stval, env->stval); + KVM_RISCV_SET_CSR(cs, env, sip, env->mip); + KVM_RISCV_SET_CSR(cs, env, satp, env->satp); + + return ret; +} + static int kvm_riscv_get_regs_fp(CPUState *cs) { int ret = 0; @@ -148,6 +199,40 @@ static int kvm_riscv_get_regs_fp(CPUState *cs) return ret; } +static int kvm_riscv_put_regs_fp(CPUState *cs) +{ + int ret = 0; + int i; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (riscv_has_ext(env, RVD)) { + uint64_t reg; + for (i = 0; i < 32; i++) { + reg = env->fpr[i]; + ret = kvm_set_one_reg(cs, RISCV_FP_D_REG(env, i), ®); + if (ret) { + return ret; + } + } + return ret; + } + + if (riscv_has_ext(env, RVF)) { + uint32_t reg; + for (i = 0; i < 32; i++) { + reg = env->fpr[i]; + ret = kvm_set_one_reg(cs, RISCV_FP_F_REG(env, i), ®); + if (ret) { + return ret; + } + } + return ret; + } + + return ret; +} + + const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO }; @@ -176,7 +261,24 @@ int kvm_arch_get_registers(CPUState *cs) int kvm_arch_put_registers(CPUState *cs, int level) { - return 0; + int ret = 0; + + ret = kvm_riscv_put_regs_core(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_put_regs_csr(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_put_regs_fp(cs); + if (ret) { + return ret; + } + + return ret; } int kvm_arch_release_virq_post(int virq) -- 2.19.1
On Mon, Jan 10, 2022 at 11:57 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Put GPR CSR and FP registers to kvm by KVM_SET_ONE_REG ioctl
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/kvm.c | 104 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 6d4df0ef6d..e695b91dc7 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -73,6 +73,14 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx } \ } while(0)
+#define KVM_RISCV_SET_CSR(cs, env, csr, reg) \ + do { \ + int ret = kvm_set_one_reg(cs, RISCV_CSR_REG(env, csr), ®); \ + if (ret) { \ + return ret; \ + } \ + } while(0)
This fails checkpatch. I know there is lots of QEMU code like this, but it probably should be `while (0)` to keep checkpatch happy. Please run checkpatch on all the patches. Alistair
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Tuesday, January 11, 2022 7:07 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; kvm-riscv@lists.infradead.org; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Anup Patel <anup@brainfault.org>; Palmer Dabbelt <palmer@dabbelt.com>; Alistair Francis <Alistair.Francis@wdc.com>; Bin Meng <bin.meng@windriver.com>; Fanliang (EulerOS) <fanliang@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Wanghaibin (D) <wanghaibin.wang@huawei.com>; wanbo (G) <wanbo13@huawei.com>; limingwang (A) <limingwang@huawei.com>; Anup Patel <anup.patel@wdc.com> Subject: Re: [PATCH v4 05/12] target/riscv: Implement kvm_arch_put_registers
On Mon, Jan 10, 2022 at 11:57 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Put GPR CSR and FP registers to kvm by KVM_SET_ONE_REG ioctl
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/kvm.c | 104 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 6d4df0ef6d..e695b91dc7 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -73,6 +73,14 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env,
uint64_t type, uint64_t idx
} \ } while(0)
+#define KVM_RISCV_SET_CSR(cs, env, csr, reg) \ + do { \ + int ret = kvm_set_one_reg(cs, RISCV_CSR_REG(env, csr), ®); \ + if (ret) { \ + return ret; \ + } \ + } while(0)
This fails checkpatch. I know there is lots of QEMU code like this, but it probably should be `while (0)` to keep checkpatch happy.
Please run checkpatch on all the patches.
Alistair
OK, it will be modified in the next series. Yifei
Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface. In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- hw/intc/sifive_plic.c | 21 +++++++--- hw/riscv/boot.c | 16 +++++++- hw/riscv/virt.c | 83 ++++++++++++++++++++++++++++------------ include/hw/riscv/boot.h | 1 + target/riscv/cpu.c | 8 ++++ target/riscv/cpu.h | 3 ++ target/riscv/kvm-stub.c | 25 ++++++++++++ target/riscv/kvm.c | 14 +++++++ target/riscv/kvm_riscv.h | 24 ++++++++++++ target/riscv/meson.build | 2 +- 10 files changed, 164 insertions(+), 33 deletions(-) create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm_riscv.h diff --git a/hw/intc/sifive_plic.c b/hw/intc/sifive_plic.c index 877e76877c..58c16881cb 100644 --- a/hw/intc/sifive_plic.c +++ b/hw/intc/sifive_plic.c @@ -30,6 +30,7 @@ #include "target/riscv/cpu.h" #include "migration/vmstate.h" #include "hw/irq.h" +#include "sysemu/kvm.h" #define RISCV_DEBUG_PLIC 0 @@ -533,6 +534,8 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, { DeviceState *dev = qdev_new(TYPE_SIFIVE_PLIC); int i; + SiFivePLICState *plic; + int s_count = 0, m_count = 0; assert(enable_stride == (enable_stride & -enable_stride)); assert(context_stride == (context_stride & -context_stride)); @@ -550,13 +553,19 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr); - for (i = 0; i < num_harts; i++) { - CPUState *cpu = qemu_get_cpu(hartid_base + i); + plic = SIFIVE_PLIC(dev); + for (i = 0; i < plic->num_addrs; i++) { + CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid); - qdev_connect_gpio_out(dev, i, - qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT)); - qdev_connect_gpio_out(dev, num_harts + i, - qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); + if (plic->addr_config[i].mode == PLICMode_S) { + qdev_connect_gpio_out(dev, s_count++, + qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT)); + } + + if (plic->addr_config[i].mode == PLICMode_M) { + qdev_connect_gpio_out(dev, num_harts + m_count++, + qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); + } } return dev; diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c index f67264374e..cae74fcbcd 100644 --- a/hw/riscv/boot.c +++ b/hw/riscv/boot.c @@ -30,6 +30,7 @@ #include "elf.h" #include "sysemu/device_tree.h" #include "sysemu/qtest.h" +#include "sysemu/kvm.h" #include <libfdt.h> @@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count) CPUState *cs = qemu_get_cpu(i); CPURISCVState *env = &RISCV_CPU(cs)->env; - if (riscv_has_ext(env, RVS)) { + if (kvm_enabled()) { + vals[i] = "S"; + } else if (riscv_has_ext(env, RVS)) { vals[i] = "MS"; } else { vals[i] = "M"; @@ -324,3 +327,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *harts return; } + +void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr) +{ + CPUState *cs; + + for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) { + RISCVCPU *riscv_cpu = RISCV_CPU(cs); + riscv_cpu->env.kernel_addr = kernel_addr; + riscv_cpu->env.fdt_addr = fdt_addr; + } +} diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 3af074148e..2643c8bc37 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -38,6 +38,7 @@ #include "chardev/char.h" #include "sysemu/device_tree.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "hw/pci/pci.h" #include "hw/pci-host/gpex.h" #include "hw/display/ramfb.h" @@ -372,13 +373,22 @@ static void create_fdt_socket_plic(RISCVVirtState *s, "sifive,plic-1.0.0", "riscv,plic0" }; - plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); + if (kvm_enabled()) { + plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2); + } else { + plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); + } for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) { - plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]); - plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT); - plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]); - plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT); + if (kvm_enabled()) { + plic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); + plic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT); + } else { + plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]); + plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT); + plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]); + plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT); + } } plic_phandles[socket] = (*phandle)++; @@ -436,10 +446,12 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap, create_fdt_socket_memory(s, memmap, socket); - if (s->have_aclint) { - create_fdt_socket_aclint(s, memmap, socket, intc_phandles); - } else { - create_fdt_socket_clint(s, memmap, socket, intc_phandles); + if (!kvm_enabled()) { + if (s->have_aclint) { + create_fdt_socket_aclint(s, memmap, socket, intc_phandles); + } else { + create_fdt_socket_clint(s, memmap, socket, intc_phandles); + } } create_fdt_socket_plic(s, memmap, socket, phandle, @@ -801,23 +813,25 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort); sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort); - /* Per-socket CLINT */ - riscv_aclint_swi_create( - memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, - base_hartid, hart_count, false); - riscv_aclint_mtimer_create( - memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size + - RISCV_ACLINT_SWI_SIZE, - RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid, hart_count, - RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME, - RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true); - - /* Per-socket ACLINT SSWI */ - if (s->have_aclint) { + if (!kvm_enabled()) { + /* Per-socket CLINT */ riscv_aclint_swi_create( - memmap[VIRT_ACLINT_SSWI].base + - i * memmap[VIRT_ACLINT_SSWI].size, - base_hartid, hart_count, true); + memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, + base_hartid, hart_count, false); + riscv_aclint_mtimer_create( + memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size + + RISCV_ACLINT_SWI_SIZE, + RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid, hart_count, + RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME, + RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true); + + /* Per-socket ACLINT SSWI */ + if (s->have_aclint) { + riscv_aclint_swi_create( + memmap[VIRT_ACLINT_SSWI].base + + i * memmap[VIRT_ACLINT_SSWI].size, + base_hartid, hart_count, true); + } } /* Per-socket PLIC hart topology configuration string */ @@ -884,6 +898,16 @@ static void virt_machine_init(MachineState *machine) memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base, mask_rom); + /* + * Only direct boot kernel is currently supported for KVM VM, + * so the "-bios" parameter is ignored and treated like "-bios none" + * when KVM is enabled. + */ + if (kvm_enabled()) { + g_free(machine->firmware); + machine->firmware = g_strdup("none"); + } + if (riscv_is_32bit(&s->soc[0])) { firmware_end_addr = riscv_find_and_load_firmware(machine, RISCV32_BIOS_BIN, start_addr, NULL); @@ -941,6 +965,15 @@ static void virt_machine_init(MachineState *machine) virt_memmap[VIRT_MROM].size, kernel_entry, fdt_load_addr, machine->fdt); + /* + * Only direct boot kernel is currently supported for KVM VM, + * So here setup kernel start address and fdt address. + * TODO:Support firmware loading and integrate to TCG start + */ + if (kvm_enabled()) { + riscv_setup_direct_kernel(kernel_entry, fdt_load_addr); + } + /* SiFive Test MMIO device */ sifive_test_create(memmap[VIRT_TEST].base); diff --git a/include/hw/riscv/boot.h b/include/hw/riscv/boot.h index baff11dd8a..5834c234aa 100644 --- a/include/hw/riscv/boot.h +++ b/include/hw/riscv/boot.h @@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState *machine, hwaddr rom_base, hwaddr rom_size, uint32_t reset_vec_size, uint64_t kernel_entry); +void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr); #endif /* RISCV_BOOT_H */ diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 6ef3314bce..8a979a3762 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -29,6 +29,8 @@ #include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h" /* RISC-V CPU definitions */ @@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index = RISCV_EXCP_NONE; env->load_res = -1; set_default_nan_mode(1, &env->fp_status); + +#ifndef CONFIG_USER_ONLY + if (kvm_enabled()) { + kvm_riscv_reset_vcpu(cpu); + } +#endif } static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info *info) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index dc10f27093..491a38de01 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -257,6 +257,9 @@ struct CPURISCVState { /* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer; /* Internal timer */ + + hwaddr kernel_addr; + hwaddr fdt_addr; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git a/target/riscv/kvm-stub.c b/target/riscv/kvm-stub.c new file mode 100644 index 0000000000..39b96fe3f4 --- /dev/null +++ b/target/riscv/kvm-stub.c @@ -0,0 +1,25 @@ +/* + * QEMU KVM RISC-V specific function stubs + * + * Copyright (c) 2020 Huawei Technologies Co., Ltd + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ +#include "qemu/osdep.h" +#include "cpu.h" +#include "kvm_riscv.h" + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu) +{ + abort(); +} diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index e695b91dc7..db6d8a5b6e 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -37,6 +37,7 @@ #include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) { @@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) return 0; } +void kvm_riscv_reset_vcpu(RISCVCPU *cpu) +{ + CPURISCVState *env = &cpu->env; + + if (!kvm_enabled()) { + return; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */ + env->gpr[11] = cpu->env.fdt_addr; /* a1 */ + env->satp = 0; +} + bool kvm_arch_cpu_check_are_resettable(void) { return true; diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h new file mode 100644 index 0000000000..f38c82bf59 --- /dev/null +++ b/target/riscv/kvm_riscv.h @@ -0,0 +1,24 @@ +/* + * QEMU KVM support -- RISC-V specific functions. + * + * Copyright (c) 2020 Huawei Technologies Co., Ltd + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu); + +#endif diff --git a/target/riscv/meson.build b/target/riscv/meson.build index 2faf08a941..fe41cc5805 100644 --- a/target/riscv/meson.build +++ b/target/riscv/meson.build @@ -19,7 +19,7 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', )) -riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files( -- 2.19.1
On Mon, Jan 10, 2022 at 11:52 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface.
In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- hw/intc/sifive_plic.c | 21 +++++++--- hw/riscv/boot.c | 16 +++++++- hw/riscv/virt.c | 83 ++++++++++++++++++++++++++++------------ include/hw/riscv/boot.h | 1 + target/riscv/cpu.c | 8 ++++ target/riscv/cpu.h | 3 ++ target/riscv/kvm-stub.c | 25 ++++++++++++ target/riscv/kvm.c | 14 +++++++ target/riscv/kvm_riscv.h | 24 ++++++++++++ target/riscv/meson.build | 2 +- 10 files changed, 164 insertions(+), 33 deletions(-) create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm_riscv.h
diff --git a/hw/intc/sifive_plic.c b/hw/intc/sifive_plic.c index 877e76877c..58c16881cb 100644 --- a/hw/intc/sifive_plic.c +++ b/hw/intc/sifive_plic.c @@ -30,6 +30,7 @@ #include "target/riscv/cpu.h" #include "migration/vmstate.h" #include "hw/irq.h" +#include "sysemu/kvm.h"
#define RISCV_DEBUG_PLIC 0
@@ -533,6 +534,8 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, { DeviceState *dev = qdev_new(TYPE_SIFIVE_PLIC); int i; + SiFivePLICState *plic; + int s_count = 0, m_count = 0;
assert(enable_stride == (enable_stride & -enable_stride)); assert(context_stride == (context_stride & -context_stride)); @@ -550,13 +553,19 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
- for (i = 0; i < num_harts; i++) { - CPUState *cpu = qemu_get_cpu(hartid_base + i); + plic = SIFIVE_PLIC(dev); + for (i = 0; i < plic->num_addrs; i++) { + CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid);
- qdev_connect_gpio_out(dev, i, - qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT)); - qdev_connect_gpio_out(dev, num_harts + i, - qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); + if (plic->addr_config[i].mode == PLICMode_S) { + qdev_connect_gpio_out(dev, s_count++, + qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT)); + } + + if (plic->addr_config[i].mode == PLICMode_M) { + qdev_connect_gpio_out(dev, num_harts + m_count++, + qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); + } }
This PLIC change breaks my 5.11.0 buildroot test case on the SiFive U board The boot process just hangs at: [ 0.542798] usbcore: registered new interface driver usbhid [ 0.543021] usbhid: USB HID core driver [ 0.544584] NET: Registered protocol family 10 [ 4.054768] Segment Routing with IPv6 [ 4.055325] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver [ 4.057956] NET: Registered protocol family 17 [ 4.059327] 9pnet: Installing 9P2000 support [ 4.059787] Key type dns_resolver registered [ 4.060515] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 4.078710] macb 10090000.ethernet eth0: PHY [10090000.ethernet-ffffffff:00] driver [Generic PHY] (irq=POLL) [ 4.079454] macb 10090000.ethernet eth0: configuring for phy/gmii link mode [ 4.087031] macb 10090000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx [ 4.094634] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Alistair
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Tuesday, January 11, 2022 8:28 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; kvm-riscv@lists.infradead.org; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Anup Patel <anup@brainfault.org>; Palmer Dabbelt <palmer@dabbelt.com>; Alistair Francis <Alistair.Francis@wdc.com>; Bin Meng <bin.meng@windriver.com>; Fanliang (EulerOS) <fanliang@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Wanghaibin (D) <wanghaibin.wang@huawei.com>; wanbo (G) <wanbo13@huawei.com>; limingwang (A) <limingwang@huawei.com> Subject: Re: [PATCH v4 06/12] target/riscv: Support start kernel directly by KVM
On Mon, Jan 10, 2022 at 11:52 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface.
In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- hw/intc/sifive_plic.c | 21 +++++++--- hw/riscv/boot.c | 16 +++++++- hw/riscv/virt.c | 83 ++++++++++++++++++++++++++++------------ include/hw/riscv/boot.h | 1 + target/riscv/cpu.c | 8 ++++ target/riscv/cpu.h | 3 ++ target/riscv/kvm-stub.c | 25 ++++++++++++ target/riscv/kvm.c | 14 +++++++ target/riscv/kvm_riscv.h | 24 ++++++++++++ target/riscv/meson.build | 2 +- 10 files changed, 164 insertions(+), 33 deletions(-) create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm_riscv.h
diff --git a/hw/intc/sifive_plic.c b/hw/intc/sifive_plic.c index 877e76877c..58c16881cb 100644 --- a/hw/intc/sifive_plic.c +++ b/hw/intc/sifive_plic.c @@ -30,6 +30,7 @@ #include "target/riscv/cpu.h" #include "migration/vmstate.h" #include "hw/irq.h" +#include "sysemu/kvm.h"
#define RISCV_DEBUG_PLIC 0
@@ -533,6 +534,8 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, { DeviceState *dev = qdev_new(TYPE_SIFIVE_PLIC); int i; + SiFivePLICState *plic; + int s_count = 0, m_count = 0;
assert(enable_stride == (enable_stride & -enable_stride)); assert(context_stride == (context_stride & -context_stride)); @@ -550,13 +553,19 @@ DeviceState *sifive_plic_create(hwaddr addr, char
*hart_config,
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
- for (i = 0; i < num_harts; i++) { - CPUState *cpu = qemu_get_cpu(hartid_base + i); + plic = SIFIVE_PLIC(dev); + for (i = 0; i < plic->num_addrs; i++) { + CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid);
- qdev_connect_gpio_out(dev, i, - qdev_get_gpio_in(DEVICE(cpu),
IRQ_S_EXT));
- qdev_connect_gpio_out(dev, num_harts + i, - qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); + if (plic->addr_config[i].mode == PLICMode_S) { + qdev_connect_gpio_out(dev, s_count++, + qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT)); + } + + if (plic->addr_config[i].mode == PLICMode_M) { + qdev_connect_gpio_out(dev, num_harts + m_count++, + qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); + } }
This PLIC change breaks my 5.11.0 buildroot test case on the SiFive U board
The boot process just hangs at:
[ 0.542798] usbcore: registered new interface driver usbhid [ 0.543021] usbhid: USB HID core driver [ 0.544584] NET: Registered protocol family 10 [ 4.054768] Segment Routing with IPv6 [ 4.055325] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver [ 4.057956] NET: Registered protocol family 17 [ 4.059327] 9pnet: Installing 9P2000 support [ 4.059787] Key type dns_resolver registered [ 4.060515] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 4.078710] macb 10090000.ethernet eth0: PHY [10090000.ethernet-ffffffff:00] driver [Generic PHY] (irq=POLL) [ 4.079454] macb 10090000.ethernet eth0: configuring for phy/gmii link mode [ 4.087031] macb 10090000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx [ 4.094634] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Alistair
SiFive-u machine cpu 0 is the management hart that does not have S-mode. The logic here causes an offset of the PLIC S-mode context. When the kernel driver enables the CPU 1 S-mode interrupt, it shifts to CPU 0. As a result, the interrupt is lost. I will fix this bug in the next series. Yifei
When KVM is enabled, set the S-mode external interrupt through kvm_riscv_set_irq function. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/cpu.c | 6 +++++- target/riscv/kvm-stub.c | 5 +++++ target/riscv/kvm.c | 17 +++++++++++++++++ target/riscv/kvm_riscv.h | 1 + 4 files changed, 28 insertions(+), 1 deletion(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 8a979a3762..39382b1d62 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -603,7 +603,11 @@ static void riscv_cpu_set_irq(void *opaque, int irq, int level) case IRQ_S_EXT: case IRQ_VS_EXT: case IRQ_M_EXT: - riscv_cpu_update_mip(cpu, 1 << irq, BOOL_TO_MASK(level)); + if (kvm_enabled()) { + kvm_riscv_set_irq(cpu, irq, level); + } else { + riscv_cpu_update_mip(cpu, 1 << irq, BOOL_TO_MASK(level)); + } break; default: g_assert_not_reached(); diff --git a/target/riscv/kvm-stub.c b/target/riscv/kvm-stub.c index 39b96fe3f4..4e8fc31a21 100644 --- a/target/riscv/kvm-stub.c +++ b/target/riscv/kvm-stub.c @@ -23,3 +23,8 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) { abort(); } + +void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level) +{ + abort(); +} diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index db6d8a5b6e..0027f11f45 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -383,6 +383,23 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) env->satp = 0; } +void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level) +{ + int ret; + unsigned virq = level ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET; + + if (irq != IRQ_S_EXT) { + perror("kvm riscv set irq != IRQ_S_EXT\n"); + abort(); + } + + ret = kvm_vcpu_ioctl(CPU(cpu), KVM_INTERRUPT, &virq); + if (ret < 0) { + perror("Set irq failed"); + abort(); + } +} + bool kvm_arch_cpu_check_are_resettable(void) { return true; diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h index f38c82bf59..ed281bdce0 100644 --- a/target/riscv/kvm_riscv.h +++ b/target/riscv/kvm_riscv.h @@ -20,5 +20,6 @@ #define QEMU_KVM_RISCV_H void kvm_riscv_reset_vcpu(RISCVCPU *cpu); +void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level); #endif -- 2.19.1
Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/kvm.c | 42 ++++++++++++++++- target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 target/riscv/sbi_ecall_interface.h diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 0027f11f45..ded2a8c29d 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -38,6 +38,8 @@ #include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" +#include "sbi_ecall_interface.h" +#include "chardev/char-fe.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) { @@ -365,9 +367,47 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs) return true; } +static int kvm_riscv_handle_sbi(CPUState *cs, struct kvm_run *run) +{ + int ret = 0; + unsigned char ch; + switch (run->riscv_sbi.extension_id) { + case SBI_EXT_0_1_CONSOLE_PUTCHAR: + ch = run->riscv_sbi.args[0]; + qemu_chr_fe_write(serial_hd(0)->be, &ch, sizeof(ch)); + break; + case SBI_EXT_0_1_CONSOLE_GETCHAR: + ret = qemu_chr_fe_read_all(serial_hd(0)->be, &ch, sizeof(ch)); + if (ret == sizeof(ch)) { + run->riscv_sbi.args[0] = ch; + } else { + run->riscv_sbi.args[0] = -1; + } + break; + default: + qemu_log_mask(LOG_UNIMP, + "%s: un-handled SBI EXIT, specific reasons is %lu\n", + __func__, run->riscv_sbi.extension_id); + ret = -1; + break; + } + return ret; +} + int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) { - return 0; + int ret = 0; + switch (run->exit_reason) { + case KVM_EXIT_RISCV_SBI: + ret = kvm_riscv_handle_sbi(cs, run); + break; + default: + qemu_log_mask(LOG_UNIMP, "%s: un-handled exit reason %d\n", + __func__, run->exit_reason); + ret = -1; + break; + } + return ret; } void kvm_riscv_reset_vcpu(RISCVCPU *cpu) diff --git a/target/riscv/sbi_ecall_interface.h b/target/riscv/sbi_ecall_interface.h new file mode 100644 index 0000000000..fb1a3fa8f2 --- /dev/null +++ b/target/riscv/sbi_ecall_interface.h @@ -0,0 +1,72 @@ +/* + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2019 Western Digital Corporation or its affiliates. + * + * Authors: + * Anup Patel <anup.patel@wdc.com> + */ + +#ifndef __SBI_ECALL_INTERFACE_H__ +#define __SBI_ECALL_INTERFACE_H__ + +/* clang-format off */ + +/* SBI Extension IDs */ +#define SBI_EXT_0_1_SET_TIMER 0x0 +#define SBI_EXT_0_1_CONSOLE_PUTCHAR 0x1 +#define SBI_EXT_0_1_CONSOLE_GETCHAR 0x2 +#define SBI_EXT_0_1_CLEAR_IPI 0x3 +#define SBI_EXT_0_1_SEND_IPI 0x4 +#define SBI_EXT_0_1_REMOTE_FENCE_I 0x5 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA 0x6 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID 0x7 +#define SBI_EXT_0_1_SHUTDOWN 0x8 +#define SBI_EXT_BASE 0x10 +#define SBI_EXT_TIME 0x54494D45 +#define SBI_EXT_IPI 0x735049 +#define SBI_EXT_RFENCE 0x52464E43 +#define SBI_EXT_HSM 0x48534D + +/* SBI function IDs for BASE extension*/ +#define SBI_EXT_BASE_GET_SPEC_VERSION 0x0 +#define SBI_EXT_BASE_GET_IMP_ID 0x1 +#define SBI_EXT_BASE_GET_IMP_VERSION 0x2 +#define SBI_EXT_BASE_PROBE_EXT 0x3 +#define SBI_EXT_BASE_GET_MVENDORID 0x4 +#define SBI_EXT_BASE_GET_MARCHID 0x5 +#define SBI_EXT_BASE_GET_MIMPID 0x6 + +/* SBI function IDs for TIME extension*/ +#define SBI_EXT_TIME_SET_TIMER 0x0 + +/* SBI function IDs for IPI extension*/ +#define SBI_EXT_IPI_SEND_IPI 0x0 + +/* SBI function IDs for RFENCE extension*/ +#define SBI_EXT_RFENCE_REMOTE_FENCE_I 0x0 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA 0x1 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID 0x2 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA 0x3 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID 0x4 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA 0x5 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID 0x6 + +/* SBI function IDs for HSM extension */ +#define SBI_EXT_HSM_HART_START 0x0 +#define SBI_EXT_HSM_HART_STOP 0x1 +#define SBI_EXT_HSM_HART_GET_STATUS 0x2 + +#define SBI_HSM_HART_STATUS_STARTED 0x0 +#define SBI_HSM_HART_STATUS_STOPPED 0x1 +#define SBI_HSM_HART_STATUS_START_PENDING 0x2 +#define SBI_HSM_HART_STATUS_STOP_PENDING 0x3 + +#define SBI_SPEC_VERSION_MAJOR_OFFSET 24 +#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f +#define SBI_SPEC_VERSION_MINOR_MASK 0xffffff +#define SBI_EXT_VENDOR_START 0x09000000 +#define SBI_EXT_VENDOR_END 0x09FFFFFF +/* clang-format on */ + +#endif -- 2.19.1
On Mon, Jan 10, 2022 at 11:49 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Alistair
--- target/riscv/kvm.c | 42 ++++++++++++++++- target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 target/riscv/sbi_ecall_interface.h
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 0027f11f45..ded2a8c29d 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -38,6 +38,8 @@ #include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" +#include "sbi_ecall_interface.h" +#include "chardev/char-fe.h"
static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) { @@ -365,9 +367,47 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs) return true; }
+static int kvm_riscv_handle_sbi(CPUState *cs, struct kvm_run *run) +{ + int ret = 0; + unsigned char ch; + switch (run->riscv_sbi.extension_id) { + case SBI_EXT_0_1_CONSOLE_PUTCHAR: + ch = run->riscv_sbi.args[0]; + qemu_chr_fe_write(serial_hd(0)->be, &ch, sizeof(ch)); + break; + case SBI_EXT_0_1_CONSOLE_GETCHAR: + ret = qemu_chr_fe_read_all(serial_hd(0)->be, &ch, sizeof(ch)); + if (ret == sizeof(ch)) { + run->riscv_sbi.args[0] = ch; + } else { + run->riscv_sbi.args[0] = -1; + } + break; + default: + qemu_log_mask(LOG_UNIMP, + "%s: un-handled SBI EXIT, specific reasons is %lu\n", + __func__, run->riscv_sbi.extension_id); + ret = -1; + break; + } + return ret; +} + int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) { - return 0; + int ret = 0; + switch (run->exit_reason) { + case KVM_EXIT_RISCV_SBI: + ret = kvm_riscv_handle_sbi(cs, run); + break; + default: + qemu_log_mask(LOG_UNIMP, "%s: un-handled exit reason %d\n", + __func__, run->exit_reason); + ret = -1; + break; + } + return ret; }
void kvm_riscv_reset_vcpu(RISCVCPU *cpu) diff --git a/target/riscv/sbi_ecall_interface.h b/target/riscv/sbi_ecall_interface.h new file mode 100644 index 0000000000..fb1a3fa8f2 --- /dev/null +++ b/target/riscv/sbi_ecall_interface.h @@ -0,0 +1,72 @@ +/* + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2019 Western Digital Corporation or its affiliates. + * + * Authors: + * Anup Patel <anup.patel@wdc.com> + */ + +#ifndef __SBI_ECALL_INTERFACE_H__ +#define __SBI_ECALL_INTERFACE_H__ + +/* clang-format off */ + +/* SBI Extension IDs */ +#define SBI_EXT_0_1_SET_TIMER 0x0 +#define SBI_EXT_0_1_CONSOLE_PUTCHAR 0x1 +#define SBI_EXT_0_1_CONSOLE_GETCHAR 0x2 +#define SBI_EXT_0_1_CLEAR_IPI 0x3 +#define SBI_EXT_0_1_SEND_IPI 0x4 +#define SBI_EXT_0_1_REMOTE_FENCE_I 0x5 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA 0x6 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID 0x7 +#define SBI_EXT_0_1_SHUTDOWN 0x8 +#define SBI_EXT_BASE 0x10 +#define SBI_EXT_TIME 0x54494D45 +#define SBI_EXT_IPI 0x735049 +#define SBI_EXT_RFENCE 0x52464E43 +#define SBI_EXT_HSM 0x48534D + +/* SBI function IDs for BASE extension*/ +#define SBI_EXT_BASE_GET_SPEC_VERSION 0x0 +#define SBI_EXT_BASE_GET_IMP_ID 0x1 +#define SBI_EXT_BASE_GET_IMP_VERSION 0x2 +#define SBI_EXT_BASE_PROBE_EXT 0x3 +#define SBI_EXT_BASE_GET_MVENDORID 0x4 +#define SBI_EXT_BASE_GET_MARCHID 0x5 +#define SBI_EXT_BASE_GET_MIMPID 0x6 + +/* SBI function IDs for TIME extension*/ +#define SBI_EXT_TIME_SET_TIMER 0x0 + +/* SBI function IDs for IPI extension*/ +#define SBI_EXT_IPI_SEND_IPI 0x0 + +/* SBI function IDs for RFENCE extension*/ +#define SBI_EXT_RFENCE_REMOTE_FENCE_I 0x0 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA 0x1 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID 0x2 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA 0x3 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID 0x4 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA 0x5 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID 0x6 + +/* SBI function IDs for HSM extension */ +#define SBI_EXT_HSM_HART_START 0x0 +#define SBI_EXT_HSM_HART_STOP 0x1 +#define SBI_EXT_HSM_HART_GET_STATUS 0x2 + +#define SBI_HSM_HART_STATUS_STARTED 0x0 +#define SBI_HSM_HART_STATUS_STOPPED 0x1 +#define SBI_HSM_HART_STATUS_START_PENDING 0x2 +#define SBI_HSM_HART_STATUS_STOP_PENDING 0x3 + +#define SBI_SPEC_VERSION_MAJOR_OFFSET 24 +#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f +#define SBI_SPEC_VERSION_MINOR_MASK 0xffffff +#define SBI_EXT_VENDOR_START 0x09000000 +#define SBI_EXT_VENDOR_END 0x09FFFFFF +/* clang-format on */ + +#endif -- 2.19.1
'host' type cpu is set isa to RV32 or RV64 simply, more isa info will obtain from KVM in kvm_arch_init_vcpu() Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/cpu.c | 15 +++++++++++++++ target/riscv/cpu.h | 1 + 2 files changed, 16 insertions(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 39382b1d62..f17cf8083d 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -213,6 +213,18 @@ static void rv32_imafcu_nommu_cpu_init(Object *obj) } #endif +#if defined(CONFIG_KVM) +static void riscv_host_cpu_init(Object *obj) +{ + CPURISCVState *env = &RISCV_CPU(obj)->env; +#if defined(TARGET_RISCV32) + set_misa(env, MXL_RV32, 0); +#elif defined(TARGET_RISCV64) + set_misa(env, MXL_RV64, 0); +#endif +} +#endif + static ObjectClass *riscv_cpu_class_by_name(const char *cpu_model) { ObjectClass *oc; @@ -818,6 +830,9 @@ static const TypeInfo riscv_cpu_type_infos[] = { .class_init = riscv_cpu_class_init, }, DEFINE_CPU(TYPE_RISCV_CPU_ANY, riscv_any_cpu_init), +#if defined(CONFIG_KVM) + DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init), +#endif #if defined(TARGET_RISCV32) DEFINE_CPU(TYPE_RISCV_CPU_BASE32, rv32_base_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_IBEX, rv32_ibex_cpu_init), diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 491a38de01..5f54fae7cc 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -45,6 +45,7 @@ #define TYPE_RISCV_CPU_SIFIVE_E51 RISCV_CPU_TYPE_NAME("sifive-e51") #define TYPE_RISCV_CPU_SIFIVE_U34 RISCV_CPU_TYPE_NAME("sifive-u34") #define TYPE_RISCV_CPU_SIFIVE_U54 RISCV_CPU_TYPE_NAME("sifive-u54") +#define TYPE_RISCV_CPU_HOST RISCV_CPU_TYPE_NAME("host") #if defined(TARGET_RISCV32) # define TYPE_RISCV_CPU_BASE TYPE_RISCV_CPU_BASE32 -- 2.19.1
Add kvm_riscv_get/put_regs_timer to synchronize virtual time context from KVM. To set register of RISCV_TIMER_REG(state) will occur a error from KVM on kvm_timer_state == 0. It's better to adapt in KVM, but it doesn't matter that adaping in QEMU. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> --- target/riscv/cpu.h | 7 +++++ target/riscv/kvm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 5f54fae7cc..9eceded96c 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -261,6 +261,13 @@ struct CPURISCVState { hwaddr kernel_addr; hwaddr fdt_addr; + + /* kvm timer */ + bool kvm_timer_dirty; + uint64_t kvm_timer_time; + uint64_t kvm_timer_compare; + uint64_t kvm_timer_state; + uint64_t kvm_timer_frequency; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index ded2a8c29d..b1f1d55f29 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -40,6 +40,7 @@ #include "kvm_riscv.h" #include "sbi_ecall_interface.h" #include "chardev/char-fe.h" +#include "migration/migration.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) { @@ -64,6 +65,9 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx #define RISCV_CSR_REG(env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_CSR, \ KVM_REG_RISCV_CSR_REG(name)) +#define RISCV_TIMER_REG(env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_TIMER, \ + KVM_REG_RISCV_TIMER_REG(name)) + #define RISCV_FP_F_REG(env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_F, idx) #define RISCV_FP_D_REG(env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_D, idx) @@ -84,6 +88,22 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx } \ } while(0) +#define KVM_RISCV_GET_TIMER(cs, env, name, reg) \ + do { \ + int ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(env, name), ®); \ + if (ret) { \ + abort(); \ + } \ + } while(0) + +#define KVM_RISCV_SET_TIMER(cs, env, name, reg) \ + do { \ + int ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(env, time), ®); \ + if (ret) { \ + abort(); \ + } \ + } while (0) + static int kvm_riscv_get_regs_core(CPUState *cs) { int ret = 0; @@ -235,6 +255,58 @@ static int kvm_riscv_put_regs_fp(CPUState *cs) return ret; } +static void kvm_riscv_get_regs_timer(CPUState *cs) +{ + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (env->kvm_timer_dirty) { + return; + } + + KVM_RISCV_GET_TIMER(cs, env, time, env->kvm_timer_time); + KVM_RISCV_GET_TIMER(cs, env, compare, env->kvm_timer_compare); + KVM_RISCV_GET_TIMER(cs, env, state, env->kvm_timer_state); + KVM_RISCV_GET_TIMER(cs, env, frequency, env->kvm_timer_frequency); + + env->kvm_timer_dirty = true; +} + +static void kvm_riscv_put_regs_timer(CPUState *cs) +{ + uint64_t reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (!env->kvm_timer_dirty) { + return; + } + + KVM_RISCV_SET_TIMER(cs, env, time, env->kvm_timer_time); + KVM_RISCV_SET_TIMER(cs, env, compare, env->kvm_timer_compare); + + /* + * To set register of RISCV_TIMER_REG(state) will occur a error from KVM + * on env->kvm_timer_state == 0, It's better to adapt in KVM, but it + * doesn't matter that adaping in QEMU now. + * TODO If KVM changes, adapt here. + */ + if (env->kvm_timer_state) { + KVM_RISCV_SET_TIMER(cs, env, state, env->kvm_timer_state); + } + + /* + * For now, migration will not work between Hosts with different timer + * frequency. Therefore, we should check whether they are the same here + * during the migration. + */ + if (migration_is_running(migrate_get_current()->state)) { + KVM_RISCV_GET_TIMER(cs, env, frequency, reg); + if (reg != env->kvm_timer_frequency) { + error_report("Dst Hosts timer frequency != Src Hosts"); + } + } + + env->kvm_timer_dirty = false; +} const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO -- 2.19.1
On Mon, Jan 10, 2022 at 11:54 AM Yifei Jiang via <qemu-devel@nongnu.org> wrote:
Add kvm_riscv_get/put_regs_timer to synchronize virtual time context from KVM.
To set register of RISCV_TIMER_REG(state) will occur a error from KVM on kvm_timer_state == 0. It's better to adapt in KVM, but it doesn't matter that adaping in QEMU.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Anup Patel <anup.patel@wdc.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com> Alistair
--- target/riscv/cpu.h | 7 +++++ target/riscv/kvm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+)
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 5f54fae7cc..9eceded96c 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -261,6 +261,13 @@ struct CPURISCVState {
hwaddr kernel_addr; hwaddr fdt_addr; + + /* kvm timer */ + bool kvm_timer_dirty; + uint64_t kvm_timer_time; + uint64_t kvm_timer_compare; + uint64_t kvm_timer_state; + uint64_t kvm_timer_frequency; };
OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index ded2a8c29d..b1f1d55f29 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -40,6 +40,7 @@ #include "kvm_riscv.h" #include "sbi_ecall_interface.h" #include "chardev/char-fe.h" +#include "migration/migration.h"
static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) { @@ -64,6 +65,9 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx #define RISCV_CSR_REG(env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_CSR, \ KVM_REG_RISCV_CSR_REG(name))
+#define RISCV_TIMER_REG(env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_TIMER, \ + KVM_REG_RISCV_TIMER_REG(name)) + #define RISCV_FP_F_REG(env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_F, idx)
#define RISCV_FP_D_REG(env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_D, idx) @@ -84,6 +88,22 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx } \ } while(0)
+#define KVM_RISCV_GET_TIMER(cs, env, name, reg) \ + do { \ + int ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(env, name), ®); \ + if (ret) { \ + abort(); \ + } \ + } while(0) + +#define KVM_RISCV_SET_TIMER(cs, env, name, reg) \ + do { \ + int ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(env, time), ®); \ + if (ret) { \ + abort(); \ + } \ + } while (0) + static int kvm_riscv_get_regs_core(CPUState *cs) { int ret = 0; @@ -235,6 +255,58 @@ static int kvm_riscv_put_regs_fp(CPUState *cs) return ret; }
+static void kvm_riscv_get_regs_timer(CPUState *cs) +{ + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (env->kvm_timer_dirty) { + return; + } + + KVM_RISCV_GET_TIMER(cs, env, time, env->kvm_timer_time); + KVM_RISCV_GET_TIMER(cs, env, compare, env->kvm_timer_compare); + KVM_RISCV_GET_TIMER(cs, env, state, env->kvm_timer_state); + KVM_RISCV_GET_TIMER(cs, env, frequency, env->kvm_timer_frequency); + + env->kvm_timer_dirty = true; +} + +static void kvm_riscv_put_regs_timer(CPUState *cs) +{ + uint64_t reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (!env->kvm_timer_dirty) { + return; + } + + KVM_RISCV_SET_TIMER(cs, env, time, env->kvm_timer_time); + KVM_RISCV_SET_TIMER(cs, env, compare, env->kvm_timer_compare); + + /* + * To set register of RISCV_TIMER_REG(state) will occur a error from KVM + * on env->kvm_timer_state == 0, It's better to adapt in KVM, but it + * doesn't matter that adaping in QEMU now. + * TODO If KVM changes, adapt here. + */ + if (env->kvm_timer_state) { + KVM_RISCV_SET_TIMER(cs, env, state, env->kvm_timer_state); + } + + /* + * For now, migration will not work between Hosts with different timer + * frequency. Therefore, we should check whether they are the same here + * during the migration. + */ + if (migration_is_running(migrate_get_current()->state)) { + KVM_RISCV_GET_TIMER(cs, env, frequency, reg); + if (reg != env->kvm_timer_frequency) { + error_report("Dst Hosts timer frequency != Src Hosts"); + } + } + + env->kvm_timer_dirty = false; +}
const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO -- 2.19.1
We hope that virtual time adjusts with vm state changing. When a vm is stopped, guest virtual time should stop counting and kvm_timer should be stopped. When the vm is resumed, guest virtual time should continue to count and kvm_timer should be restored. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/kvm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index b1f1d55f29..8d94b1c6a9 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -41,6 +41,7 @@ #include "sbi_ecall_interface.h" #include "chardev/char-fe.h" #include "migration/migration.h" +#include "sysemu/runstate.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) { @@ -377,6 +378,17 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu) return cpu->cpu_index; } +static void kvm_riscv_vm_state_change(void *opaque, bool running, RunState state) +{ + CPUState *cs = opaque; + + if (running) { + kvm_riscv_put_regs_timer(cs); + } else { + kvm_riscv_get_regs_timer(cs); + } +} + void kvm_arch_init_irq_routing(KVMState *s) { } @@ -389,6 +401,8 @@ int kvm_arch_init_vcpu(CPUState *cs) CPURISCVState *env = &cpu->env; uint64_t id; + qemu_add_vm_change_state_handler(kvm_riscv_vm_state_change, cs); + id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa)); ret = kvm_get_one_reg(cs, id, &isa); if (ret) { -- 2.19.1
Add virtual time context description to vmstate_kvmtimer. After cpu being loaded, virtual time context is updated to KVM. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Mingwang Li <limingwang@huawei.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/machine.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/target/riscv/machine.c b/target/riscv/machine.c index ad8248ebfd..95eb82792a 100644 --- a/target/riscv/machine.c +++ b/target/riscv/machine.c @@ -164,6 +164,35 @@ static const VMStateDescription vmstate_pointermasking = { } }; +static bool kvmtimer_needed(void *opaque) +{ + return kvm_enabled(); +} + +static int cpu_post_load(void *opaque, int version_id) +{ + RISCVCPU *cpu = opaque; + CPURISCVState *env = &cpu->env; + + env->kvm_timer_dirty = true; + return 0; +} + +static const VMStateDescription vmstate_kvmtimer = { + .name = "cpu/kvmtimer", + .version_id = 1, + .minimum_version_id = 1, + .needed = kvmtimer_needed, + .post_load = cpu_post_load, + .fields = (VMStateField[]) { + VMSTATE_UINT64(env.kvm_timer_time, RISCVCPU), + VMSTATE_UINT64(env.kvm_timer_compare, RISCVCPU), + VMSTATE_UINT64(env.kvm_timer_state, RISCVCPU), + + VMSTATE_END_OF_LIST() + } +}; + const VMStateDescription vmstate_riscv_cpu = { .name = "cpu", .version_id = 3, @@ -218,6 +247,7 @@ const VMStateDescription vmstate_riscv_cpu = { &vmstate_hyper, &vmstate_vector, &vmstate_pointermasking, + &vmstate_kvmtimer, NULL } }; -- 2.19.1
participants (3)
-
Alistair Francis -
Jiangyifei -
Yifei Jiang