[PATCH RFC v4 00/15] Add riscv kvm accel support
This series adds both riscv32 and riscv64 kvm support, and implements migration based on riscv. It is based on temporarily unaccepted kvm: https://github.com/kvm-riscv/linux (lastest version v15). This series depends on above pending changes which haven't yet been accepted, so this QEMU patch series is treated as RFC patches until that dependency has been dealt with. Compared to RFC v3, the time scaling is supported in this series. The new feature also requires the following patches: [1] Bugfix in kvm v15 https://lkml.org/lkml/2020/11/30/245 [2] kvm patches: [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Several steps to use this: 1. Build emulation $ ./configure --target-list=riscv64-softmmu $ make -j$(nproc) 2. Build kernel https://github.com/kvm-riscv/linux 3. Build QEMU VM Cross built in riscv toolchain. $ PKG_CONFIG_LIBDIR=<toolchain pkgconfig path> $ export PKG_CONFIG_SYSROOT_DIR=<toolchain sysroot path> $ ./configure --target-list=riscv64-softmmu --enable-kvm \ --cross-prefix=riscv64-linux-gnu- --disable-libiscsi --disable-glusterfs \ --disable-libusb --disable-usb-redir --audio-drv-list= --disable-opengl \ --disable-libxml2 $ make -j$(nproc) 4. Start emulation $ ./qemu-system-riscv64 -M virt -m 4096M -cpu rv64,x-h=true -nographic \ -name guest=riscv-hyp,debug-threads=on \ -smp 4 \ -bios ./fw_jump.bin \ -kernel ./Image \ -drive file=./hyp.img,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -append "root=/dev/vda rw console=ttyS0 earlycon=sbi" 5. Start kvm-acceled QEMU VM in emulation $ ./qemu-system-riscv64 -M virt,accel=kvm -m 1024M -cpu host -nographic \ -name guest=riscv-guset \ -smp 2 \ -bios none \ -kernel ./Image \ -drive file=./guest.img,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -append "root=/dev/vda rw console=ttyS0 earlycon=sbi" Changes since RFC v3 - Rebase on QEMU v5.2.0-rc2 and kvm-riscv linux v15. - Add time scaling support(New patches 13, 14 and 15). - Fix the bug that guest vm can't reboot. Changes since RFC v2 - Fix checkpatch error at target/riscv/sbi_ecall_interface.h. - Add riscv migration support. Changes since RFC v1 - Add separate SBI ecall interface header. - Add riscv32 kvm accel support. Yifei Jiang (15): linux-header: Update linux/kvm.h target/riscv: Add target/riscv/kvm.c to place the public kvm interface target/riscv: Implement function kvm_arch_init_vcpu target/riscv: Implement kvm_arch_get_registers target/riscv: Implement kvm_arch_put_registers target/riscv: Support start kernel directly by KVM hw/riscv: PLIC update external interrupt by KVM when kvm enabled target/riscv: Handle KVM_EXIT_RISCV_SBI exit target/riscv: Add host cpu type target/riscv: Add kvm_riscv_get/put_regs_timer target/riscv: Implement virtual time adjusting with vm state changing target/riscv: Support virtual time context synchronization target/riscv: Introduce dynamic time frequency for virt machine target/riscv: Synchronize vcpu's frequency with KVM target/riscv: Add time frequency migration support hw/intc/sifive_plic.c | 31 +- hw/riscv/virt.c | 26 +- linux-headers/linux/kvm.h | 8 + meson.build | 2 + target/riscv/cpu.c | 13 + target/riscv/cpu.h | 12 + target/riscv/kvm.c | 617 +++++++++++++++++++++++++++++ target/riscv/kvm_riscv.h | 25 ++ target/riscv/machine.c | 23 ++ target/riscv/meson.build | 1 + target/riscv/sbi_ecall_interface.h | 72 ++++ 11 files changed, 817 insertions(+), 13 deletions(-) create mode 100644 target/riscv/kvm.c create mode 100644 target/riscv/kvm_riscv.h create mode 100644 target/riscv/sbi_ecall_interface.h -- 2.19.1
Update linux-headers/linux/kvm.h from https://github.com/avpatel/linux/tree/riscv_kvm_v15. Only use this header file, so here do not update all linux headers by update-linux-headers.sh before above KVM series is accepted. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- linux-headers/linux/kvm.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index 56ce14ad20..ddeedcf3df 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -250,6 +250,7 @@ struct kvm_hyperv_exit { #define KVM_EXIT_ARM_NISV 28 #define KVM_EXIT_X86_RDMSR 29 #define KVM_EXIT_X86_WRMSR 30 +#define KVM_EXIT_RISCV_SBI 31 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -426,6 +427,13 @@ struct kvm_run { __u32 index; /* kernel -> user */ __u64 data; /* kernel <-> user */ } msr; + /* KVM_EXIT_RISCV_SBI */ + struct { + unsigned long extension_id; + unsigned long function_id; + unsigned long args[6]; + unsigned long ret[2]; + } riscv_sbi; /* Fix the size of the union. */ char padding[256]; }; -- 2.19.1
Add target/riscv/kvm.c to place kvm_arch_* function needed by kvm/kvm-all.c. Meanwhile, add kvm support in meson.build file. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- meson.build | 2 + target/riscv/kvm.c | 128 +++++++++++++++++++++++++++++++++++++++ target/riscv/meson.build | 1 + 3 files changed, 131 insertions(+) create mode 100644 target/riscv/kvm.c diff --git a/meson.build b/meson.build index e3386196ba..14728b77ab 100644 --- a/meson.build +++ b/meson.build @@ -69,6 +69,8 @@ elif cpu in ['ppc', 'ppc64'] kvm_targets = ['ppc-softmmu', 'ppc64-softmmu'] elif cpu in ['mips', 'mips64'] kvm_targets = ['mips-softmmu', 'mipsel-softmmu', 'mips64-softmmu', 'mips64el-softmmu'] +elif cpu in ['riscv32', 'riscv64'] + kvm_targets = ['riscv32-softmmu', 'riscv64-softmmu'] else kvm_targets = [] endif diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c new file mode 100644 index 0000000000..8c386d9acf --- /dev/null +++ b/target/riscv/kvm.c @@ -0,0 +1,128 @@ +/* + * RISC-V implementation of KVM hooks + * + * Copyright (c) 2020 Huawei Technologies Co., Ltd + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#include "qemu/osdep.h" +#include <sys/ioctl.h> + +#include <linux/kvm.h> + +#include "qemu-common.h" +#include "qemu/timer.h" +#include "qemu/error-report.h" +#include "qemu/main-loop.h" +#include "sysemu/sysemu.h" +#include "sysemu/kvm.h" +#include "sysemu/kvm_int.h" +#include "cpu.h" +#include "trace.h" +#include "hw/pci/pci.h" +#include "exec/memattrs.h" +#include "exec/address-spaces.h" +#include "hw/boards.h" +#include "hw/irq.h" +#include "qemu/log.h" +#include "hw/loader.h" + +const KVMCapabilityInfo kvm_arch_required_capabilities[] = { + KVM_CAP_LAST_INFO +}; + +int kvm_arch_get_registers(CPUState *cs) +{ + return 0; +} + +int kvm_arch_put_registers(CPUState *cs, int level) +{ + return 0; +} + +int kvm_arch_release_virq_post(int virq) +{ + return 0; +} + +int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route, + uint64_t address, uint32_t data, PCIDevice *dev) +{ + return 0; +} + +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + return 0; +} + +unsigned long kvm_arch_vcpu_id(CPUState *cpu) +{ + return cpu->cpu_index; +} + +void kvm_arch_init_irq_routing(KVMState *s) +{ +} + +int kvm_arch_init_vcpu(CPUState *cs) +{ + return 0; +} + +int kvm_arch_msi_data_to_gsi(uint32_t data) +{ + abort(); +} + +int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route, + int vector, PCIDevice *dev) +{ + return 0; +} + +int kvm_arch_init(MachineState *ms, KVMState *s) +{ + return 0; +} + +int kvm_arch_irqchip_create(KVMState *s) +{ + return 0; +} + +int kvm_arch_process_async_events(CPUState *cs) +{ + return 0; +} + +void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run) +{ +} + +MemTxAttrs kvm_arch_post_run(CPUState *cs, struct kvm_run *run) +{ + return MEMTXATTRS_UNSPECIFIED; +} + +bool kvm_arch_stop_on_emulation_error(CPUState *cs) +{ + return true; +} + +int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) +{ + return 0; +} diff --git a/target/riscv/meson.build b/target/riscv/meson.build index 14a5c62dac..12e9c074af 100644 --- a/target/riscv/meson.build +++ b/target/riscv/meson.build @@ -23,6 +23,7 @@ riscv_ss.add(files( 'vector_helper.c', 'translate.c', )) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files( -- 2.19.1
Get isa info from kvm while kvm init. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/kvm.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 8c386d9acf..86660ba81b 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -38,6 +38,18 @@ #include "qemu/log.h" #include "hw/loader.h" +static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) +{ + __u64 id = KVM_REG_RISCV | type | idx; + +#if defined(TARGET_RISCV32) + id |= KVM_REG_SIZE_U32; +#elif defined(TARGET_RISCV64) + id |= KVM_REG_SIZE_U64; +#endif + return id; +} + const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO }; @@ -79,7 +91,20 @@ void kvm_arch_init_irq_routing(KVMState *s) int kvm_arch_init_vcpu(CPUState *cs) { - return 0; + int ret = 0; + target_ulong isa; + RISCVCPU *cpu = RISCV_CPU(cs); + CPURISCVState *env = &cpu->env; + __u64 id; + + id = kvm_riscv_reg_id(KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa)); + ret = kvm_get_one_reg(cs, id, &isa); + if (ret) { + return ret; + } + env->misa = isa; + + return ret; } int kvm_arch_msi_data_to_gsi(uint32_t data) -- 2.19.1
On Thu, Dec 3, 2020 at 4:55 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Get isa info from kvm while kvm init.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/kvm.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 8c386d9acf..86660ba81b 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -38,6 +38,18 @@ #include "qemu/log.h" #include "hw/loader.h"
+static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) +{ + __u64 id = KVM_REG_RISCV | type | idx; + +#if defined(TARGET_RISCV32) + id |= KVM_REG_SIZE_U32; +#elif defined(TARGET_RISCV64) + id |= KVM_REG_SIZE_U64; +#endif
There is a series on list (I'll send a v2 out later today) that starts to remove these #ifdef for the RISC-V XLEN. Next time you rebase it would be great if you can use that and hopefully remove this. Alistair
+ return id; +} + const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO }; @@ -79,7 +91,20 @@ void kvm_arch_init_irq_routing(KVMState *s)
int kvm_arch_init_vcpu(CPUState *cs) { - return 0; + int ret = 0; + target_ulong isa; + RISCVCPU *cpu = RISCV_CPU(cs); + CPURISCVState *env = &cpu->env; + __u64 id; + + id = kvm_riscv_reg_id(KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa)); + ret = kvm_get_one_reg(cs, id, &isa); + if (ret) { + return ret; + } + env->misa = isa; + + return ret; }
int kvm_arch_msi_data_to_gsi(uint32_t data) -- 2.19.1
Get GPR CSR and FP registers from kvm by KVM_GET_ONE_REG ioctl. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/kvm.c | 150 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 149 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 86660ba81b..e679619e79 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -50,13 +50,161 @@ static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) return id; } +#define RISCV_CORE_REG(name) kvm_riscv_reg_id(KVM_REG_RISCV_CORE, \ + KVM_REG_RISCV_CORE_REG(name)) + +#define RISCV_CSR_REG(name) kvm_riscv_reg_id(KVM_REG_RISCV_CSR, \ + KVM_REG_RISCV_CSR_REG(name)) + +#define RISCV_FP_F_REG(idx) kvm_riscv_reg_id(KVM_REG_RISCV_FP_F, idx) + +#define RISCV_FP_D_REG(idx) kvm_riscv_reg_id(KVM_REG_RISCV_FP_D, idx) + +static int kvm_riscv_get_regs_core(CPUState *cs) +{ + int ret = 0; + int i; + target_ulong reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + ret = kvm_get_one_reg(cs, RISCV_CORE_REG(regs.pc), ®); + if (ret) { + return ret; + } + env->pc = reg; + + for (i = 1; i < 32; i++) { + __u64 id = kvm_riscv_reg_id(KVM_REG_RISCV_CORE, i); + ret = kvm_get_one_reg(cs, id, ®); + if (ret) { + return ret; + } + env->gpr[i] = reg; + } + + return ret; +} + +static int kvm_riscv_get_regs_csr(CPUState *cs) +{ + int ret = 0; + target_ulong reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(sstatus), ®); + if (ret) { + return ret; + } + env->mstatus = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(sie), ®); + if (ret) { + return ret; + } + env->mie = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(stvec), ®); + if (ret) { + return ret; + } + env->stvec = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(sscratch), ®); + if (ret) { + return ret; + } + env->sscratch = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(sepc), ®); + if (ret) { + return ret; + } + env->sepc = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(scause), ®); + if (ret) { + return ret; + } + env->scause = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(stval), ®); + if (ret) { + return ret; + } + env->sbadaddr = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(sip), ®); + if (ret) { + return ret; + } + env->mip = reg; + + ret = kvm_get_one_reg(cs, RISCV_CSR_REG(satp), ®); + if (ret) { + return ret; + } + env->satp = reg; + + return ret; +} + +static int kvm_riscv_get_regs_fp(CPUState *cs) +{ + int ret = 0; + int i; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (riscv_has_ext(env, RVD)) { + uint64_t reg; + for (i = 0; i < 32; i++) { + ret = kvm_get_one_reg(cs, RISCV_FP_D_REG(i), ®); + if (ret) { + return ret; + } + env->fpr[i] = reg; + } + return ret; + } + + if (riscv_has_ext(env, RVF)) { + uint32_t reg; + for (i = 0; i < 32; i++) { + ret = kvm_get_one_reg(cs, RISCV_FP_F_REG(i), ®); + if (ret) { + return ret; + } + env->fpr[i] = reg; + } + return ret; + } + + return ret; +} + const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO }; int kvm_arch_get_registers(CPUState *cs) { - return 0; + int ret = 0; + + ret = kvm_riscv_get_regs_core(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_get_regs_csr(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_get_regs_fp(cs); + if (ret) { + return ret; + } + + return ret; } int kvm_arch_put_registers(CPUState *cs, int level) -- 2.19.1
Put GPR CSR and FP registers to kvm by KVM_SET_ONE_REG ioctl Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/kvm.c | 142 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 141 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index e679619e79..8b206ce99c 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -85,6 +85,31 @@ static int kvm_riscv_get_regs_core(CPUState *cs) return ret; } +static int kvm_riscv_put_regs_core(CPUState *cs) +{ + int ret = 0; + int i; + target_ulong reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + reg = env->pc; + ret = kvm_set_one_reg(cs, RISCV_CORE_REG(regs.pc), ®); + if (ret) { + return ret; + } + + for (i = 1; i < 32; i++) { + __u64 id = kvm_riscv_reg_id(KVM_REG_RISCV_CORE, i); + reg = env->gpr[i]; + ret = kvm_set_one_reg(cs, id, ®); + if (ret) { + return ret; + } + } + + return ret; +} + static int kvm_riscv_get_regs_csr(CPUState *cs) { int ret = 0; @@ -148,6 +173,70 @@ static int kvm_riscv_get_regs_csr(CPUState *cs) return ret; } +static int kvm_riscv_put_regs_csr(CPUState *cs) +{ + int ret = 0; + target_ulong reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + reg = env->mstatus; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(sstatus), ®); + if (ret) { + return ret; + } + + reg = env->mie; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(sie), ®); + if (ret) { + return ret; + } + + reg = env->stvec; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(stvec), ®); + if (ret) { + return ret; + } + + reg = env->sscratch; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(sscratch), ®); + if (ret) { + return ret; + } + + reg = env->sepc; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(sepc), ®); + if (ret) { + return ret; + } + + reg = env->scause; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(scause), ®); + if (ret) { + return ret; + } + + reg = env->sbadaddr; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(stval), ®); + if (ret) { + return ret; + } + + reg = env->mip; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(sip), ®); + if (ret) { + return ret; + } + + reg = env->satp; + ret = kvm_set_one_reg(cs, RISCV_CSR_REG(satp), ®); + if (ret) { + return ret; + } + + return ret; +} + + static int kvm_riscv_get_regs_fp(CPUState *cs) { int ret = 0; @@ -181,6 +270,40 @@ static int kvm_riscv_get_regs_fp(CPUState *cs) return ret; } +static int kvm_riscv_put_regs_fp(CPUState *cs) +{ + int ret = 0; + int i; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (riscv_has_ext(env, RVD)) { + uint64_t reg; + for (i = 0; i < 32; i++) { + reg = env->fpr[i]; + ret = kvm_set_one_reg(cs, RISCV_FP_D_REG(i), ®); + if (ret) { + return ret; + } + } + return ret; + } + + if (riscv_has_ext(env, RVF)) { + uint32_t reg; + for (i = 0; i < 32; i++) { + reg = env->fpr[i]; + ret = kvm_set_one_reg(cs, RISCV_FP_F_REG(i), ®); + if (ret) { + return ret; + } + } + return ret; + } + + return ret; +} + + const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO }; @@ -209,7 +332,24 @@ int kvm_arch_get_registers(CPUState *cs) int kvm_arch_put_registers(CPUState *cs, int level) { - return 0; + int ret = 0; + + ret = kvm_riscv_put_regs_core(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_put_regs_csr(cs); + if (ret) { + return ret; + } + + ret = kvm_riscv_put_regs_fp(cs); + if (ret) { + return ret; + } + + return ret; } int kvm_arch_release_virq_post(int virq) -- 2.19.1
Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. In addition, add kvm_riscv.h to place riscv specific interface. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/riscv/virt.c | 8 ++++++++ target/riscv/cpu.c | 4 ++++ target/riscv/cpu.h | 3 +++ target/riscv/kvm.c | 15 +++++++++++++++ target/riscv/kvm_riscv.h | 24 ++++++++++++++++++++++++ 5 files changed, 54 insertions(+) create mode 100644 target/riscv/kvm_riscv.h diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 25cea7aa67..47b7018193 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -42,6 +42,7 @@ #include "sysemu/sysemu.h" #include "hw/pci/pci.h" #include "hw/pci-host/gpex.h" +#include "sysemu/kvm.h" #if defined(TARGET_RISCV32) # define BIOS_FILENAME "opensbi-riscv32-generic-fw_dynamic.bin" @@ -511,6 +512,7 @@ static void virt_machine_init(MachineState *machine) uint64_t kernel_entry; DeviceState *mmio_plic, *virtio_plic, *pcie_plic; int i, j, base_hartid, hart_count; + CPUState *cs; /* Check socket count limit */ if (VIRT_SOCKETS_MAX < riscv_socket_count(machine)) { @@ -660,6 +662,12 @@ static void virt_machine_init(MachineState *machine) virt_memmap[VIRT_MROM].size, kernel_entry, fdt_load_addr, s->fdt); + for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) { + RISCVCPU *riscv_cpu = RISCV_CPU(cs); + riscv_cpu->env.kernel_addr = kernel_entry; + riscv_cpu->env.fdt_addr = fdt_load_addr; + } + /* SiFive Test MMIO device */ sifive_test_create(memmap[VIRT_TEST].base); diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 6a0264fc6b..faee98a58c 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -29,6 +29,7 @@ #include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "kvm_riscv.h" /* RISC-V CPU definitions */ @@ -330,6 +331,9 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index = EXCP_NONE; env->load_res = -1; set_default_nan_mode(1, &env->fp_status); +#ifdef CONFIG_KVM + kvm_riscv_reset_vcpu(cpu); +#endif } static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info *info) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index c0a326c843..ad1c90f798 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -233,6 +233,9 @@ struct CPURISCVState { /* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer; /* Internal timer */ + + hwaddr kernel_addr; + hwaddr fdt_addr; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 8b206ce99c..6250ca0c7d 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -37,6 +37,7 @@ #include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h" static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) { @@ -439,3 +440,17 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) { return 0; } + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu) +{ + CPURISCVState *env = &cpu->env; + + if (!kvm_enabled()) { + return; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */ + env->gpr[11] = cpu->env.fdt_addr; /* a1 */ + env->satp = 0; +} + diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h new file mode 100644 index 0000000000..f38c82bf59 --- /dev/null +++ b/target/riscv/kvm_riscv.h @@ -0,0 +1,24 @@ +/* + * QEMU KVM support -- RISC-V specific functions. + * + * Copyright (c) 2020 Huawei Technologies Co., Ltd + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu); + +#endif -- 2.19.1
On Thu, Dec 3, 2020 at 4:58 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. In addition, add kvm_riscv.h to place riscv specific interface.
This doesn't seem right. Why do we need to do this? Other architectures don't seem to do this. Writing to the CPU from the board like this looks fishy and probably breaks some QOM rules. Alistair
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/riscv/virt.c | 8 ++++++++ target/riscv/cpu.c | 4 ++++ target/riscv/cpu.h | 3 +++ target/riscv/kvm.c | 15 +++++++++++++++ target/riscv/kvm_riscv.h | 24 ++++++++++++++++++++++++ 5 files changed, 54 insertions(+) create mode 100644 target/riscv/kvm_riscv.h
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 25cea7aa67..47b7018193 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -42,6 +42,7 @@ #include "sysemu/sysemu.h" #include "hw/pci/pci.h" #include "hw/pci-host/gpex.h" +#include "sysemu/kvm.h"
#if defined(TARGET_RISCV32) # define BIOS_FILENAME "opensbi-riscv32-generic-fw_dynamic.bin" @@ -511,6 +512,7 @@ static void virt_machine_init(MachineState *machine) uint64_t kernel_entry; DeviceState *mmio_plic, *virtio_plic, *pcie_plic; int i, j, base_hartid, hart_count; + CPUState *cs;
/* Check socket count limit */ if (VIRT_SOCKETS_MAX < riscv_socket_count(machine)) { @@ -660,6 +662,12 @@ static void virt_machine_init(MachineState *machine) virt_memmap[VIRT_MROM].size, kernel_entry, fdt_load_addr, s->fdt);
+ for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) { + RISCVCPU *riscv_cpu = RISCV_CPU(cs); + riscv_cpu->env.kernel_addr = kernel_entry; + riscv_cpu->env.fdt_addr = fdt_load_addr; + } + /* SiFive Test MMIO device */ sifive_test_create(memmap[VIRT_TEST].base);
diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 6a0264fc6b..faee98a58c 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -29,6 +29,7 @@ #include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "kvm_riscv.h"
/* RISC-V CPU definitions */
@@ -330,6 +331,9 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index = EXCP_NONE; env->load_res = -1; set_default_nan_mode(1, &env->fp_status); +#ifdef CONFIG_KVM + kvm_riscv_reset_vcpu(cpu); +#endif }
static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info *info) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index c0a326c843..ad1c90f798 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -233,6 +233,9 @@ struct CPURISCVState {
/* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer; /* Internal timer */ + + hwaddr kernel_addr; + hwaddr fdt_addr; };
OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 8b206ce99c..6250ca0c7d 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -37,6 +37,7 @@ #include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h"
static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) { @@ -439,3 +440,17 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) { return 0; } + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu) +{ + CPURISCVState *env = &cpu->env; + + if (!kvm_enabled()) { + return; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */ + env->gpr[11] = cpu->env.fdt_addr; /* a1 */ + env->satp = 0; +} + diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h new file mode 100644 index 0000000000..f38c82bf59 --- /dev/null +++ b/target/riscv/kvm_riscv.h @@ -0,0 +1,24 @@ +/* + * QEMU KVM support -- RISC-V specific functions. + * + * Copyright (c) 2020 Huawei Technologies Co., Ltd + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu); + +#endif -- 2.19.1
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Wednesday, December 9, 2020 6:19 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; Zhangxiaofeng (F) <victor.zhangxiaofeng@huawei.com>; Sagar Karandikar <sagark@eecs.berkeley.edu>; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Bastian Koppelmann <kbastian@mail.uni-paderborn.de>; Anup Patel <anup.patel@wdc.com>; yinyipeng <yinyipeng1@huawei.com>; Alistair Francis <Alistair.Francis@wdc.com>; kvm-riscv@lists.infradead.org; Palmer Dabbelt <palmer@dabbelt.com>; dengkai (A) <dengkai1@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Zhanghailiang <zhang.zhanghailiang@huawei.com> Subject: Re: [PATCH RFC v4 06/15] target/riscv: Support start kernel directly by KVM
On Thu, Dec 3, 2020 at 4:58 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. In addition, add kvm_riscv.h to place riscv specific interface.
This doesn't seem right. Why do we need to do this? Other architectures don't seem to do this.
Writing to the CPU from the board like this looks fishy and probably breaks some QOM rules.
Alistair
Sorry for the delayed reply. We need to set the starting address of the kernel and fdt to vcpu, which is implemented by firmware bootloader under other architectures and RISC-V emulators. However, the RISC-V virtual machine does not have bootloader, so we boot the kernel directly. In the future, we will support firmware loading. Before supporting the firmware bootloader, we can add a public interface instead of modifying the CPU instance directly to comply with the QOM rules. Yifei
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/riscv/virt.c | 8 ++++++++ target/riscv/cpu.c | 4 ++++ target/riscv/cpu.h | 3 +++ target/riscv/kvm.c | 15 +++++++++++++++ target/riscv/kvm_riscv.h | 24 ++++++++++++++++++++++++ 5 files changed, 54 insertions(+) create mode 100644 target/riscv/kvm_riscv.h
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 25cea7aa67..47b7018193 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -42,6 +42,7 @@ #include "sysemu/sysemu.h" #include "hw/pci/pci.h" #include "hw/pci-host/gpex.h" +#include "sysemu/kvm.h"
#if defined(TARGET_RISCV32) # define BIOS_FILENAME "opensbi-riscv32-generic-fw_dynamic.bin" @@ -511,6 +512,7 @@ static void virt_machine_init(MachineState
*machine)
uint64_t kernel_entry; DeviceState *mmio_plic, *virtio_plic, *pcie_plic; int i, j, base_hartid, hart_count; + CPUState *cs;
/* Check socket count limit */ if (VIRT_SOCKETS_MAX < riscv_socket_count(machine)) { @@ -660,6 +662,12 @@ static void virt_machine_init(MachineState *machine) virt_memmap[VIRT_MROM].size,
kernel_entry,
fdt_load_addr, s->fdt);
+ for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) { + RISCVCPU *riscv_cpu = RISCV_CPU(cs); + riscv_cpu->env.kernel_addr = kernel_entry; + riscv_cpu->env.fdt_addr = fdt_load_addr; + } + /* SiFive Test MMIO device */ sifive_test_create(memmap[VIRT_TEST].base);
diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 6a0264fc6b..faee98a58c 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -29,6 +29,7 @@ #include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "kvm_riscv.h"
/* RISC-V CPU definitions */
@@ -330,6 +331,9 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index = EXCP_NONE; env->load_res = -1; set_default_nan_mode(1, &env->fp_status); +#ifdef CONFIG_KVM + kvm_riscv_reset_vcpu(cpu); +#endif }
static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info *info) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index c0a326c843..ad1c90f798 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -233,6 +233,9 @@ struct CPURISCVState {
/* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer; /* Internal timer */ + + hwaddr kernel_addr; + hwaddr fdt_addr; };
OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 8b206ce99c..6250ca0c7d 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -37,6 +37,7 @@ #include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h"
static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) { @@ -439,3 +440,17 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) { return 0; } + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu) { + CPURISCVState *env = &cpu->env; + + if (!kvm_enabled()) { + return; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */ + env->gpr[11] = cpu->env.fdt_addr; /* a1 */ + env->satp = 0; +} + diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h new file mode 100644 index 0000000000..f38c82bf59 --- /dev/null +++ b/target/riscv/kvm_riscv.h @@ -0,0 +1,24 @@ +/* + * QEMU KVM support -- RISC-V specific functions. + * + * Copyright (c) 2020 Huawei Technologies Co., Ltd + * + * This program is free software; you can redistribute it and/or +modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but +WITHOUT + * ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY
+or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public +License for + * more details. + * + * You should have received a copy of the GNU General Public License +along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu(RISCVCPU *cpu); + +#endif -- 2.19.1
Only support supervisor external interrupt currently. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/intc/sifive_plic.c | 31 ++++++++++++++++++++++--------- target/riscv/kvm.c | 19 +++++++++++++++++++ target/riscv/kvm_riscv.h | 1 + 3 files changed, 42 insertions(+), 9 deletions(-) diff --git a/hw/intc/sifive_plic.c b/hw/intc/sifive_plic.c index 97a1a27a9a..a419ca3a3c 100644 --- a/hw/intc/sifive_plic.c +++ b/hw/intc/sifive_plic.c @@ -31,6 +31,8 @@ #include "target/riscv/cpu.h" #include "sysemu/sysemu.h" #include "migration/vmstate.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h" #define RISCV_DEBUG_PLIC 0 @@ -147,15 +149,26 @@ static void sifive_plic_update(SiFivePLICState *plic) continue; } int level = sifive_plic_irqs_pending(plic, addrid); - switch (mode) { - case PLICMode_M: - riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_MEIP, BOOL_TO_MASK(level)); - break; - case PLICMode_S: - riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_SEIP, BOOL_TO_MASK(level)); - break; - default: - break; + if (kvm_enabled()) { + if (mode == PLICMode_M) { + continue; + } +#ifdef CONFIG_KVM + kvm_riscv_set_irq(RISCV_CPU(cpu), IRQ_S_EXT, level); +#endif + } else { + switch (mode) { + case PLICMode_M: + riscv_cpu_update_mip(RISCV_CPU(cpu), + MIP_MEIP, BOOL_TO_MASK(level)); + break; + case PLICMode_S: + riscv_cpu_update_mip(RISCV_CPU(cpu), + MIP_SEIP, BOOL_TO_MASK(level)); + break; + default: + break; + } } } diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 6250ca0c7d..b01ff0754c 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -454,3 +454,22 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) env->satp = 0; } +void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level) +{ + int ret; + unsigned virq = level ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET; + + if (irq != IRQ_S_EXT) { + return; + } + + if (!kvm_enabled()) { + return; + } + + ret = kvm_vcpu_ioctl(CPU(cpu), KVM_INTERRUPT, &virq); + if (ret < 0) { + perror("Set irq failed"); + abort(); + } +} diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h index f38c82bf59..ed281bdce0 100644 --- a/target/riscv/kvm_riscv.h +++ b/target/riscv/kvm_riscv.h @@ -20,5 +20,6 @@ #define QEMU_KVM_RISCV_H void kvm_riscv_reset_vcpu(RISCVCPU *cpu); +void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level); #endif -- 2.19.1
On Thu, Dec 3, 2020 at 4:47 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Only support supervisor external interrupt currently.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/intc/sifive_plic.c | 31 ++++++++++++++++++++++--------- target/riscv/kvm.c | 19 +++++++++++++++++++ target/riscv/kvm_riscv.h | 1 + 3 files changed, 42 insertions(+), 9 deletions(-)
diff --git a/hw/intc/sifive_plic.c b/hw/intc/sifive_plic.c index 97a1a27a9a..a419ca3a3c 100644 --- a/hw/intc/sifive_plic.c +++ b/hw/intc/sifive_plic.c @@ -31,6 +31,8 @@ #include "target/riscv/cpu.h" #include "sysemu/sysemu.h" #include "migration/vmstate.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h"
#define RISCV_DEBUG_PLIC 0
@@ -147,15 +149,26 @@ static void sifive_plic_update(SiFivePLICState *plic) continue; } int level = sifive_plic_irqs_pending(plic, addrid); - switch (mode) { - case PLICMode_M: - riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_MEIP, BOOL_TO_MASK(level)); - break; - case PLICMode_S: - riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_SEIP, BOOL_TO_MASK(level)); - break; - default: - break; + if (kvm_enabled()) { + if (mode == PLICMode_M) { + continue; + } +#ifdef CONFIG_KVM + kvm_riscv_set_irq(RISCV_CPU(cpu), IRQ_S_EXT, level); +#endif
What if kvm_enalbed() is true, but CONFIG_KVM isn't defined? Alistair
+ } else { + switch (mode) { + case PLICMode_M: + riscv_cpu_update_mip(RISCV_CPU(cpu), + MIP_MEIP, BOOL_TO_MASK(level)); + break; + case PLICMode_S: + riscv_cpu_update_mip(RISCV_CPU(cpu), + MIP_SEIP, BOOL_TO_MASK(level)); + break; + default: + break; + } } }
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 6250ca0c7d..b01ff0754c 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -454,3 +454,22 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) env->satp = 0; }
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level) +{ + int ret; + unsigned virq = level ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET; + + if (irq != IRQ_S_EXT) { + return; + } + + if (!kvm_enabled()) { + return; + } + + ret = kvm_vcpu_ioctl(CPU(cpu), KVM_INTERRUPT, &virq); + if (ret < 0) { + perror("Set irq failed"); + abort(); + } +} diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h index f38c82bf59..ed281bdce0 100644 --- a/target/riscv/kvm_riscv.h +++ b/target/riscv/kvm_riscv.h @@ -20,5 +20,6 @@ #define QEMU_KVM_RISCV_H
void kvm_riscv_reset_vcpu(RISCVCPU *cpu); +void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
#endif -- 2.19.1
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Wednesday, December 9, 2020 6:30 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; Zhangxiaofeng (F) <victor.zhangxiaofeng@huawei.com>; Sagar Karandikar <sagark@eecs.berkeley.edu>; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Bastian Koppelmann <kbastian@mail.uni-paderborn.de>; Anup Patel <anup.patel@wdc.com>; yinyipeng <yinyipeng1@huawei.com>; Alistair Francis <Alistair.Francis@wdc.com>; kvm-riscv@lists.infradead.org; Palmer Dabbelt <palmer@dabbelt.com>; dengkai (A) <dengkai1@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Zhanghailiang <zhang.zhanghailiang@huawei.com> Subject: Re: [PATCH RFC v4 07/15] hw/riscv: PLIC update external interrupt by KVM when kvm enabled
On Thu, Dec 3, 2020 at 4:47 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Only support supervisor external interrupt currently.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/intc/sifive_plic.c | 31 ++++++++++++++++++++++--------- target/riscv/kvm.c | 19 +++++++++++++++++++ target/riscv/kvm_riscv.h | 1 + 3 files changed, 42 insertions(+), 9 deletions(-)
diff --git a/hw/intc/sifive_plic.c b/hw/intc/sifive_plic.c index 97a1a27a9a..a419ca3a3c 100644 --- a/hw/intc/sifive_plic.c +++ b/hw/intc/sifive_plic.c @@ -31,6 +31,8 @@ #include "target/riscv/cpu.h" #include "sysemu/sysemu.h" #include "migration/vmstate.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h"
#define RISCV_DEBUG_PLIC 0
@@ -147,15 +149,26 @@ static void sifive_plic_update(SiFivePLICState *plic) continue; } int level = sifive_plic_irqs_pending(plic, addrid); - switch (mode) { - case PLICMode_M: - riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_MEIP,
BOOL_TO_MASK(level));
- break; - case PLICMode_S: - riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_SEIP, BOOL_TO_MASK(level)); - break; - default: - break; + if (kvm_enabled()) { + if (mode == PLICMode_M) { + continue; + } +#ifdef CONFIG_KVM + kvm_riscv_set_irq(RISCV_CPU(cpu), IRQ_S_EXT, level); +#endif
What if kvm_enalbed() is true, but CONFIG_KVM isn't defined?
Alistair
Impossible. It will cause compilation failure without CONFIG_KVM. We also introduce kvm-stub.c to solve the compilation failure like other architectures. We will introduce kvm-stub.c in next series. Yifei
+ } else { + switch (mode) { + case PLICMode_M: + riscv_cpu_update_mip(RISCV_CPU(cpu), + MIP_MEIP, BOOL_TO_MASK(level)); + break; + case PLICMode_S: + riscv_cpu_update_mip(RISCV_CPU(cpu), + MIP_SEIP, BOOL_TO_MASK(level)); + break; + default: + break; + } } }
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 6250ca0c7d..b01ff0754c 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -454,3 +454,22 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) env->satp = 0; }
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level) { + int ret; + unsigned virq = level ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET; + + if (irq != IRQ_S_EXT) { + return; + } + + if (!kvm_enabled()) { + return; + } + + ret = kvm_vcpu_ioctl(CPU(cpu), KVM_INTERRUPT, &virq); + if (ret < 0) { + perror("Set irq failed"); + abort(); + } +} diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h index f38c82bf59..ed281bdce0 100644 --- a/target/riscv/kvm_riscv.h +++ b/target/riscv/kvm_riscv.h @@ -20,5 +20,6 @@ #define QEMU_KVM_RISCV_H
void kvm_riscv_reset_vcpu(RISCVCPU *cpu); +void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
#endif -- 2.19.1
Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/kvm.c | 42 ++++++++++++++++- target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 target/riscv/sbi_ecall_interface.h diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index b01ff0754c..afd99b3315 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -38,6 +38,8 @@ #include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" +#include "sbi_ecall_interface.h" +#include "chardev/char-fe.h" static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) { @@ -436,9 +438,47 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs) return true; } +static int kvm_riscv_handle_sbi(struct kvm_run *run) +{ + int ret = 0; + unsigned char ch; + switch (run->riscv_sbi.extension_id) { + case SBI_EXT_0_1_CONSOLE_PUTCHAR: + ch = run->riscv_sbi.args[0]; + qemu_chr_fe_write(serial_hd(0)->be, &ch, sizeof(ch)); + break; + case SBI_EXT_0_1_CONSOLE_GETCHAR: + ret = qemu_chr_fe_read_all(serial_hd(0)->be, &ch, sizeof(ch)); + if (ret == sizeof(ch)) { + run->riscv_sbi.args[0] = ch; + } else { + run->riscv_sbi.args[0] = -1; + } + break; + default: + qemu_log_mask(LOG_UNIMP, + "%s: un-handled SBI EXIT, specific reasons is %lu\n", + __func__, run->riscv_sbi.extension_id); + ret = -1; + break; + } + return ret; +} + int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) { - return 0; + int ret = 0; + switch (run->exit_reason) { + case KVM_EXIT_RISCV_SBI: + ret = kvm_riscv_handle_sbi(run); + break; + default: + qemu_log_mask(LOG_UNIMP, "%s: un-handled exit reason %d\n", + __func__, run->exit_reason); + ret = -1; + break; + } + return ret; } void kvm_riscv_reset_vcpu(RISCVCPU *cpu) diff --git a/target/riscv/sbi_ecall_interface.h b/target/riscv/sbi_ecall_interface.h new file mode 100644 index 0000000000..fb1a3fa8f2 --- /dev/null +++ b/target/riscv/sbi_ecall_interface.h @@ -0,0 +1,72 @@ +/* + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2019 Western Digital Corporation or its affiliates. + * + * Authors: + * Anup Patel <anup.patel@wdc.com> + */ + +#ifndef __SBI_ECALL_INTERFACE_H__ +#define __SBI_ECALL_INTERFACE_H__ + +/* clang-format off */ + +/* SBI Extension IDs */ +#define SBI_EXT_0_1_SET_TIMER 0x0 +#define SBI_EXT_0_1_CONSOLE_PUTCHAR 0x1 +#define SBI_EXT_0_1_CONSOLE_GETCHAR 0x2 +#define SBI_EXT_0_1_CLEAR_IPI 0x3 +#define SBI_EXT_0_1_SEND_IPI 0x4 +#define SBI_EXT_0_1_REMOTE_FENCE_I 0x5 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA 0x6 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID 0x7 +#define SBI_EXT_0_1_SHUTDOWN 0x8 +#define SBI_EXT_BASE 0x10 +#define SBI_EXT_TIME 0x54494D45 +#define SBI_EXT_IPI 0x735049 +#define SBI_EXT_RFENCE 0x52464E43 +#define SBI_EXT_HSM 0x48534D + +/* SBI function IDs for BASE extension*/ +#define SBI_EXT_BASE_GET_SPEC_VERSION 0x0 +#define SBI_EXT_BASE_GET_IMP_ID 0x1 +#define SBI_EXT_BASE_GET_IMP_VERSION 0x2 +#define SBI_EXT_BASE_PROBE_EXT 0x3 +#define SBI_EXT_BASE_GET_MVENDORID 0x4 +#define SBI_EXT_BASE_GET_MARCHID 0x5 +#define SBI_EXT_BASE_GET_MIMPID 0x6 + +/* SBI function IDs for TIME extension*/ +#define SBI_EXT_TIME_SET_TIMER 0x0 + +/* SBI function IDs for IPI extension*/ +#define SBI_EXT_IPI_SEND_IPI 0x0 + +/* SBI function IDs for RFENCE extension*/ +#define SBI_EXT_RFENCE_REMOTE_FENCE_I 0x0 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA 0x1 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID 0x2 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA 0x3 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID 0x4 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA 0x5 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID 0x6 + +/* SBI function IDs for HSM extension */ +#define SBI_EXT_HSM_HART_START 0x0 +#define SBI_EXT_HSM_HART_STOP 0x1 +#define SBI_EXT_HSM_HART_GET_STATUS 0x2 + +#define SBI_HSM_HART_STATUS_STARTED 0x0 +#define SBI_HSM_HART_STATUS_STOPPED 0x1 +#define SBI_HSM_HART_STATUS_START_PENDING 0x2 +#define SBI_HSM_HART_STATUS_STOP_PENDING 0x3 + +#define SBI_SPEC_VERSION_MAJOR_OFFSET 24 +#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f +#define SBI_SPEC_VERSION_MINOR_MASK 0xffffff +#define SBI_EXT_VENDOR_START 0x09000000 +#define SBI_EXT_VENDOR_END 0x09FFFFFF +/* clang-format on */ + +#endif -- 2.19.1
Currently, host cpu is inherited simply. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/cpu.c | 6 ++++++ target/riscv/cpu.h | 1 + 2 files changed, 7 insertions(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index faee98a58c..439dc89ee7 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -186,6 +186,10 @@ static void rv32_imafcu_nommu_cpu_init(Object *obj) #endif +static void riscv_host_cpu_init(Object *obj) +{ +} + static ObjectClass *riscv_cpu_class_by_name(const char *cpu_model) { ObjectClass *oc; @@ -641,10 +645,12 @@ static const TypeInfo riscv_cpu_type_infos[] = { DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E31, rvxx_sifive_e_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E34, rv32_imafcu_nommu_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_U34, rvxx_sifive_u_cpu_init), + DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init), #elif defined(TARGET_RISCV64) DEFINE_CPU(TYPE_RISCV_CPU_BASE64, riscv_base_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E51, rvxx_sifive_e_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_U54, rvxx_sifive_u_cpu_init), + DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init), #endif }; diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index ad1c90f798..4288898019 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -43,6 +43,7 @@ #define TYPE_RISCV_CPU_SIFIVE_E51 RISCV_CPU_TYPE_NAME("sifive-e51") #define TYPE_RISCV_CPU_SIFIVE_U34 RISCV_CPU_TYPE_NAME("sifive-u34") #define TYPE_RISCV_CPU_SIFIVE_U54 RISCV_CPU_TYPE_NAME("sifive-u54") +#define TYPE_RISCV_CPU_HOST RISCV_CPU_TYPE_NAME("host") #define RV32 ((target_ulong)1 << (TARGET_LONG_BITS - 2)) #define RV64 ((target_ulong)2 << (TARGET_LONG_BITS - 2)) -- 2.19.1
On Thu, Dec 3, 2020 at 4:55 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Currently, host cpu is inherited simply.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/cpu.c | 6 ++++++ target/riscv/cpu.h | 1 + 2 files changed, 7 insertions(+)
diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index faee98a58c..439dc89ee7 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -186,6 +186,10 @@ static void rv32_imafcu_nommu_cpu_init(Object *obj)
#endif
+static void riscv_host_cpu_init(Object *obj) +{ +} + static ObjectClass *riscv_cpu_class_by_name(const char *cpu_model) { ObjectClass *oc; @@ -641,10 +645,12 @@ static const TypeInfo riscv_cpu_type_infos[] = { DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E31, rvxx_sifive_e_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E34, rv32_imafcu_nommu_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_U34, rvxx_sifive_u_cpu_init), + DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init), #elif defined(TARGET_RISCV64) DEFINE_CPU(TYPE_RISCV_CPU_BASE64, riscv_base_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E51, rvxx_sifive_e_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_U54, rvxx_sifive_u_cpu_init), + DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init),
Shouldn't this only be included if KVM is configured? Also it should be shared between RV32 and RV64. Alistair
#endif };
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index ad1c90f798..4288898019 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -43,6 +43,7 @@ #define TYPE_RISCV_CPU_SIFIVE_E51 RISCV_CPU_TYPE_NAME("sifive-e51") #define TYPE_RISCV_CPU_SIFIVE_U34 RISCV_CPU_TYPE_NAME("sifive-u34") #define TYPE_RISCV_CPU_SIFIVE_U54 RISCV_CPU_TYPE_NAME("sifive-u54") +#define TYPE_RISCV_CPU_HOST RISCV_CPU_TYPE_NAME("host")
#define RV32 ((target_ulong)1 << (TARGET_LONG_BITS - 2)) #define RV64 ((target_ulong)2 << (TARGET_LONG_BITS - 2)) -- 2.19.1
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Wednesday, December 9, 2020 6:22 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; Zhangxiaofeng (F) <victor.zhangxiaofeng@huawei.com>; Sagar Karandikar <sagark@eecs.berkeley.edu>; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Bastian Koppelmann <kbastian@mail.uni-paderborn.de>; Anup Patel <anup.patel@wdc.com>; yinyipeng <yinyipeng1@huawei.com>; Alistair Francis <Alistair.Francis@wdc.com>; kvm-riscv@lists.infradead.org; Palmer Dabbelt <palmer@dabbelt.com>; dengkai (A) <dengkai1@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Zhanghailiang <zhang.zhanghailiang@huawei.com> Subject: Re: [PATCH RFC v4 09/15] target/riscv: Add host cpu type
On Thu, Dec 3, 2020 at 4:55 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Currently, host cpu is inherited simply.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/cpu.c | 6 ++++++ target/riscv/cpu.h | 1 + 2 files changed, 7 insertions(+)
diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index faee98a58c..439dc89ee7 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -186,6 +186,10 @@ static void rv32_imafcu_nommu_cpu_init(Object *obj)
#endif
+static void riscv_host_cpu_init(Object *obj) { } + static ObjectClass *riscv_cpu_class_by_name(const char *cpu_model) { ObjectClass *oc; @@ -641,10 +645,12 @@ static const TypeInfo riscv_cpu_type_infos[] = { DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E31,
rvxx_sifive_e_cpu_init),
DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E34,
rv32_imafcu_nommu_cpu_init),
DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_U34,
rvxx_sifive_u_cpu_init),
+ DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init), #elif defined(TARGET_RISCV64) DEFINE_CPU(TYPE_RISCV_CPU_BASE64, riscv_base_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_E51, rvxx_sifive_e_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_SIFIVE_U54, rvxx_sifive_u_cpu_init), + DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init),
Shouldn't this only be included if KVM is configured? Also it should be shared between RV32 and RV64.
Alistair
Yes, It should be included by CONFIG_KVM and be shared between RV32 and RV64. Yifei
#endif };
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index ad1c90f798..4288898019 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -43,6 +43,7 @@ #define TYPE_RISCV_CPU_SIFIVE_E51 RISCV_CPU_TYPE_NAME("sifive-e51") #define TYPE_RISCV_CPU_SIFIVE_U34 RISCV_CPU_TYPE_NAME("sifive-u34") #define TYPE_RISCV_CPU_SIFIVE_U54 RISCV_CPU_TYPE_NAME("sifive-u54") +#define TYPE_RISCV_CPU_HOST RISCV_CPU_TYPE_NAME("host")
#define RV32 ((target_ulong)1 << (TARGET_LONG_BITS - 2)) #define RV64 ((target_ulong)2 << (TARGET_LONG_BITS - 2)) -- 2.19.1
Add kvm_riscv_get/put_regs_timer to synchronize virtual time context from KVM. To set register of RISCV_TIMER_REG(state) will occur a error from KVM on kvm_timer_state == 0. It's better to adapt in KVM, but it doesn't matter that adaping in QEMU. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/cpu.h | 6 ++++ target/riscv/kvm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 4288898019..16d6050ead 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -237,6 +237,12 @@ struct CPURISCVState { hwaddr kernel_addr; hwaddr fdt_addr; + + /* kvm timer */ + bool kvm_timer_dirty; + uint64_t kvm_timer_time; + uint64_t kvm_timer_compare; + uint64_t kvm_timer_state; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index afd99b3315..79228eb726 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -59,6 +59,9 @@ static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) #define RISCV_CSR_REG(name) kvm_riscv_reg_id(KVM_REG_RISCV_CSR, \ KVM_REG_RISCV_CSR_REG(name)) +#define RISCV_TIMER_REG(name) kvm_riscv_reg_id(KVM_REG_RISCV_TIMER, \ + KVM_REG_RISCV_TIMER_REG(name)) + #define RISCV_FP_F_REG(idx) kvm_riscv_reg_id(KVM_REG_RISCV_FP_F, idx) #define RISCV_FP_D_REG(idx) kvm_riscv_reg_id(KVM_REG_RISCV_FP_D, idx) @@ -306,6 +309,75 @@ static int kvm_riscv_put_regs_fp(CPUState *cs) return ret; } +static void kvm_riscv_get_regs_timer(CPUState *cs) +{ + int ret; + uint64_t reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (env->kvm_timer_dirty) { + return; + } + + ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(time), ®); + if (ret) { + abort(); + } + env->kvm_timer_time = reg; + + ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(compare), ®); + if (ret) { + abort(); + } + env->kvm_timer_compare = reg; + + ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(state), ®); + if (ret) { + abort(); + } + env->kvm_timer_state = reg; + + env->kvm_timer_dirty = true; +} + +static void kvm_riscv_put_regs_timer(CPUState *cs) +{ + int ret; + uint64_t reg; + CPURISCVState *env = &RISCV_CPU(cs)->env; + + if (!env->kvm_timer_dirty) { + return; + } + + reg = env->kvm_timer_time; + ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(time), ®); + if (ret) { + abort(); + } + + reg = env->kvm_timer_compare; + ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(compare), ®); + if (ret) { + abort(); + } + + /* + * To set register of RISCV_TIMER_REG(state) will occur a error from KVM + * on env->kvm_timer_state == 0, It's better to adapt in KVM, but it + * doesn't matter that adaping in QEMU now. + * TODO If KVM changes, adapt here. + */ + if (env->kvm_timer_state) { + reg = env->kvm_timer_state; + ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(state), ®); + if (ret) { + abort(); + } + } + + env->kvm_timer_dirty = false; +} const KVMCapabilityInfo kvm_arch_required_capabilities[] = { KVM_CAP_LAST_INFO -- 2.19.1
We hope that virtual time adjusts with vm state changing. When a vm is stopped, guest virtual time should stop counting and kvm_timer should be stopped. When the vm is resumed, guest virtual time should continue to count and kvm_timer should be restored. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/kvm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 79228eb726..1e16d24544 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -40,6 +40,7 @@ #include "kvm_riscv.h" #include "sbi_ecall_interface.h" #include "chardev/char-fe.h" +#include "sysemu/runstate.h" static __u64 kvm_riscv_reg_id(__u64 type, __u64 idx) { @@ -448,6 +449,17 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu) return cpu->cpu_index; } +static void kvm_riscv_vm_state_change(void *opaque, int running, RunState state) +{ + CPUState *cs = opaque; + + if (running) { + kvm_riscv_put_regs_timer(cs); + } else { + kvm_riscv_get_regs_timer(cs); + } +} + void kvm_arch_init_irq_routing(KVMState *s) { } @@ -460,6 +472,8 @@ int kvm_arch_init_vcpu(CPUState *cs) CPURISCVState *env = &cpu->env; __u64 id; + qemu_add_vm_change_state_handler(kvm_riscv_vm_state_change, cs); + id = kvm_riscv_reg_id(KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa)); ret = kvm_get_one_reg(cs, id, &isa); if (ret) { -- 2.19.1
Add virtual time context description to vmstate_riscv_cpu. After cpu being loaded, virtual time context is updated to KVM. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/machine.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/target/riscv/machine.c b/target/riscv/machine.c index 44d4015bd6..ef2d5395a8 100644 --- a/target/riscv/machine.c +++ b/target/riscv/machine.c @@ -138,10 +138,20 @@ static const VMStateDescription vmstate_hyper = { } }; +static int cpu_post_load(void *opaque, int version_id) +{ + RISCVCPU *cpu = opaque; + CPURISCVState *env = &cpu->env; + + env->kvm_timer_dirty = true; + return 0; +} + const VMStateDescription vmstate_riscv_cpu = { .name = "cpu", .version_id = 1, .minimum_version_id = 1, + .post_load = cpu_post_load, .fields = (VMStateField[]) { VMSTATE_UINTTL_ARRAY(env.gpr, RISCVCPU, 32), VMSTATE_UINT64_ARRAY(env.fpr, RISCVCPU, 32), @@ -185,6 +195,10 @@ const VMStateDescription vmstate_riscv_cpu = { VMSTATE_UINT64(env.mtohost, RISCVCPU), VMSTATE_UINT64(env.timecmp, RISCVCPU), + VMSTATE_UINT64(env.kvm_timer_time, RISCVCPU), + VMSTATE_UINT64(env.kvm_timer_compare, RISCVCPU), + VMSTATE_UINT64(env.kvm_timer_state, RISCVCPU), + VMSTATE_END_OF_LIST() }, .subsections = (const VMStateDescription * []) { -- 2.19.1
Currently, time base frequency was fixed as SIFIVE_CLINT_TIMEBASE_FREQ. Here introduce "time-frequency" property to set time base frequency dynamically of which default value is still SIFIVE_CLINT_TIMEBASE_FREQ. The virt machine uses frequency of the first cpu to create clint and fdt. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/riscv/virt.c | 18 ++++++++++++++---- target/riscv/cpu.c | 3 +++ target/riscv/cpu.h | 2 ++ 3 files changed, 19 insertions(+), 4 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 47b7018193..788a7237b6 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -178,7 +178,7 @@ static void create_pcie_irq_map(void *fdt, char *nodename, } static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap, - uint64_t mem_size, const char *cmdline) + uint64_t mem_size, const char *cmdline, uint64_t timebase_frequency) { void *fdt; int i, cpu, socket; @@ -225,7 +225,7 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap, qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", - SIFIVE_CLINT_TIMEBASE_FREQ); + timebase_frequency); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); @@ -510,6 +510,7 @@ static void virt_machine_init(MachineState *machine) target_ulong firmware_end_addr, kernel_start_addr; uint32_t fdt_load_addr; uint64_t kernel_entry; + uint64_t timebase_frequency = 0; DeviceState *mmio_plic, *virtio_plic, *pcie_plic; int i, j, base_hartid, hart_count; CPUState *cs; @@ -553,12 +554,20 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort); sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort); + if (!timebase_frequency) { + timebase_frequency = RISCV_CPU(first_cpu)->env.frequency; + } + /* If vcpu's time frequency is not specified, we use default frequency */ + if (!timebase_frequency) { + timebase_frequency = SIFIVE_CLINT_TIMEBASE_FREQ; + } + /* Per-socket CLINT */ sifive_clint_create( memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, memmap[VIRT_CLINT].size, base_hartid, hart_count, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, - SIFIVE_CLINT_TIMEBASE_FREQ, true); + timebase_frequency, true); /* Per-socket PLIC hart topology configuration string */ plic_hart_config_len = @@ -610,7 +619,8 @@ static void virt_machine_init(MachineState *machine) main_mem); /* create device tree */ - create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline); + create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline, + timebase_frequency); /* boot rom */ memory_region_init_rom(mask_rom, NULL, "riscv_virt_board.mrom", diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 439dc89ee7..66f35bcbbf 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -494,6 +494,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) riscv_cpu_register_gdb_regs_for_features(cs); + env->user_frequency = env->frequency; + qemu_init_vcpu(cs); cpu_reset(cs); @@ -531,6 +533,7 @@ static Property riscv_cpu_properties[] = { DEFINE_PROP_BOOL("mmu", RISCVCPU, cfg.mmu, true), DEFINE_PROP_BOOL("pmp", RISCVCPU, cfg.pmp, true), DEFINE_PROP_UINT64("resetvec", RISCVCPU, cfg.resetvec, DEFAULT_RSTVEC), + DEFINE_PROP_UINT64("time-frequency", RISCVCPU, env.frequency, 0), DEFINE_PROP_END_OF_LIST(), }; diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 16d6050ead..f5b6c34176 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -243,6 +243,8 @@ struct CPURISCVState { uint64_t kvm_timer_time; uint64_t kvm_timer_compare; uint64_t kvm_timer_state; + uint64_t user_frequency; + uint64_t frequency; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, -- 2.19.1
On Thu, Dec 3, 2020 at 4:57 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Currently, time base frequency was fixed as SIFIVE_CLINT_TIMEBASE_FREQ. Here introduce "time-frequency" property to set time base frequency dynamically of which default value is still SIFIVE_CLINT_TIMEBASE_FREQ. The virt machine uses frequency of the first cpu to create clint and fdt.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/riscv/virt.c | 18 ++++++++++++++---- target/riscv/cpu.c | 3 +++ target/riscv/cpu.h | 2 ++ 3 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 47b7018193..788a7237b6 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -178,7 +178,7 @@ static void create_pcie_irq_map(void *fdt, char *nodename, }
static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap, - uint64_t mem_size, const char *cmdline) + uint64_t mem_size, const char *cmdline, uint64_t timebase_frequency) { void *fdt; int i, cpu, socket; @@ -225,7 +225,7 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap,
qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", - SIFIVE_CLINT_TIMEBASE_FREQ); + timebase_frequency); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); @@ -510,6 +510,7 @@ static void virt_machine_init(MachineState *machine) target_ulong firmware_end_addr, kernel_start_addr; uint32_t fdt_load_addr; uint64_t kernel_entry; + uint64_t timebase_frequency = 0; DeviceState *mmio_plic, *virtio_plic, *pcie_plic; int i, j, base_hartid, hart_count; CPUState *cs; @@ -553,12 +554,20 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort); sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
+ if (!timebase_frequency) { + timebase_frequency = RISCV_CPU(first_cpu)->env.frequency; + } + /* If vcpu's time frequency is not specified, we use default frequency */ + if (!timebase_frequency) { + timebase_frequency = SIFIVE_CLINT_TIMEBASE_FREQ; + } + /* Per-socket CLINT */ sifive_clint_create( memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, memmap[VIRT_CLINT].size, base_hartid, hart_count, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, - SIFIVE_CLINT_TIMEBASE_FREQ, true); + timebase_frequency, true);
/* Per-socket PLIC hart topology configuration string */ plic_hart_config_len = @@ -610,7 +619,8 @@ static void virt_machine_init(MachineState *machine) main_mem);
/* create device tree */ - create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline); + create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline, + timebase_frequency);
/* boot rom */ memory_region_init_rom(mask_rom, NULL, "riscv_virt_board.mrom", diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 439dc89ee7..66f35bcbbf 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -494,6 +494,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
riscv_cpu_register_gdb_regs_for_features(cs);
+ env->user_frequency = env->frequency; + qemu_init_vcpu(cs); cpu_reset(cs);
@@ -531,6 +533,7 @@ static Property riscv_cpu_properties[] = { DEFINE_PROP_BOOL("mmu", RISCVCPU, cfg.mmu, true), DEFINE_PROP_BOOL("pmp", RISCVCPU, cfg.pmp, true), DEFINE_PROP_UINT64("resetvec", RISCVCPU, cfg.resetvec, DEFAULT_RSTVEC), + DEFINE_PROP_UINT64("time-frequency", RISCVCPU, env.frequency, 0),
Why not set the default to SIFIVE_CLINT_TIMEBASE_FREQ? Also, QEMU now has a clock API, is using that instead a better option? Alistair
DEFINE_PROP_END_OF_LIST(), };
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 16d6050ead..f5b6c34176 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -243,6 +243,8 @@ struct CPURISCVState { uint64_t kvm_timer_time; uint64_t kvm_timer_compare; uint64_t kvm_timer_state; + uint64_t user_frequency; + uint64_t frequency; };
OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, -- 2.19.1
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Wednesday, December 9, 2020 6:26 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; Zhangxiaofeng (F) <victor.zhangxiaofeng@huawei.com>; Sagar Karandikar <sagark@eecs.berkeley.edu>; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Bastian Koppelmann <kbastian@mail.uni-paderborn.de>; Anup Patel <anup.patel@wdc.com>; yinyipeng <yinyipeng1@huawei.com>; Alistair Francis <Alistair.Francis@wdc.com>; kvm-riscv@lists.infradead.org; Palmer Dabbelt <palmer@dabbelt.com>; dengkai (A) <dengkai1@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Zhanghailiang <zhang.zhanghailiang@huawei.com> Subject: Re: [PATCH RFC v4 13/15] target/riscv: Introduce dynamic time frequency for virt machine
On Thu, Dec 3, 2020 at 4:57 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Currently, time base frequency was fixed as SIFIVE_CLINT_TIMEBASE_FREQ. Here introduce "time-frequency" property to set time base frequency dynamically of which default value is still SIFIVE_CLINT_TIMEBASE_FREQ. The virt machine uses frequency of the first
cpu to create clint and fdt.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/riscv/virt.c | 18 ++++++++++++++---- target/riscv/cpu.c | 3 +++ target/riscv/cpu.h | 2 ++ 3 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 47b7018193..788a7237b6 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -178,7 +178,7 @@ static void create_pcie_irq_map(void *fdt, char *nodename, }
static void create_fdt(RISCVVirtState *s, const struct MemmapEntry
*memmap,
- uint64_t mem_size, const char *cmdline) + uint64_t mem_size, const char *cmdline, uint64_t + timebase_frequency) { void *fdt; int i, cpu, socket; @@ -225,7 +225,7 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap,
qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", - SIFIVE_CLINT_TIMEBASE_FREQ); + timebase_frequency); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); @@ -510,6 +510,7 @@ static void virt_machine_init(MachineState *machine) target_ulong firmware_end_addr, kernel_start_addr; uint32_t fdt_load_addr; uint64_t kernel_entry; + uint64_t timebase_frequency = 0; DeviceState *mmio_plic, *virtio_plic, *pcie_plic; int i, j, base_hartid, hart_count; CPUState *cs; @@ -553,12 +554,20 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort); sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
+ if (!timebase_frequency) { + timebase_frequency = RISCV_CPU(first_cpu)->env.frequency; + } + /* If vcpu's time frequency is not specified, we use default frequency */ + if (!timebase_frequency) { + timebase_frequency = SIFIVE_CLINT_TIMEBASE_FREQ; + } + /* Per-socket CLINT */ sifive_clint_create( memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, memmap[VIRT_CLINT].size, base_hartid, hart_count, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, - SIFIVE_CLINT_TIMEBASE_FREQ, true); + timebase_frequency, true);
/* Per-socket PLIC hart topology configuration string */ plic_hart_config_len = @@ -610,7 +619,8 @@ static void virt_machine_init(MachineState *machine) main_mem);
/* create device tree */ - create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline); + create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline, + timebase_frequency);
/* boot rom */ memory_region_init_rom(mask_rom, NULL, "riscv_virt_board.mrom", diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 439dc89ee7..66f35bcbbf 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -494,6 +494,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
riscv_cpu_register_gdb_regs_for_features(cs);
+ env->user_frequency = env->frequency; + qemu_init_vcpu(cs); cpu_reset(cs);
@@ -531,6 +533,7 @@ static Property riscv_cpu_properties[] = { DEFINE_PROP_BOOL("mmu", RISCVCPU, cfg.mmu, true), DEFINE_PROP_BOOL("pmp", RISCVCPU, cfg.pmp, true), DEFINE_PROP_UINT64("resetvec", RISCVCPU, cfg.resetvec, DEFAULT_RSTVEC), + DEFINE_PROP_UINT64("time-frequency", RISCVCPU, env.frequency, 0),
Why not set the default to SIFIVE_CLINT_TIMEBASE_FREQ?
When the time frequency is not specified, it will follow the host or the migration source. And we define 0 as equivalent to not specified time frequency.
Also, QEMU now has a clock API, is using that instead a better option?
Sorry, I didn't find the clock API. Could you tell me what the API is. I think that the time frequency is option of KVM VCPU. So it is appropriate to put this option in the CPU. Yifei
Alistair
DEFINE_PROP_END_OF_LIST(), };
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 16d6050ead..f5b6c34176 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -243,6 +243,8 @@ struct CPURISCVState { uint64_t kvm_timer_time; uint64_t kvm_timer_compare; uint64_t kvm_timer_state; + uint64_t user_frequency; + uint64_t frequency; };
OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, -- 2.19.1
On Mon, Dec 14, 2020 at 11:31 PM Jiangyifei <jiangyifei@huawei.com> wrote:
-----Original Message----- From: Alistair Francis [mailto:alistair23@gmail.com] Sent: Wednesday, December 9, 2020 6:26 AM To: Jiangyifei <jiangyifei@huawei.com> Cc: qemu-devel@nongnu.org Developers <qemu-devel@nongnu.org>; open list:RISC-V <qemu-riscv@nongnu.org>; Zhangxiaofeng (F) <victor.zhangxiaofeng@huawei.com>; Sagar Karandikar <sagark@eecs.berkeley.edu>; open list:Overall <kvm@vger.kernel.org>; libvir-list@redhat.com; Bastian Koppelmann <kbastian@mail.uni-paderborn.de>; Anup Patel <anup.patel@wdc.com>; yinyipeng <yinyipeng1@huawei.com>; Alistair Francis <Alistair.Francis@wdc.com>; kvm-riscv@lists.infradead.org; Palmer Dabbelt <palmer@dabbelt.com>; dengkai (A) <dengkai1@huawei.com>; Wubin (H) <wu.wubin@huawei.com>; Zhanghailiang <zhang.zhanghailiang@huawei.com> Subject: Re: [PATCH RFC v4 13/15] target/riscv: Introduce dynamic time frequency for virt machine
On Thu, Dec 3, 2020 at 4:57 AM Yifei Jiang <jiangyifei@huawei.com> wrote:
Currently, time base frequency was fixed as SIFIVE_CLINT_TIMEBASE_FREQ. Here introduce "time-frequency" property to set time base frequency dynamically of which default value is still SIFIVE_CLINT_TIMEBASE_FREQ. The virt machine uses frequency of the first
cpu to create clint and fdt.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- hw/riscv/virt.c | 18 ++++++++++++++---- target/riscv/cpu.c | 3 +++ target/riscv/cpu.h | 2 ++ 3 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 47b7018193..788a7237b6 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -178,7 +178,7 @@ static void create_pcie_irq_map(void *fdt, char *nodename, }
static void create_fdt(RISCVVirtState *s, const struct MemmapEntry
*memmap,
- uint64_t mem_size, const char *cmdline) + uint64_t mem_size, const char *cmdline, uint64_t + timebase_frequency) { void *fdt; int i, cpu, socket; @@ -225,7 +225,7 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap,
qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", - SIFIVE_CLINT_TIMEBASE_FREQ); + timebase_frequency); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); @@ -510,6 +510,7 @@ static void virt_machine_init(MachineState *machine) target_ulong firmware_end_addr, kernel_start_addr; uint32_t fdt_load_addr; uint64_t kernel_entry; + uint64_t timebase_frequency = 0; DeviceState *mmio_plic, *virtio_plic, *pcie_plic; int i, j, base_hartid, hart_count; CPUState *cs; @@ -553,12 +554,20 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort); sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
+ if (!timebase_frequency) { + timebase_frequency = RISCV_CPU(first_cpu)->env.frequency; + } + /* If vcpu's time frequency is not specified, we use default frequency */ + if (!timebase_frequency) { + timebase_frequency = SIFIVE_CLINT_TIMEBASE_FREQ; + } + /* Per-socket CLINT */ sifive_clint_create( memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, memmap[VIRT_CLINT].size, base_hartid, hart_count, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, - SIFIVE_CLINT_TIMEBASE_FREQ, true); + timebase_frequency, true);
/* Per-socket PLIC hart topology configuration string */ plic_hart_config_len = @@ -610,7 +619,8 @@ static void virt_machine_init(MachineState *machine) main_mem);
/* create device tree */ - create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline); + create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline, + timebase_frequency);
/* boot rom */ memory_region_init_rom(mask_rom, NULL, "riscv_virt_board.mrom", diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 439dc89ee7..66f35bcbbf 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -494,6 +494,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
riscv_cpu_register_gdb_regs_for_features(cs);
+ env->user_frequency = env->frequency; + qemu_init_vcpu(cs); cpu_reset(cs);
@@ -531,6 +533,7 @@ static Property riscv_cpu_properties[] = { DEFINE_PROP_BOOL("mmu", RISCVCPU, cfg.mmu, true), DEFINE_PROP_BOOL("pmp", RISCVCPU, cfg.pmp, true), DEFINE_PROP_UINT64("resetvec", RISCVCPU, cfg.resetvec, DEFAULT_RSTVEC), + DEFINE_PROP_UINT64("time-frequency", RISCVCPU, env.frequency, 0),
Why not set the default to SIFIVE_CLINT_TIMEBASE_FREQ?
When the time frequency is not specified, it will follow the host or the migration source. And we define 0 as equivalent to not specified time frequency.
Also, QEMU now has a clock API, is using that instead a better option?
Sorry, I didn't find the clock API. Could you tell me what the API is. I think that the time frequency is option of KVM VCPU. So it is appropriate to put this option in the CPU.
The clock API is documented here: https://gitlab.com/qemu-project/qemu/-/blob/master/docs/devel/clocks.rst I'm not sure if it applies to KVM, but it is at least worth considering. Alistair
Yifei
Alistair
DEFINE_PROP_END_OF_LIST(), };
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 16d6050ead..f5b6c34176 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -243,6 +243,8 @@ struct CPURISCVState { uint64_t kvm_timer_time; uint64_t kvm_timer_compare; uint64_t kvm_timer_state; + uint64_t user_frequency; + uint64_t frequency; };
OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, -- 2.19.1
If vcpu's frequency is specified by cpu option 'frequency', it will be set into KVM by KVM_SET_ONE_REG ioctl. Otherwise, vcpu's frequency will follow KVM by KVM_GET_ONE_REG ioctl. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/kvm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 1e16d24544..3499efd4eb 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -409,6 +409,8 @@ int kvm_arch_get_registers(CPUState *cs) int kvm_arch_put_registers(CPUState *cs, int level) { int ret = 0; + RISCVCPU *cpu = RISCV_CPU(cs); + CPURISCVState *env = &cpu->env; ret = kvm_riscv_put_regs_core(cs); if (ret) { @@ -425,6 +427,10 @@ int kvm_arch_put_registers(CPUState *cs, int level) return ret; } + if (env->frequency) { + ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(frequency), &env->frequency); + } + return ret; } @@ -481,6 +487,17 @@ int kvm_arch_init_vcpu(CPUState *cs) } env->misa = isa; + /* + * Synchronize vcpu's frequency with KVM. If vcpu's frequency is specified + * by cpu option 'frequency', this will be set to KVM. Otherwise, vcpu's + * frequency will follow KVM. + */ + if (env->user_frequency) { + ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(frequency), &env->frequency); + } else { + ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(frequency), &env->frequency); + } + return ret; } -- 2.19.1
If vcpu's time frequency is not specified by CPU option 'time-frequency' on the destination, the time frequency of destination will follow the source. If vcpu's time frequency specified by CPU option 'time-frequency' on the destination is different from migrated time frequency. The migration will be abort. Signed-off-by: Yifei Jiang <jiangyifei@huawei.com> Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com> --- target/riscv/machine.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/target/riscv/machine.c b/target/riscv/machine.c index ef2d5395a8..6955542fef 100644 --- a/target/riscv/machine.c +++ b/target/riscv/machine.c @@ -144,6 +144,13 @@ static int cpu_post_load(void *opaque, int version_id) CPURISCVState *env = &cpu->env; env->kvm_timer_dirty = true; + + if (env->user_frequency && env->user_frequency != env->frequency) { + error_report("Mismatch between user-specified time frequency and " + "migrated time frequency"); + return -EINVAL; + } + return 0; } @@ -198,6 +205,7 @@ const VMStateDescription vmstate_riscv_cpu = { VMSTATE_UINT64(env.kvm_timer_time, RISCVCPU), VMSTATE_UINT64(env.kvm_timer_compare, RISCVCPU), VMSTATE_UINT64(env.kvm_timer_state, RISCVCPU), + VMSTATE_UINT64(env.frequency, RISCVCPU), VMSTATE_END_OF_LIST() }, -- 2.19.1
participants (3)
-
Alistair Francis -
Jiangyifei -
Yifei Jiang