[RFC PATCH] docs/about/deprecated: Deprecate 32-bit host systems
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit, keeping the 32-bit support alive is an inadequate burden for the QEMU project. Let's mark the 32-bit support as deprecated so we can drop it after a while - this will help us to cut down our limited CI minutes in the gitlab CI, for example. Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 9f1bbc495d..ce6463e72b 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -181,9 +181,20 @@ As Debian 10 ("Buster") moved into LTS the big endian 32 bit version of MIPS moved out of support making it hard to maintain our cross-compilation CI tests of the architecture. As we no longer have CI coverage support may bitrot away before the deprecation process -completes. The little endian variants of MIPS (both 32 and 64 bit) are +completes. The little endian variants of MIPS are still a supported host architecture. +32-bit host operating systems (since 8.0) +''''''''''''''''''''''''''''''''''''''''' + +Testing 32-bit host OS support takes a lot of precious time during the QEMU +contiguous integration tests, and considering that many OS vendors stopped +shipping 32-bit variants of their OS distributions and most hardware from +the past >10 years is capable of 64-bit, keeping the 32-bit support alive +is an inadequate burden for the QEMU project. Thus QEMU will soon drop the +support for 32-bit host systems. + + QEMU API (QAPI) events ---------------------- -- 2.31.1
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support. thanks -- PMM
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile. I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ? With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 30/01/2023 13.01, Daniel P. Berrangé wrote:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Even mobile phones have 64-bit processors nowadays, Rasberry PIs are 64-bit nowadays ... which arm hosts scenarios are still limited to 32-bit ? Also, as far as I know, 32-bit KVM arm support has been removed from the Linux kernel a while ago already, so it's just about TCG now ... is there really still that much interest in running emulation on a non-beefy 32-bit host? Anyway, we could add the deprecation notice now to find out if there are still 32-bit users out there who will then start complaining about this.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
The burden are the CI minutes of the shared CI runners. We've got quite a bunch of 32-bit jobs in the CI: - cross-armel-system - cross-armel-user - cross-armhf-system - cross-armhf-user - cross-i386-system - cross-i386-user - cross-i386-tci - cross-mipsel-system - cross-mipsel-user - cross-win32-system If we could finally drop supporting 32-bit hosts, that would help with our CI minutes problem quite a lot, I think. Thomas
On Mon, Jan 30, 2023 at 01:22:22PM +0100, Thomas Huth wrote:
On 30/01/2023 13.01, Daniel P. Berrangé wrote:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Even mobile phones have 64-bit processors nowadays, Rasberry PIs are 64-bit nowadays ... which arm hosts scenarios are still limited to 32-bit ?
Also, as far as I know, 32-bit KVM arm support has been removed from the Linux kernel a while ago already, so it's just about TCG now ... is there really still that much interest in running emulation on a non-beefy 32-bit host?
Anyway, we could add the deprecation notice now to find out if there are still 32-bit users out there who will then start complaining about this.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
The burden are the CI minutes of the shared CI runners. We've got quite a bunch of 32-bit jobs in the CI:
- cross-armel-system - cross-armel-user - cross-armhf-system - cross-armhf-user - cross-i386-system - cross-i386-user - cross-i386-tci - cross-mipsel-system - cross-mipsel-user - cross-win32-system
If we could finally drop supporting 32-bit hosts, that would help with our CI minutes problem quite a lot, I think.
CI is a non-technical constraint, that's more about support level. If we want to save CI minutes, we can declare that some or all of the 32-bit hosts scenarios are now tier 2, rather than tier 1. So the 32-bit host support still exists, but we're not doing gating CI on every combination. eg could declare 32-bit for linux-user is tier 1 and fully tested but 32-bit machine emul is tier 2 and adhoc tested. Or make it more nuanced per arch target We only need to deprecate and delete if we have some wins at the code level that we can't do with while it exists. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 30/01/2023 13.43, Daniel P. Berrangé wrote:
On Mon, Jan 30, 2023 at 01:22:22PM +0100, Thomas Huth wrote:
On 30/01/2023 13.01, Daniel P. Berrangé wrote:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
... Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
The burden are the CI minutes of the shared CI runners. We've got quite a bunch of 32-bit jobs in the CI:
- cross-armel-system - cross-armel-user - cross-armhf-system - cross-armhf-user - cross-i386-system - cross-i386-user - cross-i386-tci - cross-mipsel-system - cross-mipsel-user - cross-win32-system
If we could finally drop supporting 32-bit hosts, that would help with our CI minutes problem quite a lot, I think.
CI is a non-technical constraint, that's more about support level.
If we want to save CI minutes, we can declare that some or all of the 32-bit hosts scenarios are now tier 2, rather than tier 1. So the 32-bit host support still exists, but we're not doing gating CI on every combination. eg could declare 32-bit for linux-user is tier 1 and fully tested but 32-bit machine emul is tier 2 and adhoc tested. Or make it more nuanced per arch target
Well, while the burden currently mostly comes from the CI minutes (since it catches problems immediately, they don't get committed), you would need people's time instead who have to look after the problems once they've been committed to the repository (which will surely happen if we don't check 32-bit host support in the CI anymore). Who's volunteering in spending his time with analyzing (e.g. bisecting) and fixing those problems? At least I don't.
We only need to deprecate and delete if we have some wins at the code level that we can't do with while it exists.
We also would have quite some wins at the code level, I think: At least the whole "tcg/arm/" folder would go away, and we could simplify all spots that are using HOST_LONG_BITS in the code base. Thomas
On 1/30/23 02:01, Daniel P. Berrangé wrote:
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
TCG_OVERSIZED_GUEST, where 32-bit host running 64-bit guest cannot run the guest except in round-robin mode. It's not *that* much of a support burden, since we're not going to eliminate round-robin mode. But I do question whether we need to support 64-bit guests on 32-bit hosts at all. Retaining 32-bit on 32-bit allows arm32 to emulate i686, which I suspect, but have no proof, is the limit of what users actually want. r~
On 30/1/23 20:19, Richard Henderson wrote:
But I do question whether we need to support 64-bit guests on 32-bit hosts at all. Retaining 32-bit on 32-bit allows arm32 to emulate i686, which I suspect, but have no proof, is the limit of what users actually want.
I presume you implicitly restrict that to user emulation, right? WRT i686, if your example is "i686 useremu on non-x86 embedde router" then any 32-bit host is potentially interested, not only arm32. I remember being able to run armhf binaries on armel hosts (and vice versa) was useful 7 years ago. Today I have no clue, we could poll the community and some distributions.
On 1/30/23 13:14, Philippe Mathieu-Daudé wrote:
On 30/1/23 20:19, Richard Henderson wrote:
But I do question whether we need to support 64-bit guests on 32-bit hosts at all. Retaining 32-bit on 32-bit allows arm32 to emulate i686, which I suspect, but have no proof, is the limit of what users actually want.
I presume you implicitly restrict that to user emulation, right?
No, there's no specific reason to eliminate e.g. qemu-system-i386. or any other 32-bit guest. Though quite often such hardware doesn't really have enough ram to do a good job, that's not a technical argument against.
WRT i686, if your example is "i686 useremu on non-x86 embedde router" then any 32-bit host is potentially interested, not only arm32.
arm32 was merely an example -- the other 32-bit hosts are i686, mips, ppc. But we don't have many of them.
I remember being able to run armhf binaries on armel hosts (and vice versa) was useful 7 years ago.
Fair enough.
Today I have no clue, we could poll the community and some distributions.
Sure. r~
On 31/1/23 00:33, Richard Henderson wrote:
On 1/30/23 13:14, Philippe Mathieu-Daudé wrote:
On 30/1/23 20:19, Richard Henderson wrote:
But I do question whether we need to support 64-bit guests on 32-bit hosts at all. Retaining 32-bit on 32-bit allows arm32 to emulate i686, which I suspect, but have no proof, is the limit of what users actually want.
I presume you implicitly restrict that to user emulation, right?
No, there's no specific reason to eliminate e.g. qemu-system-i386. or any other 32-bit guest. Though quite often such hardware doesn't really have enough ram to do a good job, that's not a technical argument against.
I am not concerned by the RAM limit but by the community maintenance cost. As of 2023, QEMU v7.2.0 is usable on 32-bit. Fixes will be backported in the v7.2.x stable releases. Maybe this is enough for 32-bit host users; and planning the unavailability of features released in v8.2 or v9.0 for 32-bit hosts doesn't seem unreasonable. Stefan Weil once posted stats of Win32 vs Win64 binary downloads from last year IIRC, and Win32 is still consumed (but maybe the difference comes from mirrors or users always downloading both versions).
WRT i686, if your example is "i686 useremu on non-x86 embedded router" then any 32-bit host is potentially interested, not only arm32.
arm32 was merely an example -- the other 32-bit hosts are i686, mips, ppc. But we don't have many of them.
I remember being able to run armhf binaries on armel hosts (and vice versa) was useful 7 years ago.
Fair enough.
Today I have no clue, we could poll the community and some distributions.
Sure.
r~
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
With regards, Daniel
-- Alex Bennée Virtualisation Tech Lead @ Linaro
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware. From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues. ATB, Mark.
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark! Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And which QEMU machine are you using for running macOS? The mac99 or the g3beige machine? Unrelated to KVM: Do you happen to know whether there are any problems when running 32-bit guests with TCG with the mac99 or g3beige machine while using qemu-system-ppc64 ? Thanks, Thomas
Unrelated to KVM: Do you happen to know whether there are any problems when running 32-bit guests with TCG with the mac99 or g3beige machine while using qemu-system-ppc64 ?
We removed rfi support for 64bit machine in QEMU 2.7. Commit a2e71b28e8 ("ppc: Fix rfi/rfid/hrfi/... emulation"). So a mac99 machine using a 970 CPU needs a 64 bit kernel. The rfi insn was deleted from the ISA ... C.
On Tue, 4 Apr 2023, Cédric Le Goater wrote:
Unrelated to KVM: Do you happen to know whether there are any problems when running 32-bit guests with TCG with the mac99 or g3beige machine while using qemu-system-ppc64 ?
We removed rfi support for 64bit machine in QEMU 2.7. Commit a2e71b28e8 ("ppc: Fix rfi/rfid/hrfi/... emulation"). So a mac99 machine using a 970 CPU needs a 64 bit kernel. The rfi insn was deleted from the ISA ...
I don't know about that issue but maybe this does not really answer the question. AFAIK you could still run 32 bit guests with qemu-system-ppc64 and they seem to work. One could still get the qemu-system-ppc -M mac99 machine as qemu-system-ppc64 -M mac99 -cpu G4 but this is very confusing and nobody could guess it. Other machines should work the same in qemu-system-ppc64, the only concern is that some defines such as TARGET_ULONG are different between qemu-system-ppc and qemu-system-ppc64 and it's not known if this could cause any problems. It's more frequently happens that people try qemu-system-ppc64 -M mac99 when they want a G4 Mac and get confused that it emulates a G5 Mac instead which does not even work for anything than a Linux guest. Regards, BALATON Zoltan
[ adding Zoltan ] On 4/4/23 16:00, Thomas Huth wrote:
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark!
Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And which QEMU machine are you using for running macOS? The mac99 or the g3beige machine?
Zoltan, what about the pegasos2 and sam460ex machines ? can they be run under KVM ? Thanks, C.
On Tue, 4 Apr 2023, Cédric Le Goater wrote:
[ adding Zoltan ]
On 4/4/23 16:00, Thomas Huth wrote:
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote: > > Testing 32-bit host OS support takes a lot of precious time during the > QEMU > contiguous integration tests, and considering that many OS vendors > stopped > shipping 32-bit variants of their OS distributions and most hardware > from > the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark!
Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And which QEMU machine are you using for running macOS? The mac99 or the g3beige machine?
Zoltan, what about the pegasos2 and sam460ex machines ? can they be run under KVM ?
I don't know as I don't have PPC hardware to test on but theoretically they should work. Although BookE KVM was dropped from Linux I think so sam460ex could only work with an old kernel on a BookE host which is now rare but pegasos2 uses G4 CPU which is more likely to work on a host with the same CPU but I don't know anybody tested it yet. I know some people are interested to use it especially on G4 and G5 and some tested the latter but it does not work due to some differences that should be handled by KVM-PR but apparently they aren't (e.g. dcbz cache line size which I've seen debugged and identified by at least two sources that I sent references to this list before but to my knowledge no fix got upstream in Linux for this). AFAIK Mark has a G4 Mac mini on which KVM-PR works and it may also work on G5 when using 32 bit Kernel but running G4 guest on G5 with 64 bit kernel does not work. G5 host with 64 bit kernel may work with 64 bit guests but all Amiga like OSes I'm interested in are 32 bit so they either need 32 bit host kernel, preferably with same CPU as the guest (so G4 for pegasos2 or PPC440 for sam460ex) or fixing the patching and emulation of instructions in KVM-PR that behave differently between G4 and G5 which should be possible but nobody seems to have done it or bothered to upstream it and what's there may be incomplete or buggy. The only reports I've seen that said it works were either running 32bit guest on 32bit BookS host or 64bit guest on 64bit BookS host. (A special case is running 64bit aware 32bit guest such as newer MacOS or MorphOS versions on 64bit hosts which can run on real G5 so these may run as long as they correctly identify the CPU they are running on but the G5 Mac emulation in QEMU currently can't boot these because the machine is not emulated precisely enough for them.) Regards, BALATON Zoltan
On 04/04/2023 17.42, BALATON Zoltan wrote:
On Tue, 4 Apr 2023, Cédric Le Goater wrote:
[ adding Zoltan ]
On 4/4/23 16:00, Thomas Huth wrote:
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote: > On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote: >> >> Testing 32-bit host OS support takes a lot of precious time during >> the QEMU >> contiguous integration tests, and considering that many OS vendors >> stopped >> shipping 32-bit variants of their OS distributions and most hardware >> from >> the past >10 years is capable of 64-bit > > True for x86, not necessarily true for other architectures. > Are you proposing to deprecate x86 32-bit, or all 32-bit? > I'm not entirely sure about whether we're yet at a point where > I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark!
Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And which QEMU machine are you using for running macOS? The mac99 or the g3beige machine?
Zoltan, what about the pegasos2 and sam460ex machines ? can they be run under KVM ?
I don't know as I don't have PPC hardware to test on but theoretically they should work. Although BookE KVM was dropped from Linux I think so sam460ex could only work with an old kernel on a BookE host which is now rare [...]
Thanks for your explanations, that indeed helps to understand the situation! But are you sure about the BookE KVM removal in the Linux kernel? ... when I look at the arch/powerpc/kvm/ folder there, I can still see some files there with "booke" in the name? Thomas
On Wed, 5 Apr 2023, Thomas Huth wrote:
On 04/04/2023 17.42, BALATON Zoltan wrote:
On Tue, 4 Apr 2023, Cédric Le Goater wrote:
[ adding Zoltan ]
On 4/4/23 16:00, Thomas Huth wrote:
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
> On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote: >> On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote: >>> >>> Testing 32-bit host OS support takes a lot of precious time during >>> the QEMU >>> contiguous integration tests, and considering that many OS vendors >>> stopped >>> shipping 32-bit variants of their OS distributions and most hardware >>> from >>> the past >10 years is capable of 64-bit >> >> True for x86, not necessarily true for other architectures. >> Are you proposing to deprecate x86 32-bit, or all 32-bit? >> I'm not entirely sure about whether we're yet at a point where >> I'd want to deprecate-and-drop 32-bit arm host support. > > Do we have a feeling on which aspects of 32-bit cause us the support > burden ? The boring stuff like compiler errors from mismatched integer > sizes is mostly quick & easy to detect simply through a cross compile. > > I vaguely recall someone mentioned problems with atomic ops in the > past, > or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark!
Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And which QEMU machine are you using for running macOS? The mac99 or the g3beige machine?
Zoltan, what about the pegasos2 and sam460ex machines ? can they be run under KVM ?
I don't know as I don't have PPC hardware to test on but theoretically they should work. Although BookE KVM was dropped from Linux I think so sam460ex could only work with an old kernel on a BookE host which is now rare [...]
Thanks for your explanations, that indeed helps to understand the situation!
But are you sure about the BookE KVM removal in the Linux kernel? ... when I look at the arch/powerpc/kvm/ folder there, I can still see some files there with "booke" in the name?
No, I'm not sure but I think KVM on PPC440 (which is used by sam460ex) is likely not working properly. What's there may work on newer cores such as e500 and later but not sure if that can run PPC440 code. I never heard anyone successfully getting sam460ex work with KVM but that may also be because real PPC440 hosts are rare. But if the question is if we still need 32 bit PPC host I think we do for now as that's the only way to run 32bit guests with G3 and G4 until the issues which prevent them to run on 64bit host kernel are fixed. Regards, BALATON Zoltan
On 05/04/2023 13.54, BALATON Zoltan wrote:
On Wed, 5 Apr 2023, Thomas Huth wrote:
On 04/04/2023 17.42, BALATON Zoltan wrote:
On Tue, 4 Apr 2023, Cédric Le Goater wrote:
[ adding Zoltan ]
On 4/4/23 16:00, Thomas Huth wrote:
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes: > >> On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote: >>> On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote: >>>> >>>> Testing 32-bit host OS support takes a lot of precious time during >>>> the QEMU >>>> contiguous integration tests, and considering that many OS vendors >>>> stopped >>>> shipping 32-bit variants of their OS distributions and most >>>> hardware from >>>> the past >10 years is capable of 64-bit >>> >>> True for x86, not necessarily true for other architectures. >>> Are you proposing to deprecate x86 32-bit, or all 32-bit? >>> I'm not entirely sure about whether we're yet at a point where >>> I'd want to deprecate-and-drop 32-bit arm host support. >> >> Do we have a feeling on which aspects of 32-bit cause us the support >> burden ? The boring stuff like compiler errors from mismatched integer >> sizes is mostly quick & easy to detect simply through a cross compile. >> >> I vaguely recall someone mentioned problems with atomic ops in the >> past, >> or was it 128-bit ints, caused implications for the codebase ? > > Atomic operations on > TARGET_BIT_SIZE and cputlb when > TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the > backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered > throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark!
Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And which QEMU machine are you using for running macOS? The mac99 or the g3beige machine?
Zoltan, what about the pegasos2 and sam460ex machines ? can they be run under KVM ?
I don't know as I don't have PPC hardware to test on but theoretically they should work. Although BookE KVM was dropped from Linux I think so sam460ex could only work with an old kernel on a BookE host which is now rare [...]
Thanks for your explanations, that indeed helps to understand the situation!
But are you sure about the BookE KVM removal in the Linux kernel? ... when I look at the arch/powerpc/kvm/ folder there, I can still see some files there with "booke" in the name?
No, I'm not sure but I think KVM on PPC440 (which is used by sam460ex) is likely not working properly. What's there may work on newer cores such as e500 and later but not sure if that can run PPC440 code. I never heard anyone successfully getting sam460ex work with KVM but that may also be because real PPC440 hosts are rare.
Ok, if nobody is really using KVM on PPC440 anymore, we should mark that as deprecated in QEMU, I think.
But if the question is if we still need 32 bit PPC host I think we do for now as that's the only way to run 32bit guests with G3 and G4 until the issues which prevent them to run on 64bit host kernel are fixed.
Yes, understood. As long as 32-bit KVM is in use on ppc, we've got to keep the code around. Thomas
On Tue, 4 Apr 2023, Thomas Huth wrote:
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark!
Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And
Another issue I know is that mac99 behaves differently in qemu-system-ppc and qemu-system-ppc64. This is not only confusing users but makes it more difficult to get rid of qemu-system-ppc. I've tried to solve that and sumbitted a patch to start deprecating these but I could not get that merged. That thread ended here: https://lists.nongnu.org/archive/html/qemu-ppc/2023-01/msg00406.html Regards, BALATON Zoltan
On 04/04/2023 15:00, Thomas Huth wrote:
On 05/02/2023 23.12, Mark Cave-Ayland wrote:
On 30/01/2023 20:45, Alex Bennée wrote:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
I am one of an admittedly small group of people still interested in using KVM-PR on ppc32 to boot MacOS, although there is some interest on using 64-bit KVM-PR to run super-fast MacOS on modern Talos hardware.
From my perspective losing the ability to run 64-bit guests on 32-bit hardware with TCG wouldn't be an issue, as long as it were still possible to use qemu-system-ppc on 32-bit hardware using both TCG and KVM to help debug the remaining issues.
Hi Mark!
Just out of curiosity (since we briefly talked about 32-bit KVM on ppc in today's QEMU/KVM call - in the context of whether qemu-system-ppc64 is a proper superset of qemu-system-ppc when it comes to building a unified qemu-system binary): What host machine are you using for running KVM-PR? And which QEMU machine are you using for running macOS? The mac99 or the g3beige machine?
I'm using a G4 Mac Mini for my KVM experiments running Debian ports which is fairly up to date e.g. gcc-12 for building QEMU. Both the mac99 and g3beige machines will boot up to the OS X 10.2 installer on my current (custom) 5.1 kernel.
Unrelated to KVM: Do you happen to know whether there are any problems when running 32-bit guests with TCG with the mac99 or g3beige machine while using qemu-system-ppc64 ?
I'm not sure as I don't run qemu-system-ppc64 on a regular basis, but I have heard from people in the past who have happily used it to boot 64-bit Linux. (goes and turns on the Mac Mini) A quick test this evening shows that latest git master builds qemu-system-ppc and runs fine with -accel=kvm on the Mac Mini, however something now asserts with -accel=tcg on startup which didn't happen before: Thread 1 "qemu-system-ppc" received signal SIGSEGV, Segmentation fault. tcg_tb_alloc (s=0x1) at ../tcg/tcg.c:1025 1025 tb = (void *)ROUND_UP((uintptr_t)s->code_gen_ptr, align); (gdb) bt #0 0x00ad4148 in tcg_tb_alloc (s=0x1) at ../tcg/tcg.c:1025 #1 0x00ad34c8 in tcg_context_init (max_cpus=1) at ../tcg/tcg.c:959 #2 0x00ad34c8 in tcg_init (tb_size=0, splitwx=0, max_cpus=max_cpus@entry=1) at ../tcg/tcg.c:1010 #3 0x00b3b784 in tcg_init_machine (ms=<optimized out>) at ../accel/tcg/tcg-all.c:127 #4 0x008dc784 in accel_init_machine (accel=accel@entry=0x159f8c0, ms=0x1538000) at ../accel/accel-softmmu.c:39 #5 0x00869184 in do_configure_accelerator (opaque=opaque@entry=0xbffff2ac, opts=opts@entry=0x159f4f0, errp=errp@entry=0x13694cc <error_fatal>) at ../softmmu/vl.c:2224 #6 0x00d8e13c in qemu_opts_foreach (list=<optimized out>, func=func@entry=0x869010 <do_configure_accelerator>, opaque=opaque@entry=0xbffff2ac, errp=errp@entry=0x13694cc <error_fatal>) at ../util/qemu-option.c:1135 #7 0x0086c3b4 in configure_accelerators (progname=<optimized out>) at ../softmmu/vl.c:2293 #8 0x0086c3b4 in qemu_init (argc=<optimized out>, argv=<optimized out>) at ../softmmu/vl.c:3561 #9 0x0060acd4 in main (argc=<optimized out>, argv=<optimized out>) at ../softmmu/main.c:47 Richard, any thoughts on this at all? ATB, Mark.
Am 30. Januar 2023 20:45:47 UTC schrieb "Alex Bennée" <alex.bennee@linaro.org>:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
Are there any plans or ideas to support 128 bit architectures such as CHERI in the future? There is already a QEMU fork implementing CHERI for RISC V [1]. Also ARM has developed an experimental hardware implementation of CHERI within the Morello project where Linaro is involved as well, although the QEMU implementation is performed by the University of Cambridge [2]. I'm asking because once we deeply bake in the assumption that host size >= guest size then adding such architectures will become much harder. Best regards, Bernhard [1] https://github.com/CTSRD-CHERI/qemu [2] https://git.morello-project.org/university-of-cambridge/mirrors/qemu/-/tree/...
With regards, Daniel
On Wed, Feb 22, 2023 at 09:11:13AM +0000, Bernhard Beschow wrote:
Am 30. Januar 2023 20:45:47 UTC schrieb "Alex Bennée" <alex.bennee@linaro.org>:
Daniel P. Berrangé <berrange@redhat.com> writes:
On Mon, Jan 30, 2023 at 11:47:02AM +0000, Peter Maydell wrote:
On Mon, 30 Jan 2023 at 11:44, Thomas Huth <thuth@redhat.com> wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit
True for x86, not necessarily true for other architectures. Are you proposing to deprecate x86 32-bit, or all 32-bit? I'm not entirely sure about whether we're yet at a point where I'd want to deprecate-and-drop 32-bit arm host support.
Do we have a feeling on which aspects of 32-bit cause us the support burden ? The boring stuff like compiler errors from mismatched integer sizes is mostly quick & easy to detect simply through a cross compile.
I vaguely recall someone mentioned problems with atomic ops in the past, or was it 128-bit ints, caused implications for the codebase ?
Atomic operations on > TARGET_BIT_SIZE and cputlb when TCG_OVERSIZED_GUEST is set. Also the core TCG code and a bunch of the backends have TARGET_LONG_BITS > TCG_TARGET_REG_BITS ifdefs peppered throughout.
Are there any plans or ideas to support 128 bit architectures such as CHERI in the future? There is already a QEMU fork implementing CHERI for RISC V [1]. Also ARM has developed an experimental hardware implementation of CHERI within the Morello project where Linaro is involved as well, although the QEMU implementation is performed by the University of Cambridge [2].
If 128 bit hardware exists and has real world non-toy usage, then a request to support it in QEMU is essentially inevitable.
I'm asking because once we deeply bake in the assumption that host size >= guest size then adding such architectures will become much harder.
Yep, that is a risk. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Feb 22, 2023 at 09:51:57AM +0000, Daniel P. Berrangé wrote:
On Wed, Feb 22, 2023 at 09:11:13AM +0000, Bernhard Beschow wrote:
Are there any plans or ideas to support 128 bit architectures such as CHERI in the future? There is already a QEMU fork implementing CHERI for RISC V [1]. Also ARM has developed an experimental hardware implementation of CHERI within the Morello project where Linaro is involved as well, although the QEMU implementation is performed by the University of Cambridge [2].
If 128 bit hardware exists and has real world non-toy usage, then a request to support it in QEMU is essentially inevitable.
I'm asking because once we deeply bake in the assumption that host size >= guest size then adding such architectures will become much harder.
Yep, that is a risk.
I can second that. Better keep it in the code and deal with it. Maybe those specific parts can be implemented in such a way that it can easily become a noop on host size >= guest size. With regards, Reinoud
Reinoud Zandijk <reinoud@NetBSD.org> writes:
On Wed, Feb 22, 2023 at 09:51:57AM +0000, Daniel P. Berrangé wrote:
On Wed, Feb 22, 2023 at 09:11:13AM +0000, Bernhard Beschow wrote:
Are there any plans or ideas to support 128 bit architectures such as CHERI in the future? There is already a QEMU fork implementing CHERI for RISC V [1]. Also ARM has developed an experimental hardware implementation of CHERI within the Morello project where Linaro is involved as well, although the QEMU implementation is performed by the University of Cambridge [2].
If 128 bit hardware exists and has real world non-toy usage, then a request to support it in QEMU is essentially inevitable.
I'm asking because once we deeply bake in the assumption that host size >= guest size then adding such architectures will become much harder.
Yep, that is a risk.
I can second that. Better keep it in the code and deal with it. Maybe those specific parts can be implemented in such a way that it can easily become a noop on host size >= guest size.
AIUI these don't expose bigger addresses or natural atomic sizes which is where things get tricky for the TCG. All the extra bits are used for authentication or permission checks. -- Alex Bennée Virtualisation Tech Lead @ Linaro
I feel the discussion petered out without a conclusion. I don't think letting the status quo win by inertia is a good outcome here. Which 32-bit hosts are still useful, and why? Please note my question is not about the cost of keeping them (or savings from not keeping them), it's about the value they provide. When value rounds to zero, cost is irrelevant, so let's get a firm idea of value *first*.
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Please note my question is not about the cost of keeping them (or savings from not keeping them), it's about the value they provide. When value rounds to zero, cost is irrelevant, so let's get a firm idea of value *first*.
With my OS maintainer hat on, 32-bit host support is not relevant to Fedora[1] or RHEL. Both have dropped all 32-bit host install support, only maintaining 32-bit builds of some libaries for use on 64-bit host install, which is not relevant to QEMU's deliverables. With regards, Daniel [1] Arm7 was the last 32-bit host in Fedora, dropped after F36 -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 2/17/23 11:36, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Hi Markus, if the question is very very general, my opinion is that some of the most useful 32-bit systems are things like ARMv8-R capable systems for deeply embedded real time applications. Ciao, Claudio
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Which 32-bit hosts does Linux still provide KVM support for. If any, is there an EOL date for Linux 32-bit KVM support ? With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
Which 32-bit hosts are still useful, and why?
Citing my previous mail: I now checked all downloads of the latests installers since 2022-12-30. qemu-w32-setup-20221230.exe – 509 different IP addresses qemu-w64-setup-20221230.exe - 5471 different IP addresses 339 unique IP addresses are common for 32- and 64-bit, either crawlers or people who simply got both variants. So there remain 170 IP addresses which only downloaded the 32-bit variant in the last week. I see 437 different strings for the browser type, but surprisingly none of them looks like a crawler. So there still seems to be a certain small need for QEMU installers for 32-bit Windows: 170 users für 32 bit only, 339 users for both 32 and 64 bit, 5132 users for 64 bit only. Stefan
Stefan Weil <sw@weilnetz.de> writes:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
Which 32-bit hosts are still useful, and why?
Citing my previous mail:
I now checked all downloads of the latests installers since 2022-12-30.
qemu-w32-setup-20221230.exe – 509 different IP addresses qemu-w64-setup-20221230.exe - 5471 different IP addresses
339 unique IP addresses are common for 32- and 64-bit, either crawlers or people who simply got both variants. So there remain 170 IP addresses which only downloaded the 32-bit variant in the last week.
I see 437 different strings for the browser type, but surprisingly none of them looks like a crawler.
So there still seems to be a certain small need for QEMU installers for 32-bit Windows: 170 users für 32 bit only, 339 users for both 32 and 64 bit, 5132 users for 64 bit only.
Actual data is always welcome! I wonder, though... given how decrepit 32-bit-only PCs must now be, and how well Windows 10 (the only version still supporting them) runs on these, how many downloaders of the w32 version could (and quite probably should) use the w64 version instead?
On Fri, Feb 17, 2023 at 12:05:46PM +0100, Stefan Weil wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
Which 32-bit hosts are still useful, and why?
Citing my previous mail:
I now checked all downloads of the latests installers since 2022-12-30.
qemu-w32-setup-20221230.exe – 509 different IP addresses qemu-w64-setup-20221230.exe - 5471 different IP addresses
339 unique IP addresses are common for 32- and 64-bit, either crawlers or people who simply got both variants. So there remain 170 IP addresses which only downloaded the 32-bit variant in the last week.
I see 437 different strings for the browser type, but surprisingly none of them looks like a crawler.
So there still seems to be a certain small need for QEMU installers for 32-bit Windows: 170 users für 32 bit only, 339 users for both 32 and 64 bit, 5132 users for 64 bit only.
The question which is hard/impossible to answer is whether the people who downloaded the 32-bit build genuinely needed a 32-bit build or just did so out of habit or confusion. I know you can't believe everything you see with statistics, but as an example, the chart at the bottom of this page suggests new deployments of 32-bit Windows are negligible today: https://www.pcbenchmarks.net/os-marketshare.html there are existing deployments not accounted for there, but that may still suggest many of the 32-bit downloads of QEMU will end up being run on 64-bit hosts. If we were to apply our support platform rule of only targetting the latest 2 versions of the OS, this limits our targets to Win 10 and Win 11. Windows 11 dropped 32-bit IIUC, so we're talking about 32-bit installs of Windows 10 only - even in Win10 days all new physical hardware would have been 64-bit capable. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Feb 17, 2023 at 12:05:46PM +0100, Stefan Weil wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
Which 32-bit hosts are still useful, and why?
Citing my previous mail:
I now checked all downloads of the latests installers since 2022-12-30.
qemu-w32-setup-20221230.exe – 509 different IP addresses qemu-w64-setup-20221230.exe - 5471 different IP addresses
339 unique IP addresses are common for 32- and 64-bit, either crawlers or people who simply got both variants. So there remain 170 IP addresses which only downloaded the 32-bit variant in the last week.
I see 437 different strings for the browser type, but surprisingly none of them looks like a crawler.
So there still seems to be a certain small need for QEMU installers for 32-bit Windows: 170 users für 32 bit only, 339 users for both 32 and 64 bit, 5132 users for 64 bit only.
As you seem to have access to download stats could you check generic download stats too i.e. not only for Windows installers? Also, how do you account for all 32 bit host downloads from say mirrors or the source code downloads that are compiled and redistributed by 3rd parties for 32 bit hosts? Although I am not that happy with dropping support on any platform I don't think such a decision should be taken that lightly. Maybe a change in the test setup would be handier? I.e. only test 32 bit hosts say once a 2 days and the 64 bit hosts each day? Reinoud
Am 19.02.23 um 12:27 schrieb Reinoud Zandijk:
On Fri, Feb 17, 2023 at 12:05:46PM +0100, Stefan Weil wrote:
So there still seems to be a certain small need for QEMU installers for 32-bit Windows: 170 users für 32 bit only, 339 users for both 32 and 64 bit, 5132 users for 64 bit only. As you seem to have access to download stats could you check generic download stats too i.e. not only for Windows installers?
Sorry, but I only have statistics for my own server https://qemu.weilnetz.de/ which provides Windows installers. Stefan
On 2/17/23 11:47, Daniel P. Berrangé wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Which 32-bit hosts does Linux still provide KVM support for.
All except ARM: MIPS, x86, PPC and RISC-V. I would like to remove x86, but encountered some objections. MIPS, nobody is really using it I think. So that leaves PPC and RISC-V.
If any, is there an EOL date for Linux 32-bit KVM support ?
No, and I don't think there's going to be one. Paolo
(Cc'ing Huacai & Jiaxun). On 17/2/23 17:38, Paolo Bonzini wrote:
On 2/17/23 11:47, Daniel P. Berrangé wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Which 32-bit hosts does Linux still provide KVM support for.
All except ARM: MIPS, x86, PPC and RISC-V.
I would like to remove x86, but encountered some objections.
MIPS, nobody is really using it I think.
32-bit was added in 2014, commit 222e7d11e7 ("target-mips: Enable KVM support in build system"). I'm not aware of anybody using it (even testing it). I don't have hardware to test it (neither time). We are still cross-compiling it although. 64-bit support was added recently (see commit aa2953fd16 "configure: Add KVM target support for MIPS64") and is used (see commit fbc5884ce2 "meson.build: Re-enable KVM support for MIPS" from 2020), however I tend to see it more as hobbyist use than production one. Besides it is listed as 'Odd Fixes' in MAINTAINERS (still 2020, commit 134f7f7da1 "MAINTAINERS: Reactivate MIPS KVM CPUs").
So that leaves PPC and RISC-V.
If any, is there an EOL date for Linux 32-bit KVM support ?
No, and I don't think there's going to be one.
Paolo
On 17/02/2023 18.43, Philippe Mathieu-Daudé wrote:
(Cc'ing Huacai & Jiaxun).
On 17/2/23 17:38, Paolo Bonzini wrote:
On 2/17/23 11:47, Daniel P. Berrangé wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Which 32-bit hosts does Linux still provide KVM support for.
All except ARM: MIPS, x86, PPC and RISC-V.
I would like to remove x86, but encountered some objections.
MIPS, nobody is really using it I think.
32-bit was added in 2014, commit 222e7d11e7 ("target-mips: Enable KVM support in build system"). I'm not aware of anybody using it (even testing it). I don't have hardware to test it (neither time).
Could you maybe suggest a kernel patch to remove it, to see what happens? ... if nobody objects to the removal of the 32-bit MIPS KVM kernel support and the patch gets merged, that would help us in the long run, I think. Thanks, Thomas
在2023年2月17日二月 下午6:57，Thomas Huth写道：
On 17/02/2023 18.43, Philippe Mathieu-Daudé wrote:
(Cc'ing Huacai & Jiaxun).
On 17/2/23 17:38, Paolo Bonzini wrote:
On 2/17/23 11:47, Daniel P. Berrangé wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Which 32-bit hosts does Linux still provide KVM support for.
All except ARM: MIPS, x86, PPC and RISC-V.
I would like to remove x86, but encountered some objections.
MIPS, nobody is really using it I think.
32-bit was added in 2014, commit 222e7d11e7 ("target-mips: Enable KVM support in build system"). I'm not aware of anybody using it (even testing it). I don't have hardware to test it (neither time).
Could you maybe suggest a kernel patch to remove it, to see what happens? ... if nobody objects to the removal of the 32-bit MIPS KVM kernel support and the patch gets merged, that would help us in the long run, I think.
I’m still occasionally testing 32-bit MIPS KVM support with MIPS P5600. It works just fine so there is no need for further maintenance work. I’d be sad to see the support go but I can live with it. There are commercial users for MIPS 32 KVM but they’re all running customized downstream QEMU so I guess it’s fine. Thanks - Jiaxun
Thanks, Thomas
-- - Jiaxun
On 17/02/2023 17.38, Paolo Bonzini wrote:
On 2/17/23 11:47, Daniel P. Berrangé wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
Which 32-bit hosts does Linux still provide KVM support for.
All except ARM: MIPS, x86, PPC and RISC-V.
I would like to remove x86, but encountered some objections.
So if I got that right: https://lore.kernel.org/all/b8fa9561295bb6af2b7fcaa8125c6a3b89b305c7.camel@r... ... the objection is mainly that some kernel developers want to keep the code around for easier testing of nested 32-bit guests L1 hypervisors. If that's the only use case that is still around for the 32-bit KVM x86 kernel code, I guess it should also be fine to use older versions of QEMU in those L1 hypervisor guests (assuming you have to use an older 32-bit Linux distro for this anyway). So unless I got that wrong, there is really nobody around anymore who needs an *upstream* QEMU for running 32-bit x86 KVM hosts, is there? Please correct me if I'm wrong, but I think we can really deprecated at least qemu-system-i386 and qemu-system-arm now, can't we? Thanks, Thomas
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
NetBSD runs on a bunch of 32 bit-only hosts (sparc32, ppc32, armv7, vax, mips32 etc.) that all run Qemu fine. They are all actively maintained and released as part of the main releases. Maintaining 32 bit host support is thus vital for those machines; not everyone runs a 64 bit system. with regards, Reinoud Zandijk, NetBSD developer.
On Fri, Feb 17, 2023 at 05:06:42PM +0100, Reinoud Zandijk wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
NetBSD runs on a bunch of 32 bit-only hosts (sparc32, ppc32, armv7, vax, mips32 etc.) that all run Qemu fine. They are all actively maintained and released as part of the main releases.
Maintaining 32 bit host support is thus vital for those machines; not everyone runs a 64 bit system.
In the context of NetBSD, is QEMU a leaf node package, or is it a dependancy of anything involved in delivery of the distro ? ie, if QEMU were to drop 32-bit support, if it is merely a leaf node package, then end users of NetBSD would not have access to QEMU, but you would still be able to deliver the rest of NetBSD on those platforms without the QEMU package present ? With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 2/17/23 06:06, Reinoud Zandijk wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
NetBSD runs on a bunch of 32 bit-only hosts (sparc32, ppc32, armv7, vax, mips32 etc.) that all run Qemu fine. They are all actively maintained and released as part of the main releases.
Are you sure about that? TCG doesn't support sparc32 or vax. I suppose you could be using TCI, but I can't even imagine how slow that would be on vax. r~
On Fri, Feb 17, 2023 at 08:22:43AM -1000, Richard Henderson wrote:
On 2/17/23 06:06, Reinoud Zandijk wrote:
On Fri, Feb 17, 2023 at 11:36:41AM +0100, Markus Armbruster wrote:
I feel the discussion petered out without a conclusion.
I don't think letting the status quo win by inertia is a good outcome here.
Which 32-bit hosts are still useful, and why?
NetBSD runs on a bunch of 32 bit-only hosts (sparc32, ppc32, armv7, vax, mips32 etc.) that all run Qemu fine. They are all actively maintained and released as part of the main releases.
Are you sure about that? TCG doesn't support sparc32 or vax. I suppose you could be using TCI, but I can't even imagine how slow that would be on vax.
I've asked around amongst active developers and they steted that they had used it in the past but not ianymore recently. I doubt normal users would use them often though since most hardware is 64 bit capable anyways. Reinoud
On 1/30/23 12:44, Thomas Huth wrote:
Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit, keeping the 32-bit support alive is an inadequate burden for the QEMU project. Let's mark the 32-bit support as deprecated so we can drop it after a while - this will help us to cut down our limited CI minutes in the gitlab CI, for example.
Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 9f1bbc495d..ce6463e72b 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -181,9 +181,20 @@ As Debian 10 ("Buster") moved into LTS the big endian 32 bit version of MIPS moved out of support making it hard to maintain our cross-compilation CI tests of the architecture. As we no longer have CI coverage support may bitrot away before the deprecation process -completes. The little endian variants of MIPS (both 32 and 64 bit) are +completes. The little endian variants of MIPS are still a supported host architecture.
+32-bit host operating systems (since 8.0) +''''''''''''''''''''''''''''''''''''''''' + +Testing 32-bit host OS support takes a lot of precious time during the QEMU +contiguous integration tests, and considering that many OS vendors stopped +shipping 32-bit variants of their OS distributions and most hardware from +the past >10 years is capable of 64-bit, keeping the 32-bit support alive +is an inadequate burden for the QEMU project. Thus QEMU will soon drop the +support for 32-bit host systems. + + QEMU API (QAPI) events ----------------------
From our (SUSE) support perspective, we do not support 32bit virtualization hosts downstream,
and in general I am in favor of the change. Ciao, Claudio
participants (16)
-
Alex Bennée -
BALATON Zoltan -
Bernhard Beschow -
Claudio Fontana -
Cédric Le Goater -
Daniel P. Berrangé -
Jiaxun Yang -
Mark Cave-Ayland -
Markus Armbruster -
Paolo Bonzini -
Peter Maydell -
Philippe Mathieu-Daudé -
Reinoud Zandijk -
Richard Henderson -
Stefan Weil -
Thomas Huth