[PATCH v4 0/5] Deprecate system emulation support for 32-bit x86 and arm hosts
We're struggling quite badly with our CI minutes on the shared gitlab runners, so we urgently need to think of ways to cut down our supported build and target environments. Hardly anybody uses QEMU system emulation on 32-bit x86 and arm hosts anymore, so it's time to deprecate these environments now. This is a follow-up patch series from the previous discussion here: https://lore.kernel.org/qemu-devel/20230130114428.1297295-1-thuth@redhat.com... where people still mentioned that there is still interest in certain support for 32-bit host hardware. But as far as I could see, there is no real need for system emulation on 32-bit x86 host and 32-bit arm hosts anymore, so it should be fine if we drop these host environments soon (these are also the two architectures that contribute the most to the long test times in our CI, so we would benefit a lot by dropping those). v4: - Drop the patch that deprecated qemu-system-arm since it still might be required to enforce 32-bit guests with TCG - Only deprecate system emulation on 32-bit x86 hosts since user-mode emulation might still be useful in certain scenarios - Add a sentence how to enforce 32-bit mode with qemu-system-x86_64 v3: - Update some commit descriptions according to the suggestions in v2 - Added the Reviewed-bys from v2 v2: - Split binary and host deprecation into separate patches - Added patches to immediately drop the jobs from the CI Thomas Huth (5): docs/about/deprecated: Deprecate 32-bit x86 hosts for system emulation docs/about/deprecated: Deprecate the qemu-system-i386 binary gitlab-ci.d/crossbuilds: Drop the i386 system emulation job docs/about/deprecated: Deprecate 32-bit arm hosts for system emulation gitlab-ci.d/crossbuilds: Drop the 32-bit arm system emulation jobs docs/about/deprecated.rst | 34 ++++++++++++++++++++++++++++++++++ .gitlab-ci.d/crossbuilds.yml | 24 ------------------------ 2 files changed, 34 insertions(+), 24 deletions(-) -- 2.31.1
Hardly anybody still uses 32-bit x86 hosts today, so we should start deprecating them to stop wasting our time and CI minutes here. For example, there are also still some unresolved problems with these: When emulating 64-bit binaries in user mode, TCG does not honor atomicity for 64-bit accesses, which is "perhaps worse than not working at all" (quoting Richard). Let's simply make it clear that people should use 64-bit x86 hosts nowadays and we do not intend to fix/maintain the old 32-bit stuff. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 15084f7bea..1ca9dc33d6 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -196,6 +196,17 @@ CI coverage support may bitrot away before the deprecation process completes. The little endian variants of MIPS (both 32 and 64 bit) are still a supported host architecture. +System emulation on 32-bit x86 hosts (since 8.0) +'''''''''''''''''''''''''''''''''''''''''''''''' + +Support for 32-bit x86 host deployments is increasingly uncommon in mainstream +OS distributions given the widespread availability of 64-bit x86 hardware. +The QEMU project no longer considers 32-bit x86 support for system emulation to +be an effective use of its limited resources, and thus intends to discontinue +it. Since all recent x86 hardware from the past >10 years is capable of the +64-bit x86 extensions, a corresponding 64-bit OS should be used instead. + + QEMU API (QAPI) events ---------------------- -- 2.31.1
Aside from not supporting KVM on 32-bit hosts, the qemu-system-x86_64 binary is a proper superset of the qemu-system-i386 binary. With the 32-bit host support being deprecated, it is now also possible to deprecate the qemu-system-i386 binary. With regards to 32-bit KVM support in the x86 Linux kernel, the developers confirmed that they do not need a recent qemu-system-i386 binary here: https://lore.kernel.org/kvm/Y%2ffkTs5ajFy0hP1U@google.com/ Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1ca9dc33d6..c4fcc6b33c 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -34,6 +34,20 @@ deprecating the build option and no longer defend it in CI. The ``--enable-gcov`` build option remains for analysis test case coverage. +``qemu-system-i386`` binary (since 8.0) +''''''''''''''''''''''''''''''''''''''' + +The ``qemu-system-i386`` binary was mainly useful for running with KVM +on 32-bit x86 hosts, but most Linux distributions already removed their +support for 32-bit x86 kernels, so hardly anybody still needs this. The +``qemu-system-x86_64`` binary is a proper superset and can be used to +run 32-bit guests by selecting a 32-bit CPU model, including KVM support +on x86_64 hosts. Thus users are recommended to reconfigure their systems +to use the ``qemu-system-x86_64`` binary instead. If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``. + + System emulator command line arguments -------------------------------------- -- 2.31.1
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
Aside from not supporting KVM on 32-bit hosts, the qemu-system-x86_64 binary is a proper superset of the qemu-system-i386 binary. With the 32-bit host support being deprecated, it is now also possible to deprecate the qemu-system-i386 binary.
With regards to 32-bit KVM support in the x86 Linux kernel, the developers confirmed that they do not need a recent qemu-system-i386 binary here:
https://lore.kernel.org/kvm/Y%2ffkTs5ajFy0hP1U@google.com/
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1ca9dc33d6..c4fcc6b33c 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -34,6 +34,20 @@ deprecating the build option and no longer defend it in CI. The ``--enable-gcov`` build option remains for analysis test case coverage.
+``qemu-system-i386`` binary (since 8.0) +''''''''''''''''''''''''''''''''''''''' + +The ``qemu-system-i386`` binary was mainly useful for running with KVM +on 32-bit x86 hosts, but most Linux distributions already removed their +support for 32-bit x86 kernels, so hardly anybody still needs this. The +``qemu-system-x86_64`` binary is a proper superset and can be used to +run 32-bit guests by selecting a 32-bit CPU model, including KVM support +on x86_64 hosts. Thus users are recommended to reconfigure their systems +to use the ``qemu-system-x86_64`` binary instead. If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, because we have code that changes the family/model/stepping for 'max' which is target dependent: #ifdef TARGET_X86_64 object_property_set_int(OBJECT(cpu), "family", 15, &error_abort); object_property_set_int(OBJECT(cpu), "model", 107, &error_abort); object_property_set_int(OBJECT(cpu), "stepping", 1, &error_abort); #else object_property_set_int(OBJECT(cpu), "family", 6, &error_abort); object_property_set_int(OBJECT(cpu), "model", 6, &error_abort); object_property_set_int(OBJECT(cpu), "stepping", 3, &error_abort); #endif The former is a 64-bit AMD model and the latter is a 32-bit model. Seems LLVM was sensitive to this distinction to some extent: https://gitlab.com/qemu-project/qemu/-/issues/191 A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
Aside from not supporting KVM on 32-bit hosts, the qemu-system-x86_64 binary is a proper superset of the qemu-system-i386 binary. With the 32-bit host support being deprecated, it is now also possible to deprecate the qemu-system-i386 binary.
With regards to 32-bit KVM support in the x86 Linux kernel, the developers confirmed that they do not need a recent qemu-system-i386 binary here:
https://lore.kernel.org/kvm/Y%2ffkTs5ajFy0hP1U@google.com/
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1ca9dc33d6..c4fcc6b33c 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -34,6 +34,20 @@ deprecating the build option and no longer defend it in CI. The ``--enable-gcov`` build option remains for analysis test case coverage.
+``qemu-system-i386`` binary (since 8.0) +''''''''''''''''''''''''''''''''''''''' + +The ``qemu-system-i386`` binary was mainly useful for running with KVM +on 32-bit x86 hosts, but most Linux distributions already removed their +support for 32-bit x86 kernels, so hardly anybody still needs this. The +``qemu-system-x86_64`` binary is a proper superset and can be used to +run 32-bit guests by selecting a 32-bit CPU model, including KVM support +on x86_64 hosts. Thus users are recommended to reconfigure their systems +to use the ``qemu-system-x86_64`` binary instead. If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, because we have code that changes the family/model/stepping for 'max' which is target dependent:
#ifdef TARGET_X86_64 object_property_set_int(OBJECT(cpu), "family", 15, &error_abort); object_property_set_int(OBJECT(cpu), "model", 107, &error_abort); object_property_set_int(OBJECT(cpu), "stepping", 1, &error_abort); #else object_property_set_int(OBJECT(cpu), "family", 6, &error_abort); object_property_set_int(OBJECT(cpu), "model", 6, &error_abort); object_property_set_int(OBJECT(cpu), "stepping", 3, &error_abort); #endif
The former is a 64-bit AMD model and the latter is a 32-bit model.
Seems LLVM was sensitive to this distinction to some extent:
https://gitlab.com/qemu-project/qemu/-/issues/191
A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
Ugh, ok. I gave it a quick try with a patch like this: diff --git a/target/i386/cpu.c b/target/i386/cpu.c --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -4344,15 +4344,15 @@ static void max_x86_cpu_initfn(Object *obj) */ object_property_set_str(OBJECT(cpu), "vendor", CPUID_VENDOR_AMD, &error_abort); -#ifdef TARGET_X86_64 - object_property_set_int(OBJECT(cpu), "family", 15, &error_abort); - object_property_set_int(OBJECT(cpu), "model", 107, &error_abort); - object_property_set_int(OBJECT(cpu), "stepping", 1, &error_abort); -#else - object_property_set_int(OBJECT(cpu), "family", 6, &error_abort); - object_property_set_int(OBJECT(cpu), "model", 6, &error_abort); - object_property_set_int(OBJECT(cpu), "stepping", 3, &error_abort); -#endif + if (object_property_get_bool(obj, "lm", &error_abort)) { + object_property_set_int(obj, "family", 15, &error_abort); + object_property_set_int(obj, "model", 107, &error_abort); + object_property_set_int(obj, "stepping", 1, &error_abort); + } else { + object_property_set_int(obj, "family", 6, &error_abort); + object_property_set_int(obj, "model", 6, &error_abort); + object_property_set_int(obj, "stepping", 3, &error_abort); + } object_property_set_str(OBJECT(cpu), "model-id", "QEMU TCG CPU version " QEMU_HW_VERSION, &error_abort); ... but it seems like the "lm" property is not initialized there yet, so this does not work... :-/ Giving that we have soft-freeze tomorrow, let's ignore this patch for now and revisit this topic during the 8.1 cycle. But I'll queue the other 4 patches to get some pressure out of our CI during the freeze time. Thomas
On Mon, Mar 06, 2023 at 10:54:15AM +0100, Thomas Huth wrote:
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
Aside from not supporting KVM on 32-bit hosts, the qemu-system-x86_64 binary is a proper superset of the qemu-system-i386 binary. With the 32-bit host support being deprecated, it is now also possible to deprecate the qemu-system-i386 binary.
With regards to 32-bit KVM support in the x86 Linux kernel, the developers confirmed that they do not need a recent qemu-system-i386 binary here:
https://lore.kernel.org/kvm/Y%2ffkTs5ajFy0hP1U@google.com/
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1ca9dc33d6..c4fcc6b33c 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -34,6 +34,20 @@ deprecating the build option and no longer defend it in CI. The ``--enable-gcov`` build option remains for analysis test case coverage. +``qemu-system-i386`` binary (since 8.0) +''''''''''''''''''''''''''''''''''''''' + +The ``qemu-system-i386`` binary was mainly useful for running with KVM +on 32-bit x86 hosts, but most Linux distributions already removed their +support for 32-bit x86 kernels, so hardly anybody still needs this. The +``qemu-system-x86_64`` binary is a proper superset and can be used to +run 32-bit guests by selecting a 32-bit CPU model, including KVM support +on x86_64 hosts. Thus users are recommended to reconfigure their systems +to use the ``qemu-system-x86_64`` binary instead. If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, because we have code that changes the family/model/stepping for 'max' which is target dependent:
#ifdef TARGET_X86_64 object_property_set_int(OBJECT(cpu), "family", 15, &error_abort); object_property_set_int(OBJECT(cpu), "model", 107, &error_abort); object_property_set_int(OBJECT(cpu), "stepping", 1, &error_abort); #else object_property_set_int(OBJECT(cpu), "family", 6, &error_abort); object_property_set_int(OBJECT(cpu), "model", 6, &error_abort); object_property_set_int(OBJECT(cpu), "stepping", 3, &error_abort); #endif
The former is a 64-bit AMD model and the latter is a 32-bit model.
Seems LLVM was sensitive to this distinction to some extent:
https://gitlab.com/qemu-project/qemu/-/issues/191
A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
Ugh, ok. I gave it a quick try with a patch like this:
diff --git a/target/i386/cpu.c b/target/i386/cpu.c --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -4344,15 +4344,15 @@ static void max_x86_cpu_initfn(Object *obj) */ object_property_set_str(OBJECT(cpu), "vendor", CPUID_VENDOR_AMD, &error_abort); -#ifdef TARGET_X86_64 - object_property_set_int(OBJECT(cpu), "family", 15, &error_abort); - object_property_set_int(OBJECT(cpu), "model", 107, &error_abort); - object_property_set_int(OBJECT(cpu), "stepping", 1, &error_abort); -#else - object_property_set_int(OBJECT(cpu), "family", 6, &error_abort); - object_property_set_int(OBJECT(cpu), "model", 6, &error_abort); - object_property_set_int(OBJECT(cpu), "stepping", 3, &error_abort); -#endif + if (object_property_get_bool(obj, "lm", &error_abort)) { + object_property_set_int(obj, "family", 15, &error_abort); + object_property_set_int(obj, "model", 107, &error_abort); + object_property_set_int(obj, "stepping", 1, &error_abort); + } else { + object_property_set_int(obj, "family", 6, &error_abort); + object_property_set_int(obj, "model", 6, &error_abort); + object_property_set_int(obj, "stepping", 3, &error_abort); + } object_property_set_str(OBJECT(cpu), "model-id", "QEMU TCG CPU version " QEMU_HW_VERSION, &error_abort);
... but it seems like the "lm" property is not initialized there yet, so this does not work... :-/
Giving that we have soft-freeze tomorrow, let's ignore this patch for now and revisit this topic during the 8.1 cycle. But I'll queue the other 4 patches to get some pressure out of our CI during the freeze time.
Yep, makes sense. More generally the whole impl of the 'max' CPU feels somewhat questionable even for qemu-system-i386. It exposes all features that TCG supports. A large set of these features never existed on *any* 32-bit silicon. Hands up who has seen 32-bit silicon with AVX2 support ? From a correctness POV we should have capped CPU features in some manner. Given the lack of interest in 32-bit though, we've ignored the problem and it likely does not affect apps anyway as they're not likely to be looking for newish features. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
[...] If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, [...] A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
I think I just spotted this by accident in target/i386/cpu.c around line 637: #ifdef TARGET_X86_64 #define TCG_EXT2_X86_64_FEATURES (CPUID_EXT2_SYSCALL | CPUID_EXT2_LM) #else #define TCG_EXT2_X86_64_FEATURES 0 #endif Thomas
On Mon, Mar 06, 2023 at 02:48:16PM +0100, Thomas Huth wrote:
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
[...] If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, [...] A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
I think I just spotted this by accident in target/i386/cpu.c around line 637:
#ifdef TARGET_X86_64 #define TCG_EXT2_X86_64_FEATURES (CPUID_EXT2_SYSCALL | CPUID_EXT2_LM) #else #define TCG_EXT2_X86_64_FEATURES 0 #endif
Hmm, so right now the difference between qemu-system-i386 and qemu-system-x86_64 is based on compile time conditionals. So we have the burden of building everything twice and also a burden of testing everything twice. If we eliminate qemu-system-i386 we get rid of our own burden, but users/mgmt apps need to adapt to force qemu-system-x86_64 to present a 32-bit system. What about if we had qemu-system-i386 be a hardlink to qemu-system-x86_64, and then changed behaviour based off the executed binary name ? ie if running qemu-system-i386, we could present a 32-bit CPU by default. We eliminate all of our double compilation burden still. We still have extra testing burden, but it is in a fairly narrow area, so does not imply x2 the testing burden just $small-percentage extra testing ? That would means apps/users would not need to change at all, but we still get most of the win we're after on the QEMU side Essentially #ifdef TARGET_X86_64 would be change 'if (is_64bit) {...}' in a handful of places, with 'bool is_64bit' initialized in main() from argv[0] ? With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 06/03/2023 15.06, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 02:48:16PM +0100, Thomas Huth wrote:
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
[...] If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, [...] A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
I think I just spotted this by accident in target/i386/cpu.c around line 637:
#ifdef TARGET_X86_64 #define TCG_EXT2_X86_64_FEATURES (CPUID_EXT2_SYSCALL | CPUID_EXT2_LM) #else #define TCG_EXT2_X86_64_FEATURES 0 #endif
Hmm, so right now the difference between qemu-system-i386 and qemu-system-x86_64 is based on compile time conditionals. So we have the burden of building everything twice and also a burden of testing everything twice.
If we eliminate qemu-system-i386 we get rid of our own burden, but users/mgmt apps need to adapt to force qemu-system-x86_64 to present a 32-bit system.
What about if we had qemu-system-i386 be a hardlink to qemu-system-x86_64, and then changed behaviour based off the executed binary name ?
We could also simply provide a shell script that runs: qemu-system-x86_64 -cpu qemu32 $* ... that'd sounds like the simplest solution to me. Thomas
On Mon, Mar 06, 2023 at 03:18:23PM +0100, Thomas Huth wrote:
On 06/03/2023 15.06, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 02:48:16PM +0100, Thomas Huth wrote:
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
[...] If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, [...] A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
I think I just spotted this by accident in target/i386/cpu.c around line 637:
#ifdef TARGET_X86_64 #define TCG_EXT2_X86_64_FEATURES (CPUID_EXT2_SYSCALL | CPUID_EXT2_LM) #else #define TCG_EXT2_X86_64_FEATURES 0 #endif
Hmm, so right now the difference between qemu-system-i386 and qemu-system-x86_64 is based on compile time conditionals. So we have the burden of building everything twice and also a burden of testing everything twice.
If we eliminate qemu-system-i386 we get rid of our own burden, but users/mgmt apps need to adapt to force qemu-system-x86_64 to present a 32-bit system.
What about if we had qemu-system-i386 be a hardlink to qemu-system-x86_64, and then changed behaviour based off the executed binary name ?
We could also simply provide a shell script that runs:
qemu-system-x86_64 -cpu qemu32 $*
... that'd sounds like the simplest solution to me.
That woudn't do the right thing if the user ran 'qemu-system-i386 -cpu max' because their '-cpu max' would override the -cpu arg in the shell script that forced 32-bit mode. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Mar 06, 2023 at 02:25:46PM +0000, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 03:18:23PM +0100, Thomas Huth wrote:
On 06/03/2023 15.06, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 02:48:16PM +0100, Thomas Huth wrote:
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
[...] If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, [...] A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
I think I just spotted this by accident in target/i386/cpu.c around line 637:
#ifdef TARGET_X86_64 #define TCG_EXT2_X86_64_FEATURES (CPUID_EXT2_SYSCALL | CPUID_EXT2_LM) #else #define TCG_EXT2_X86_64_FEATURES 0 #endif
Hmm, so right now the difference between qemu-system-i386 and qemu-system-x86_64 is based on compile time conditionals. So we have the burden of building everything twice and also a burden of testing everything twice.
If we eliminate qemu-system-i386 we get rid of our own burden, but users/mgmt apps need to adapt to force qemu-system-x86_64 to present a 32-bit system.
What about if we had qemu-system-i386 be a hardlink to qemu-system-x86_64, and then changed behaviour based off the executed binary name ?
We could also simply provide a shell script that runs:
qemu-system-x86_64 -cpu qemu32 $*
... that'd sounds like the simplest solution to me.
That woudn't do the right thing if the user ran 'qemu-system-i386 -cpu max' because their '-cpu max' would override the -cpu arg in the shell script that forced 32-bit mode.
It would also fail to work with SELinux, because policy restrictions doesn't allow for an intermediate wrapper script to exec binaries. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 6/3/23 15:06, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 02:48:16PM +0100, Thomas Huth wrote:
On 06/03/2023 10.27, Daniel P. Berrangé wrote:
On Mon, Mar 06, 2023 at 09:46:55AM +0100, Thomas Huth wrote:
[...] If a 32-bit CPU guest +environment should be enforced, you can switch off the "long mode" CPU +flag, e.g. with ``-cpu max,lm=off``.
I had the idea to check this today and this is not quite sufficient, [...] A further difference is that qemy-system-i686 does not appear to enable the 'syscall' flag, but I've not figured out where that difference is coming from in the code.
I think I just spotted this by accident in target/i386/cpu.c around line 637:
#ifdef TARGET_X86_64 #define TCG_EXT2_X86_64_FEATURES (CPUID_EXT2_SYSCALL | CPUID_EXT2_LM) #else #define TCG_EXT2_X86_64_FEATURES 0 #endif
Hmm, so right now the difference between qemu-system-i386 and qemu-system-x86_64 is based on compile time conditionals. So we have the burden of building everything twice and also a burden of testing everything twice.
If we eliminate qemu-system-i386 we get rid of our own burden, but users/mgmt apps need to adapt to force qemu-system-x86_64 to present a 32-bit system.
What about if we had qemu-system-i386 be a hardlink to qemu-system-x86_64, and then changed behaviour based off the executed binary name ?
ie if running qemu-system-i386, we could present a 32-bit CPU by default. We eliminate all of our double compilation burden still. We still have extra testing burden, but it is in a fairly narrow area, so does not imply x2 the testing burden just $small-percentage extra testing ? That would means apps/users would not need to change at all, but we still get most of the win we're after on the QEMU side
Essentially #ifdef TARGET_X86_64 would be change 'if (is_64bit) {...}' in a handful of places, with 'bool is_64bit' initialized in main() from argv[0] ?
That is what Alex suggested me to do with ARM binaries as a prototype of unifying 32/64-bit binaries, avoiding to break users scripts.
Hardly anybody still uses 32-bit x86 environments for running QEMU with full system emulation, so let's stop wasting our scarce CI minutes with this job. (There are still the 32-bit MinGW and TCI jobs around for having some compile test coverage on 32-bit) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- .gitlab-ci.d/crossbuilds.yml | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml index d3a31a2112..a25cb87ae4 100644 --- a/.gitlab-ci.d/crossbuilds.yml +++ b/.gitlab-ci.d/crossbuilds.yml @@ -43,16 +43,6 @@ cross-arm64-user: variables: IMAGE: debian-arm64-cross -cross-i386-system: - extends: - - .cross_system_build_job - - .cross_test_artifacts - needs: - job: i386-fedora-cross-container - variables: - IMAGE: fedora-i386-cross - MAKE_CHECK_ARGS: check-qtest - cross-i386-user: extends: - .cross_user_build_job -- 2.31.1
On 6/3/23 09:46, Thomas Huth wrote:
Hardly anybody still uses 32-bit x86 environments for running QEMU with full system emulation, so let's stop wasting our scarce CI minutes with this job.
(There are still the 32-bit MinGW and TCI jobs around for having some compile test coverage on 32-bit)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- .gitlab-ci.d/crossbuilds.yml | 10 ---------- 1 file changed, 10 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
For running QEMU in system emulation mode, the user needs a rather strong host system, i.e. not only an embedded low-frequency controller. All recent beefy arm host machines should support 64-bit now, it's unlikely that anybody is still seriously using QEMU on a 32-bit arm CPU, so we deprecate the 32-bit arm hosts here to finally save use some time and precious CI minutes. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index c4fcc6b33c..f0de517dc2 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -220,6 +220,15 @@ be an effective use of its limited resources, and thus intends to discontinue it. Since all recent x86 hardware from the past >10 years is capable of the 64-bit x86 extensions, a corresponding 64-bit OS should be used instead. +System emulation on 32-bit arm hosts (since 8.0) +'''''''''''''''''''''''''''''''''''''''''''''''' + +Since QEMU needs a strong host machine for running full system emulation, and +all recent powerful arm hosts support 64-bit, the QEMU project deprecates the +support for running any system emulation on 32-bit arm hosts in general. Use +64-bit arm hosts for system emulation instead. (Note: "user" mode emulation +continuous to be supported on 32-bit arm hosts, too) + QEMU API (QAPI) events ---------------------- -- 2.31.1
On 6/3/23 09:46, Thomas Huth wrote:
For running QEMU in system emulation mode, the user needs a rather strong host system, i.e. not only an embedded low-frequency controller. All recent beefy arm host machines should support 64-bit now, it's unlikely that anybody is still seriously using QEMU on a 32-bit arm CPU, so we deprecate the 32-bit arm hosts here to finally save use some time and precious CI minutes.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- docs/about/deprecated.rst | 9 +++++++++ 1 file changed, 9 insertions(+)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Hardly anybody still uses 32-bit arm environments for running QEMU, so let's stop wasting our scarce CI minutes with these jobs. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Signed-off-by: Thomas Huth <thuth@redhat.com> --- .gitlab-ci.d/crossbuilds.yml | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml index a25cb87ae4..61b8ac86ee 100644 --- a/.gitlab-ci.d/crossbuilds.yml +++ b/.gitlab-ci.d/crossbuilds.yml @@ -1,13 +1,6 @@ include: - local: '/.gitlab-ci.d/crossbuild-template.yml' -cross-armel-system: - extends: .cross_system_build_job - needs: - job: armel-debian-cross-container - variables: - IMAGE: debian-armel-cross - cross-armel-user: extends: .cross_user_build_job needs: @@ -15,13 +8,6 @@ cross-armel-user: variables: IMAGE: debian-armel-cross -cross-armhf-system: - extends: .cross_system_build_job - needs: - job: armhf-debian-cross-container - variables: - IMAGE: debian-armhf-cross - cross-armhf-user: extends: .cross_user_build_job needs: -- 2.31.1
participants (4)
-
Daniel P. Berrangé -
Philippe Mathieu-Daudé -
Richard Henderson -
Thomas Huth