8:34 a.m.
On 2/14/25 02:06, Akio Kakuno (Fujitsu) via Devel wrote:
Hi, all!
I'm adding three test for CCA compatibility:
domaincapstest, qemucapabilitiestest, and qemuxmlconftest.
This is because SEV-SNP added these three tests.
I have three questions regarding these tests:
1. How to add tests to qemuxmlconftest
2. How to create launch-security-cca.xml
3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
1. How to add tests to qemuxmlconftest
Following the example of launch-security-sev-snp tests, I've done the
following.
Is this correct?
(1) Placed the following three files in tests/qemuxmlconfdata:
launch-security-cca.xml
launch-security-cca.aarch64-latest.xml
launch-security-cca.aarch64-latest.args
The placement is correct, yes. BUT ...
(2) Added the test processing to qemuxmlconftest.c's
mymain() function:
DO_TEST_CAPS_ARCH_LATEST_FULL("launch-security-cca",
"aarch64",
ARG_QEMU_CAPS,
QEMU_CAPS_CCA_GUEST,
QEMU_CAPS_LAST);
... this can be simplified to:
DO_TEST_CAPS_ARCH_LATEST("launch-security-cca", "aarch64");
2. How to create launch-security-cca.xml
Do I need to handmade this from scratch or is there an automated method?
Basically it's hand written. What I usually do is I copy-paste the
domain XML I used when developing and testing a feature. And then cut
off all unnecessary elements.
3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
I created launch-security-cca.aarch64-latest.* using the method described in
doc/advanced-tests.rst.
And, I created the test data for qemucapabilitiestest and domaincapstest using
the method described in tests/qemucapabilitiesdata/README.rst.
VIR_TEST_REGENERATE_OUTPUT=1 ./qemuxmlconftest
VIR_TEST_REGENERATE_OUTPUT=1 ./domaincapstest
VIR_TEST_REGENERATE_OUTPUT=1 ./qemucapabilitiestest
Can I use the generated file for testing as is?
Because doc/advanced-tests.rst says:
"VERY CAREFULLY to ensure they are correct"
> I assume that automatically generated expected values are
checked for
accuracy.
Not really. It takes a machine brain to decide whether those files
follow some syntax (e.g. whether JSON is valid), but it takes human
brain to decide whether full combination of cmd line arguments actually
makes sense. I'd say - if you're able to start generated cmd line
(modulo some FD passing stuff - see my point above about cutting off
unnecessary elements), then you're probably fine.
If correct, they are adopted; otherwise, investigation and
remediation are undertaken.
However, due to the lack of explicit documentation, we require confirmation.
Also, it appears to be generating expected values based on the execution
environment.
Do we need to worry about variations in execution environments?
No. Our tests should generate stable enough (and reproducible!) environment.
For example, executing qemuxmlconftest detects the following
comparison errors,
such as with aarch64-virt-minimal.aarch64-latest, etc.
Expect [sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-m]
Actual [m]
Yeah, printing diffs is not very userfriendly. You can get better
results with VIR_TEST_DEBUG=2.
Best Regards.
Michal
8:40 p.m.
New subject: [PATCH 1/1] RFC: Add Arm CCA support for getting capability
information and running Realm VM
Dear Michal
Thank you for your reply.
I will create a test based on your comment and post v2.
-----Original Message-----
From: Michal Prívozník <mprivozn(a)redhat.com>
Sent: Monday, February 17, 2025 11:34 PM
To: Kakuno, Akio/角野 秋男 <fj3333bs(a)fujitsu.com>; 'devel(a)lists.libvirt.org'
<devel(a)lists.libvirt.org>
Subject: Re: [PATCH 1/1] RFC: Add Arm CCA support for getting capability information and
running Realm VM
On 2/14/25 02:06, Akio Kakuno (Fujitsu) via Devel wrote:
Hi, all!
I'm adding three test for CCA compatibility:
domaincapstest, qemucapabilitiestest, and qemuxmlconftest.
This is because SEV-SNP added these three tests.
I have three questions regarding these tests:
1. How to add tests to qemuxmlconftest
2. How to create launch-security-cca.xml
3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
1. How to add tests to qemuxmlconftest
Following the example of launch-security-sev-snp tests, I've done the
following.
Is this correct?
(1) Placed the following three files in tests/qemuxmlconfdata:
launch-security-cca.xml
launch-security-cca.aarch64-latest.xml
launch-security-cca.aarch64-latest.args
The placement is correct, yes. BUT ...
(2) Added the test processing to qemuxmlconftest.c's
mymain() function:
DO_TEST_CAPS_ARCH_LATEST_FULL("launch-security-cca",
"aarch64",
ARG_QEMU_CAPS,
QEMU_CAPS_CCA_GUEST,
QEMU_CAPS_LAST);
... this can be simplified to:
DO_TEST_CAPS_ARCH_LATEST("launch-security-cca", "aarch64");
2. How to create launch-security-cca.xml
Do I need to handmade this from scratch or is there an automated method?
Basically it's hand written. What I usually do is I copy-paste the domain XML I used
when developing and testing a feature. And then cut off all unnecessary elements.
3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
I created launch-security-cca.aarch64-latest.* using the method described in
doc/advanced-tests.rst.
And, I created the test data for qemucapabilitiestest and domaincapstest using
the method described in tests/qemucapabilitiesdata/README.rst.
VIR_TEST_REGENERATE_OUTPUT=1 ./qemuxmlconftest
VIR_TEST_REGENERATE_OUTPUT=1 ./domaincapstest
VIR_TEST_REGENERATE_OUTPUT=1 ./qemucapabilitiestest
Can I use the generated file for testing as is?
Because doc/advanced-tests.rst says:
"VERY CAREFULLY to ensure they are correct"
> I assume that automatically generated expected values are
checked for accuracy.
Not really. It takes a machine brain to decide whether those files follow some syntax
(e.g. whether JSON is valid), but it takes human brain to decide whether full combination
of cmd line arguments actually makes sense. I'd say - if you're able to start
generated cmd line (modulo some FD passing stuff - see my point above about cutting off
unnecessary elements), then you're probably fine.
If correct, they are adopted; otherwise, investigation and
remediation are undertaken.
However, due to the lack of explicit documentation, we require confirmation.
Also, it appears to be generating expected values based on the execution
environment.
Do we need to worry about variations in execution environments?
No. Our tests should generate stable enough (and reproducible!) environment.
For example, executing qemuxmlconftest detects the following
comparison errors,
such as with aarch64-virt-minimal.aarch64-latest, etc.
Expect [sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -m]
Actual [m]
Yeah, printing diffs is not very userfriendly. You can get better results with
VIR_TEST_DEBUG=2.
Best Regards.
Michal
21
days inactive
24
days old
1 comments
2 participants
participants (2)
-
Akio Kakuno (Fujitsu) -
Michal Prívozník