On 2/10/20 4:01 PM, Collin Walling wrote:

Hello,

I am working on implementing libvirt test cases for the hypervisor-cpu-compare and -baseline commands. Ideally, I would like to take advantage of the qemuProcess API to spin up a QEMU instance and run these commands to test real data queried from the hypervisor.

The tests in libvirt's tests directory are all unit tests, and don't run external binaries like qemu - they attempt to test various pieces of code within libvirt in a limited, controlled environment, not requiring any special privileges, and without being subject to the vagaries of external programs. qemuxml2argvtest, for example, calls the code that converts a libvirt domain xml into a qemu commandline, but rather than starting up a qemu process with that commandline, it then compares the generated commandline string to the "known correct" commandline that is stored in a file in a subdirectory of the tests directory (for this example, look at tests/qemuxml2argvdata/*.xml vs tests/qemuxml2argvdata/*.args); this way the success of the test doesn't depend on qemu being installed, the user having the proper privileges to perform all the other things that qemu needs to run, or be subject to a failure due to a bug in the version of qemu running on the build system. Other examples of tests are *xml2xmltest, which parse a test input file, then format the resulting object back to xml, and compare that result with known-correct XML that, again, is stored in a file. Sometimes the code that is being tested calls a function that can't be realistically called from a unit test that is run on random hardware by an unprivileged user (e.g. unbinding a PCI device from its host driver and re-binding it to the vfio-pci driver). In that case, various library functions are "mocked" (by linking in the .o created from *mock.c files - virpcimock.c in the case of PCI device driver binding) to return reasonable values that will allow the libvirt code to be tested without actually needing to perform the "impossible" operation. Anyway, the short answer to your questions is that libvirt's tests directory isn't the proper location (nor is "make check" the proper time) for a test that starts up a qemu process and sends commands to it. Something like that would be better suited for an integration test suite, like the ones that are in the libvirt-tck package (which is a bit dated, but still very useful, as long as you don't mind writing perl scripts :-P) For a unit test in the tests directory, you would want to setup a test harness (blahtest.c) that links in the files containing your code, calls those functions with a fixed set of inputs, then compares the results to fixed known outputs. Along the way you may need to "stub-out" or "mock" various functions that are called by the code under test but can't actually be called in the environment of a unit test.

However, I am having issues with my libvirt tests communicating with a QEMU instance. The API can successfully spin an instance, but no commands can be sent to QEMU -- not even the qmp_capabilities handshake. The test case hangs forever with no indication that something went wrong.

The hang occurs specifically within the qemuProcessQMPLaunch phase of the qemuProcessQMPStart function. Eventually the libvirt API will get to qemuMonitorSend, and at this loop...

while (!mon->msg->finished) { if (virCondWait(&mon->notify, &mon->parent.lock) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Unable to wait on monitor condition")); goto cleanup; } }

...the program will hang at conditional wait. I kept the QEMU instance alive after killing my test and connected to the monitor socket that was still lingering. The qmp_capabilities command was never sent, thus leading me to believe that the libvirt tests cannot communicate with a QEMU instance.

As s390x is currently the only arch to have implemented these commands, I believe it would be beneficial to have an easy way to test both the expected QEMU response and libvirt parsing within a single test case if/when other archs decide to implement these commands.

I'd like to ask two questions:

1: does it make sense for libvirt tests to communicate with an external binary (specifically QEMU)?

2: if so, is there some sort of conflict between a libvirt test and a QEMU binary? I afraid to say that I am at a loss how to repair this or perhaps how to use the API properly.

I appreciate anyone's help with looking into this.

Note: in case I am not clear, by "libvirt test" I am referring to a test implemented in the tests directory of the libvirt project.