- Devel - libvirt List Archives

[libvirt PATCHv2 00/15] Switch to json-c from yajl
by Ján Tomko 05 Sep '24

05 Sep '24

Wow. Such patch. Much series. Ján Tomko (15): util: json: introduce virJSONStringPrettifyBlanks tests: switch to compact empty JSON object formatting build: introduce WITH_JSON ci: install json-c too meson: add option for building with json-c meson: switch checks to depend on json-c as well as yajl build: do not depend on yajl build: link with json_c util: json: write a json-c implementation nss: convert findLeases to use json-c nss: convert findMACs to use json-c meson: do not link anything with yajl anymore meson: options: drop yajl meson: drop yajl detection ci: drop yajl completely ci/buildenv/almalinux-9.sh | 4 +- ci/buildenv/alpine-319.sh | 4 +- ci/buildenv/alpine-edge.sh | 4 +- ci/buildenv/centos-stream-9.sh | 4 +- ci/buildenv/debian-11-cross-aarch64.sh | 2 +- ci/buildenv/debian-11-cross-armv6l.sh | 2 +- ci/buildenv/debian-11-cross-armv7l.sh | 2 +- ci/buildenv/debian-11-cross-i686.sh | 2 +- ci/buildenv/debian-11-cross-mips64el.sh | 2 +- ci/buildenv/debian-11-cross-mipsel.sh | 2 +- ci/buildenv/debian-11-cross-ppc64le.sh | 2 +- ci/buildenv/debian-11-cross-s390x.sh | 2 +- ci/buildenv/debian-11.sh | 2 +- ci/buildenv/debian-12-cross-aarch64.sh | 2 +- ci/buildenv/debian-12-cross-armv6l.sh | 2 +- ci/buildenv/debian-12-cross-armv7l.sh | 2 +- ci/buildenv/debian-12-cross-i686.sh | 2 +- ci/buildenv/debian-12-cross-mips64el.sh | 2 +- ci/buildenv/debian-12-cross-mipsel.sh | 2 +- ci/buildenv/debian-12-cross-ppc64le.sh | 2 +- ci/buildenv/debian-12-cross-s390x.sh | 2 +- ci/buildenv/debian-12.sh | 2 +- ci/buildenv/debian-sid-cross-aarch64.sh | 2 +- ci/buildenv/debian-sid-cross-armv6l.sh | 2 +- ci/buildenv/debian-sid-cross-armv7l.sh | 2 +- ci/buildenv/debian-sid-cross-i686.sh | 2 +- ci/buildenv/debian-sid-cross-mips64el.sh | 2 +- ci/buildenv/debian-sid-cross-ppc64le.sh | 2 +- ci/buildenv/debian-sid-cross-s390x.sh | 2 +- ci/buildenv/debian-sid.sh | 2 +- ci/buildenv/fedora-39.sh | 4 +- ci/buildenv/fedora-40.sh | 4 +- ci/buildenv/fedora-rawhide.sh | 4 +- ci/buildenv/opensuse-leap-15.sh | 2 +- ci/buildenv/opensuse-tumbleweed.sh | 2 +- ci/buildenv/ubuntu-2204.sh | 2 +- ci/buildenv/ubuntu-2404.sh | 2 +- ci/cirrus/freebsd-13.vars | 2 +- ci/cirrus/freebsd-14.vars | 2 +- ci/cirrus/macos-13.vars | 2 +- ci/cirrus/macos-14.vars | 2 +- ci/containers/almalinux-9.Dockerfile | 4 +- ci/containers/alpine-319.Dockerfile | 4 +- ci/containers/alpine-edge.Dockerfile | 4 +- ci/containers/centos-stream-9.Dockerfile | 4 +- .../debian-11-cross-aarch64.Dockerfile | 2 +- .../debian-11-cross-armv6l.Dockerfile | 2 +- .../debian-11-cross-armv7l.Dockerfile | 2 +- ci/containers/debian-11-cross-i686.Dockerfile | 2 +- .../debian-11-cross-mips64el.Dockerfile | 2 +- .../debian-11-cross-mipsel.Dockerfile | 2 +- .../debian-11-cross-ppc64le.Dockerfile | 2 +- .../debian-11-cross-s390x.Dockerfile | 2 +- ci/containers/debian-11.Dockerfile | 2 +- .../debian-12-cross-aarch64.Dockerfile | 2 +- .../debian-12-cross-armv6l.Dockerfile | 2 +- .../debian-12-cross-armv7l.Dockerfile | 2 +- ci/containers/debian-12-cross-i686.Dockerfile | 2 +- .../debian-12-cross-mips64el.Dockerfile | 2 +- .../debian-12-cross-mipsel.Dockerfile | 2 +- .../debian-12-cross-ppc64le.Dockerfile | 2 +- .../debian-12-cross-s390x.Dockerfile | 2 +- ci/containers/debian-12.Dockerfile | 2 +- .../debian-sid-cross-aarch64.Dockerfile | 2 +- .../debian-sid-cross-armv6l.Dockerfile | 2 +- .../debian-sid-cross-armv7l.Dockerfile | 2 +- .../debian-sid-cross-i686.Dockerfile | 2 +- .../debian-sid-cross-mips64el.Dockerfile | 2 +- .../debian-sid-cross-ppc64le.Dockerfile | 2 +- .../debian-sid-cross-s390x.Dockerfile | 2 +- ci/containers/debian-sid.Dockerfile | 2 +- ci/containers/fedora-39.Dockerfile | 4 +- ci/containers/fedora-40.Dockerfile | 4 +- ci/containers/fedora-rawhide.Dockerfile | 4 +- ci/containers/opensuse-leap-15.Dockerfile | 2 +- ci/containers/opensuse-tumbleweed.Dockerfile | 2 +- ci/containers/ubuntu-2204.Dockerfile | 2 +- ci/containers/ubuntu-2404.Dockerfile | 2 +- ci/lcitool/projects/libvirt.yml | 2 +- libvirt.spec.in | 6 +- meson.build | 62 +-- meson_options.txt | 8 +- src/libvirt_private.syms | 1 + src/meson.build | 2 +- src/util/meson.build | 2 +- src/util/virjson.c | 485 +++++------------- src/util/virjson.h | 2 + tests/meson.build | 8 +- tests/qemublocktest.c | 5 +- .../backupmerge/empty-out.json | 4 +- tests/qemumigparamsdata/empty.json | 4 +- tests/qemumigparamstest.c | 5 +- tests/virmacmaptest.c | 5 +- tests/virmacmaptestdata/empty.json | 4 +- tests/virnetdaemontest.c | 2 +- tests/virstoragetest.c | 4 +- tools/nss/libvirt_nss_leases.c | 370 +++++-------- tools/nss/libvirt_nss_macs.c | 278 +++------- tools/nss/meson.build | 4 +- 99 files changed, 475 insertions(+), 972 deletions(-) -- 2.46.0

1 2

[PATCH] qemu: backup: Use 'async' monitor in 'qemuBackupDiskDataCleanupOne'
by Peter Krempa 05 Sep '24

05 Sep '24

'qemuBackupDiskDataCleanupOne()' is entering the monitor while we're in the async backup job inside 'qemuBackupBegin()' which is semantically wrong and per upstream report causes crashes if some monitoring commands are run in parallel. Use qemuDomainObjEnterMonitorAsync() instead. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/qemu/qemu_backup.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c index 81391c29f7..5eb2cbe306 100644 --- a/src/qemu/qemu_backup.c +++ b/src/qemu/qemu_backup.c @@ -125,7 +125,8 @@ qemuBackupDiskDataCleanupOne(virDomainObj *vm, if (!dd->started) { if (dd->added) { - qemuDomainObjEnterMonitor(vm); + if (qemuDomainObjEnterMonitorAsync(vm, VIR_ASYNC_JOB_BACKUP) < 0) + return; qemuBlockStorageSourceAttachRollback(priv->mon, dd->crdata->srcdata[0]); qemuDomainObjExitMonitor(vm); } -- 2.46.0

2 1

[PATCH 0/2] tests: qemucapabilities: Final update of 9.1 caps
by Peter Krempa 05 Sep '24

05 Sep '24

Update the qemu-9.1 caps on x86_64 to final version after yesterday's qemu release and add caps for riscv64 captured on an x86_64 host to manifest that ACPI is already supported: https://gitlab.com/libvirt/libvirt/-/issues/665 Peter Krempa (2): tests: qemucapabilities: Update 'caps_9.1.0_x86_64' for final release tests: qemucapabilitiesdata: Add caps for 'caps_9.1.0_riscv64' (captured on x86_64) .../qemu_9.1.0-tcg-virt.riscv64.xml | 196 + .../qemu_9.1.0-virt.riscv64.xml | 185 + .../caps_9.1.0_riscv64.replies | 31586 ++++++++++++++++ .../caps_9.1.0_riscv64.xml | 193 + .../caps_9.1.0_x86_64.replies | 14 +- .../caps_9.1.0_x86_64.xml | 12 +- ...ult-video-type-riscv64.riscv64-latest.args | 3 +- ...ault-video-type-riscv64.riscv64-latest.xml | 3 + ...efi-riscv64.riscv64-latest.abi-update.args | 3 +- ...-efi-riscv64.riscv64-latest.abi-update.xml | 3 + .../riscv64-virt-acpi.riscv64-latest.args | 3 +- .../riscv64-virt-acpi.riscv64-latest.xml | 3 + ...ault-models.riscv64-latest.abi-update.args | 3 +- ...fault-models.riscv64-latest.abi-update.xml | 3 + ...64-virt-default-models.riscv64-latest.args | 3 +- ...v64-virt-default-models.riscv64-latest.xml | 3 + .../riscv64-virt-graphics.riscv64-latest.args | 3 +- .../riscv64-virt-graphics.riscv64-latest.xml | 3 + ...v64-virt-headless-mmio.riscv64-latest.args | 3 +- ...cv64-virt-headless-mmio.riscv64-latest.xml | 3 + .../riscv64-virt-headless.riscv64-latest.args | 3 +- .../riscv64-virt-headless.riscv64-latest.xml | 3 + ...irt-minimal.riscv64-latest.abi-update.args | 3 +- ...virt-minimal.riscv64-latest.abi-update.xml | 3 + .../riscv64-virt-minimal.riscv64-latest.args | 3 +- .../riscv64-virt-minimal.riscv64-latest.xml | 3 + 26 files changed, 32223 insertions(+), 23 deletions(-) create mode 100644 tests/domaincapsdata/qemu_9.1.0-tcg-virt.riscv64.xml create mode 100644 tests/domaincapsdata/qemu_9.1.0-virt.riscv64.xml create mode 100644 tests/qemucapabilitiesdata/caps_9.1.0_riscv64.replies create mode 100644 tests/qemucapabilitiesdata/caps_9.1.0_riscv64.xml -- 2.46.0

2 5

[PATCH 0/3] virsh: Fix regression in script-friendly output of 'virsh list --uuid'
by Peter Krempa 05 Sep '24

05 Sep '24

Peter Krempa (3): virsh: cmdList: Revert to script-friendly output for 'virsh list --uuid' NEWS: Mention breakage of scrip-friendly virsh output virshtest: Add test case for script friendly options NEWS.rst | 11 +++++++ tests/virshtest.c | 2 ++ .../virshtestdata/script-friendly-options.in | 11 +++++++ .../virshtestdata/script-friendly-options.out | 33 +++++++++++++++++++ tools/virsh-domain-monitor.c | 2 +- 5 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 tests/virshtestdata/script-friendly-options.in create mode 100644 tests/virshtestdata/script-friendly-options.out -- 2.46.0

2 4

Libvirtd sends SIGTERM to old qemu processes after restart
by Dmitrii Abramov 05 Sep '24

05 Sep '24

Hello, Libvirt community. We have one strange issue with libivrtd. We’ve been using Libvirtd in docker for several years. This year we switched to the new generation of processes AMD 7663 and we started to use the new version(for us) of libviirtd 8.0.0. Before this we used Libvirt 6.0 Right now we have such situation: if we restart the container with Libvirt or Libvirt crashes and Docker engine restarts it the new process of Libvirt send SIGTERM to all running QEMU processes. This highlight from strace of qemu process: --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=2045513, si_uid=0} --- This is output from bash where you can see that PID 2045513 is a new process of Libvirt(look at the uptime of container and PID) milan15 : ~ [0] # ps afxj | grep 2045513 2044884 2045700 2045699 2044884 pts/3 2045699 S+ 0 0:00 \_ grep --color=auto 2045513 2045492 2045513 2045513 2045513 ? -1 Ssl 0 0:04 \_ libvirtd -l milan15 : ~ [0] # milan15 : ~ [0] # docker ps | grep "libvirt$" 5b9e7d81a2f3 registry.beget.ru/vps/docker-libvirt/ubuntu/jammy:20240124-1 "libvirtd -l" 6 days ago Up 2 minutes libvirt milan15 : ~ [0] # docker top libvirt UID PID PPID C STIME TTY TIME CMD root 2045513 2045492 2 21:20 ? 00:00:04 libvirtd -l milan15 : ~ [0] # We found that in logs Libvirtd says that he is unable to access to /sys/fs/cgroups. {"log":"2024-09-04 17:40:02.831+0000: 2041803: error : virCgroupV2ParseControllersFile:282 : Unable to read from '/sys/fs/cgroup/../../machine/qemu-1394-mameluk-59ad0e58-732e-4468-9d3a-9be2cbac4931.libvirt-qemu/cgroup.controllers': No such file or directory\n","stream":"stderr","time":"2024-09-04T17:40:02.831308736Z"} {"log":"2024-09-04 17:40:02.831+0000: 2041804: error : virFileReadAll:1447 : Failed to open file '/sys/fs/cgroup/../../machine/qemu-1393-pavelaw3-215f218a-48d4-4b22-b15d-90ee0665f643.libvirt-qemu/cgroup.controllers': No such file or directory\n","stream":"stderr","time":"2024-09-04T17:40:02.83143703Z"} {"log":"2024-09-04 17:40:02.831+0000: 2041804: error : virCgroupV2ParseControllersFile:282 : Unable to read from ‘/sys/fs/cgroup/../../machine/qemu-1393-pavelaw3-215f218a-48d4-4b22-b15d-90ee0665f643.libvirt-qemu/cgroup.controllers': No such file or directory\n","stream":"stderr","time":"2024-09-04T17:40:02.831453382Z"} We made several tests(added sleep and listed /sys/fs/cgroup directory before launching the new process ) — it seems that every process in container has ability to access to /sys/fs/cgroups. It seems that the path /sys/fs/cgroup/../../machine/ isn’t correct. We also tried Libvirt 10.0 version and many other. The result is the same. We have the same scheme on other generations of processors — and everything works excellent. This is an example of you docker-compose where we run Libvirt(just a part) version: "3.0" services: libvirt: image: mameluk_libvirtd build: context: . privileged: true volumes: - /etc/docker/libvirt/etc/libvirt:/etc/libvirt:rw - /lib/modules:/lib/modules:ro - /dev:/dev - /sys:/sys - /run:/run:rw - /var/lib/libvirt:/var/lib/libvirt:rslave - /var/log/libvirt:/var/log/libvirt - /home/docker/svc-libvirt/images:/home/svc-libvirt/images - /etc/docker/libvirt/etc/lvm:/etc/lvm - /home/docker/svc-libvirt/cidata:/home/svc-libvirt/cidata ipc: host network_mode: host environment: - TZ=Europe/Moscow pid: host restart: on-failure entrypoint: ["/bin/start.sh", ""] # depends_on: #- virtlogd container_name: mameluk_libvirtd System: Ubuntu 22.04 core: Linux milan15 6.5.0-35-generic #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May 7 09:00:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux Can anyone help us to solve this problem -- Best Regards, Dmitrii Abramov

3 5

[RFC PATCH] deprecation: don't enable TCG plugins by default on 32 bit hosts
by Alex Bennée 05 Sep '24

05 Sep '24

The existing plugins already liberally use host pointer stuffing for passing user data which will fail when doing 64 bit guests on 32 bit hosts. We should discourage this by officially deprecating support and adding another nail to the 32 bit host coffin. Signed-off-by: Alex Bennée <alex.bennee(a)linaro.org> --- docs/about/deprecated.rst | 11 +++++++++++ configure | 11 ++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 88f0f03786..8a4e249717 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -184,6 +184,17 @@ be an effective use of its limited resources, and thus intends to discontinue it. Since all recent x86 hardware from the past >10 years is capable of the 64-bit x86 extensions, a corresponding 64-bit OS should be used instead. +TCG Plugin support not enabled by default on 32-bit hosts (since 9.2) +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +While it is still possible to enable TCG plugin support for 32-bit +hosts there are a number of potential pitfalls when instrumenting +64-bit guests. The plugin APIs typically pass most addresses as +uint64_t but practices like encoding that address in a host pointer +for passing as user-data will loose data. As most software analysis +benefits from having plenty of host memory it seems reasonable to +encourage users to use 64 bit builds of QEMU for analysis work +whatever targets they are instrumenting. System emulator CPUs -------------------- diff --git a/configure b/configure index d08b71f14b..8acb311527 100755 --- a/configure +++ b/configure @@ -424,6 +424,7 @@ fi # Note that this case should only have supported host CPUs, not guests. # Please keep it sorted and synchronized with meson.build's host_arch. host_arch= +host_bits=64 linux_arch= case "$cpu" in aarch64) @@ -434,12 +435,14 @@ case "$cpu" in armv*b|armv*l|arm) cpu=arm host_arch=arm + host_bits=32 linux_arch=arm ;; i386|i486|i586|i686) cpu="i386" host_arch=i386 + host_bits=32 linux_arch=x86 CPU_CFLAGS="-m32" ;; @@ -458,11 +461,13 @@ case "$cpu" in mips*) cpu=mips host_arch=mips + host_bits=32 linux_arch=mips ;; ppc) host_arch=ppc + host_bits=32 linux_arch=powerpc CPU_CFLAGS="-m32" ;; @@ -480,11 +485,13 @@ case "$cpu" in riscv32 | riscv64) host_arch=riscv + host_bits=32 linux_arch=riscv ;; s390) linux_arch=s390 + host_bits=32 CPU_CFLAGS="-m31" ;; s390x) @@ -495,6 +502,7 @@ case "$cpu" in sparc|sun4[cdmuv]) cpu=sparc + host_bits=32 CPU_CFLAGS="-m32 -mv8plus -mcpu=ultrasparc" ;; sparc64) @@ -505,6 +513,7 @@ case "$cpu" in x32) cpu="x86_64" host_arch=x86_64 + host_bits=32 linux_arch=x86 CPU_CFLAGS="-mx32" ;; @@ -1028,7 +1037,7 @@ if test "$static" = "yes" ; then fi plugins="no" fi -if test "$plugins" != "no"; then +if test "$plugins" != "no" && test host_bits = 64; then plugins=yes subdirs="$subdirs contrib/plugins" fi -- 2.39.2

3 2

[RFC PATCH] docs: add documentation for the AFL Fuzzing method
by Abhiram Tilak 04 Sep '24

04 Sep '24

I have worked on fuzzing libvirt with AFLPlusPlus over the summer and documented my fuzzing setup [1], to help replicate my setup for anyone who wants to take on fuzzing libvirt. There are few ideas on how to move forward: - Have the gitlab repo setup as it is, and if anyone is looking to take forward fuzzing libvirt, they will find my repo in a search-engine. - Add a brief documented summary of my setup in the docs section for libvirt repo. This would make it easier for anyone to find my repo. The following patch adds the above mentioned summary to the 'kbase/internals' section. Adds a basic documentation required to replicate the fuzzing setup and explains usage of various scripts. The contents of this page is a ReStructured version of the README found in the original setup repo [1]. [1]: https://gitlab.com/edupua/libvirt-aflfuzz Signed-off-by: Abhiram Tilak <atp.exp(a)gmail.com> --- docs/kbase/index.rst | 3 + docs/kbase/internals/afl-fuzzing.rst | 275 +++++++++++++++++++++++++++ docs/kbase/internals/meson.build | 1 + 3 files changed, 279 insertions(+) create mode 100644 docs/kbase/internals/afl-fuzzing.rst diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index e51b35cbfc..1cbd2fc762 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -116,3 +116,6 @@ Internals `QEMU monitor event handling <internals/qemu-event-handlers.html>`__ Brief outline how events emitted by qemu on the monitor are handlded. + +`XML Fuzzing using AFLPlusPlus <internals/afl-fuzzing.html>`__ + Guide to use AFLPlusPlus for fuzzing libvirt's XML parsers. diff --git a/docs/kbase/internals/afl-fuzzing.rst b/docs/kbase/internals/afl-fuzzing.rst new file mode 100644 index 0000000000..2c37847476 --- /dev/null +++ b/docs/kbase/internals/afl-fuzzing.rst @@ -0,0 +1,275 @@ +============================ +AFL Fuzzing method +============================ + +Setup +===== + +Basic Setup +----------- + +Current setup uses a debian12 (bookworm) virtual machine. On a fresh +installation, you will need the following packages installed on your +system. Some really basic debian setup instructions are not mentioned +here like adding user to sudo group etc. + +The helper scripts and other required files mentioned here can be found in +[this](https://gitlab.com/edupua/libvirt-aflfuzz) repository. This +repo will be referred as 'helper repo' in the following guide. + +- Libvirt Dependencies + (`source <https://gitlab.com/libvirt/libvirt/-/blob/master/ci/containers/debian-12.Do…>`__) + +.. code:: sh + + sudo apt install \ + augeas-lenses augeas-tools bash-completion black ca-certificates \ + ccache clang codespell cpp diffutils dnsmasq dwarves ebtables \ + flake8 gcc gettext git grep iproute2 iptables kmod libacl1-dev \ + libapparmor-dev libattr1-dev libaudit-dev libblkid-dev libc6-dev \ + libcap-ng-dev libclang-rt-dev libcurl4-gnutls-dev libdevmapper-dev \ + libfuse-dev libglib2.0-dev libglusterfs-dev libgnutls28-dev libiscsi-dev \ + libnbd-dev libnl-3-dev libnl-route-3-dev libnuma-dev libparted-dev \ + libpcap0.8-dev libpciaccess-dev librbd-dev libreadline-dev libsanlock-dev \ + libsasl2-dev libselinux1-dev libssh-gcrypt-dev libssh2-1-dev libtirpc-dev \ + libudev-dev libxen-dev libxml2-dev libxml2-utils libyajl-dev locales \ + lvm2 make meson nfs-common ninja-build numad open-iscsi perl-base \ + pkgconf policykit-1 python3 python3-docutils python3-pytest qemu-utils \ + sed systemtap-sdt-dev wireshark-dev xsltproc apparmor + +- AFLPlusPlus Dependencies + (`source <https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/INSTALL.md>`__) + +.. code:: sh + + sudo apt-get install \ + build-essential python3-dev automake cmake git flex bison \ + libglib2.0-dev libpixman-1-dev python3-setuptools cargo \ + libgtk-3-dev lld llvm llvm-dev clang ninja-build + + sudo apt-get install \ + gcc-$(gcc --version|head -n1|sed 's/\..*//'|sed 's/.* //')-plugin-dev \ + libstdc++-$(gcc --version|head -n1|sed 's/\..*//'|sed 's/.* //')-dev + +(separated the commands because the gcc-plugin-dev command requires gcc +to be installed.) + +- By default the debian path doesn’t contain ``/sbin`` and + ``/usr/sbin``, its good to update the path to have them + +.. code:: sh + + export PATH=/sbin:/usr/sbin/:/usr/local/sbin:$PATH + +Building AFLPlusPlus +-------------------- + +Build all the modules using the ``distrib`` target. + +.. code:: sh + + git clone https://github.com/AFLplusplus/AFLplusplus + cd AFLplusplus + make PERFORMANCE=1 CODE_COVERAGE=1 PROFILING=1 INTROSPECTION=1 LLVM_CONFIG=llvm-config-14 distrib + sudo make install + +The following variables are the recommended one, but it is possible to have a +more simpler build if one wants to. + +Bulding AFL-Cov +--------------- + +.. code:: sh + + git clone https://github.com/vanhauser-thc/afl-cov.git + cd afl-cov + sudo make install + +Install tmux +------------ + +.. code:: sh + + sudo apt install tmux + +Building Libvirt with AFL harness +================================= + +Currently there is a fork of +`libvirt <https://gitlab.com/edupua/libvirt>`__ which adds support for +compiling the required harnesses. + +[Note: This repo might not be upto date with upstream, please rebase] + +The directory structure is really important because all the helper +scripts rely on it. + +- Libvirt is expected to be cloned in the home directory. + + .. code:: sh + + cd ~/libvirt + git clone https://gitlab.com/libvirt/libvirt + +Alternatively there is a patch series with required commits. You can +find them in the ``patches`` directory in the helper repo. Again, this might +not be upto date with the current repo and might need conflict handling. + +Apply all the patches in order, except the cover-letter. + +.. code:: sh + + git am patches/0001-afl-fuzzing-add-meson-support-for-fuzzing.patch + git am patches/0002-afl-fuzzzing-add-harness-for-net-create-and-net-port.patch + git am patches/0003-afl-fuzzzing-add-harness-for-nwfilter.patch + git am patches/0004-afl-fuzzzing-add-harness-for-vol-create-and-pool-cre.patch + git am patches/0005-afl-fuzzzing-add-harness-for-domain-create.patch + +Build Script +------------ + +The first script to be used is the build script which can be found in the +helper repo. It can build individual builds with support for different +compilers and different sanitizers. It also has a cmplog build which +requires a corresponding default build to run. (Using cmplog requires +2 repos, the cmplog build and a non-cmplog AFL build). + +.. code:: sh + + scripts/build.sh --all + +The full build is about 15 Gigs, please make sure you have provision for +it... code:: sh + + cd ~/libvirt + git clone https://gitlab.com/libvirt/libvirt + +The default build contains ``undefined sanitizer`` and +``address sanitizer``, there are other options like ``tsan`` and +``AFL_HARDEN`` are also produced. + +Corpus generation +================= + +There is a python generator tool +```lxmlgen`` <https://gitlab.com/edupua/lxmlgen>`__ which can be used to +generate a decent corpus useful for fuzzing most commands. The required +documentation is in the repo itself and consists of just running two +scripts ``run.sh`` and ``cmin.sh``. + +.. code:: sh + + git clone https://gitlab.com/edupua/lxmlgen + cd lxmlgen + ./run.sh + ./cmin.sh --all + +The cmin.sh expects the default build to exist (``build_default``), +which should be generated in the previous step. + +The fuzzing scripts expect the final corpus to be in the ``corpus`` +directory in the root of the current repo. + +Fuzzing Individual builds +========================= + +Up from this point, a casual fuzzer can start fuzzing runs. Use the +helper scripts ``base.sh`` or ``custom_mutator.sh`` that pass extra +arguments to ``afl-fuzz`` thus, there are +`plethora <https://www.mankier.com/8/afl-fuzz>`__ of options to play +around with. + +.. code:: sh + + scripts/base.sh net-create build_default + +Custom Mutator +-------------- + +There is a simple `custom mutator <https://gitlab.com/edupua/mutator>`__ +write in python. To take full advantage of this mutator, please use it +singly with ``AFL_CUSTOM_MUTATOR_ONLY`` option. + +Once you clone the repo don’t forget to install the ``requirements.txt`` +and you can use the script ``custom_mutator.sh`` just like ``base.sh``. + +This script expects the mutator to be cloned into a folder called +``custom_mutator``, with the python libraries installed. It also expects +the corresponding RelaxNG schema files to be placed in ``schemas`` dir, +which should be present by default. + +Generate a tmux command +======================= + +If you have access to a large amount of computational power with +multiple cores it is recommended to run a parallel fuzz with multiple +instances running at once. Its meant to only run one afl subcommand at +once. + +The script ``gen_tmux.sh`` automatically detects all the build +directories in the libvirt directory and uses a relevant combination of +``afl-fuzz`` options with different weights for each option. + +It also has supports for ``--custom_mutator`` argument which uses +``custom_mutator.sh`` instead of ``base.sh``. + +.. code:: sh + + scripts/gen_tmux.sh net-create --custom_mutator + +Finally this script should generate a ``mega.sh`` which deploys multiple +instances. The script generates approximately 30 instances and each run +uses a core of CPU, and may require manually adding or removing +instances accordingly. + +Gather Coverage +=============== + +There is again a script to generate coverage for a given subcommand. + +.. code:: sh + + scripts/cov.sh net-create + +Note: This command can take a really long time since it generates +coverage for each and every queue file in the corpus and does so for all +the master plus slave instances of a fuzzing run. + +You can use the argument ``--cover-corpus`` which avoids running each +file and is lot faster. + +The final coverage can be viewed in HTML format in the ``cov/web`` +directory inside the specified output folder. + +See Also +======== + +Setting up AFLPlusPlus in a new machine may require a tuning. Please use +the following resources: + +- Ultimate fuzzing using AFLPlusPlus guide - + https://aflplus.plus/docs/fuzzing_in_depth/ +- Performance Tips in AFLPlusPlus - + https://aflplus.plus/docs/perf_tips/ +- Parallel fuzzing - https://aflplus.plus/docs/parallel_fuzzing/ + +There is also detailed documentation that can be easily found in AFL’s +github repo: - `Custom +mutators <https://github.com/AFLplusplus/AFLplusplus/tree/stable/custom_mutators>`__ +- Use this guide to help craft custom mutators for AFLPlusPlus, also +take a look at the reference links to find some related links. - +`Instrumentation <https://github.com/AFLplusplus/AFLplusplus/tree/stable/instrumentation>`__ +- This directory contains a detailed list of popular instrumentation +methods used and some related examples. + +It is also really important to have a great understand of how AFL works +under the hood before using it to do something useful. Refer to `this +blog <https://blog.ritsec.club/posts/afl-under-hood/>`__ + +Please consider looking at other efforts to fuzz libvirt as well: + +- AltLinux: `Fuzzing libvirt + RPC <https://www.altlinux.org/Libvirt_fuzzing_technique>`__ +- Libfuzzer + Libprotobuf-mutator: `Fuzzing libvirt using grammar aware + mutator <https://gitlab.com/Skryptonyte/libvirt/-/tree/gsoc_fuzz_final_forreal/tests…>`__ + diff --git a/docs/kbase/internals/meson.build b/docs/kbase/internals/meson.build index f1e9122f8f..b944c50e0b 100644 --- a/docs/kbase/internals/meson.build +++ b/docs/kbase/internals/meson.build @@ -9,6 +9,7 @@ docs_kbase_internals_files = [ 'qemu-migration', 'qemu-threads', 'rpc', + 'afl-fuzzing', ] -- 2.39.2

2 1

[PATCH 00/14] Introduce fuzzing of XML formats
by Rayhan Faizel 04 Sep '24

04 Sep '24

This series introduces multiple fuzzers developed as part of Google Summer of Code 2024. We adopt a structure-aware fuzzing approach to fuzz libvirt XML formats. The fuzzing methodology makes use of libFuzzer and libprotobuf-mutator. The fuzzers work by mutating intermediate protobufs and converting them to XML. The fuzzing method in use requires inclusion of C++ sources. However, C++ compilation will be done only if '-Dfuzz' is enabled. Otherwise, libvirt will compile normally as before. The fuzzing method works only on clang compilers which support libFuzzer. This series introduces a total of six fuzzers: 1. QEMU XML domain 2. QEMU XML hotplug 3. CH XML domain 4. VMX XML domain 5. libXL XML domain 6. NWFilter XML In terms of the number of crashes discovered, QEMU XML domain, QEMU XML hotplug and libXL fuzzers are the most interesting ones. The setup process is documented at the end of the series (patch 14). Rayhan Faizel (14): src: Tweak source code to allow C++ compilation meson: Add support for clang/LLVM coverage instrumentation tests: Export handlers for fake secondary drivers schemas: Refactor relaxNG schema to ease protobuf conversion scripts: Add script to convert relaxNG to protobuf fuzz: Implement base fuzzing setup for XML domain fuzz: Implement QEMU XML domain fuzzer fuzz: Implement QEMU XML hotplug fuzzer ch: Remove unused variables fuzz: Implement CH XML domain fuzzer fuzz: Implement VMX XML domain fuzzer fuzz: Implement libXL XML domain fuzzer fuzz: Implement NWFilter XML fuzzer docs: Document the fuzzers build-aux/syntax-check.mk | 1 + docs/kbase/index.rst | 3 + docs/kbase/internals/meson.build | 1 + docs/kbase/internals/xml-fuzzing.rst | 120 ++++ meson.build | 55 ++ meson_options.txt | 5 +- scripts/meson.build | 1 + scripts/relaxng-to-proto.py | 521 ++++++++++++++++++ src/ch/ch_monitor.c | 2 +- src/ch/ch_monitor.h | 3 + src/ch/ch_process.c | 2 - src/conf/domain_conf.c | 18 +- src/conf/domain_conf.h | 6 +- src/conf/netdev_vport_profile_conf.c | 2 +- src/conf/schemas/basictypes.rng | 20 +- src/conf/schemas/domaincommon.rng | 11 +- src/conf/schemas/networkcommon.rng | 14 +- src/qemu/qemu_hotplug.c | 4 + src/qemu/qemu_monitor.c | 6 +- src/qemu/qemu_monitor.h | 2 +- src/util/virfile.h | 2 +- src/util/virnetdev.h | 12 +- src/util/virnetdevip.h | 2 +- src/util/virnetdevmacvlan.h | 2 +- src/util/virnetdevvportprofile.c | 2 +- src/util/virnetdevvportprofile.h | 2 +- src/util/virnvme.c | 4 +- src/util/virnvme.h | 2 +- src/util/viruuid.h | 2 +- tests/commandhelper.c | 8 +- tests/fuzz/README.rst | 131 +++++ tests/fuzz/ch_xml_domain_fuzz.cc | 157 ++++++ tests/fuzz/libxl_xml_domain_fuzz.cc | 159 ++++++ tests/fuzz/llvm_symbolizer_wrapper.c | 11 + tests/fuzz/meson.build | 183 ++++++ tests/fuzz/proto_custom_datatypes.cc | 234 ++++++++ tests/fuzz/proto_custom_datatypes.h | 30 + tests/fuzz/proto_header_common.h | 51 ++ tests/fuzz/proto_to_xml.cc | 277 ++++++++++ tests/fuzz/proto_to_xml.h | 39 ++ tests/fuzz/protos/meson.build | 46 ++ tests/fuzz/protos/xml_datatypes.proto | 93 ++++ tests/fuzz/protos/xml_domain.proto | 62 +++ tests/fuzz/protos/xml_domain_disk_only.proto | 21 + .../protos/xml_domain_interface_only.proto | 21 + tests/fuzz/protos/xml_hotplug.proto | 38 ++ tests/fuzz/protos/xml_nwfilter.proto | 9 + tests/fuzz/qemu_xml_domain_fuzz.cc | 277 ++++++++++ tests/fuzz/qemu_xml_hotplug_fuzz.cc | 340 ++++++++++++ tests/fuzz/run_fuzz.in | 142 +++++ tests/fuzz/vmx_xml_domain_fuzz.cc | 208 +++++++ tests/fuzz/xml_nwfilter_fuzz.cc | 149 +++++ tests/meson.build | 5 + tests/qemumonitortestutils.c | 48 ++ tests/qemumonitortestutils.h | 6 + tests/qemuxmlconftest.c | 249 --------- tests/testutilsqemu.c | 256 +++++++++ tests/testutilsqemu.h | 57 ++ 58 files changed, 3832 insertions(+), 302 deletions(-) create mode 100644 docs/kbase/internals/xml-fuzzing.rst create mode 100644 scripts/relaxng-to-proto.py create mode 100644 tests/fuzz/README.rst create mode 100644 tests/fuzz/ch_xml_domain_fuzz.cc create mode 100644 tests/fuzz/libxl_xml_domain_fuzz.cc create mode 100644 tests/fuzz/llvm_symbolizer_wrapper.c create mode 100644 tests/fuzz/meson.build create mode 100644 tests/fuzz/proto_custom_datatypes.cc create mode 100644 tests/fuzz/proto_custom_datatypes.h create mode 100644 tests/fuzz/proto_header_common.h create mode 100644 tests/fuzz/proto_to_xml.cc create mode 100644 tests/fuzz/proto_to_xml.h create mode 100644 tests/fuzz/protos/meson.build create mode 100644 tests/fuzz/protos/xml_datatypes.proto create mode 100644 tests/fuzz/protos/xml_domain.proto create mode 100644 tests/fuzz/protos/xml_domain_disk_only.proto create mode 100644 tests/fuzz/protos/xml_domain_interface_only.proto create mode 100644 tests/fuzz/protos/xml_hotplug.proto create mode 100644 tests/fuzz/protos/xml_nwfilter.proto create mode 100644 tests/fuzz/qemu_xml_domain_fuzz.cc create mode 100644 tests/fuzz/qemu_xml_hotplug_fuzz.cc create mode 100644 tests/fuzz/run_fuzz.in create mode 100644 tests/fuzz/vmx_xml_domain_fuzz.cc create mode 100644 tests/fuzz/xml_nwfilter_fuzz.cc -- 2.34.1

3 22

[PATCH] spec: Fix dtrace BuildRequires on fedora 41+
by Cole Robinson 04 Sep '24

04 Sep '24

/usr/bin/dtrace has been split from `systemtap-sdt-devel` into `systemtap-sdt-dtrace` It's forward and backward compatible to require the dtrace binary directly. We still need the latter dep though, for sdt.h in generated libvirt_probes.h Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- libvirt.spec.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index 29101e74fe..86af13a861 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -425,8 +425,8 @@ BuildRequires: libcurl-devel BuildRequires: libwsman-devel >= 2.6.3 %endif BuildRequires: audit-libs-devel -# we need /usr/sbin/dtrace BuildRequires: systemtap-sdt-devel +BuildRequires: /usr/bin/dtrace # For mount/umount in FS driver BuildRequires: util-linux # For showmount in FS driver (netfs discovery) -- 2.46.0

2 1

[PATCH 0/2] virnetdevtap: Output nicer message for common user error
by Martin Kletzander 04 Sep '24

04 Sep '24

See PATCH 2/2 (duh!). Martin Kletzander (2): virnetdevtap: Do (not) use NULLSTR consistently virnetdevtap: Add better error message for a possible common user error src/util/virnetdevtap.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) -- 2.46.0

2 4