- Devel - libvirt List Archives

[libvirt] [PATCH v2 0/9] use VIR_AUTO* all around qemu_driver.c
by Daniel Henrique Barboza 30 Sep '19

30 Sep '19

I am messing a lot with qemu_driver.c and other QEMU files due to a multifunction PCI feature I'm working on, ending up sometimes moving code here and there, sometimes copy and pasting parts of it, and spreading all these 'vices', namely using char* and VIR_FREE() and virQEMUDriverConfigPtr with virObjectUnref(), knowing that there is a better way of doing it. Instead of changing just the code I was working on, I took a leap and ended up changing the whole qemu_driver.c file. So here it is. The changes were split to make it reviewing a bit saner. The commiter can squash these in fewer patches if desirable. This was done on top of master commit 6ffb8fff9e. changes in v2: - addressed review concerns made by Erik - added more cleanups, as suggested by Erik v1: https://www.redhat.com/archives/libvir-list/2019-September/msg00719.html Daniel Henrique Barboza (9): qemu_driver: use VIR_AUTOUNREF() with virQEMUDriverConfigPtr 1/3 qemu_driver: use VIR_AUTOUNREF() with virQEMUDriverConfigPtr 2/3 qemu_driver: use VIR_AUTOUNREF() with virQEMUDriverConfigPtr 3/3 qemu_driver: use VIR_AUTOUNREF() with virCapsPtr qemu_driver: use VIR_AUTOUNREF() with more pointer types qemu_driver: use VIR_AUTOFREE() with strings 1/3 qemu_driver: use VIR_AUTOFREE() with strings 2/3 qemu_driver: use VIR_AUTOFREE() with strings 3/3 qemu_driver: VIR_AUTOFREE on other scalar pointers src/qemu/qemu_driver.c | 709 ++++++++++++++--------------------------- 1 file changed, 235 insertions(+), 474 deletions(-) -- 2.21.0

1 9

[libvirt] [PATCH v2] docs: attempt to document the general libvirt dev strategy
by Daniel P. Berrangé 30 Sep '19

30 Sep '19

There are various ideas / plans floating around for future libvirt work, some of which is actively in progress. Historically we've never captured this kind of information anywhere, except in mailing list discussions. In particular guidelines in hacking.html.in don't appear until a policy is actively applied. This patch attempts to fill the documentation gap, by creating a new "strategy" page which outlines the general vision for some notable future changes. The key thing to note is that none of the stuff on this page is guaranteed, plans may change as new information arises. IOW this is a "best guess" as to the desired future. This doc has focused on three areas, related to the topic of language usage / consolidation - Use of non-C languages for the library, daemons or helper tools - Replacement of autotools with meson - Use of RST and Sphinx for documentation (website + man pages) Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/docs.html.in | 3 + docs/strategy.html.in | 144 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) create mode 100644 docs/strategy.html.in diff --git a/docs/docs.html.in b/docs/docs.html.in index c1741c89b4..6cf09f51bc 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -128,6 +128,9 @@ <dt><a href="hacking.html">Contributor guidelines</a></dt> <dd>General hacking guidelines for contributors</dd> + <dt><a href="strategy.html">Project strategy</a></dt> + <dd>Sets a vision for future direction & technical choices</dd> + <dt><a href="bugs.html">Bug reports</a></dt> <dd>How and where to report bugs and request features</dd> diff --git a/docs/strategy.html.in b/docs/strategy.html.in new file mode 100644 index 0000000000..2ed5deeec6 --- /dev/null +++ b/docs/strategy.html.in @@ -0,0 +1,144 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + <h1>Libvirt Project Strategy</h1> + + + This document attempts to outline the libvirt project strategy for + the near future. Think of this as a high level vision or todo list + setting the direction for the project and its developers to take. + + + <h2>Language consolidation</h2> + + + At time of writing libvirt uses the following languages + + + <dl> + <dt>C</dt> + <dd>The core libvirt library, daemons, and helper tools are all written + in the C language.</dd> + <dt>Python</dt> + <dd>Various supporting build/test scripts are written in Python, with + compatibility for Python 2 and 3.</dd> + <dt>Perl</dt> + <dd>Various supporting build/test scripts are written in Perl. It is + also used for many syntax-check inline rules</dd> + <dt>Shell</dt> + <dd>The autoconf configure script, is essentially shell script. Shell + is also used for some simple build/test scripts. At runtime libvirt + avoids shell except when using SSH tunnels to a remote host</dd> + <dt>XSLT</dt> + <dd>The website uses XSLT for its templating system. The API + documentation is also autogenerated from an XML description + using XSLT</dd> + <dt>HTML</dt> + <dd>The website documentation is all written in plain HTML. Some HTML + is also auto-generated for API documentation</dd> + <dt>M4</dt> + <dd>The autoconf configure script uses a large number of M4 macros to + generate its content</dd> + <dt>make</dt> + <dd>The core build system uses the traditional GNU make recipes</dd> + <dt>automake</dt> + <dd>The make recipes use automake's language extensions which are + then turned into regular make rules</dd> + <dt>awk/sed</dt> + <dd>A number of the syntax-check inline rules involve use of awk/sed + scripts</dd> + <dt>POD</dt> + <dd>The command line manual pages are typically written in Perl's POD + format, and converted to troff</dd> + </dl> + + + The wide range of languages used present a knowledge burden for + developers involved in libvirt, especially when there are multiple + languages all used in the same problem spaces. This is most notable + in the build system which uses a combination of shell, m4, make, + automake, awk, sed, perl and python, with debugging requiring + understanding of the interactions between many languages. The + relative popularity of the languages has implications for how easily + the project can attract new contributors, and the quality of the code + they'll be able to submit. This is most notable when comparing Perl + to Python, as since the start of the libvirt project, the popularity + and knowledge of Perl has declined, while Python has risen. + + + The C language has served libvirt well over the years, but its age shows + giving rise to limitations which negatively impact the project in terms + of code quality, reliability, and efficiency of development. Most notably + its lack of memory safety means that many code bugs become trivially + exploitable security flaws or denial of service. The lack of a high + level portable runtime, resulting in a lot of effort being spent to + ensure cross platform portability. The modern languages Rust and Go + provide viable options for low level systems programming, in a way that + was not practical with other common languages such as Python and Java. + There is thus a desire to make use of either Rust or Go, or a combination + of both, to incrementally replace existing use of C, and also for + greenfield development. + + + With this in mind the libvirt project has set a vision for language + usage in the future + + + <dl> + <dt>C</dt> + <dd>Large parts of the core libvirt library, daemons, and helper tools + will continue to make use in the C language. Integration of other + languages will be an incremental, targetted process where they can + bring the greatest benefit.</dd> + <dt>Rust / Go</dt> + <dd>Parts of the core libvirt library, daemons and helper tools are to + leverage Rust or Go or both to replace C.</dd> + <dt>Meson</dt> + <dd>The core build system is to be written in meson.</dd> + <dt>Python</dt> + <dd>Various supporting build/test scripts are written in Python 3 + compatible mode only.</dd> + <dt>reStructuredText</dt> + <dd>The website and command man pages are to be written in RST, using + Sphinx as the engine to convert to end user formats like HTML, troff, + etc</dd> + </dl> + + + Some notable points from the above. Whether the core library / daemons + will use Rust or Go internally is still to be decided based on more + detailed evaluation to identify the best fit. The need to link and embed + this functionality in other processes has complex interactions both at a + technical and non-technical level. For standalone helper tools, either + language is viable, but there are fewer concerns around interactions with + other in-process code from 3rd parties. Thus a different decision may be + made for daemons/libraries vs tools. Any rewrite proposed for existing + functionality will have to weigh up the benefits of the new code, + against the risk of introducing regressions wrt the previous code. + + + + The meson build system is written in Python 3. This directly informs the + choice of Python 3 as the language for all supporting build scripts, + re-inforcing the other benefits of Python, over Perl, Shell, M4, + automake, etc. There is no intention to support Python 2 given meson's + requirement for Python 3. + + + + Using the RST format for documentation allows for the use of XSLT to be + eliminated from the build process. RST and the Sphinx toolkit are widely + used, as seen by the huge repository of content on + https://readthedocs.org/. The ability to embed raw HTML in the RST docs + will greatly facilitate its adoption, avoiding the need for a big bang + conversion of existing content. Given the desire to eliminate Perl + usage, replacing the use of POD documentation for manual pages is an + obvious followup task. RST is the obvious choice to achieve alignment + with the website, allowing the man pages to be easily published online + with other docs. It is further anticipated that the current API docs + generator which uses XSLT to convert the XML API description would be + converted to something which generates RST using Python instead of XSLT. + + </body> +</html> -- 2.21.0

3 3

[libvirt] [PATCH] qemu_capabilities: Put only unique FW images into domcaps
by Michal Privoznik 30 Sep '19

30 Sep '19

In the domain capabilities XML there are FW image paths printed. There are two sources for the image paths (in order of preference): 1) firmware descriptor files - as returned by qemuFirmwareGetSupported() 2) a compile time list of FW:NRAM pairs which can be overridden in qemu.conf If either of those contains a duplicate FW image path (which is a valid use case) it is printed twice in the capabilities XML. While it's technically not a bug, it doesn't look good. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_capabilities.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 9b19930964..489a6872c4 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -5136,12 +5136,22 @@ virQEMUCapsFillDomainLoaderCaps(virDomainCapsLoaderPtr capsLoader, for (i = 0; i < nfirmwares; i++) { const char *filename = firmwares[i]->name; + size_t j; if (!virFileExists(filename)) { VIR_DEBUG("loader filename=%s does not exist", filename); continue; } + /* Put only unique FW images onto the list */ + for (j = 0; j < capsLoader->values.nvalues; j++) { + if (STREQ(filename, capsLoader->values.values[j])) + break; + } + + if (j != capsLoader->values.nvalues) + continue; + if (VIR_STRDUP(capsLoader->values.values[capsLoader->values.nvalues], filename) < 0) return -1; -- 2.21.0

3 2

[libvirt] [PATCH 0/2] qemu: Sanitize update of 'current' checkpoint/snapshot
by Peter Krempa 30 Sep '19

30 Sep '19

See 1/2. Peter Krempa (2): qemu: snapshot: Don't update current snapshot until we're done qemu: checkpoint: Don't update current checkpoint until we are done src/qemu/qemu_checkpoint.c | 7 ------- src/qemu/qemu_driver.c | 19 ------------------- 2 files changed, 26 deletions(-) -- 2.21.0

3 4

[libvirt] [PATCH 0/3] Add vhost-user-scsi/blk support
by Li Feng 30 Sep '19

30 Sep '19

The latest qemu has support the vhost-user-scsi and vhost-user-blk backend for better performance. This patch let the libvirt could manage the vhost-user disks. Usage in xml like this: <vhost-user-blk-pci type='unix'> <source type='bind' path='/tmp/vhost-blk.sock'> <reconnect enabled='yes' timeout='5' /> </source> <queue num='4'/> </vhost-user-blk-pci> Li Feng (3): qemu: Add vhost-user-scsi/blk support vhost-user-scsi/blk: add xml validation check qemu: Add multiqueue support for vhost-user-scsi/blk docs/schemas/domaincommon.rng | 48 ++++++++++ src/conf/device_conf.h | 5 + src/conf/domain_conf.c | 93 +++++++++++++++++- src/conf/domain_conf.h | 9 ++ src/qemu/qemu_alias.c | 16 ++++ src/qemu/qemu_command.c | 105 +++++++++++++++++++++ src/qemu/qemu_domain.c | 2 + .../vhost-user-blk.x86_64-latest.args | 37 ++++++++ tests/qemuxml2argvdata/vhost-user-blk.xml | 38 ++++++++ .../vhost-user-scsi.x86_64-latest.args | 37 ++++++++ tests/qemuxml2argvdata/vhost-user-scsi.xml | 37 ++++++++ tests/qemuxml2argvtest.c | 3 + 12 files changed, 429 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/vhost-user-blk.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-blk.xml create mode 100644 tests/qemuxml2argvdata/vhost-user-scsi.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-scsi.xml -- 2.11.0 -- The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.

2 4

[libvirt] [PATCH] clarify the xml example for NVDIMM more clealy
by Luyao Zhong 30 Sep '19

30 Sep '19

The NVDIMM backend file can be a normal file or a real device file, Current xml example and explainations may mislead users. So add more info about the NVDIMM related elements and update the xml examples. Signed-off-by: Luyao Zhong <luyao.zhong(a)intel.com> --- docs/formatdomain.html.in | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 86a5261..556a39f 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -8533,7 +8533,6 @@ qemu-kvm -net nic,model=? /dev/null <memory model='nvdimm'> <source> <path>/tmp/nvdimm</path> - <alignsize unit='KiB'>2048</alignsize> </source> <target> <size unit='KiB'>524288</size> @@ -8544,9 +8543,10 @@ qemu-kvm -net nic,model=? /dev/null <readonly/> </target> </memory> - <memory model='nvdimm'> + <memory model='nvdimm' access='shared'> <source> <path>/dev/dax0.0</path> + <alignsize unit='KiB'>2048</alignsize> <pmem/> </source> <target> @@ -8580,6 +8580,8 @@ qemu-kvm -net nic,model=? /dev/null module basis. Values are the same as <a href="#elementsMemoryBacking">Memory Backing</a>: <code>shared</code> and <code>private</code>. + For <code>nvdimm</code> model, if using real NVDIMM DAX device as + backend, <code>shared</code> is required. </dd> @@ -8643,7 +8645,8 @@ qemu-kvm -net nic,model=? /dev/null alignment used to mmap the address range for the backend <code>path</code>. If not supplied the host page size is used. For example, to mmap a real NVDIMM device a 2M-aligned page may - be required. + be required, and host page size is 4KB, then we need to set this + element to 2MB. Since 5.0.0 </dd> -- 2.7.4

2 1

[libvirt] [PATCH 0/4] Automatic lockable object unlocking
by Peter Krempa 30 Sep '19

30 Sep '19

A spin-off from the refactors to checkpoints where Eric suggested that we might want to replace virDomainObjEndAPI. See patch 2 for the implementation and patch 3 for example use. Peter Krempa (4): util: object: Note that VIR_AUTOUNREF variables must have the reference util: object: Add VIR_AUTOUNLOCK and VIR_AUTORELEASE qemu: driver: Convert qemuDomainGetInfo to use VIR_AUTORELEASE qemu: Don't repeat virDomainObjEndAPI in qemuDomainBlockPull src/libvirt_private.syms | 2 ++ src/qemu/qemu_driver.c | 29 +++++++++++------------------ src/util/virobject.c | 29 +++++++++++++++++++++++++++++ src/util/virobject.h | 34 +++++++++++++++++++++++++++++++++- 4 files changed, 75 insertions(+), 19 deletions(-) -- 2.21.0

2 7

[libvirt] [PATCH] security: AppArmor profile fixes for swtpm
by Chris Coulson 30 Sep '19

30 Sep '19

The AppArmor profile generated by virt-aa-helper is too strict for swtpm. This change contains 2 small fixes: - Relax append access to swtpm's log file to permit write access instead. Append access is insufficient because the log is opened with O_CREAT. - Permit swtpm to acquire a lock on its lock file. --- src/security/virt-aa-helper.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index 326cfaf52a..3d7cc32459 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -1238,10 +1238,10 @@ get_files(vahControl * ctl) * directory, log, and PID files. */ virBufferAsprintf(&buf, - " \"%s/lib/libvirt/swtpm/%s/%s/**\" rw,\n", + " \"%s/lib/libvirt/swtpm/%s/%s/**\" rwk,\n", LOCALSTATEDIR, uuidstr, tpmpath); virBufferAsprintf(&buf, - " \"%s/log/swtpm/libvirt/qemu/%s-swtpm.log\" a,\n", + " \"%s/log/swtpm/libvirt/qemu/%s-swtpm.log\" w,\n", LOCALSTATEDIR, ctl->def->name); virBufferAsprintf(&buf, " \"%s/libvirt/qemu/swtpm/%s-swtpm.pid\" rw,\n", -- 2.20.1

5 5

[libvirt] [PATCH v1 0/3] Adding the 'ccf-assist' pSeries capability
by Daniel Henrique Barboza 30 Sep '19

30 Sep '19

Hi, This short series exposes, implements and documents a pSeries guest capability called Count Cache Flush Assist (ccf-assist), which was added in the Linux kernel since 5.1 and in QEMU since 4.0.0. The reason why this capability needs to exposed via Libvirt is because it is defaulted to 'off' in QEMU to not break migration compatibility in the Power 9 processor class. However, there is a performance gain in activating it, thus the user might want to choose less migration options for more power. More details can be found in the commit message of patch 1. Daniel Henrique Barboza (3): qemu: Add capability for the ccf-assist pSeries feature qemu: Implement the ccf-assist pSeries feature news: Update for the ccf-assist pSeries feature docs/formatdomain.html.in | 9 +++++++++ docs/news.xml | 9 +++++++++ docs/schemas/domaincommon.rng | 5 +++++ src/conf/domain_conf.c | 4 ++++ src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 20 +++++++++++++++++++ src/qemu/qemu_domain.c | 1 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + tests/qemuxml2argvdata/pseries-features.args | 2 +- tests/qemuxml2argvdata/pseries-features.xml | 1 + tests/qemuxml2argvtest.c | 1 + tests/qemuxml2xmloutdata/pseries-features.xml | 1 + tests/qemuxml2xmltest.c | 1 + 15 files changed, 58 insertions(+), 1 deletion(-) -- 2.21.0

1 4

[libvirt] [PATCH] qemu: sanity check vhost user FD before passing to QEMU
by Daniel P. Berrangé 30 Sep '19

30 Sep '19

Ensure that the FD we're passing to QEMU is actually open, so we get a sane error message upfront instead of telling QEMU to use a closed FD. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/qemu/qemu_command.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 77470a6037..8c979fdf65 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4704,6 +4704,12 @@ qemuBuildVhostUserChardevStr(const char *alias, { char *chardev = NULL; + if (*fd == -1) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Attempt to pass closed vhostuser FD")); + return NULL; + } + if (virAsprintf(&chardev, "socket,id=chr-vu-%s,fd=%d", alias, *fd) < 0) return NULL; -- 2.21.0

2 1