- Devel - libvirt List Archives

[libvirt] [PATCH] docs: document that C & Python are the preferred languages
by Daniel P. Berrangé 01 Oct '19

01 Oct '19

Blacklist Perl and Shell code in favour of Python for sake of readability and portability. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/hacking.html.in | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/hacking.html.in b/docs/hacking.html.in index 90376968c3..8572eb65a6 100644 --- a/docs/hacking.html.in +++ b/docs/hacking.html.in @@ -343,6 +343,36 @@ Richard Jones' guide to working with open source projects</a>. </p> + <h2><a id="lang">Language Usage</a></h2> + + <p> + The libvirt repository makes use of a large number of programming + languages. There is a general desire to phase out some of the + existing languages used to reduce the knowledge burden on + developers, and facilitate introduction of new languages in + the future. + </p> + + <p> + The preferred languages at this time are + </p> + + <ul> + <li>C - for the main libvirt codebase. Dialect supported by + GCC/CLang only.</li> + <li>Python - for supporting build scripts / tools. Code must + run with both version 2.7 and 3.x at this time.</li> + </ul> + + <p> + Languages that should not be used for any new contributions. + </p> + + <ul> + <li>Perl - build scripts must be written in Python instead.</li> + <li>Shell - build scripts must be written in Python instead.</li> + </ul> + <h2><a id="tooling">Tooling</a></h2> <p> -- 2.21.0

5 11

[libvirt] Libvirt release ...
by Daniel Veillard 01 Oct '19

01 Oct '19

Seems I totally forgot to work on the release in the last week. I suggest to enter freeze in a few hours, then have RC2 on Thursday and the release this Saturday ! Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0

[libvirt] [PATCH 0/2] qemu_process: use VIR_AUTO* all around
by Daniel Henrique Barboza 30 Sep '19

30 Sep '19

The usual AUTOFREE() and AUTOUNREF() changes that allows for a bit of cleanup. Daniel Henrique Barboza (2): qemu_process: use VIR_AUTOFREE() qemu_process: use VIR_AUTOUNREF() src/qemu/qemu_process.c | 413 +++++++++++++++------------------------- 1 file changed, 149 insertions(+), 264 deletions(-) -- 2.21.0

1 2

[libvirt] [PATCH] rpc: fix escaping of shell path for netcat binary
by Daniel P. Berrangé 30 Sep '19

30 Sep '19

Consider having a nc binary in the path with a space in its name, for example '/tmp/fo o/nc' This results in libvirt running SSH with the following arg value "'if ''/tmp/fo o/nc'' -q 2>&1 | grep \"requires an argument\" >/dev/null 2>&1; then ARG=-q0; else ARG=;fi;''/tmp/fo o/nc'' $ARG -U /var/run/libvirt/libvirt-sock'" The use of the single quote escaping was introduced by commit 6ac6238de33fc74e7545b245ae273d1bfd658808 Author: Guido Günther <agx(a)sigxcpu.org> Date: Thu Oct 13 21:49:01 2011 +0200 Use virBufferEscapeShell in virNetSocketNewConnectSSH to escape the netcat command since it's passed to the shell. Adjust expected test case output accordingly. While the intention of this change was good, the result is broken as it is still underquoted. On the SSH server side, SSH itself runs the command via the shell. Our command is then invoking the shell again. Thus we see $ virsh -c qemu+ssh://root@domokun/system?netcat=%2Ftmp%2Ffo%20o%2Fnc list error: failed to connect to the hypervisor error: End of file while reading data: sh: /tmp/fo: No such file or directory: Input/output error With the second level of escaping added we can now successfully use a nc binary with a space in the path. The original test case added was misleading as it illustrated using a binary path of 'nc -4' which is not a path, it is a command with a separate argument, which is getting interpreted as a path. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/rpc/virnetclient.c | 8 ++++++++ src/rpc/virnetsocket.c | 9 +++++++++ tests/virnetsockettest.c | 6 +++--- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c index 64855fb8d6..53d8b219ea 100644 --- a/src/rpc/virnetclient.c +++ b/src/rpc/virnetclient.c @@ -490,6 +490,10 @@ virNetClientPtr virNetClientNewLibSSH2(const char *host, DEFAULT_VALUE(knownHostsVerify, "normal"); virBufferEscapeShell(&buf, netcatPath); + if (!(nc = virBufferContentAndReset(&buf))) + goto no_memory; + virBufferEscapeShell(&buf, nc); + VIR_FREE(nc); if (!(nc = virBufferContentAndReset(&buf))) goto no_memory; @@ -596,6 +600,10 @@ virNetClientPtr virNetClientNewLibssh(const char *host, DEFAULT_VALUE(knownHostsVerify, "normal"); virBufferEscapeShell(&buf, netcatPath); + if (!(nc = virBufferContentAndReset(&buf))) + goto no_memory; + virBufferEscapeShell(&buf, nc); + VIR_FREE(nc); if (!(nc = virBufferContentAndReset(&buf))) goto no_memory; diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c index ebd304707a..a469907779 100644 --- a/src/rpc/virnetsocket.c +++ b/src/rpc/virnetsocket.c @@ -903,6 +903,15 @@ int virNetSocketNewConnectSSH(const char *nodename, return -1; } quoted = virBufferContentAndReset(&buf); + + virBufferEscapeShell(&buf, quoted); + VIR_FREE(quoted); + if (virBufferCheckError(&buf) < 0) { + virCommandFree(cmd); + return -1; + } + quoted = virBufferContentAndReset(&buf); + /* * This ugly thing is a shell script to detect availability of * the -q option for 'nc': debian and suse based distros need this diff --git a/tests/virnetsockettest.c b/tests/virnetsockettest.c index bb8357f7cd..8cad351605 100644 --- a/tests/virnetsockettest.c +++ b/tests/virnetsockettest.c @@ -661,15 +661,15 @@ mymain(void) struct testSSHData sshData7 = { .nodename = "somehost", - .netcat = "nc -4", + .netcat = "/tmp/fo o/nc", .path = "/tmp/socket", .expectOut = "-T -e none -- somehost sh -c '" - "if ''nc -4'' -q 2>&1 | grep \"requires an argument\" >/dev/null 2>&1; then " + "if \'''\\''/tmp/fo o/nc'\\'''' -q 2>&1 | grep \"requires an argument\" >/dev/null 2>&1; then " "ARG=-q0;" "else " "ARG=;" "fi;" - "''nc -4'' $ARG -U /tmp/socket'\n", + "'''\\''/tmp/fo o/nc'\\'''' $ARG -U /tmp/socket'\n", }; if (virTestRun("SSH test 7", testSocketSSH, &sshData7) < 0) ret = -1; -- 2.21.0

2 1

[libvirt] [PATCH] admin: fix memory leak of typed parameters getting client info
by Daniel P. Berrangé 30 Sep '19

30 Sep '19

In the error code path, the temporary parameters are not freed. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/admin/admin_server.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/admin/admin_server.c b/src/admin/admin_server.c index 248df3f795..0d6091937d 100644 --- a/src/admin/admin_server.c +++ b/src/admin/admin_server.c @@ -310,6 +310,8 @@ adminClientGetInfo(virNetServerClientPtr client, ret = 0; cleanup: + if (tmpparams) + virTypedParamsFree(tmpparams, *nparams); virObjectUnref(identity); VIR_FREE(sock_addr); return ret; -- 2.21.0

2 1

[libvirt] [PATCH v2 0/9] use VIR_AUTO* all around qemu_driver.c
by Daniel Henrique Barboza 30 Sep '19

30 Sep '19

I am messing a lot with qemu_driver.c and other QEMU files due to a multifunction PCI feature I'm working on, ending up sometimes moving code here and there, sometimes copy and pasting parts of it, and spreading all these 'vices', namely using char* and VIR_FREE() and virQEMUDriverConfigPtr with virObjectUnref(), knowing that there is a better way of doing it. Instead of changing just the code I was working on, I took a leap and ended up changing the whole qemu_driver.c file. So here it is. The changes were split to make it reviewing a bit saner. The commiter can squash these in fewer patches if desirable. This was done on top of master commit 6ffb8fff9e. changes in v2: - addressed review concerns made by Erik - added more cleanups, as suggested by Erik v1: https://www.redhat.com/archives/libvir-list/2019-September/msg00719.html Daniel Henrique Barboza (9): qemu_driver: use VIR_AUTOUNREF() with virQEMUDriverConfigPtr 1/3 qemu_driver: use VIR_AUTOUNREF() with virQEMUDriverConfigPtr 2/3 qemu_driver: use VIR_AUTOUNREF() with virQEMUDriverConfigPtr 3/3 qemu_driver: use VIR_AUTOUNREF() with virCapsPtr qemu_driver: use VIR_AUTOUNREF() with more pointer types qemu_driver: use VIR_AUTOFREE() with strings 1/3 qemu_driver: use VIR_AUTOFREE() with strings 2/3 qemu_driver: use VIR_AUTOFREE() with strings 3/3 qemu_driver: VIR_AUTOFREE on other scalar pointers src/qemu/qemu_driver.c | 709 ++++++++++++++--------------------------- 1 file changed, 235 insertions(+), 474 deletions(-) -- 2.21.0

1 9

[libvirt] [PATCH v2] docs: attempt to document the general libvirt dev strategy
by Daniel P. Berrangé 30 Sep '19

30 Sep '19

There are various ideas / plans floating around for future libvirt work, some of which is actively in progress. Historically we've never captured this kind of information anywhere, except in mailing list discussions. In particular guidelines in hacking.html.in don't appear until a policy is actively applied. This patch attempts to fill the documentation gap, by creating a new "strategy" page which outlines the general vision for some notable future changes. The key thing to note is that none of the stuff on this page is guaranteed, plans may change as new information arises. IOW this is a "best guess" as to the desired future. This doc has focused on three areas, related to the topic of language usage / consolidation - Use of non-C languages for the library, daemons or helper tools - Replacement of autotools with meson - Use of RST and Sphinx for documentation (website + man pages) Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/docs.html.in | 3 + docs/strategy.html.in | 144 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) create mode 100644 docs/strategy.html.in diff --git a/docs/docs.html.in b/docs/docs.html.in index c1741c89b4..6cf09f51bc 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -128,6 +128,9 @@ <dt><a href="hacking.html">Contributor guidelines</a></dt> <dd>General hacking guidelines for contributors</dd> + <dt><a href="strategy.html">Project strategy</a></dt> + <dd>Sets a vision for future direction & technical choices</dd> + <dt><a href="bugs.html">Bug reports</a></dt> <dd>How and where to report bugs and request features</dd> diff --git a/docs/strategy.html.in b/docs/strategy.html.in new file mode 100644 index 0000000000..2ed5deeec6 --- /dev/null +++ b/docs/strategy.html.in @@ -0,0 +1,144 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + <h1>Libvirt Project Strategy</h1> + + <p> + This document attempts to outline the libvirt project strategy for + the near future. Think of this as a high level vision or todo list + setting the direction for the project and its developers to take. + </p> + + <h2>Language consolidation</h2> + + <p> + At time of writing libvirt uses the following languages + </p> + + <dl> + <dt>C</dt> + <dd>The core libvirt library, daemons, and helper tools are all written + in the C language.</dd> + <dt>Python</dt> + <dd>Various supporting build/test scripts are written in Python, with + compatibility for Python 2 and 3.</dd> + <dt>Perl</dt> + <dd>Various supporting build/test scripts are written in Perl. It is + also used for many syntax-check inline rules</dd> + <dt>Shell</dt> + <dd>The autoconf configure script, is essentially shell script. Shell + is also used for some simple build/test scripts. At runtime libvirt + avoids shell except when using SSH tunnels to a remote host</dd> + <dt>XSLT</dt> + <dd>The website uses XSLT for its templating system. The API + documentation is also autogenerated from an XML description + using XSLT</dd> + <dt>HTML</dt> + <dd>The website documentation is all written in plain HTML. Some HTML + is also auto-generated for API documentation</dd> + <dt>M4</dt> + <dd>The autoconf configure script uses a large number of M4 macros to + generate its content</dd> + <dt>make</dt> + <dd>The core build system uses the traditional GNU make recipes</dd> + <dt>automake</dt> + <dd>The make recipes use automake's language extensions which are + then turned into regular make rules</dd> + <dt>awk/sed</dt> + <dd>A number of the syntax-check inline rules involve use of awk/sed + scripts</dd> + <dt>POD</dt> + <dd>The command line manual pages are typically written in Perl's POD + format, and converted to troff</dd> + </dl> + + <p> + The wide range of languages used present a knowledge burden for + developers involved in libvirt, especially when there are multiple + languages all used in the same problem spaces. This is most notable + in the build system which uses a combination of shell, m4, make, + automake, awk, sed, perl and python, with debugging requiring + understanding of the interactions between many languages. The + relative popularity of the languages has implications for how easily + the project can attract new contributors, and the quality of the code + they'll be able to submit. This is most notable when comparing Perl + to Python, as since the start of the libvirt project, the popularity + and knowledge of Perl has declined, while Python has risen. + </p> + <p> + The C language has served libvirt well over the years, but its age shows + giving rise to limitations which negatively impact the project in terms + of code quality, reliability, and efficiency of development. Most notably + its lack of memory safety means that many code bugs become trivially + exploitable security flaws or denial of service. The lack of a high + level portable runtime, resulting in a lot of effort being spent to + ensure cross platform portability. The modern languages Rust and Go + provide viable options for low level systems programming, in a way that + was not practical with other common languages such as Python and Java. + There is thus a desire to make use of either Rust or Go, or a combination + of both, to incrementally replace existing use of C, and also for + greenfield development. + </p> + <p> + With this in mind the libvirt project has set a vision for language + usage in the future + </p> + + <dl> + <dt>C</dt> + <dd>Large parts of the core libvirt library, daemons, and helper tools + will continue to make use in the C language. Integration of other + languages will be an incremental, targetted process where they can + bring the greatest benefit.</dd> + <dt>Rust / Go</dt> + <dd>Parts of the core libvirt library, daemons and helper tools are to + leverage Rust or Go or both to replace C.</dd> + <dt>Meson</dt> + <dd>The core build system is to be written in meson.</dd> + <dt>Python</dt> + <dd>Various supporting build/test scripts are written in Python 3 + compatible mode only.</dd> + <dt>reStructuredText</dt> + <dd>The website and command man pages are to be written in RST, using + Sphinx as the engine to convert to end user formats like HTML, troff, + etc</dd> + </dl> + + <p> + Some notable points from the above. Whether the core library / daemons + will use Rust or Go internally is still to be decided based on more + detailed evaluation to identify the best fit. The need to link and embed + this functionality in other processes has complex interactions both at a + technical and non-technical level. For standalone helper tools, either + language is viable, but there are fewer concerns around interactions with + other in-process code from 3rd parties. Thus a different decision may be + made for daemons/libraries vs tools. Any rewrite proposed for existing + functionality will have to weigh up the benefits of the new code, + against the risk of introducing regressions wrt the previous code. + </p> + + <p> + The meson build system is written in Python 3. This directly informs the + choice of Python 3 as the language for all supporting build scripts, + re-inforcing the other benefits of Python, over Perl, Shell, M4, + automake, etc. There is no intention to support Python 2 given meson's + requirement for Python 3. + </p> + + <p> + Using the RST format for documentation allows for the use of XSLT to be + eliminated from the build process. RST and the Sphinx toolkit are widely + used, as seen by the huge repository of content on + https://readthedocs.org/. The ability to embed raw HTML in the RST docs + will greatly facilitate its adoption, avoiding the need for a big bang + conversion of existing content. Given the desire to eliminate Perl + usage, replacing the use of POD documentation for manual pages is an + obvious followup task. RST is the obvious choice to achieve alignment + with the website, allowing the man pages to be easily published online + with other docs. It is further anticipated that the current API docs + generator which uses XSLT to convert the XML API description would be + converted to something which generates RST using Python instead of XSLT. + </p> + </body> +</html> -- 2.21.0

3 3

[libvirt] [PATCH] qemu_capabilities: Put only unique FW images into domcaps
by Michal Privoznik 30 Sep '19

30 Sep '19

In the domain capabilities XML there are FW image paths printed. There are two sources for the image paths (in order of preference): 1) firmware descriptor files - as returned by qemuFirmwareGetSupported() 2) a compile time list of FW:NRAM pairs which can be overridden in qemu.conf If either of those contains a duplicate FW image path (which is a valid use case) it is printed twice in the capabilities XML. While it's technically not a bug, it doesn't look good. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_capabilities.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 9b19930964..489a6872c4 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -5136,12 +5136,22 @@ virQEMUCapsFillDomainLoaderCaps(virDomainCapsLoaderPtr capsLoader, for (i = 0; i < nfirmwares; i++) { const char *filename = firmwares[i]->name; + size_t j; if (!virFileExists(filename)) { VIR_DEBUG("loader filename=%s does not exist", filename); continue; } + /* Put only unique FW images onto the list */ + for (j = 0; j < capsLoader->values.nvalues; j++) { + if (STREQ(filename, capsLoader->values.values[j])) + break; + } + + if (j != capsLoader->values.nvalues) + continue; + if (VIR_STRDUP(capsLoader->values.values[capsLoader->values.nvalues], filename) < 0) return -1; -- 2.21.0

3 2

[libvirt] [PATCH 0/2] qemu: Sanitize update of 'current' checkpoint/snapshot
by Peter Krempa 30 Sep '19

30 Sep '19

See 1/2. Peter Krempa (2): qemu: snapshot: Don't update current snapshot until we're done qemu: checkpoint: Don't update current checkpoint until we are done src/qemu/qemu_checkpoint.c | 7 ------- src/qemu/qemu_driver.c | 19 ------------------- 2 files changed, 26 deletions(-) -- 2.21.0

3 4

[libvirt] [PATCH 0/3] Add vhost-user-scsi/blk support
by Li Feng 30 Sep '19

30 Sep '19

The latest qemu has support the vhost-user-scsi and vhost-user-blk backend for better performance. This patch let the libvirt could manage the vhost-user disks. Usage in xml like this: <vhost-user-blk-pci type='unix'> <source type='bind' path='/tmp/vhost-blk.sock'> <reconnect enabled='yes' timeout='5' /> </source> <queue num='4'/> </vhost-user-blk-pci> Li Feng (3): qemu: Add vhost-user-scsi/blk support vhost-user-scsi/blk: add xml validation check qemu: Add multiqueue support for vhost-user-scsi/blk docs/schemas/domaincommon.rng | 48 ++++++++++ src/conf/device_conf.h | 5 + src/conf/domain_conf.c | 93 +++++++++++++++++- src/conf/domain_conf.h | 9 ++ src/qemu/qemu_alias.c | 16 ++++ src/qemu/qemu_command.c | 105 +++++++++++++++++++++ src/qemu/qemu_domain.c | 2 + .../vhost-user-blk.x86_64-latest.args | 37 ++++++++ tests/qemuxml2argvdata/vhost-user-blk.xml | 38 ++++++++ .../vhost-user-scsi.x86_64-latest.args | 37 ++++++++ tests/qemuxml2argvdata/vhost-user-scsi.xml | 37 ++++++++ tests/qemuxml2argvtest.c | 3 + 12 files changed, 429 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/vhost-user-blk.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-blk.xml create mode 100644 tests/qemuxml2argvdata/vhost-user-scsi.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-scsi.xml -- 2.11.0 -- The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.

2 4