April 2020 - Devel - libvirt List Archives

[PATCH] util: virdaemon: fix waiting for child processes
by Rafael Fonseca 08 Apr '20

08 Apr '20

Unlike `waitpid`, `virProcessWait` only returns -1 (error) or 0 (success), so comparing that to `pid` will always be false and the parent will report failure with: error : main:851 : Failed to fork as daemon: No such file or directory even though the grandchild process is succesfully running. Note that the errno message is misleading: it was last set when trying to find a restart state file. Signed-off-by: Rafael Fonseca <r4f4rfs(a)gmail.com> --- src/util/virdaemon.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/util/virdaemon.c b/src/util/virdaemon.c index 5d92c7def7..99530fd146 100644 --- a/src/util/virdaemon.c +++ b/src/util/virdaemon.c @@ -97,15 +97,14 @@ virDaemonForkIntoBackground(const char *argv0) default: { /* parent */ - int got, exitstatus = 0; + int exitstatus = 0; int ret; char status; VIR_FORCE_CLOSE(statuspipe[1]); /* We wait to make sure the first child forked successfully */ - if ((got = virProcessWait(pid, &exitstatus, 0)) < 0 || - got != pid || + if (virProcessWait(pid, &exitstatus, 0) < 0 || exitstatus != 0) { goto error; } -- 2.25.1

2 2

A few XML modeling questions
by Jim Fehlig 07 Apr '20

07 Apr '20

Hi All, I've been struggling a bit deciding how to model Xen's max_event_channels and e820_host xl.cfg(5) settings. For max_event_channels the man page says: Limit the guest to using at most N event channels (PV interrupts). Guests use hypervisor resources for each event channel they use. The default of 1023 should be sufficient for typical guests. The maximum value depends on what the guest supports. Guests supporting the FIFO-based event channel ABI support up to 131,071 event channels. Other guests are limited to 4095 (64-bit x86 and ARM) or 1023 (32-bit x86). So event channels are PV interrupts that are used by PV devices and timers, and as inter-processor interrupts. My initial thought was to add a maxEventChannels attribute to the xenbus controller, similar maxGrantFrames. The latter was added to allow specifying the max shared pages a guest could use, which in conjunction with event channels provides data transfer mechanism between front and backends for PV drivers. My only hesitation is that event channels are also used for inter-processor interrupts, which is a bit outside the notion of a virtual controller for Xen PV devices. However, the first sentence in the xenbus wiki [1] states: In practice, the bus is used for configuration negotiation, leaving most data transfer to be done via an interdomain channel composed of a shared page and an event channel. So perhaps it is tolerable to consider max_event_channels an attribute of xenbus that is specified at domain creation, to then be used by the guest however it pleases up to the max value? e820_host is a bit trickier. For this setting, which is PV-specific, the man page says: Selects whether to expose the host e820 (memory map) to the guest via the virtual e820. When this option is false (0) the guest pseudo-physical address space consists of a single contiguous RAM region. When this option is specified the virtual e820 instead reflects the host e820 and contains the same PCI holes. The total amount of RAM represented by the memory map is always the same, this option configures only how it is laid out. Exposing the host e820 to the guest gives the guest kernel the opportunity to set aside the required part of its pseudo-physical address space in order to provide address space to map passedthrough PCI devices. It is guest Operating System dependent whether this option is required, specifically it is required when using a mainline Linux ("pvops") kernel. This option defaults to true (1) if any PCI passthrough devices are configured and false (0) otherwise. If you do not configure any passthrough devices at domain creation time but expect to hotplug devices later then you should set this option. Conversely if your particular guest kernel does not require this behavior then it is safe to allow this to be enabled but you may wish to disable it anyway. I'm tempted to unconditionally enable this setting. It is required for pvops kernels and apparently harmless for other PV kernels. I asked one of the Xen devs about any downsides to always enabling e820_host, to which he replied "Scattered memory blocks inside the guest, possibly leading to slightly higher overhead. But nothing really severe afaics.". If there is a strong desire to model this setting it might best be placed under hypervisor features, e.g. <features> <xen> <hoste820 state='on'/> </xen> </features> Opinions and/or comments are much appreciated. Regards, Jim [1] https://wiki.xen.org/wiki/XenBus

1 1

[PATCH ocaml] build: skip github lockdown file in manifest check
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

The github lockdown file is not intended to be included in any built dist, so should be excluded when checking the manifest. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- Pushed as a CI build fix Makefile.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.in b/Makefile.in index 4357ba6..54ecaf3 100644 --- a/Makefile.in +++ b/Makefile.in @@ -86,7 +86,7 @@ dist: ChangeLog ls -l $(PACKAGE)-$(VERSION).tar.gz check-manifest: - git ls-files > .check-manifest; \ + git ls-files | grep -v .github/lockdown.yml > .check-manifest; \ echo aclocal.m4 >> .check-manifest; \ sort -o .check-manifest .check-manifest; \ sort MANIFEST > .orig-manifest; \ -- 2.24.1

2 1

Re: Domain backup file explodes on s3fs
by Eric Blake 07 Apr '20

07 Apr '20

[adding libvirt list] On 4/7/20 2:13 PM, Tim Haley wrote: > Hi all, > > Have been playing with `virsh backup-begin` of late and think it's an > excellent feature. I've noticed one behavior I'm not sure I understand. It looks like https://bugzilla.redhat.com/show_bug.cgi?id=1814664 is a similar description of the same problem: namely, if qemu is not able to determine that the destination already reads as zero, then it forcefully zeroes the destination of a backup job. We may want to copy the fact that qemu 5.0 is adding 'qemu-img convert --target-is-zero' to add a similar knob to the QMP commands that trigger disk copying (blockdev-backup, blockdev-mirror, possibly others) as well as logic to avoid writing zeroes when the destination is already treated as zero (whether by a probe, or by the knob being set). ... > If my /backups directory is just XFS, I get a backup file that looks > like it is just the size of data blocks in use > > -rw------- 1 root root 2769551360 Mar 19 16:56 > vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a For a local file, qemu is easily able to probe whether the destination starts as all zeroes (thanks to lseek(SEEK_DATA)); > > but if I write to an s3fs (object storage backend) the file blows up to > the whole size of the disk > > -rw------- 1 root root 8591507456 Mar 18 19:03 > vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a whereas for s3fs, it looks like qemu does not have access to a quick test to learn if the image starts all zero (POSIX does not provide a quick way for doing this on a generic block device, but if you are aware of an ioctl or otherwise that qemu could use, that might be helpful). Or maybe the s3fs really is random contents rather than all zero, in which case forcefully writing zeroes is the only correct behavior. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

1 0

[virttools-planet PATCH 0/4] Switch planet.virt-tools.org over to use GitLab Pages
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

This introduces use of GitLab CI + Pages to replace the current OpenShift application which is only admin accessible by myself. It also has automatic integration with LetsEncrypt guaranteeing that we'll never have expired certificates. Daniel P. Berrangé (4): Convert README to markdown format Introduce use of GitLab CI for publishing to GitLab Pages Remove obsolete openshift hosting configuration Drop Amit Shah and Nathan Gauër .gitlab-ci.yml | 14 + README | 39 -- README.md | 43 ++ openshift/templates/.gitignore | 2 - openshift/templates/update-tls.sh | 16 - openshift/templates/virttools-planet-tls.json | 82 ---- openshift/templates/virttools-planet.json | 425 ------------------ updater/planet-cache.py => planet-cache.py | 0 updater/planet.py => planet.py | 0 {updater/planet => planet}/__init__.py | 0 {updater/planet => planet}/atomstyler.py | 0 {updater/planet => planet}/cache.py | 0 .../compat_logging/__init__.py | 0 .../compat_logging/config.py | 0 .../compat_logging/handlers.py | 0 {updater/planet => planet}/feedparser.py | 0 {updater/planet => planet}/htmltmpl.py | 0 {updater/planet => planet}/sanitize.py | 0 {updater/planet => planet}/tests/__init__.py | 0 .../planet => planet}/tests/test_channel.py | 0 {updater/planet => planet}/tests/test_main.py | 0 .../planet => planet}/tests/test_sanitize.py | 0 {updater/planet => planet}/timeoutsocket.py | 0 .../images/alexbennee.jpeg | Bin .../virt-tools => public}/images/berrange.png | Bin .../virt-tools => public}/images/cole.png | Bin .../images/ehabkost.jpeg | Bin .../images/header-bg.png | Bin .../virt-tools => public}/images/kashyap.jpeg | Bin .../virt-tools => public}/images/logo.png | Bin .../virt-tools => public}/images/logo.xcf | Bin .../virt-tools => public}/images/otubo.png | Bin .../virt-tools => public}/images/qemu.png | Bin .../virt-tools => public}/images/rjones.jpeg | Bin .../virt-tools => public}/images/sgarzare.png | Bin .../virt-tools => public}/images/teuf.png | Bin .../images/thomashuth.png | Bin .../virt-tools => public}/images/ybettan.png | Bin .../virt-tools => public}/images/zeenix.png | Bin updater/app.py | 18 - .../virt-tools => virt-tools}/atom.xml.tmpl | 0 .../basic/index.html.tmpl | 7 +- .../basic/style.css.tmpl | 0 {updater/virt-tools => virt-tools}/config.ini | 10 +- .../foafroll.xml.tmpl | 0 .../virt-tools => virt-tools}/opml.xml.tmpl | 0 .../virt-tools => virt-tools}/rss10.xml.tmpl | 0 .../virt-tools => virt-tools}/rss20.xml.tmpl | 0 web/httpd-cfg/cors.conf | 3 - 49 files changed, 62 insertions(+), 597 deletions(-) create mode 100644 .gitlab-ci.yml delete mode 100644 README create mode 100644 README.md delete mode 100644 openshift/templates/.gitignore delete mode 100755 openshift/templates/update-tls.sh delete mode 100644 openshift/templates/virttools-planet-tls.json delete mode 100644 openshift/templates/virttools-planet.json rename updater/planet-cache.py => planet-cache.py (100%) rename updater/planet.py => planet.py (100%) rename {updater/planet => planet}/__init__.py (100%) rename {updater/planet => planet}/atomstyler.py (100%) rename {updater/planet => planet}/cache.py (100%) rename {updater/planet => planet}/compat_logging/__init__.py (100%) rename {updater/planet => planet}/compat_logging/config.py (100%) rename {updater/planet => planet}/compat_logging/handlers.py (100%) rename {updater/planet => planet}/feedparser.py (100%) rename {updater/planet => planet}/htmltmpl.py (100%) rename {updater/planet => planet}/sanitize.py (100%) rename {updater/planet => planet}/tests/__init__.py (100%) rename {updater/planet => planet}/tests/test_channel.py (100%) rename {updater/planet => planet}/tests/test_main.py (100%) rename {updater/planet => planet}/tests/test_sanitize.py (100%) rename {updater/planet => planet}/timeoutsocket.py (100%) rename {updater/virt-tools => public}/images/alexbennee.jpeg (100%) rename {updater/virt-tools => public}/images/berrange.png (100%) rename {updater/virt-tools => public}/images/cole.png (100%) rename {updater/virt-tools => public}/images/ehabkost.jpeg (100%) rename {updater/virt-tools => public}/images/header-bg.png (100%) rename {updater/virt-tools => public}/images/kashyap.jpeg (100%) rename {updater/virt-tools => public}/images/logo.png (100%) rename {updater/virt-tools => public}/images/logo.xcf (100%) rename {updater/virt-tools => public}/images/otubo.png (100%) rename {updater/virt-tools => public}/images/qemu.png (100%) rename {updater/virt-tools => public}/images/rjones.jpeg (100%) rename {updater/virt-tools => public}/images/sgarzare.png (100%) rename {updater/virt-tools => public}/images/teuf.png (100%) rename {updater/virt-tools => public}/images/thomashuth.png (100%) rename {updater/virt-tools => public}/images/ybettan.png (100%) rename {updater/virt-tools => public}/images/zeenix.png (100%) delete mode 100755 updater/app.py rename {updater/virt-tools => virt-tools}/atom.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/basic/index.html.tmpl (91%) rename {updater/virt-tools => virt-tools}/basic/style.css.tmpl (100%) rename {updater/virt-tools => virt-tools}/config.ini (96%) rename {updater/virt-tools => virt-tools}/foafroll.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/opml.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/rss10.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/rss20.xml.tmpl (100%) delete mode 100644 web/httpd-cfg/cors.conf -- 2.24.1

2 10

[libvirt PATCH 00/10] docs: Convert hacking to rst, split and rework it
by Andrea Bolognani 07 Apr '20

07 Apr '20

hacking.html is the canonical entry point for people looking to help with libvirt's development, but its sheer size makes it a farily ineffective tool. Move most non-critical information off to separate pages and brush up what's left so that it works as a quick introduction. As a prerequisite, convert the existing page to reStructuredText. I have compared the HTML version and the conversion visually and didn't spot any massive differences, save for a few spacing changes that are not specific to the page itself but rather to the HTML generated by docutils and which will require CSS tweaks to address, which is a job I'm leaving to someone else. I have also spent very little time actually changing the contents, and mostly just moved them around. There's plenty of opportunities for improvement there, but once again that's someone else's job. Andrea Bolognani (10): docs: Convert hacking.html to reStructuredText docs: Add glib-adoption.rst docs: Add coding-style.rst docs: Add programming-languages.rst docs: Add developer-tooling.rst docs: Add committer-guidelines.rst docs: Add advanced-tests.rst docs: Add submitting-patches.rst docs: Add best-practices.rst docs: Update hacking.rst build-aux/syntax-check.mk | 4 +- docs/advanced-tests.rst | 178 ++++ docs/best-practices.rst | 37 + docs/coding-style.rst | 919 +++++++++++++++++++ docs/committer-guidelines.rst | 33 + docs/developer-tooling.rst | 13 + docs/glib-adoption.rst | 96 ++ docs/hacking.html.in | 1555 -------------------------------- docs/hacking.rst | 79 ++ docs/programming-languages.rst | 21 + docs/submitting-patches.rst | 88 ++ 11 files changed, 1466 insertions(+), 1557 deletions(-) create mode 100644 docs/advanced-tests.rst create mode 100644 docs/best-practices.rst create mode 100644 docs/coding-style.rst create mode 100644 docs/committer-guidelines.rst create mode 100644 docs/developer-tooling.rst create mode 100644 docs/glib-adoption.rst delete mode 100644 docs/hacking.html.in create mode 100644 docs/hacking.rst create mode 100644 docs/programming-languages.rst create mode 100644 docs/submitting-patches.rst -- 2.25.1

2 13

ALERT: cimtest.git repository history recreated
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

In attempting to import the cimtest.git repository to gitlab I discovered some serious problems in the current git history. Specifically the author and committer email addresses were badly mangled for many commits. For example an author of: Deepti B. Kalakeri<deeptik(a)linux.vnet.ibm.com> <Deepti B. Kalakeri<deeptik(a)linux.vnet.ibm.com>> This is so badly malformed that both gitlab and github refuse to allow the commits in question to be pushed. Not only that, but "git filter-branch" and "git fast-import" will both crash and burn when trying to parse these, so if you try to rewrite email addresses with "git filter-branch" it makes it even worse. The only solution was to use "git fast-export --all" to dump the entire history to a file, then hand edit this to fix the broken addresses that were confusing filter-branch, then use "git fast-import" to initialize a new repository. cimtest.git is a ancient repo that hasn't seen changes since 2014, so I'm guessing it is unlikely any current contributor has a checkout of it. If someone does though, you'll have to throw away your current checkout and get a new clone, as the history has been fully rewritten to fix this problem. I suspect the root cause dates from when we imported the original Mercurial repo into Git. Our conversion process probably mangled some addresses without us realizing at the time. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

1 0

[libvirt-go-xml PATCH 0/2] Switch CI from Travis to GitLab
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

Daniel P. Berrangé (2): gitlab: add CI definition for GitLab travis: delete CI configuration .gitlab-ci.yml | 25 +++++++++++++++++++++++++ .travis.yml | 13 ------------- README.md | 2 +- 3 files changed, 26 insertions(+), 14 deletions(-) create mode 100644 .gitlab-ci.yml delete mode 100644 .travis.yml -- 2.24.1

2 5

[virttools-web PATCH 0/3] Switch www.virt-tools.org over to use GitLab Pages
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

This introduces use of GitLab CI + Pages to replace the current OpenShift application which is only admin accessible by myself. It also has automatic integration with LetsEncrypt guaranteeing that we'll never have expired certificates. Daniel P. Berrangé (3): Convert README to markdown format Introduce use of GitLab CI for publishing to GitLab Pages Remove obsolete openshift hosting configuration .gitlab-ci.yml | 9 + README | 19 -- README.md | 15 ++ openshift/templates/.gitignore | 2 - openshift/templates/update-tls.sh | 16 -- openshift/templates/virttools-web-tls.json | 82 ------- openshift/templates/virttools-web.json | 272 --------------------- {web => public}/header-bg.png | Bin {web => public}/index.css | 0 {web => public}/index.html | 4 +- {web => public}/logo-kvm.png | Bin {web => public}/logo-libguestfs.png | Bin {web => public}/logo-libosinfo.png | Bin {web => public}/logo-libvirt.png | Bin {web => public}/logo-qemu.png | Bin {web => public}/logo-virt-manager.png | Bin {web => public}/logo.png | Bin {web => public}/logo.xcf | Bin 18 files changed, 25 insertions(+), 394 deletions(-) create mode 100644 .gitlab-ci.yml delete mode 100644 README create mode 100644 README.md delete mode 100644 openshift/templates/.gitignore delete mode 100755 openshift/templates/update-tls.sh delete mode 100644 openshift/templates/virttools-web-tls.json delete mode 100644 openshift/templates/virttools-web.json rename {web => public}/header-bg.png (100%) rename {web => public}/index.css (100%) rename {web => public}/index.html (93%) rename {web => public}/logo-kvm.png (100%) rename {web => public}/logo-libguestfs.png (100%) rename {web => public}/logo-libosinfo.png (100%) rename {web => public}/logo-libvirt.png (100%) rename {web => public}/logo-qemu.png (100%) rename {web => public}/logo-virt-manager.png (100%) rename {web => public}/logo.png (100%) rename {web => public}/logo.xcf (100%) -- 2.24.1

2 4

[PATCHv2 0/1] Use g_autoptr and g_autofree in cpu.c
by Seeteena Thoufeek 07 Apr '20

07 Apr '20

return NULL for all the failure cases for the function virCPUDataParse(). Previous discussion can be found at https://www.redhat.com/archives/libvir-list/2020-April/msg00191.html Seeteena Thoufeek (1): cpu: Use g_autoptr and g_autofree in cpu.c src/cpu/cpu.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) -- 1.8.3.1

2 2