- Devel - libvirt List Archives

Re: Domain backup file explodes on s3fs
by Eric Blake 07 Apr '20

07 Apr '20

[adding libvirt list] On 4/7/20 2:13 PM, Tim Haley wrote: > Hi all, > > Have been playing with `virsh backup-begin` of late and think it's an > excellent feature. I've noticed one behavior I'm not sure I understand. It looks like https://bugzilla.redhat.com/show_bug.cgi?id=1814664 is a similar description of the same problem: namely, if qemu is not able to determine that the destination already reads as zero, then it forcefully zeroes the destination of a backup job. We may want to copy the fact that qemu 5.0 is adding 'qemu-img convert --target-is-zero' to add a similar knob to the QMP commands that trigger disk copying (blockdev-backup, blockdev-mirror, possibly others) as well as logic to avoid writing zeroes when the destination is already treated as zero (whether by a probe, or by the knob being set). ... > If my /backups directory is just XFS, I get a backup file that looks > like it is just the size of data blocks in use > > -rw------- 1 root root 2769551360 Mar 19 16:56 > vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a For a local file, qemu is easily able to probe whether the destination starts as all zeroes (thanks to lseek(SEEK_DATA)); > > but if I write to an s3fs (object storage backend) the file blows up to > the whole size of the disk > > -rw------- 1 root root 8591507456 Mar 18 19:03 > vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a whereas for s3fs, it looks like qemu does not have access to a quick test to learn if the image starts all zero (POSIX does not provide a quick way for doing this on a generic block device, but if you are aware of an ioctl or otherwise that qemu could use, that might be helpful). Or maybe the s3fs really is random contents rather than all zero, in which case forcefully writing zeroes is the only correct behavior. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

1 0

[virttools-planet PATCH 0/4] Switch planet.virt-tools.org over to use GitLab Pages
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

This introduces use of GitLab CI + Pages to replace the current OpenShift application which is only admin accessible by myself. It also has automatic integration with LetsEncrypt guaranteeing that we'll never have expired certificates. Daniel P. Berrangé (4): Convert README to markdown format Introduce use of GitLab CI for publishing to GitLab Pages Remove obsolete openshift hosting configuration Drop Amit Shah and Nathan Gauër .gitlab-ci.yml | 14 + README | 39 -- README.md | 43 ++ openshift/templates/.gitignore | 2 - openshift/templates/update-tls.sh | 16 - openshift/templates/virttools-planet-tls.json | 82 ---- openshift/templates/virttools-planet.json | 425 ------------------ updater/planet-cache.py => planet-cache.py | 0 updater/planet.py => planet.py | 0 {updater/planet => planet}/__init__.py | 0 {updater/planet => planet}/atomstyler.py | 0 {updater/planet => planet}/cache.py | 0 .../compat_logging/__init__.py | 0 .../compat_logging/config.py | 0 .../compat_logging/handlers.py | 0 {updater/planet => planet}/feedparser.py | 0 {updater/planet => planet}/htmltmpl.py | 0 {updater/planet => planet}/sanitize.py | 0 {updater/planet => planet}/tests/__init__.py | 0 .../planet => planet}/tests/test_channel.py | 0 {updater/planet => planet}/tests/test_main.py | 0 .../planet => planet}/tests/test_sanitize.py | 0 {updater/planet => planet}/timeoutsocket.py | 0 .../images/alexbennee.jpeg | Bin .../virt-tools => public}/images/berrange.png | Bin .../virt-tools => public}/images/cole.png | Bin .../images/ehabkost.jpeg | Bin .../images/header-bg.png | Bin .../virt-tools => public}/images/kashyap.jpeg | Bin .../virt-tools => public}/images/logo.png | Bin .../virt-tools => public}/images/logo.xcf | Bin .../virt-tools => public}/images/otubo.png | Bin .../virt-tools => public}/images/qemu.png | Bin .../virt-tools => public}/images/rjones.jpeg | Bin .../virt-tools => public}/images/sgarzare.png | Bin .../virt-tools => public}/images/teuf.png | Bin .../images/thomashuth.png | Bin .../virt-tools => public}/images/ybettan.png | Bin .../virt-tools => public}/images/zeenix.png | Bin updater/app.py | 18 - .../virt-tools => virt-tools}/atom.xml.tmpl | 0 .../basic/index.html.tmpl | 7 +- .../basic/style.css.tmpl | 0 {updater/virt-tools => virt-tools}/config.ini | 10 +- .../foafroll.xml.tmpl | 0 .../virt-tools => virt-tools}/opml.xml.tmpl | 0 .../virt-tools => virt-tools}/rss10.xml.tmpl | 0 .../virt-tools => virt-tools}/rss20.xml.tmpl | 0 web/httpd-cfg/cors.conf | 3 - 49 files changed, 62 insertions(+), 597 deletions(-) create mode 100644 .gitlab-ci.yml delete mode 100644 README create mode 100644 README.md delete mode 100644 openshift/templates/.gitignore delete mode 100755 openshift/templates/update-tls.sh delete mode 100644 openshift/templates/virttools-planet-tls.json delete mode 100644 openshift/templates/virttools-planet.json rename updater/planet-cache.py => planet-cache.py (100%) rename updater/planet.py => planet.py (100%) rename {updater/planet => planet}/__init__.py (100%) rename {updater/planet => planet}/atomstyler.py (100%) rename {updater/planet => planet}/cache.py (100%) rename {updater/planet => planet}/compat_logging/__init__.py (100%) rename {updater/planet => planet}/compat_logging/config.py (100%) rename {updater/planet => planet}/compat_logging/handlers.py (100%) rename {updater/planet => planet}/feedparser.py (100%) rename {updater/planet => planet}/htmltmpl.py (100%) rename {updater/planet => planet}/sanitize.py (100%) rename {updater/planet => planet}/tests/__init__.py (100%) rename {updater/planet => planet}/tests/test_channel.py (100%) rename {updater/planet => planet}/tests/test_main.py (100%) rename {updater/planet => planet}/tests/test_sanitize.py (100%) rename {updater/planet => planet}/timeoutsocket.py (100%) rename {updater/virt-tools => public}/images/alexbennee.jpeg (100%) rename {updater/virt-tools => public}/images/berrange.png (100%) rename {updater/virt-tools => public}/images/cole.png (100%) rename {updater/virt-tools => public}/images/ehabkost.jpeg (100%) rename {updater/virt-tools => public}/images/header-bg.png (100%) rename {updater/virt-tools => public}/images/kashyap.jpeg (100%) rename {updater/virt-tools => public}/images/logo.png (100%) rename {updater/virt-tools => public}/images/logo.xcf (100%) rename {updater/virt-tools => public}/images/otubo.png (100%) rename {updater/virt-tools => public}/images/qemu.png (100%) rename {updater/virt-tools => public}/images/rjones.jpeg (100%) rename {updater/virt-tools => public}/images/sgarzare.png (100%) rename {updater/virt-tools => public}/images/teuf.png (100%) rename {updater/virt-tools => public}/images/thomashuth.png (100%) rename {updater/virt-tools => public}/images/ybettan.png (100%) rename {updater/virt-tools => public}/images/zeenix.png (100%) delete mode 100755 updater/app.py rename {updater/virt-tools => virt-tools}/atom.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/basic/index.html.tmpl (91%) rename {updater/virt-tools => virt-tools}/basic/style.css.tmpl (100%) rename {updater/virt-tools => virt-tools}/config.ini (96%) rename {updater/virt-tools => virt-tools}/foafroll.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/opml.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/rss10.xml.tmpl (100%) rename {updater/virt-tools => virt-tools}/rss20.xml.tmpl (100%) delete mode 100644 web/httpd-cfg/cors.conf -- 2.24.1

2 10

[libvirt PATCH 00/10] docs: Convert hacking to rst, split and rework it
by Andrea Bolognani 07 Apr '20

07 Apr '20

hacking.html is the canonical entry point for people looking to help with libvirt's development, but its sheer size makes it a farily ineffective tool. Move most non-critical information off to separate pages and brush up what's left so that it works as a quick introduction. As a prerequisite, convert the existing page to reStructuredText. I have compared the HTML version and the conversion visually and didn't spot any massive differences, save for a few spacing changes that are not specific to the page itself but rather to the HTML generated by docutils and which will require CSS tweaks to address, which is a job I'm leaving to someone else. I have also spent very little time actually changing the contents, and mostly just moved them around. There's plenty of opportunities for improvement there, but once again that's someone else's job. Andrea Bolognani (10): docs: Convert hacking.html to reStructuredText docs: Add glib-adoption.rst docs: Add coding-style.rst docs: Add programming-languages.rst docs: Add developer-tooling.rst docs: Add committer-guidelines.rst docs: Add advanced-tests.rst docs: Add submitting-patches.rst docs: Add best-practices.rst docs: Update hacking.rst build-aux/syntax-check.mk | 4 +- docs/advanced-tests.rst | 178 ++++ docs/best-practices.rst | 37 + docs/coding-style.rst | 919 +++++++++++++++++++ docs/committer-guidelines.rst | 33 + docs/developer-tooling.rst | 13 + docs/glib-adoption.rst | 96 ++ docs/hacking.html.in | 1555 -------------------------------- docs/hacking.rst | 79 ++ docs/programming-languages.rst | 21 + docs/submitting-patches.rst | 88 ++ 11 files changed, 1466 insertions(+), 1557 deletions(-) create mode 100644 docs/advanced-tests.rst create mode 100644 docs/best-practices.rst create mode 100644 docs/coding-style.rst create mode 100644 docs/committer-guidelines.rst create mode 100644 docs/developer-tooling.rst create mode 100644 docs/glib-adoption.rst delete mode 100644 docs/hacking.html.in create mode 100644 docs/hacking.rst create mode 100644 docs/programming-languages.rst create mode 100644 docs/submitting-patches.rst -- 2.25.1

2 13

ALERT: cimtest.git repository history recreated
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

In attempting to import the cimtest.git repository to gitlab I discovered some serious problems in the current git history. Specifically the author and committer email addresses were badly mangled for many commits. For example an author of: Deepti B. Kalakeri<deeptik(a)linux.vnet.ibm.com> <Deepti B. Kalakeri<deeptik(a)linux.vnet.ibm.com>> This is so badly malformed that both gitlab and github refuse to allow the commits in question to be pushed. Not only that, but "git filter-branch" and "git fast-import" will both crash and burn when trying to parse these, so if you try to rewrite email addresses with "git filter-branch" it makes it even worse. The only solution was to use "git fast-export --all" to dump the entire history to a file, then hand edit this to fix the broken addresses that were confusing filter-branch, then use "git fast-import" to initialize a new repository. cimtest.git is a ancient repo that hasn't seen changes since 2014, so I'm guessing it is unlikely any current contributor has a checkout of it. If someone does though, you'll have to throw away your current checkout and get a new clone, as the history has been fully rewritten to fix this problem. I suspect the root cause dates from when we imported the original Mercurial repo into Git. Our conversion process probably mangled some addresses without us realizing at the time. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

1 0

[libvirt-go-xml PATCH 0/2] Switch CI from Travis to GitLab
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

Daniel P. Berrangé (2): gitlab: add CI definition for GitLab travis: delete CI configuration .gitlab-ci.yml | 25 +++++++++++++++++++++++++ .travis.yml | 13 ------------- README.md | 2 +- 3 files changed, 26 insertions(+), 14 deletions(-) create mode 100644 .gitlab-ci.yml delete mode 100644 .travis.yml -- 2.24.1

2 5

[virttools-web PATCH 0/3] Switch www.virt-tools.org over to use GitLab Pages
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

This introduces use of GitLab CI + Pages to replace the current OpenShift application which is only admin accessible by myself. It also has automatic integration with LetsEncrypt guaranteeing that we'll never have expired certificates. Daniel P. Berrangé (3): Convert README to markdown format Introduce use of GitLab CI for publishing to GitLab Pages Remove obsolete openshift hosting configuration .gitlab-ci.yml | 9 + README | 19 -- README.md | 15 ++ openshift/templates/.gitignore | 2 - openshift/templates/update-tls.sh | 16 -- openshift/templates/virttools-web-tls.json | 82 ------- openshift/templates/virttools-web.json | 272 --------------------- {web => public}/header-bg.png | Bin {web => public}/index.css | 0 {web => public}/index.html | 4 +- {web => public}/logo-kvm.png | Bin {web => public}/logo-libguestfs.png | Bin {web => public}/logo-libosinfo.png | Bin {web => public}/logo-libvirt.png | Bin {web => public}/logo-qemu.png | Bin {web => public}/logo-virt-manager.png | Bin {web => public}/logo.png | Bin {web => public}/logo.xcf | Bin 18 files changed, 25 insertions(+), 394 deletions(-) create mode 100644 .gitlab-ci.yml delete mode 100644 README create mode 100644 README.md delete mode 100644 openshift/templates/.gitignore delete mode 100755 openshift/templates/update-tls.sh delete mode 100644 openshift/templates/virttools-web-tls.json delete mode 100644 openshift/templates/virttools-web.json rename {web => public}/header-bg.png (100%) rename {web => public}/index.css (100%) rename {web => public}/index.html (93%) rename {web => public}/logo-kvm.png (100%) rename {web => public}/logo-libguestfs.png (100%) rename {web => public}/logo-libosinfo.png (100%) rename {web => public}/logo-libvirt.png (100%) rename {web => public}/logo-qemu.png (100%) rename {web => public}/logo-virt-manager.png (100%) rename {web => public}/logo.png (100%) rename {web => public}/logo.xcf (100%) -- 2.24.1

2 4

[PATCHv2 0/1] Use g_autoptr and g_autofree in cpu.c
by Seeteena Thoufeek 07 Apr '20

07 Apr '20

return NULL for all the failure cases for the function virCPUDataParse(). Previous discussion can be found at https://www.redhat.com/archives/libvir-list/2020-April/msg00191.html Seeteena Thoufeek (1): cpu: Use g_autoptr and g_autofree in cpu.c src/cpu/cpu.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) -- 1.8.3.1

2 2

[libvirt PATCH 00/15] RFC: basic CGroup support with qemu:///session
by marcandre.lureau＠redhat.com 07 Apr '20

07 Apr '20

From: Marc-André Lureau <marcandre.lureau(a)redhat.com> Hi, This is a small series that allows basic QEMU VM CGroup support with the help of machined --user: https://github.com/systemd/systemd/pull/15312 The first few patches are fixes to register dbus and slirp-helper correctly with the VM cgroup. A few changes are done to the machined support, adding session support, and registering the VM to get a systemd scope cgroup under user machine.slice. Marc-André Lureau (15): slirp: leave the dbus daemon running on error slirp: add helper to VM cgroup qemu-dbus: prevent double start of the bus daemon qemu-dbus: remove unused variable qemu-dbus: register DBus bus to the VM cgroup qemu-cgroup: remove unnecessary include dbus: rename virDBusIs* -> virDBusSystemIs* dbus: add virDBusIsService{Enabled,Registered} with @conn argument systemd: check org.freedesktop.machine1 registration systemd: only check for activitable machine1 service systemd: register machines against session service cgroup: return directly if there is nothing to remove qemu-cgroup: register ext devices when cpu/cpuset controller are missing qemu: create cgroup regardless of controller support qemu-cgroup: drop the need for privileges to use cgroup src/qemu/qemu_cgroup.c | 13 ------- src/qemu/qemu_dbus.c | 29 +++++++++++++- src/qemu/qemu_dbus.h | 3 ++ src/qemu/qemu_extdevice.c | 12 ++++++ src/qemu/qemu_process.c | 6 +-- src/qemu/qemu_slirp.c | 16 +++++++- src/qemu/qemu_slirp.h | 3 ++ src/util/vircgroup.c | 5 +++ src/util/vircgroupv2devices.c | 7 ++-- src/util/virdbus.c | 71 ++++++++++++++++++++++++++--------- src/util/virdbus.h | 7 +++- src/util/virfirewalld.c | 2 +- src/util/virsystemd.c | 27 +++++-------- tests/virsystemdtest.c | 35 ----------------- 14 files changed, 140 insertions(+), 96 deletions(-) -- 2.26.0.rc2.42.g98cedd0233

5 23

ANNOUNCE: primary git repos switched to https://gitlab.com/libvirt/
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

Hello All, As of this email, the primary host for libvirt project git repositories has been switched to gitlab.com with all projects under the following group: https://gitlab.com/libvirt The development process remains the same for now, with patches still to be submitted to this mailing list as before. The repositories on libvirt.org will continue to exist, but as an automated read-only mirror of the primary gitlab.com repositories, so may be delayed 30-60 minutes vs the primary repos. To update your current checkouts to point to the new primary repos use the following: git remote set-url origin https://gitlab.com/libvirt/$REPO.git eg git remote set-url origin https://gitlab.com/libvirt/libvirt.git (if you are a person with commit rights you'll need to use the ssh URL instead which I've emailed you about directly) Switching over to require merge requests is a separate task that will be done incrementally for repos over a period of time, with further announcements made on this at a later date. As long as a ".gitpublish" config file exists in a repo, then patches for that repo should continue to be sent to this mailing list. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

1 0

backport new changes to 4.0.0
by Jiatong Shen 07 Apr '20

07 Apr '20

Hello, Sorry in advance if I am asking questions at the wrong place. I would like know if we have plans to backport new features to old releases, for example block copy remote disks. If the answer is no, then where can I find some guidance on building a libvirt debian packages? Thank you very much for the help. -- Best Regards, Jiatong Shen

3 2