A few XML modeling questions
by Jim Fehlig
Hi All,
I've been struggling a bit deciding how to model Xen's max_event_channels and
e820_host xl.cfg(5) settings. For max_event_channels the man page says:
Limit the guest to using at most N event channels (PV interrupts). Guests
use hypervisor resources for each event channel they use.
The default of 1023 should be sufficient for typical guests. The maximum
value depends on what the guest supports. Guests supporting the FIFO-based
event channel ABI support up to 131,071 event channels. Other guests are
limited to 4095 (64-bit x86 and ARM) or 1023 (32-bit x86).
So event channels are PV interrupts that are used by PV devices and timers, and
as inter-processor interrupts. My initial thought was to add a maxEventChannels
attribute to the xenbus controller, similar maxGrantFrames. The latter was added
to allow specifying the max shared pages a guest could use, which in conjunction
with event channels provides data transfer mechanism between front and backends
for PV drivers. My only hesitation is that event channels are also used for
inter-processor interrupts, which is a bit outside the notion of a virtual
controller for Xen PV devices. However, the first sentence in the xenbus wiki
[1] states:
In practice, the bus is used for configuration negotiation, leaving most
data transfer to be done via an interdomain channel composed of a shared
page and an event channel.
So perhaps it is tolerable to consider max_event_channels an attribute of xenbus
that is specified at domain creation, to then be used by the guest however it
pleases up to the max value?
e820_host is a bit trickier. For this setting, which is PV-specific, the man
page says:
Selects whether to expose the host e820 (memory map) to the guest via the
virtual e820. When this option is false (0) the guest pseudo-physical
address space consists of a single contiguous RAM region. When this option
is specified the virtual e820 instead reflects the host e820 and contains
the same PCI holes. The total amount of RAM represented by the memory map
is always the same, this option configures only how it is laid out.
Exposing the host e820 to the guest gives the guest kernel the opportunity
to set aside the required part of its pseudo-physical address space in order
to provide address space to map passedthrough PCI devices. It is guest
Operating System dependent whether this option is required, specifically it
is required when using a mainline Linux ("pvops") kernel. This option
defaults to true (1) if any PCI passthrough devices are configured and
false (0) otherwise. If you do not configure any passthrough devices at
domain creation time but expect to hotplug devices later then you should
set this option. Conversely if your particular guest kernel does not
require this behavior then it is safe to allow this to be enabled but
you may wish to disable it anyway.
I'm tempted to unconditionally enable this setting. It is required for pvops
kernels and apparently harmless for other PV kernels. I asked one of the Xen
devs about any downsides to always enabling e820_host, to which he replied
"Scattered memory blocks inside the guest, possibly leading to slightly higher
overhead. But nothing really severe afaics.".
If there is a strong desire to model this setting it might best be placed under
hypervisor features, e.g.
<features>
<xen>
<hoste820 state='on'/>
</xen>
</features>
Opinions and/or comments are much appreciated.
Regards,
Jim
[1] https://wiki.xen.org/wiki/XenBus
4 years, 6 months
[PATCH ocaml] build: skip github lockdown file in manifest check
by Daniel P. Berrangé
The github lockdown file is not intended to be included in any built
dist, so should be excluded when checking the manifest.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
Pushed as a CI build fix
Makefile.in | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Makefile.in b/Makefile.in
index 4357ba6..54ecaf3 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -86,7 +86,7 @@ dist: ChangeLog
ls -l $(PACKAGE)-$(VERSION).tar.gz
check-manifest:
- git ls-files > .check-manifest; \
+ git ls-files | grep -v .github/lockdown.yml > .check-manifest; \
echo aclocal.m4 >> .check-manifest; \
sort -o .check-manifest .check-manifest; \
sort MANIFEST > .orig-manifest; \
--
2.24.1
4 years, 6 months
Re: Domain backup file explodes on s3fs
by Eric Blake
[adding libvirt list]
On 4/7/20 2:13 PM, Tim Haley wrote:
> Hi all,
>
> Have been playing with `virsh backup-begin` of late and think it's an
> excellent feature. I've noticed one behavior I'm not sure I understand.
It looks like https://bugzilla.redhat.com/show_bug.cgi?id=1814664 is a
similar description of the same problem: namely, if qemu is not able to
determine that the destination already reads as zero, then it forcefully
zeroes the destination of a backup job. We may want to copy the fact
that qemu 5.0 is adding 'qemu-img convert --target-is-zero' to add a
similar knob to the QMP commands that trigger disk copying
(blockdev-backup, blockdev-mirror, possibly others) as well as logic to
avoid writing zeroes when the destination is already treated as zero
(whether by a probe, or by the knob being set).
...
> If my /backups directory is just XFS, I get a backup file that looks
> like it is just the size of data blocks in use
>
> -rw------- 1 root root 2769551360 Mar 19 16:56
> vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a
For a local file, qemu is easily able to probe whether the destination
starts as all zeroes (thanks to lseek(SEEK_DATA));
>
> but if I write to an s3fs (object storage backend) the file blows up to
> the whole size of the disk
>
> -rw------- 1 root root 8591507456 Mar 18 19:03
> vda.2aa450cc-6d2e-11ea-8de0-52542e0d008a
whereas for s3fs, it looks like qemu does not have access to a quick
test to learn if the image starts all zero (POSIX does not provide a
quick way for doing this on a generic block device, but if you are aware
of an ioctl or otherwise that qemu could use, that might be helpful).
Or maybe the s3fs really is random contents rather than all zero, in
which case forcefully writing zeroes is the only correct behavior.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
4 years, 6 months
[virttools-planet PATCH 0/4] Switch planet.virt-tools.org over to use GitLab Pages
by Daniel P. Berrangé
This introduces use of GitLab CI + Pages to replace the current
OpenShift application which is only admin accessible by myself.
It also has automatic integration with LetsEncrypt guaranteeing
that we'll never have expired certificates.
Daniel P. Berrangé (4):
Convert README to markdown format
Introduce use of GitLab CI for publishing to GitLab Pages
Remove obsolete openshift hosting configuration
Drop Amit Shah and Nathan Gauër
.gitlab-ci.yml | 14 +
README | 39 --
README.md | 43 ++
openshift/templates/.gitignore | 2 -
openshift/templates/update-tls.sh | 16 -
openshift/templates/virttools-planet-tls.json | 82 ----
openshift/templates/virttools-planet.json | 425 ------------------
updater/planet-cache.py => planet-cache.py | 0
updater/planet.py => planet.py | 0
{updater/planet => planet}/__init__.py | 0
{updater/planet => planet}/atomstyler.py | 0
{updater/planet => planet}/cache.py | 0
.../compat_logging/__init__.py | 0
.../compat_logging/config.py | 0
.../compat_logging/handlers.py | 0
{updater/planet => planet}/feedparser.py | 0
{updater/planet => planet}/htmltmpl.py | 0
{updater/planet => planet}/sanitize.py | 0
{updater/planet => planet}/tests/__init__.py | 0
.../planet => planet}/tests/test_channel.py | 0
{updater/planet => planet}/tests/test_main.py | 0
.../planet => planet}/tests/test_sanitize.py | 0
{updater/planet => planet}/timeoutsocket.py | 0
.../images/alexbennee.jpeg | Bin
.../virt-tools => public}/images/berrange.png | Bin
.../virt-tools => public}/images/cole.png | Bin
.../images/ehabkost.jpeg | Bin
.../images/header-bg.png | Bin
.../virt-tools => public}/images/kashyap.jpeg | Bin
.../virt-tools => public}/images/logo.png | Bin
.../virt-tools => public}/images/logo.xcf | Bin
.../virt-tools => public}/images/otubo.png | Bin
.../virt-tools => public}/images/qemu.png | Bin
.../virt-tools => public}/images/rjones.jpeg | Bin
.../virt-tools => public}/images/sgarzare.png | Bin
.../virt-tools => public}/images/teuf.png | Bin
.../images/thomashuth.png | Bin
.../virt-tools => public}/images/ybettan.png | Bin
.../virt-tools => public}/images/zeenix.png | Bin
updater/app.py | 18 -
.../virt-tools => virt-tools}/atom.xml.tmpl | 0
.../basic/index.html.tmpl | 7 +-
.../basic/style.css.tmpl | 0
{updater/virt-tools => virt-tools}/config.ini | 10 +-
.../foafroll.xml.tmpl | 0
.../virt-tools => virt-tools}/opml.xml.tmpl | 0
.../virt-tools => virt-tools}/rss10.xml.tmpl | 0
.../virt-tools => virt-tools}/rss20.xml.tmpl | 0
web/httpd-cfg/cors.conf | 3 -
49 files changed, 62 insertions(+), 597 deletions(-)
create mode 100644 .gitlab-ci.yml
delete mode 100644 README
create mode 100644 README.md
delete mode 100644 openshift/templates/.gitignore
delete mode 100755 openshift/templates/update-tls.sh
delete mode 100644 openshift/templates/virttools-planet-tls.json
delete mode 100644 openshift/templates/virttools-planet.json
rename updater/planet-cache.py => planet-cache.py (100%)
rename updater/planet.py => planet.py (100%)
rename {updater/planet => planet}/__init__.py (100%)
rename {updater/planet => planet}/atomstyler.py (100%)
rename {updater/planet => planet}/cache.py (100%)
rename {updater/planet => planet}/compat_logging/__init__.py (100%)
rename {updater/planet => planet}/compat_logging/config.py (100%)
rename {updater/planet => planet}/compat_logging/handlers.py (100%)
rename {updater/planet => planet}/feedparser.py (100%)
rename {updater/planet => planet}/htmltmpl.py (100%)
rename {updater/planet => planet}/sanitize.py (100%)
rename {updater/planet => planet}/tests/__init__.py (100%)
rename {updater/planet => planet}/tests/test_channel.py (100%)
rename {updater/planet => planet}/tests/test_main.py (100%)
rename {updater/planet => planet}/tests/test_sanitize.py (100%)
rename {updater/planet => planet}/timeoutsocket.py (100%)
rename {updater/virt-tools => public}/images/alexbennee.jpeg (100%)
rename {updater/virt-tools => public}/images/berrange.png (100%)
rename {updater/virt-tools => public}/images/cole.png (100%)
rename {updater/virt-tools => public}/images/ehabkost.jpeg (100%)
rename {updater/virt-tools => public}/images/header-bg.png (100%)
rename {updater/virt-tools => public}/images/kashyap.jpeg (100%)
rename {updater/virt-tools => public}/images/logo.png (100%)
rename {updater/virt-tools => public}/images/logo.xcf (100%)
rename {updater/virt-tools => public}/images/otubo.png (100%)
rename {updater/virt-tools => public}/images/qemu.png (100%)
rename {updater/virt-tools => public}/images/rjones.jpeg (100%)
rename {updater/virt-tools => public}/images/sgarzare.png (100%)
rename {updater/virt-tools => public}/images/teuf.png (100%)
rename {updater/virt-tools => public}/images/thomashuth.png (100%)
rename {updater/virt-tools => public}/images/ybettan.png (100%)
rename {updater/virt-tools => public}/images/zeenix.png (100%)
delete mode 100755 updater/app.py
rename {updater/virt-tools => virt-tools}/atom.xml.tmpl (100%)
rename {updater/virt-tools => virt-tools}/basic/index.html.tmpl (91%)
rename {updater/virt-tools => virt-tools}/basic/style.css.tmpl (100%)
rename {updater/virt-tools => virt-tools}/config.ini (96%)
rename {updater/virt-tools => virt-tools}/foafroll.xml.tmpl (100%)
rename {updater/virt-tools => virt-tools}/opml.xml.tmpl (100%)
rename {updater/virt-tools => virt-tools}/rss10.xml.tmpl (100%)
rename {updater/virt-tools => virt-tools}/rss20.xml.tmpl (100%)
delete mode 100644 web/httpd-cfg/cors.conf
--
2.24.1
4 years, 6 months
[libvirt PATCH 00/10] docs: Convert hacking to rst, split and rework it
by Andrea Bolognani
hacking.html is the canonical entry point for people looking to help
with libvirt's development, but its sheer size makes it a farily
ineffective tool.
Move most non-critical information off to separate pages and brush
up what's left so that it works as a quick introduction.
As a prerequisite, convert the existing page to reStructuredText.
I have compared the HTML version and the conversion visually and
didn't spot any massive differences, save for a few spacing changes
that are not specific to the page itself but rather to the HTML
generated by docutils and which will require CSS tweaks to address,
which is a job I'm leaving to someone else.
I have also spent very little time actually changing the contents,
and mostly just moved them around. There's plenty of opportunities
for improvement there, but once again that's someone else's job.
Andrea Bolognani (10):
docs: Convert hacking.html to reStructuredText
docs: Add glib-adoption.rst
docs: Add coding-style.rst
docs: Add programming-languages.rst
docs: Add developer-tooling.rst
docs: Add committer-guidelines.rst
docs: Add advanced-tests.rst
docs: Add submitting-patches.rst
docs: Add best-practices.rst
docs: Update hacking.rst
build-aux/syntax-check.mk | 4 +-
docs/advanced-tests.rst | 178 ++++
docs/best-practices.rst | 37 +
docs/coding-style.rst | 919 +++++++++++++++++++
docs/committer-guidelines.rst | 33 +
docs/developer-tooling.rst | 13 +
docs/glib-adoption.rst | 96 ++
docs/hacking.html.in | 1555 --------------------------------
docs/hacking.rst | 79 ++
docs/programming-languages.rst | 21 +
docs/submitting-patches.rst | 88 ++
11 files changed, 1466 insertions(+), 1557 deletions(-)
create mode 100644 docs/advanced-tests.rst
create mode 100644 docs/best-practices.rst
create mode 100644 docs/coding-style.rst
create mode 100644 docs/committer-guidelines.rst
create mode 100644 docs/developer-tooling.rst
create mode 100644 docs/glib-adoption.rst
delete mode 100644 docs/hacking.html.in
create mode 100644 docs/hacking.rst
create mode 100644 docs/programming-languages.rst
create mode 100644 docs/submitting-patches.rst
--
2.25.1
4 years, 6 months
ALERT: cimtest.git repository history recreated
by Daniel P. Berrangé
In attempting to import the cimtest.git repository to gitlab I discovered
some serious problems in the current git history.
Specifically the author and committer email addresses were badly mangled
for many commits.
For example an author of:
Deepti B. Kalakeri<deeptik(a)linux.vnet.ibm.com> <Deepti B. Kalakeri<deeptik(a)linux.vnet.ibm.com>>
This is so badly malformed that both gitlab and github refuse to allow
the commits in question to be pushed.
Not only that, but "git filter-branch" and "git fast-import" will both
crash and burn when trying to parse these, so if you try to rewrite
email addresses with "git filter-branch" it makes it even worse.
The only solution was to use "git fast-export --all" to dump the entire
history to a file, then hand edit this to fix the broken addresses that
were confusing filter-branch, then use "git fast-import" to initialize
a new repository.
cimtest.git is a ancient repo that hasn't seen changes since 2014, so
I'm guessing it is unlikely any current contributor has a checkout of
it. If someone does though, you'll have to throw away your current
checkout and get a new clone, as the history has been fully rewritten
to fix this problem.
I suspect the root cause dates from when we imported the original
Mercurial repo into Git. Our conversion process probably mangled
some addresses without us realizing at the time.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
4 years, 6 months
[libvirt-go-xml PATCH 0/2] Switch CI from Travis to GitLab
by Daniel P. Berrangé
Daniel P. Berrangé (2):
gitlab: add CI definition for GitLab
travis: delete CI configuration
.gitlab-ci.yml | 25 +++++++++++++++++++++++++
.travis.yml | 13 -------------
README.md | 2 +-
3 files changed, 26 insertions(+), 14 deletions(-)
create mode 100644 .gitlab-ci.yml
delete mode 100644 .travis.yml
--
2.24.1
4 years, 6 months
[virttools-web PATCH 0/3] Switch www.virt-tools.org over to use GitLab Pages
by Daniel P. Berrangé
This introduces use of GitLab CI + Pages to replace the current
OpenShift application which is only admin accessible by myself.
It also has automatic integration with LetsEncrypt guaranteeing
that we'll never have expired certificates.
Daniel P. Berrangé (3):
Convert README to markdown format
Introduce use of GitLab CI for publishing to GitLab Pages
Remove obsolete openshift hosting configuration
.gitlab-ci.yml | 9 +
README | 19 --
README.md | 15 ++
openshift/templates/.gitignore | 2 -
openshift/templates/update-tls.sh | 16 --
openshift/templates/virttools-web-tls.json | 82 -------
openshift/templates/virttools-web.json | 272 ---------------------
{web => public}/header-bg.png | Bin
{web => public}/index.css | 0
{web => public}/index.html | 4 +-
{web => public}/logo-kvm.png | Bin
{web => public}/logo-libguestfs.png | Bin
{web => public}/logo-libosinfo.png | Bin
{web => public}/logo-libvirt.png | Bin
{web => public}/logo-qemu.png | Bin
{web => public}/logo-virt-manager.png | Bin
{web => public}/logo.png | Bin
{web => public}/logo.xcf | Bin
18 files changed, 25 insertions(+), 394 deletions(-)
create mode 100644 .gitlab-ci.yml
delete mode 100644 README
create mode 100644 README.md
delete mode 100644 openshift/templates/.gitignore
delete mode 100755 openshift/templates/update-tls.sh
delete mode 100644 openshift/templates/virttools-web-tls.json
delete mode 100644 openshift/templates/virttools-web.json
rename {web => public}/header-bg.png (100%)
rename {web => public}/index.css (100%)
rename {web => public}/index.html (93%)
rename {web => public}/logo-kvm.png (100%)
rename {web => public}/logo-libguestfs.png (100%)
rename {web => public}/logo-libosinfo.png (100%)
rename {web => public}/logo-libvirt.png (100%)
rename {web => public}/logo-qemu.png (100%)
rename {web => public}/logo-virt-manager.png (100%)
rename {web => public}/logo.png (100%)
rename {web => public}/logo.xcf (100%)
--
2.24.1
4 years, 6 months
[libvirt PATCH 00/15] RFC: basic CGroup support with qemu:///session
by marcandre.lureau＠redhat.com
From: Marc-André Lureau <marcandre.lureau(a)redhat.com>
Hi,
This is a small series that allows basic QEMU VM CGroup support with
the help of machined --user:
https://github.com/systemd/systemd/pull/15312
The first few patches are fixes to register dbus and slirp-helper
correctly with the VM cgroup.
A few changes are done to the machined support, adding session
support, and registering the VM to get a systemd scope cgroup under
user machine.slice.
Marc-André Lureau (15):
slirp: leave the dbus daemon running on error
slirp: add helper to VM cgroup
qemu-dbus: prevent double start of the bus daemon
qemu-dbus: remove unused variable
qemu-dbus: register DBus bus to the VM cgroup
qemu-cgroup: remove unnecessary include
dbus: rename virDBusIs* -> virDBusSystemIs*
dbus: add virDBusIsService{Enabled,Registered} with @conn argument
systemd: check org.freedesktop.machine1 registration
systemd: only check for activitable machine1 service
systemd: register machines against session service
cgroup: return directly if there is nothing to remove
qemu-cgroup: register ext devices when cpu/cpuset controller are
missing
qemu: create cgroup regardless of controller support
qemu-cgroup: drop the need for privileges to use cgroup
src/qemu/qemu_cgroup.c | 13 -------
src/qemu/qemu_dbus.c | 29 +++++++++++++-
src/qemu/qemu_dbus.h | 3 ++
src/qemu/qemu_extdevice.c | 12 ++++++
src/qemu/qemu_process.c | 6 +--
src/qemu/qemu_slirp.c | 16 +++++++-
src/qemu/qemu_slirp.h | 3 ++
src/util/vircgroup.c | 5 +++
src/util/vircgroupv2devices.c | 7 ++--
src/util/virdbus.c | 71 ++++++++++++++++++++++++++---------
src/util/virdbus.h | 7 +++-
src/util/virfirewalld.c | 2 +-
src/util/virsystemd.c | 27 +++++--------
tests/virsystemdtest.c | 35 -----------------
14 files changed, 140 insertions(+), 96 deletions(-)
--
2.26.0.rc2.42.g98cedd0233
4 years, 6 months
