[libvirt] [PATCH v2 00/15] docs: some refactoring and new docs about RPM deployment
by Daniel P. Berrangé
This refactors existing docs related to the remote driver/daemon and
URIs. It then also adds a kbase page about RPM package options.
This introduces the use of RST for docs as a replacement for HTML.
The intent is that all new docs should use RST from this point.
The POD man pages will also be converted to RST. I'll post this as
a different series.
Note this series only touches the main HTML content, not the
styling / templating using XSL. Tackling that is a separate
job I've not attempted. My guess is that we'd use either Sphinx
or Pelican to do the templating.
Existing HTML docs are candidates for conversion to RST, however,
people should *NOT* attempt todo this manually.
In this series I've converted docs/kbase/ files, using the pandoc
tool. This does a pretty good job in general but does need some
manual cleanups. First I post-processed its output to change the
heading highlighting to follow the documented style
$ cat convert.pl
my @in = <>;
my @out;
for (my $i ; $i <= $#in; $i++) {
my $line = $in[$i];
if ($line =~ /^=+$/) {
my @newout = splice @out, 0, $i - 1;
push @newout, $line;
push @newout, $out[$i-1];
push @newout, $out[$i];
push @newout, $line;
push @newout, "\n";
push @newout, ".. contents::\n";
@out = @newout;
} elsif ($line =~ /^(-|~|^|')+$/) {
$line =~ s/-/=/g;
$line =~ s/~/-/g;
$line =~ s/\^/~/g;
$line =~ s/'/^/g;
push @out, $line;
} else {
push @out, $line;
}
}
print @out;
So I'm converting with
pandoc -f html -t rst < foo.html.in | perl convert.pl > foo.rst
$EDITOR foot.rst
git rm -f foot.html.in
git add foo.rst
After pandoc & the headings convert, there's usually manual fixes
needed to deal with a few oddities pandoc gets wrong, or looks
ugly. This is still way simpler than converting the doc to rst
manually.
Changed in v2:
- Added a style guide for RST docs
- Probe for rst2html's different names
- Added make rules for building rst
- Fixed permalinks in generated docs
- Misc CSS fixes
- Auto-converted all of kbase/ directory
Daniel P. Berrangé (15):
docs: split TLS certificate setup into its own file
docs: move docs about remote driver URIs into URI docs
docs: introduce rst2html as a mandatory tool for building docs
docs: adapt filling of <head> section for rst2html output
docs: generate permalinks correctly for rst2html output
docs: relax CSS context match for pretty tables
docs: add styling for <tt> element
docs: add a minimal style guide for writing RST docs
docs: convert kbase/domainstatecapture.html.in to RST
docs: convert kbase/launch_security_sev.html.in to RST
docs: convert kbase/secureusage.html.in to RST
docs: convert kbase/locking.html.in to RST
docs: convert kbase/locking-lockd.html.in to RST
docs: convert kbase/locking-sanlock.html.in to RST
docs: add a kbase page about RPM packaging options
docs/Makefile.am | 33 +-
docs/aclpolkit.html.in | 20 +-
docs/docs.html.in | 6 +
docs/genaclperms.pl | 2 +-
docs/generic.css | 4 +-
docs/kbase.html.in | 4 +
docs/kbase/domainstatecapture.html.in | 303 -----------
docs/kbase/domainstatecapture.rst | 255 +++++++++
docs/kbase/launch_security_sev.html.in | 533 -------------------
docs/kbase/launch_security_sev.rst | 529 +++++++++++++++++++
docs/kbase/locking-lockd.html.in | 160 ------
docs/kbase/locking-lockd.rst | 121 +++++
docs/kbase/locking-sanlock.html.in | 247 ---------
docs/kbase/locking-sanlock.rst | 193 +++++++
docs/kbase/locking.html.in | 48 --
docs/kbase/locking.rst | 33 ++
docs/kbase/rpm-deployment.rst | 410 +++++++++++++++
docs/kbase/secureusage.html.in | 171 -------
docs/kbase/secureusage.rst | 131 +++++
docs/libvirt.css | 76 +--
docs/migration.html.in | 2 +-
docs/newapi.xsl | 4 +-
docs/page.xsl | 9 +-
docs/remote.html.in | 684 +------------------------
docs/styleguide.rst | 66 +++
docs/tlscerts.html.in | 413 +++++++++++++++
docs/uri.html.in | 263 ++++++++--
libvirt.spec.in | 2 +
m4/virt-external-programs.m4 | 5 +
mingw-libvirt.spec.in | 1 +
30 files changed, 2478 insertions(+), 2250 deletions(-)
delete mode 100644 docs/kbase/domainstatecapture.html.in
create mode 100644 docs/kbase/domainstatecapture.rst
delete mode 100644 docs/kbase/launch_security_sev.html.in
create mode 100644 docs/kbase/launch_security_sev.rst
delete mode 100644 docs/kbase/locking-lockd.html.in
create mode 100644 docs/kbase/locking-lockd.rst
delete mode 100644 docs/kbase/locking-sanlock.html.in
create mode 100644 docs/kbase/locking-sanlock.rst
delete mode 100644 docs/kbase/locking.html.in
create mode 100644 docs/kbase/locking.rst
create mode 100644 docs/kbase/rpm-deployment.rst
delete mode 100644 docs/kbase/secureusage.html.in
create mode 100644 docs/kbase/secureusage.rst
create mode 100644 docs/styleguide.rst
create mode 100644 docs/tlscerts.html.in
--
2.23.0
5 years, 4 months
[libvirt] [PATCH RFC V2] guests: Add support for openSUSE
by Jim Fehlig
This change adds support for the latest openSUSE Leap and
openSUSE Tumbleweed guests.
Signed-off-by: Jim Fehlig <jfehlig(a)suse.com>
---
This version of the patch overcomes the 'lci build' issues noted in V1
and works well for openSUSE Leap 15.1 and Tumbleweed. I'm still of the
opinion of only supporting the latest Leap since there is only a six
month support overlap between versions. Version x-1 is only supported
six months after x is released.
'lci update ... libvirt' fails in task "Configure hostname":
fatal: [libvirt-opensuse-tumbleweed]: FAILED! => {"changed": false, "msg": "hostname module cannot be used on platform Linux (Opensuse-tumbleweed)"}
Not sure why that is the case. But more importantly, even though it
doesn't fail, the "Update installed packages" task doesn't install any
of the packages required to build libvirt. I suppose that agrees with
guests/REAMDME, but how then are the packages required to build $project
installed? How/when is the list of packages required to build $project
conveyed to $package_manager?
'lcitool build libvirt-opensuse-tumbleweed libvirt' fails in the
"Gathering Facts" task:
fatal: [libvirt-opensuse-tumbleweed]: UNREACHABLE! => {"changed": false, "msg": "Invalid/incorrect password: ", "unreachable": true}
Passing --debug to the command didn't help with determining what needed
the password. ssh? I can't find where the "Gathering Facts" task is
defined. The $root_password_file passed to /usr/bin/ansible-playbook
contains the correct password for the root user. /me needs to figure
out how to better debug this stuff :-).
guests/configs/autoinst.xml | 75 +++++++++++++++++++
.../host_vars/libvirt-opensuse-15/docker.yml | 2 +
.../host_vars/libvirt-opensuse-15/install.yml | 2 +
guests/host_vars/libvirt-opensuse-15/main.yml | 22 ++++++
.../libvirt-opensuse-tumbleweed/docker.yml | 2 +
.../libvirt-opensuse-tumbleweed/install.yml | 2 +
.../libvirt-opensuse-tumbleweed/main.yml | 22 ++++++
guests/inventory | 2 +
guests/lcitool | 2 +
guests/playbooks/update/tasks/base.yml | 15 ++++
guests/vars/mappings.yml | 41 +++++++++-
11 files changed, 185 insertions(+), 2 deletions(-)
diff --git a/guests/configs/autoinst.xml b/guests/configs/autoinst.xml
new file mode 100644
index 0000000..42c42f8
--- /dev/null
+++ b/guests/configs/autoinst.xml
@@ -0,0 +1,75 @@
+<?xml version="1.0"?>
+<!DOCTYPE profile>
+<profile
+ xmlns="http://www.suse.com/1.0/yast2ns"
+ xmlns:config="http://www.suse.com/1.0/configns">
+ <general>
+ <mode>
+ <confirm config:type="boolean">false</confirm>
+ </mode>
+ </general>
+ <partitioning config:type="list">
+ <drive>
+ <device>/dev/vda</device>
+ <use>all</use>
+ <partitions config:type="list">
+ <partition>
+ <filesystem config:type="symbol">swap</filesystem>
+ <size>256M</size>
+ <mount>swap</mount>
+ </partition>
+ <partition>
+ <filesystem config:type="symbol">ext4</filesystem>
+ <mount>/</mount>
+ <size>max</size>
+ </partition>
+ </partitions>
+ </drive>
+ </partitioning>
+ <bootloader>
+ <global>
+ <terminal>console serial</terminal>
+ </global>
+ </bootloader>
+ <timezone>
+ <hwclock>UTC</hwclock>
+ <timezone>UTC</timezone>
+ </timezone>
+ <software>
+ <install_recommended config:type="boolean">false</install_recommended>
+ <products config:type="list">
+ <product>openSUSE</product>
+ </products>
+ <patterns config:type="list">
+ <pattern>base</pattern>
+ <pattern>minimal_base</pattern>
+ <pattern>yast2_basis</pattern>
+ </patterns>
+ <packages config:type="list">
+ <package>openssh</package>
+ </packages>
+ </software>
+ <networking>
+ <keep_install_network config:type="boolean">true</keep_install_network>
+ </networking>
+ <users config:type="list">
+ <user>
+ <username>root</username>
+ <user_password>root</user_password>
+ <encrypted config:type="boolean">false</encrypted>
+ <uid>0</uid>
+ <gid>0</gid>
+ <home>/root</home>
+ <shell>/bin/bash</shell>
+ </user>
+ </users>
+ <services-manager>
+ <default_target>multi-user</default_target>
+ <services config:type="list">
+ <service>
+ <service_name>sshd</service_name>
+ <service_status>enable</service_status>
+ </service>
+ </services>
+ </services-manager>
+</profile>
diff --git a/guests/host_vars/libvirt-opensuse-15/docker.yml b/guests/host_vars/libvirt-opensuse-15/docker.yml
new file mode 100644
index 0000000..8f32d06
--- /dev/null
+++ b/guests/host_vars/libvirt-opensuse-15/docker.yml
@@ -0,0 +1,2 @@
+---
+docker_base: opensuse/leap:15
diff --git a/guests/host_vars/libvirt-opensuse-15/install.yml b/guests/host_vars/libvirt-opensuse-15/install.yml
new file mode 100644
index 0000000..d0fdbe5
--- /dev/null
+++ b/guests/host_vars/libvirt-opensuse-15/install.yml
@@ -0,0 +1,2 @@
+---
+install_url: http://download.opensuse.org/distribution/leap/15.1/repo/oss/
diff --git a/guests/host_vars/libvirt-opensuse-15/main.yml b/guests/host_vars/libvirt-opensuse-15/main.yml
new file mode 100644
index 0000000..abd83c5
--- /dev/null
+++ b/guests/host_vars/libvirt-opensuse-15/main.yml
@@ -0,0 +1,22 @@
+---
+projects:
+ - libosinfo
+ - libvirt
+ - libvirt-cim
+ - libvirt-dbus
+ - libvirt-glib
+ - libvirt-perl
+ - libvirt-python
+ - libvirt-sandbox
+ - libvirt-tck
+ - osinfo-db
+ - osinfo-db-tools
+ - virt-manager
+ - virt-viewer
+
+package_format: 'rpm'
+package_manager: 'zypper'
+os_name: 'OpenSUSE'
+os_version: '15'
+
+ansible_python_interpreter: /usr/bin/python3
diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml
new file mode 100644
index 0000000..cc79a22
--- /dev/null
+++ b/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml
@@ -0,0 +1,2 @@
+---
+docker_base: opensuse/tumbleweed
diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml
new file mode 100644
index 0000000..36cea68
--- /dev/null
+++ b/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml
@@ -0,0 +1,2 @@
+---
+install_url: http://download.opensuse.org/tumbleweed/repo/oss/
diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml
new file mode 100644
index 0000000..828df81
--- /dev/null
+++ b/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml
@@ -0,0 +1,22 @@
+---
+projects:
+ - libosinfo
+ - libvirt
+ - libvirt-cim
+ - libvirt-dbus
+ - libvirt-glib
+ - libvirt-perl
+ - libvirt-python
+ - libvirt-sandbox
+ - libvirt-tck
+ - osinfo-db
+ - osinfo-db-tools
+ - virt-manager
+ - virt-viewer
+
+package_format: 'rpm'
+package_manager: 'zypper'
+os_name: 'OpenSUSE'
+os_version: 'Tumbleweed'
+
+ansible_python_interpreter: /usr/bin/python3
diff --git a/guests/inventory b/guests/inventory
index 3b15513..71d3c91 100644
--- a/guests/inventory
+++ b/guests/inventory
@@ -8,5 +8,7 @@ libvirt-fedora-rawhide
libvirt-freebsd-11
libvirt-freebsd-12
libvirt-freebsd-current
+libvirt-opensuse-15
+libvirt-opensuse-tumbleweed
libvirt-ubuntu-16
libvirt-ubuntu-18
diff --git a/guests/lcitool b/guests/lcitool
index a630971..7f26731 100755
--- a/guests/lcitool
+++ b/guests/lcitool
@@ -531,6 +531,8 @@ class Application:
install_config = "preseed.cfg"
elif facts["os_name"] in ["CentOS", "Fedora"]:
install_config = "kickstart.cfg"
+ elif facts["os_name"] == "OpenSUSE":
+ install_config = "autoinst.xml"
else:
raise Exception(
"Host {} doesn't support installation".format(host)
diff --git a/guests/playbooks/update/tasks/base.yml b/guests/playbooks/update/tasks/base.yml
index 3d83e78..b6a2cd2 100644
--- a/guests/playbooks/update/tasks/base.yml
+++ b/guests/playbooks/update/tasks/base.yml
@@ -65,12 +65,27 @@
when:
- package_format == 'pkg'
+- name: Update installed packages
+ command: '{{ package_manager }} update -y -l --force-resolution --no-recommends'
+ args:
+ warn: no
+ when:
+ - os_name == 'OpenSUSE'
+
- name: Clean up packages after update
shell: '{{ package_manager }} clean packages -y && {{ package_manager }} autoremove -y'
args:
warn: no
when:
- package_format == 'rpm'
+ - not os_name == "OpenSUSE"
+
+- name: Clean up packages after update
+ shell: '{{ package_manager }} clean'
+ args:
+ warn: no
+ when:
+ - os_name == "OpenSUSE"
- name: Clean up packages after update
apt:
diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml
index bd9b161..8359ce2 100644
--- a/guests/vars/mappings.yml
+++ b/guests/vars/mappings.yml
@@ -19,10 +19,10 @@
# - deb, pkg, rpm
#
# Valid OS names are:
-# - CentOS, Debian, Fedora, FreeBSD, Ubuntu
+# - CentOS, Debian, Fedora, FreeBSD, OpenSUSE, Ubuntu
#
# Valid OS versions are:
-# - CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on
+# - CentOS7, Debian9, FedoraRawhide, OpenSUSE15, Ubuntu18 and so on
#
# The arch specific rules use a prefix "$ARCH-" where $ARCH
# is a libvirt arch name.
@@ -70,6 +70,7 @@ mappings:
apparmor:
deb: libapparmor-dev
+ OpenSUSE: libapparmor-devel
cross-policy-deb: foreign
augeas:
@@ -117,6 +118,7 @@ mappings:
cppi:
Fedora: cppi
FreeBSD: cppi
+ OpenSUSE: cppi
cyrus-sasl:
deb: libsasl2-dev
@@ -127,6 +129,7 @@ mappings:
dbus-daemon:
default: dbus
Fedora: dbus-daemon
+ OpenSUSE: dbus-1
device-mapper:
deb: libdevmapper-dev
@@ -196,6 +199,7 @@ mappings:
deb: libglusterfs-dev
rpm: glusterfs-api-devel
Debian9: glusterfs-common
+ OpenSUSE: glusterfs-devel
Ubuntu16: glusterfs-common
Ubuntu18: glusterfs-common
cross-policy-deb: foreign
@@ -204,6 +208,7 @@ mappings:
deb: libgnutls28-dev
pkg: gnutls
rpm: gnutls-devel
+ OpenSUSE: libgnutls-devel
cross-policy-deb: foreign
go:
@@ -228,6 +233,7 @@ mappings:
gtk-update-icon-cache:
default: gtk-update-icon-cache
+ OpenSUSE: gtk3-tools
Ubuntu16: libgtk2.0-bin
gtk-vnc2:
@@ -248,14 +254,17 @@ mappings:
ip:
deb: iproute2
rpm: iproute
+ OpenSUSE: iproute2
iscsiadm:
deb: open-iscsi
rpm: iscsi-initiator-utils
+ OpenSUSE: open-iscsi
isoinfo:
default: genisoimage
FreeBSD: cdrkit
+ OpenSUSE: mkisofs
java:
deb: openjdk-11-jre-headless
@@ -289,6 +298,7 @@ mappings:
libaudit:
deb: libaudit-dev
rpm: audit-libs-devel
+ OpenSUSE: audit-devel
cross-policy-deb: foreign
libblkid:
@@ -320,6 +330,7 @@ mappings:
deb: libdbus-1-dev
pkg: dbus
rpm: dbus-devel
+ OpenSUSE: dbus-1-devel
cross-policy-deb: foreign
libgovirt:
@@ -347,6 +358,7 @@ mappings:
rpm: numactl-devel
armv6l-deb:
armv7l-deb:
+ OpenSUSE: libnuma-devel
cross-policy-deb: foreign
libparted:
@@ -370,6 +382,7 @@ mappings:
deb: librbd-dev
Fedora: librbd-devel
CentOS7: librbd1-devel
+ OpenSUSE: librbd-devel
cross-policy-deb: foreign
libselinux:
@@ -436,6 +449,7 @@ mappings:
deb: locales
Fedora: glibc-langpack-en
FreeBSD:
+ OpenSUSE: glibc-locale
lsof:
default: lsof
@@ -458,6 +472,7 @@ mappings:
ninja:
default: ninja-build
FreeBSD: ninja
+ OpenSUSE: ninja
mingw32-curl:
Fedora: mingw32-curl
@@ -639,6 +654,7 @@ mappings:
netcf:
deb: libnetcf-dev
rpm: netcf-devel
+ OpenSUSE:
cross-policy-deb: skip
net-tools:
@@ -678,6 +694,7 @@ mappings:
deb: perl
pkg: perl5
rpm: perl-Archive-Tar
+ OpenSUSE: perl-Archive-Tar-Wrapper
perl-CPAN-Changes:
deb: libcpan-changes-perl
@@ -709,6 +726,7 @@ mappings:
deb: libio-compress-perl
pkg: p5-IO-Compress
rpm: perl-IO-Compress-Bzip2
+ OpenSUSE: perl-Compress-Bzip2
perl-IO-String:
deb: libio-string-perl
@@ -747,6 +765,7 @@ mappings:
deb: libtest-lwp-useragent-perl
pkg: p5-Test-LWP-UserAgent
Fedora: perl-Test-LWP-UserAgent
+ OpenSUSE: perl-Test-LWP-UserAgent
perl-Test-Pod:
deb: libtest-pod-perl
@@ -809,20 +828,24 @@ mappings:
deb: python-dev
pkg: python2
rpm: python2-devel
+ OpenSUSE: python-devel
cross-policy-deb: foreign
python2-lxml:
default: python-lxml
Fedora: python2-lxml
FreeBSD: py27-lxml
+ OpenSUSE: python2-lxml
python2-nose:
default: python-nose
Fedora: python2-nose
FreeBSD: py27-nose
+ OpenSUSE: python2-nose
python2-setuptools:
CentOS7: python2-setuptools
+ OpenSUSE: python2-setuptools
python3:
default: python3
@@ -831,6 +854,7 @@ mappings:
default: python3-dbus
FreeBSD: py36-dbus
CentOS7: python36-dbus
+ OpenSUSE: python3-dbus-python
python3-devel:
deb: python3-dev
@@ -838,17 +862,20 @@ mappings:
Fedora: python3-devel
CentOS7: python36-devel
cross-policy-deb: foreign
+ OpenSUSE: python3-devel
python3-gi:
deb: python3-gi
pkg: py36-gobject3
rpm: python3-gobject
CentOS7: python36-gobject
+ OpenSUSE: python3-gobject
python3-libxml2:
default: python3-libxml2
FreeBSD: py36-libxml2
CentOS7:
+ OpenSUSE: python3-libxml2-python
Ubuntu16:
python3-lxml:
@@ -864,6 +891,7 @@ mappings:
python3-pip:
CentOS7: python3-pip
Debian9: python3-pip
+ OpenSUSE: python3-pip
Ubuntu16: python3-pip
Ubuntu18: python3-pip
@@ -884,6 +912,7 @@ mappings:
qemu-img:
default: qemu-utils
rpm: qemu-img
+ OpenSUSE: qemu-tools
radvd:
default: radvd
@@ -929,6 +958,7 @@ mappings:
deb: libspice-client-gtk-3.0-dev
pkg: spice-gtk
rpm: spice-gtk3-devel
+ OpenSUSE: spice-gtk-devel
cross-policy-deb: foreign
strace:
@@ -942,6 +972,7 @@ mappings:
deb: iproute2
rpm: iproute-tc
CentOS7: iproute
+ OpenSUSE: iproute2
unzip:
default: unzip
@@ -964,10 +995,12 @@ mappings:
wireshark:
deb: wireshark-dev
Fedora: wireshark-devel
+ OpenSUSE: wireshark-devel
cross-policy-deb: skip
xen:
Fedora: xen-devel
+ OpenSUSE: xen-devel
x86_64-deb: libxen-dev
armv7l-deb: libxen-dev
aarch64-deb: libxen-dev
@@ -994,18 +1027,21 @@ mappings:
xz-static:
deb: liblzma-dev
Fedora: xz-static
+ OpenSUSE: xz-static-devel
cross-policy-deb: foreign
yajl:
deb: libyajl-dev
pkg: yajl
rpm: yajl-devel
+ OpenSUSE: libyajl-devel
cross-policy-deb: foreign
zfs:
default: zfs-fuse
CentOS:
FreeBSD:
+ OpenSUSE:
zlib:
deb: zlib1g-dev
@@ -1015,6 +1051,7 @@ mappings:
zlib-static:
deb: zlib1g-dev
rpm: zlib-static
+ OpenSUSE: zlib-devel-static
cross-policy-deb: foreign
--
2.24.0
5 years, 4 months
[libvirt] [PATCH v3 0/4] qemu: block: implement optional removal of committed snapshot images
by Pavel Mores
v3 aims to incorporate Peter's feedback to v2, mainly:
- patch 1 was split into two - the delete flag propagation stays in patch 1,
the actual enabling the feature was moved to what's now patch 4
- patch 2 was modified to handle NFS and to use g_strerror() instead of
strerror_r()
- the former patch 4 was squashed into patch 3.
Pavel Mores (4):
qemu: block: propagate the delete flag to where it can actually be
used
qemu: block: use the delete flag to delete snapshot images if
requested
qemu: block: store the delete flag in libvirtd's status XML
qemu: block: enable the snapshot image deletion feature
src/qemu/qemu_blockjob.c | 43 ++++++++++++++++++-
src/qemu/qemu_blockjob.h | 4 +-
src/qemu/qemu_domain.c | 6 ++-
src/qemu/qemu_domain.h | 6 +++
src/qemu/qemu_driver.c | 5 ++-
.../blockjob-blockdev-in.xml | 1 +
6 files changed, 60 insertions(+), 5 deletions(-)
--
2.21.0
5 years, 4 months
[libvirt] [PATCH 00/21] Incremental backup support for qemu
by Peter Krempa
This is a finished version of my RFC posted here:
https://www.redhat.com/archives/libvir-list/2019-October/msg01193.html
All cleanups of that series are now merged and this series has new
cleanups and depending patches that I posted earlier this week. For
convenience everything can be fetched at:
git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-backup
This series requires qemu 4.2 to work properly. Please make sure to
fetch the current head.
Compared to previous versions:
- The API is similar to what was posted last time or in Eric's version
- the backup job ID was removed, this can't be integrated with qemu
async jobs as there is only one async jobs.
- the virDomainBackupEnd API was removed as virDomainAbortJob can be
used instead
- the usage of domain job APIs is fully implemented as it was documented
in previous eric's postings
- the domain job completion event and completed job statistics are used
to notify about the state and successful finishing of the job
(see virDomainGetJobStats/virsh domjobinfo)
- unfortunately block jobs and snapshots are still not supported, thus
this feature will stay disabled for now:
If you want to give the feature a spin you must use the following XML
override:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
...
<qemu:capabilities>
<qemu:add capability='incremental-backup'/>
</qemu:capabilities>
</domain>
Eric Blake (5):
backup: Document new XML for backups
backup: Introduce virDomainBackup APIs
backup: Implement backup APIs for remote driver
backup: Parse and output backup XML
backup: Implement virsh support for backup
Peter Krempa (16):
API: Introduce field for reporting temporary disk space usage of a
domain job
virsh: Implement VIR_DOMAIN_JOB_DISK_TEMP_(USED|TOTAL) in
cmdDomjobinfo
API: Add domain job operation for backups
qemu: Add infrastructure for statistics of a backup job
qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type
Add 'backup' block job type
qemu: monitor: Add support for blockdev-backup via 'transaction'
qemu: domain: Track backup job data in the status XML
qemu: blockjob: Track internal data for 'backup' blockjob
tests: qemustatusxml2xml: Add test for 'pull' type backup job
conf: backup: Add fields for tracking stats of completed sub-jobs
doc: Document quirk of getting block job info for a 'backup' blockjob
qemu: Implement backup job APIs and qemu handling
qemu: backup: Implement stats gathering while the job is running
qemu: driver: Allow cancellation of the backup job
qemu: blockjob: Implement concluded blockjob handler for backup
blockjobs
docs/docs.html.in | 3 +-
docs/format.html.in | 1 +
docs/formatbackup.html.in | 161 +++
docs/formatcheckpoint.html.in | 12 +-
docs/index.html.in | 3 +-
docs/schemas/domainbackup.rng | 214 ++++
examples/c/misc/event-test.c | 3 +
include/libvirt/libvirt-domain.h | 32 +-
libvirt.spec.in | 1 +
mingw-libvirt.spec.in | 2 +
po/POTFILES.in | 3 +
src/conf/Makefile.inc.am | 2 +
src/conf/backup_conf.c | 499 ++++++++
src/conf/backup_conf.h | 108 ++
src/conf/domain_conf.c | 2 +-
src/conf/virconftypes.h | 3 +
src/driver-hypervisor.h | 12 +
src/libvirt-domain-checkpoint.c | 7 +-
src/libvirt-domain.c | 142 +++
src/libvirt_private.syms | 8 +
src/libvirt_public.syms | 6 +
src/qemu/Makefile.inc.am | 2 +
src/qemu/qemu_backup.c | 1017 +++++++++++++++++
src/qemu/qemu_backup.h | 46 +
src/qemu/qemu_blockjob.c | 96 +-
src/qemu/qemu_blockjob.h | 17 +
src/qemu/qemu_domain.c | 140 +++
src/qemu/qemu_domain.h | 14 +
src/qemu/qemu_driver.c | 68 +-
src/qemu/qemu_migration.c | 2 +
src/qemu/qemu_monitor.c | 13 +
src/qemu/qemu_monitor.h | 15 +
src/qemu/qemu_monitor_json.c | 33 +
src/qemu/qemu_monitor_json.h | 8 +
src/qemu/qemu_process.c | 25 +
src/remote/remote_driver.c | 2 +
src/remote/remote_protocol.x | 33 +-
src/remote_protocol-structs | 15 +
tests/Makefile.am | 3 +
tests/domainbackupxml2xmlin/backup-pull.xml | 9 +
tests/domainbackupxml2xmlin/backup-push.xml | 9 +
tests/domainbackupxml2xmlin/empty.xml | 1 +
tests/domainbackupxml2xmlout/backup-pull.xml | 9 +
tests/domainbackupxml2xmlout/backup-push.xml | 9 +
tests/domainbackupxml2xmlout/empty.xml | 7 +
tests/qemumonitorjsontest.c | 8 +-
.../qemustatusxml2xmldata/backup-pull-in.xml | 607 ++++++++++
.../qemustatusxml2xmldata/backup-pull-out.xml | 1 +
tests/qemuxml2xmltest.c | 2 +
tests/virschematest.c | 2 +
tools/Makefile.am | 1 +
tools/virsh-backup.c | 144 +++
tools/virsh-backup.h | 21 +
tools/virsh-domain.c | 26 +-
tools/virsh.c | 2 +
tools/virsh.h | 1 +
tools/virsh.pod | 32 +
57 files changed, 3639 insertions(+), 25 deletions(-)
create mode 100644 docs/formatbackup.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 src/conf/backup_conf.c
create mode 100644 src/conf/backup_conf.h
create mode 100644 src/qemu/qemu_backup.c
create mode 100644 src/qemu/qemu_backup.h
create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlin/empty.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlout/empty.xml
create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml
create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml
create mode 100644 tools/virsh-backup.c
create mode 100644 tools/virsh-backup.h
--
2.23.0
5 years, 4 months
[libvirt] CLZERO support for AMD
by Ani Sinha
Hi :
I see a commit in qemu that adds support for CLZERO but in CPUIDs. :
commit e900135dcfb676a4019dc206395e02db5d1de0d6
Author: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de>
Date: Wed Sep 25 23:49:48 2019 +0200
i386: Add CPUID bit for CLZERO and XSAVEERPTR
The CPUID bits CLZERO and XSAVEERPTR are availble on AMD's ZEN platform
and could be passed to the guest.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de>
Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com>
Is there a corresponding change in libvirt adding this capability? Libvirt complains when I add it manually in the vm xml:
# virsh create newvm.xml
error: Failed to create domain from newvm.xml
error: unsupported configuration: unknown CPU feature: clzero
# grep clzero newvm.xml
<feature policy='require' name='clzero'/>
Should we add it?
thanks
ani
5 years, 4 months
[libvirt] [PATCH 0/9] qemu: clean up capability handling during startup (incremental backup prequels)
by Peter Krempa
For libvirt to use incremental backup we will require that blockdev is
enabled. Clean up how we handle caps during startup since blockdev may
be masked out there and add infrastructure to interlock flags.
Peter Krempa (9):
qemu: process: Make it obvious that virDomainDefPostParse is called
with NULL opaque
qemu: process: Don't try to redetect missing qemuCaps on reconnect
qemu: Move and rename qemuDomainUpdateQEMUCaps
qemu: process: Move clearing of the BLOCKDEV capability to
qemuProcessPrepareQEMUCaps
qemu: process: Move clearing of QEMU_CAPS_CHARDEV_FD_PASS to
qemuProcessPrepareQEMUCaps
qemu: process: Move handling of qemu capability overrides
qemu: caps: Don't check capability before clearing it
qemu: capabilities: Lock out incremental backup capability without
blockdev
qemu: process: Re-process qemu capability lockout in
qemuProcessPrepareQEMUCaps
src/qemu/qemu_capabilities.c | 20 +++++++++-
src/qemu/qemu_capabilities.h | 2 +
src/qemu/qemu_domain.c | 26 ------------
src/qemu/qemu_domain.h | 4 --
src/qemu/qemu_process.c | 76 ++++++++++++++++++++++++------------
5 files changed, 72 insertions(+), 56 deletions(-)
--
2.23.0
5 years, 4 months
[libvirt] [PATCH 0/2] qemu: blockcopy: fix regression with block devices
by Peter Krempa
Peter Krempa (2):
qemu: blockcopy: Report error on image format detection failure
qemu: blockcopy: Fix conditions when virStorageSource should be
initialized
src/qemu/qemu_driver.c | 34 +++++++++++++++++++---------------
1 file changed, 19 insertions(+), 15 deletions(-)
--
2.23.0
5 years, 4 months
[libvirt] [PATCH] spec: Disable gluster on i686 on RHEL-8
by Jiri Denemark
Gluster 6.0 is not built on i686 for RHEL-8, which prevents libvirt from
building. Let's just disable gluster there as all we need are client
libraries anyway.
Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com>
---
libvirt.spec.in | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/libvirt.spec.in b/libvirt.spec.in
index a6219da604..c7fac104d9 100644
--- a/libvirt.spec.in
+++ b/libvirt.spec.in
@@ -65,7 +65,15 @@
%else
%define with_storage_sheepdog 0
%endif
+
%define with_storage_gluster 0%{!?_without_storage_gluster:1}
+%ifnarch %{qemu_kvm_arches}
+ # gluster is only built where qemu driver is enabled on RHEL 8
+ %if 0%{?rhel} >= 8
+ %define with_storage_gluster 0
+ %endif
+%endif
+
%define with_numactl 0%{!?_without_numactl:1}
# F25+ has zfs-fuse
--
2.24.0
5 years, 4 months
[libvirt] [PATCH] remote_daemon_stream: Hold an extra reference to stream in daemonStreamFilter
by Michal Privoznik
In v5.9.0-273-g8ecab214de I've tried to fix a lock ordering
problem, but introduced a crasher. Problem is that because the
client lock is unlocked (in order to honour lock ordering) the
stream we are currently checking in daemonStreamFilter() might be
freed and thus stream->priv might not even exist when the control
get to virMutexLock() call.
To resolve this, grab an extra reference to the stream and handle
its cleanup should the refcounter reach zero after the deref.
If that's the case and we are the only ones holding a reference
to the stream, we MUST return a positive value to make
virNetServerClientDispatchRead() break its loop where it iterates
over filters. The problem is, if we did not do so, then
"filter = filter->next" line will read from a memory that was
just freed (freeing a stream also unregisters its filter).
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
Reproducing this issue is very easy:
1) put sleep(5) right after virObjectUnlock(client) in the fist hunk,
2) virsh console --force $dom and type something so that the stream
has some data to process
3) while 2) is still running, run the same command from another terminal
4) observe libvirtd crash
src/remote/remote_daemon_stream.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c
index 82cadb67ac..73e4d7befb 100644
--- a/src/remote/remote_daemon_stream.c
+++ b/src/remote/remote_daemon_stream.c
@@ -293,10 +293,25 @@ daemonStreamFilter(virNetServerClientPtr client,
daemonClientStream *stream = opaque;
int ret = 0;
+ /* We must honour lock ordering here. Client private data lock must
+ * be acquired before client lock. Bu we are already called with
+ * client locked. To avoid stream disappearing while we unlock
+ * everything, let's increase its refcounter. This has some
+ * implications though. */
+ stream->refs++;
virObjectUnlock(client);
virMutexLock(&stream->priv->lock);
virObjectLock(client);
+ if (stream->refs == 1) {
+ /* So we are the only ones holding the reference to the stream.
+ * Return 1 to signal to the caller that we've processed the
+ * message. And to "process" means free. */
+ virNetMessageFree(msg);
+ ret = 1;
+ goto cleanup;
+ }
+
if (msg->header.type != VIR_NET_STREAM &&
msg->header.type != VIR_NET_STREAM_HOLE)
goto cleanup;
@@ -318,6 +333,10 @@ daemonStreamFilter(virNetServerClientPtr client,
cleanup:
virMutexUnlock(&stream->priv->lock);
+ /* Don't pass client here, because client is locked here and this
+ * function might try to lock it again which would result in a
+ * deadlock. */
+ daemonFreeClientStream(NULL, stream);
return ret;
}
--
2.23.0
5 years, 4 months
