[PATCH Libvirt 00/11] Support dirty page rate upper limit
by ~hyman
QEMU introduced the dirty page rate limit feature in 7.1.0, see the
details in the following link:
https://lore.kernel.org/qemu-
devel/cover.1656177590.git.huangy81(a)chinatelecom.cn/
So maybe it's the right time to enable this feature in libvirt and the
upper user can play with it, expecting the upper app can use this
feature to do a virtual CPU Qos or whatever else.
Introduce the virsh API as follows:
# virsh limit-dirty-page-rate <domain> [--rate <number>] [--vcpu
<number>] [--cancel]
Examples:
To set the dirty page rate upper limit 60MB/s for all virtual CPUs in
c81_node1, use:
# virsh limit-dirty-page-rate c81_node1 --rate 60
Set dirty page rate limit 60(MB/s) on all virtual CPUs successfully
To set the dirty page rate upper limit 35MB/s for virtual CPU 1 in
c81_node1, use:
# virsh limit-dirty-page-rate c81_node1 --rate 35 --vcpu 1
Set vcpu[1] dirty page rate upper limit 35(MB/s) successfully
Specify the 'cancel' option to do the reverse, the optional option
'vcpu' is used to specify the CPU index to be set.
To query the dirty page rate upper limit, use:
# virsh vcpuinfo c81_node1
VCPU: 0
CPU: 14
State: running
CPU time: 27.1s
CPU Affinity: yyyyyyyyyyyyyyyy
DirtyRate limit: 60
DirtyRate current: 0
VCPU: 1
CPU: 1
State: running
CPU time: 25.1s
CPU Affinity: yyyyyyyyyyyyyyyy
DirtyRate limit: 35
DirtyRate current: 0
VCPU: 2
CPU: 7
State: running
CPU time: 6.0s
CPU Affinity: yyyyyyyyyyyyyyyy
DirtyRate limit: 60
DirtyRate current: 0
VCPU: 3
CPU: 8
State: running
CPU time: 3.5s
CPU Affinity: yyyyyyyyyyyyyyyy
DirtyRate limit: 60
DirtyRate current: 0
The patch set adds two new APIs to implement a dirty page rate limit:
1. virDomainSetVcpuDirtyLimit, which set virtual CPU dirty page rate
limit. virsh command 'limit-dirty-page-rate' correspondingly.
2. virDomainCancelVcpuDirtyLimit, which cancel virtual CPU dirty page
rate
limit. 'cancel' option was introduced to 'limit-dirty-page-rate' to
cancel
the limit correspondingly.
In addition, function 'qemuMonitorQueryVcpuDirtyLimit' was implemented
to query the dirty page rate upper limit, the virsh command 'vcpuinfo'
was extended. So that the user can query dirty page rate limit info via
'vcpuinfo'.
This series makes the main modifications as follows:
- introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability so that libvirt
can probe before using dirty page rate upper limit feature.
- implement virsh command 'limit-dirty-page-rate' to set/cancel dirty
page rate upper limit.
- extend 'vcpuinfo' API so that it can display dirty page rate upper
limit.
- document dirty page rate limit feature.
Please review, and hoping the comments, thanks !
Yong
Hyman Huang(黄勇) (11):
qemu_capabilities: Introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability
libvirt: Add virDomainSetVcpuDirtyLimit API
qemu_driver: Implement qemuDomainSetVcpuDirtyLimit
virsh: Introduce limit-dirty-page-rate api
qemu_monitor: Implement qemuMonitorQueryVcpuDirtyLimit
qemu_driver: Extend qemuDomainGetVcpus
virsh: Extend vcpuinfo api
libvirt: Add virDomainCancelVcpuDirtyLimit API
qemu_driver: Implement qemuDomainCancelVcpuDirtyLimit
virsh: Add cancel option of limit-dirty-page-rate api
NEWS: Document limit dirty page rate APIs
NEWS.rst | 16 ++
include/libvirt/libvirt-domain.h | 22 +++
src/driver-hypervisor.h | 13 ++
src/libvirt-domain.c | 106 ++++++++++++
src/libvirt_public.syms | 6 +
src/qemu/qemu_capabilities.c | 2 +
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_driver.c | 154 ++++++++++++++++++
src/qemu/qemu_monitor.c | 36 ++++
src/qemu/qemu_monitor.h | 26 +++
src/qemu/qemu_monitor_json.c | 150 +++++++++++++++++
src/qemu/qemu_monitor_json.h | 13 ++
src/remote/remote_daemon_dispatch.c | 2 +
src/remote/remote_driver.c | 4 +
src/remote/remote_protocol.x | 28 +++-
src/remote_protocol-structs | 13 ++
.../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 +
.../caps_7.1.0_x86_64.xml | 1 +
tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 +
.../caps_7.2.0_x86_64+hvf.xml | 1 +
.../caps_7.2.0_x86_64.xml | 1 +
.../caps_8.0.0_riscv64.xml | 1 +
.../caps_8.0.0_x86_64.xml | 1 +
.../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 +
.../caps_8.1.0_x86_64.xml | 1 +
tools/virsh-domain.c | 123 ++++++++++++++
26 files changed, 723 insertions(+), 1 deletion(-)
--
2.38.5
1 year, 3 months
[PATCH v2] NEWS: Document my contributions for upcoming release
by Michal Privoznik
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
v2 of:
https://listman.redhat.com/archives/libvir-list/2023-July/241020.html
diff to v2:
- Reworded, basically every paragraph, to make them sound more user
friendly. Thanks Martin for his suggestions!
NEWS.rst | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/NEWS.rst b/NEWS.rst
index 1de8314a61..8f0c67f779 100644
--- a/NEWS.rst
+++ b/NEWS.rst
@@ -33,8 +33,23 @@ v9.6.0 (unreleased)
``/disk/target@bus='scsi'``) supports the ``removable`` attribute at
``/disk/target@removable```.
+ * qemu: Add NUMA node automatically for memory hotplug
+
+ Users no longer need to specify guest NUMA node in the domain XML when
+ enabling memory hotplug, libvirt automatically adds one when it is missing.
+
* **Bug fixes**
+ * qemu: Adapt to new way of specifying PC speaker
+
+ PC speaker is now usable again with newer QEMU since the change of how it
+ is specified on the command line.
+
+ * qemu_tpm: Try harder to create emulator state
+
+ Libvirt no longer considers empty directory valid SWTPM state and setup is
+ now run properly in such case.
+
v9.5.0 (2023-07-03)
===================
--
2.41.0
1 year, 3 months
[PATCH] NEWS: Document my contributions for upcoming release
by Michal Privoznik
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
NEWS.rst | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/NEWS.rst b/NEWS.rst
index 1de8314a61..0dc909c849 100644
--- a/NEWS.rst
+++ b/NEWS.rst
@@ -33,8 +33,25 @@ v9.6.0 (unreleased)
``/disk/target@bus='scsi'``) supports the ``removable`` attribute at
``/disk/target@removable```.
+ * qemu: Add NUMA node automatically for memory hotplug
+
+ When enabling memory hotplug, libvirt required at least one guest NUMA to
+ be specified in the domain XML. With this release, libvirt adds it
+ automatically when needed.
+
* **Bug fixes**
+ * qemu: Adapt to new way of specifying PC speaker
+
+ Because the way PC speaker is specified on QEMU command line changed,
+ libvirt was unable to start a domain with PC speaker. This is now fixed.
+
+ * qemu_tpm: Try harder to create emulator state
+
+ If a per-domain SWTPM state directory exists but is empty libvirt would
+ consider it a valid state and skipped running 'swtpm_setup'. This is now
+ fixed.
+
v9.5.0 (2023-07-03)
===================
--
2.41.0
1 year, 3 months
[PATCH 00/26] Migration PULL 2023-07-24
by Juan Quintela
Hi
This is the migration PULL request. It has:
- Fabiano rosas trheadinfo cleanups
- Hyman Huang dirtylimit changes
- Part of my changes
- Peter Xu documentation
- Tejus updato to migration descriptions
- Wei want improvements for postocpy and multifd setup
Please apply.
Now a not on CI, thas has been really bad. After too many problems
with last PULLS, I decided to learn to use qemu CI. On one hand, it
is not so difficult, even I can use it O:-)
On the other hand, the amount of problems that I got is inmense. Some
of them dissapear when I rerun the checks, but I never know if it is
my PULL request, the CI system or the tests themselves.
So it ends going something like:
while (true); do
- git pull
- git rebase
- git push ci blah, blah
- Next day cames, and too many errors, so I rebase again
The last step takes more time than expected and not always trivial to
know how the failure is.
This (last) patch is not part of the PULL request, but I have found
that it _always_ makes gcov fail. I had to use bisect to find where
the problem was.
https://gitlab.com/juan.quintela/qemu/-/jobs/4571878922
I could use help to know how a change in test/qtest/migration-test.c
can break block layer tests, I am all ears.
Yes, I tried several times. It always fails on that patch. The
passes with flying colors.
Later, Juan.
Fabiano Rosas (2):
migration/multifd: Rename threadinfo.c functions
migration/multifd: Protect accesses to migration_threads
Hyman Huang(黄勇) (8):
softmmu/dirtylimit: Add parameter check for hmp "set_vcpu_dirty_limit"
qapi/migration: Introduce x-vcpu-dirty-limit-period parameter
qapi/migration: Introduce vcpu-dirty-limit parameters
migration: Introduce dirty-limit capability
migration: Refactor auto-converge capability logic
migration: Put the detection logic before auto-converge checking
migration: Implement dirty-limit convergence algo
migration: Extend query-migrate to provide dirty page limit info
Juan Quintela (12):
migration-test: Be consistent for ppc
migration-test: Make machine_opts regular with other options
migration-test: Create arch_opts
migration-test: machine_opts is really arch specific
migration.json: Don't use space before colon
migration: skipped field is really obsolete.
qemu-file: Rename qemu_file_transferred_ fast -> noflush
migration: Change qemu_file_transferred to noflush
qemu_file: Make qemu_file_is_writable() static
qemu-file: Simplify qemu_file_shutdown()
qemu-file: Make qemu_file_get_error_obj() static
migration/rdma: Split qemu_fopen_rdma() into input/output functions
Peter Xu (1):
docs/migration: Update postcopy bits
Tejus GK (1):
migration: Update error description whenever migration fails
Wei Wang (2):
migration: enforce multifd and postcopy preempt to be set before
incoming
qtest/migration-tests.c: use "-incoming defer" for postcopy tests
docs/about/deprecated.rst | 10 +++
docs/devel/migration.rst | 94 ++++++++++++++++++++---------
qapi/migration.json | 107 ++++++++++++++++++++++++++-------
include/sysemu/dirtylimit.h | 2 +
migration/options.h | 1 +
migration/qemu-file.h | 14 ++---
migration/threadinfo.h | 7 +--
migration/migration-hmp-cmds.c | 26 ++++++++
migration/migration.c | 36 ++++++++---
migration/multifd.c | 4 +-
migration/options.c | 87 ++++++++++++++++++++++++++-
migration/qemu-file.c | 24 ++------
migration/ram.c | 59 +++++++++++++++---
migration/rdma.c | 39 ++++++------
migration/savevm.c | 6 +-
migration/threadinfo.c | 19 +++++-
migration/vmstate.c | 4 +-
softmmu/dirtylimit.c | 97 +++++++++++++++++++++++++++---
tests/qtest/migration-test.c | 48 +++++++--------
migration/trace-events | 1 +
20 files changed, 520 insertions(+), 165 deletions(-)
--
2.40.1
1 year, 3 months
Entering freeze for libvirt-9.6.0
by Jiri Denemark
I have just tagged v9.6.0-rc1 in the repository and pushed signed
tarballs and source RPMs to https://download.libvirt.org/
Please give the release candidate some testing and in case you find a
serious issue which should have a fix in the upcoming release, feel
free to reply to this thread to make sure the issue is more visible.
If you have not done so yet, please update NEWS.rst to document any
significant change you made since the last release.
Thanks,
Jirka
1 year, 3 months
[PATCH 0/3] Update the capability and doc for device igb
by Han Han
The interface model igb is added since v9.3.0, but the qemu capability
and qemu supported version is missing. So I add them in this series.
Han Han (3):
docs: formatdomain: Mention the QEMU version for igb
qemu: capabilities: Add the capability for the device igb
qemu/qemu_validate: Add capability check for the model igb
docs/formatdomain.rst | 2 +-
src/qemu/qemu_capabilities.c | 2 ++
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_validate.c | 7 +++++++
tests/qemucapabilitiesdata/caps_8.0.0_x86_64.xml | 1 +
tests/qemucapabilitiesdata/caps_8.1.0_x86_64.xml | 1 +
6 files changed, 13 insertions(+), 1 deletion(-)
--
2.41.0
1 year, 3 months
[libvirt PATCH 0/8] rpm: Fix handling of systemd units
by Andrea Bolognani
Plus some extras I'm throwing in for free :)
To understand why these changes are needed, see the original bug
report[1] as well as the discussion triggered by Martin's initial
attempt at addressing it[2].
Getting this right is quite tricky, so in order to convince myself
that I'm not just going to break everyone's deployment I've tested
things fairly extensively.
In particular, I've verified that things work as expected when
upgrading from libvirt 9.0.0 (e.g. pre-split) on AlmaLinux 8 when the
initial configuration is
* a default one (socket-activated monolithic daemon);
* monolithic daemon with --listen;
* modular daemons;
* modular daemons with virtproxyd-tcp.socket enabled;
as well as when installing from scratch with
* no particular configuration;
* local systemd preset overrides that result in the modular daemons
being preferred to the monolithic one.
Everything seems to work fine, but further testing would certainly be
more than welcome!
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2210058
[2] https://listman.redhat.com/archives/libvir-list/2023-June/240226.html
Andrea Bolognani (8):
rpm: Bump min_fedora
rpm: Style/alignment tweaks
rpm: Reorder scriptlets
rpm: Reduce use of with_modular_daemons
rpm: Remove custom libvirtd restart logic
rpm: Introduce new macros for handling of systemd units
rpm: Switch to new macros for handling of systemd units
rpm: Delete unused macros
libvirt.spec.in | 453 +++++++++++++++++++++++++-----------------------
1 file changed, 232 insertions(+), 221 deletions(-)
--
2.41.0
1 year, 3 months
[PATCH] qemu: don't add --mac-addr option to passt commandline
by Laine Stump
When I implemented passt support in libvirt, I saw the --mac-addr
option on the passt commandline, immediately assumed that this was
used for setting the guest interface's mac address somewhere within
passt, and read no further. As a result, "--mac-addr" is always added
to the passt commandline, specifying the setting from
<mac addr='blah'/> in the guest's interface config.
But as pointed out in this bugzilla comment:
https://bugzilla.redhat.com/2184967#c8
That is *not at all* what passt's --mac-addr option does. Instead, it
is used to force the *remote* mac address for incoming traffic to a
specific value. So setting --mac-addr results in all traffic on the
interface having the same (the guest's) mac address for both source
and destination in all traffic. Surprisingly, this still works, so
nobody noticed it during testing.
The proper thing is to not specify any mac address to passt - the
remote MAC addresses can and should remain untouched, and the local
MAC address will end up being known to passt and beyond just by the
guest sending out packets with that MAC address.
Reported-by: Laszlo Ersek <lersek(a)redhat.com>
Signed-off-by: Laine Stump <laine(a)redhat.com>
---
src/qemu/qemu_passt.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/qemu/qemu_passt.c b/src/qemu/qemu_passt.c
index 3679bf75fc..d36856e92e 100644
--- a/src/qemu/qemu_passt.c
+++ b/src/qemu/qemu_passt.c
@@ -176,7 +176,6 @@ qemuPasstStart(virDomainObj *vm,
g_autofree char *passtSocketName = qemuPasstCreateSocketPath(vm, net);
g_autoptr(virCommand) cmd = NULL;
g_autofree char *pidfile = qemuPasstCreatePidFilename(vm, net);
- char macaddr[VIR_MAC_STRING_BUFLEN];
bool needUnlink = false;
size_t i;
@@ -187,7 +186,6 @@ qemuPasstStart(virDomainObj *vm,
virCommandAddArgList(cmd,
"--one-off",
"--socket", passtSocketName,
- "--mac-addr", virMacAddrFormat(&net->mac, macaddr),
"--pid", pidfile,
NULL);
--
2.41.0
1 year, 3 months
[PULL 00/25] Migration 20230726 patches
by Juan Quintela
The following changes since commit 6cb2011fedf8c4e7b66b4a3abd6b42c1bae99ce6:
Update version for v8.1.0-rc1 release (2023-07-25 20:09:05 +0100)
are available in the Git repository at:
https://gitlab.com/juan.quintela/qemu.git tags/migration-20230726-pull-request
for you to fetch changes up to 697c4c86ab515a728ffb2adc2c3c04b22fa9210f:
migration/rdma: Split qemu_fopen_rdma() into input/output functions (2023-07-26 10:55:56 +0200)
----------------------------------------------------------------
Migration Pull request
Hi
This is the migration PULL request. It is the same than yesterday with proper PULL headers.
It pass CI. It contains:
- Fabiano rosas trheadinfo cleanups
- Hyman Huang dirtylimit changes
- Part of my changes
- Peter Xu documentation
- Tejus updato to migration descriptions
- Wei want improvements for postocpy and multifd setup
Please apply.
Thanks, Juan.
----------------------------------------------------------------
Fabiano Rosas (2):
migration/multifd: Rename threadinfo.c functions
migration/multifd: Protect accesses to migration_threads
Hyman Huang(黄勇) (8):
softmmu/dirtylimit: Add parameter check for hmp "set_vcpu_dirty_limit"
qapi/migration: Introduce x-vcpu-dirty-limit-period parameter
qapi/migration: Introduce vcpu-dirty-limit parameters
migration: Introduce dirty-limit capability
migration: Refactor auto-converge capability logic
migration: Put the detection logic before auto-converge checking
migration: Implement dirty-limit convergence algo
migration: Extend query-migrate to provide dirty page limit info
Juan Quintela (11):
migration-test: Be consistent for ppc
migration-test: Make machine_opts regular with other options
migration-test: Create arch_opts
migration-test: machine_opts is really arch specific
migration: skipped field is really obsolete.
qemu-file: Rename qemu_file_transferred_ fast -> noflush
migration: Change qemu_file_transferred to noflush
qemu_file: Make qemu_file_is_writable() static
qemu-file: Simplify qemu_file_shutdown()
qemu-file: Make qemu_file_get_error_obj() static
migration/rdma: Split qemu_fopen_rdma() into input/output functions
Peter Xu (1):
docs/migration: Update postcopy bits
Tejus GK (1):
migration: Update error description whenever migration fails
Wei Wang (2):
migration: enforce multifd and postcopy preempt to be set before
incoming
qtest/migration-tests.c: use "-incoming defer" for postcopy tests
docs/about/deprecated.rst | 10 ++++
docs/devel/migration.rst | 94 ++++++++++++++++++++++----------
qapi/migration.json | 87 ++++++++++++++++++++++++++----
include/sysemu/dirtylimit.h | 2 +
migration/options.h | 1 +
migration/qemu-file.h | 14 ++---
migration/threadinfo.h | 7 +--
migration/migration-hmp-cmds.c | 26 +++++++++
migration/migration.c | 36 +++++++++----
migration/multifd.c | 4 +-
migration/options.c | 87 +++++++++++++++++++++++++++++-
migration/qemu-file.c | 24 ++-------
migration/ram.c | 59 +++++++++++++++++----
migration/rdma.c | 39 +++++++-------
migration/savevm.c | 6 +--
migration/threadinfo.c | 19 +++++--
migration/vmstate.c | 4 +-
softmmu/dirtylimit.c | 97 ++++++++++++++++++++++++++++++----
tests/qtest/migration-test.c | 48 ++++++++---------
migration/trace-events | 1 +
20 files changed, 510 insertions(+), 155 deletions(-)
--
2.40.1
1 year, 3 months
[PATCH] remote: handle partial data transmission
by Oleg Vasilev
A new bug was introduced as a part of use-after-free fix below:
commit 411cbe7199ce533ae5fa78f5558dddca6f88ef1a
Author: Oleg Vasilev <oleg.vasilev(a)virtuozzo.com>
Date: Tue Jul 4 13:10:22 2023 +0600
remote: fix stream use-after-free
When the message was processed partially, it is actually supposed to
stay in the queue to be processed again. In such case, reinsert it back.
Signed-off-by: Oleg Vasilev <oleg.vasilev(a)virtuozzo.com>
---
src/remote/remote_daemon_stream.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c
index 345c40b48c..f52af790c1 100644
--- a/src/remote/remote_daemon_stream.c
+++ b/src/remote/remote_daemon_stream.c
@@ -775,8 +775,12 @@ daemonStreamHandleWrite(virNetServerClient *client,
ret = -1;
}
- if (ret > 0)
- break; /* still processing data from msg */
+ if (ret > 0) {
+ /* still processing data from msg, put it back into queue */
+ msg->next = stream->rx;
+ stream->rx = msg;
+ break;
+ }
if (ret < 0) {
virNetMessageFree(msg);
--
2.41.0
1 year, 3 months
