On 3/7/24 10:51, Martin Kletzander wrote:
On Wed, Mar 06, 2024 at 05:17:36PM +0100, Christian Rohmann via Users wrote:
Hallo libvirt-users!
Hi, I'll try to reply in the simplest possible way.
we observe lock-ups / timeouts with in prometheus-libvirt-exporter (https://github.com/inovex/prometheus-libvirt-exporter) when libvirt is live-migrating domains:
Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePrepare3Params)
All of the source code can be found at: https://github.com/inovex/prometheus-libvirt-exporter/blob/master/pkg/export.... Basically the error happens when DomainMemoryStats or other operational domain info is queried via the libvirt socket.
Yes, the domain is being modified by the migration, so it is locked.
While this is true, the "lock" - or job I should rather say is an async one, meaning a QUERY job can be acquired. It's only MODIFY job that should wait in the queue. What's rather weird is - the thread holding the job is 'MigratePrepare' which usually isn't that long. @Christian, what is the libvirt version? Are you able to reproduce with either libvirt-10.1.0 or (even better) current master? Michal