Re: Error Cannot acquire state change lock from remoteDispatchDomainMigratePrepare3Params during live migration of domains

On 3/7/24 10:51, Martin Kletzander wrote: > On Wed, Mar 06, 2024 at 05:17:36PM +0100, Christian Rohmann via Users > wrote: >> Hallo libvirt-users! >> > > Hi, I'll try to reply in the simplest possible way. > >> we observe lock-ups / timeouts with in prometheus-libvirt-exporter >> (https://github.com/inovex/prometheus-libvirt-exporter) when >> libvirt is live-migrating domains: >> >>> Timed out during operation: cannot acquire state change lock (held by >>> monitor=remoteDispatchDomainMigratePrepare3Params) >> >> >> All of the source code can be found at: >> https://github.com/inovex/prometheus-libvirt-exporter/blob/master/pkg/exp.... >> Basically the error happens when DomainMemoryStats or other operational >> domain info is queried via the libvirt socket. >> > > Yes, the domain is being modified by the migration, so it is locked. While this is true, the "lock" - or job I should rather say is an async one, meaning a QUERY job can be acquired. It's only MODIFY job that should wait in the queue. What's rather weird is - the thread holding the job is 'MigratePrepare' which usually isn't that long.