Re: [libvirt] [PATCH 00/16] snapshot refactoring (incremental backup saga)

Friday, 22 March 2019

On Wed, 2019-03-20 at 00:40 -0500, Eric Blake wrote:
...
 A couple of these have seen the list before, but most of them are
 new. The bulk of this series is about refactoring snapshot_conf.c into
 smaller pieces that I can then reuse for implementing checkpoints,
 without having to open-code the hierarchy algorithms a second time
 (one of John's complaints against my v4 incremental backup series).

 I suspect that src/vz/ builds may break on one or more of these
 patches; I could not get a working vz build environment.

 Eric Blake (16):
   test: Avoid use-after-free on virDomainSnapshotDelete
   snapshot: Use accessors for virDomainSnapshot members
   snapshot: Create virDomainMoment base class
   vbox: Clean up some snapshot usage
   snapshot: Drop virDomainSnapshotDef.current
   snapshot: Track current snapshot in virDomainSnapshotObjList
   snapshot: Add accessors for updating snapshot list relations
   snapshot: Access snapshot def directly when needed
   snapshot: Refactor list filtering
   snapshot: Factor out virDomainMomentDef class
   snapshot: Switch type of virDomainSnapshotObj.def
   snapshot: Rename virDomainSnapshotObjPtr
   snapshot: Rename file for virDomainMomentObj
   snapshot: Move snapshot list code into generic file
   snapshot: Tweaks to support new bulk dumpxml/import API
   backup: Introduce virDomainCheckpointPtr 
This causes libvirtd to crash at startup on my machine.

Have a backtrace:

  Thread 19 (Thread 0x7fffaa4e0700 (LWP 31651)):
  #0  0x00007ffff72ecd31 in open64 () from /lib64/libc.so.6
  #1  0x00007ffff727d3f6 in _IO_file_open () from /lib64/libc.so.6
  #2  0x00007ffff727d5ad in __GI__IO_file_fopen () from /lib64/libc.so.6
  #3  0x00007ffff727132d in __fopen_internal () from /lib64/libc.so.6
  #4  0x00007ffff71199d8 in ?? () from /lib64/libudev.so.1
  #5  0x00007ffff71146dd in ?? () from /lib64/libudev.so.1
  #6  0x00007ffff71173ed in ?? () from /lib64/libudev.so.1
  #7  0x00007ffff71179d9 in ?? () from /lib64/libudev.so.1
  #8  0x00007ffff710be77 in udev_device_get_property_value () from /lib64/libudev.so.1
  #9  0x00007fffc2e8a3ef in udevGetDeviceProperty
(udev_device=udev_device@entry=0x7fff900606e0,
property_key=property_key@entry=0x7fffc2ea3504 "DRIVER") at
node_device/node_device_udev.c:140
  #10 0x00007fffc2e8a459 in udevGetStringProperty
(udev_device=udev_device@entry=0x7fff900606e0,
property_key=property_key@entry=0x7fffc2ea3504 "DRIVER", value=0x7fff900f98d8)
at node_device/node_device_udev.c:154
  #11 0x00007fffc2e8b3ad in udevAddOneDevice (device=device@entry=0x7fff900606e0) at
node_device/node_device_udev.c:1369
  #12 0x00007fffc2e8d27e in udevProcessDeviceListEntry (list_entry=0x7fff9004f350,
udev=0x7fff98120f00) at node_device/node_device_udev.c:1435
  #13 udevEnumerateDevices (udev=0x7fff98120f00) at node_device/node_device_udev.c:1489
  #14 nodeStateInitializeEnumerate (opaque=0x7fff98120f00) at
node_device/node_device_udev.c:1798
  #15 0x00007ffff7c5a2c2 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #16 0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #17 0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 18 (Thread 0x7fffaace1700 (LWP 31650)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x7fff98120640,
m=m@entry=0x7fff98120600) at util/virthread.c:154
  #2  0x00007fffc2e8d404 in udevEventHandleThread (opaque=<optimized out>) at
node_device/node_device_udev.c:1618
  #3  0x00007ffff7c5a2c2 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 17 (Thread 0x7fffab4e2700 (LWP 31607)):
  #0  0x00007ffff7cc1c85 in virDomainMomentAssignDef (moments=0x7fff981af8a0,
def=def@entry=0x7fff98227870) at conf/virdomainmomentobjlist.c:230
  #1  0x00007ffff7cc22b8 in virDomainSnapshotAssignDef (snapshots=<optimized out>,
def=def@entry=0x7fff98227870) at conf/virdomainsnapshotobjlist.c:212
  #2  0x00007fffc2b3dd6f in qemuDomainSnapshotLoad (vm=0x7fff982dfcc0, data=<optimized
out>) at qemu/qemu_driver.c:481
  #3  0x00007ffff7cbfeba in virDomainObjListHelper (payload=<optimized out>,
name=<optimized out>, opaque=0x7fffab4e1960) at conf/virdomainobjlist.c:803
  #4  0x00007ffff7c06200 in virHashForEach (data=<optimized out>, iter=<optimized
out>, table=<optimized out>) at util/virhash.c:575
  #5  virHashForEach (table=0x7fff981226c0, iter=iter@entry=0x7ffff7cbfeb0
<virDomainObjListHelper>, data=data@entry=0x7fffab4e1960) at util/virhash.c:563
  #6  0x00007ffff7cc13a1 in virDomainObjListForEach (doms=0x7fff98122660,
callback=callback@entry=0x7fffc2b3db40 <qemuDomainSnapshotLoad>,
opaque=<optimized out>) at conf/virdomainobjlist.c:818
  #7  0x00007fffc2b3ebc3 in qemuStateInitialize (privileged=true, callback=<optimized
out>, opaque=<optimized out>) at qemu/qemu_driver.c:898
  #8  0x00007ffff7e07fed in virStateInitialize (privileged=true, callback=0x555555578480
<daemonInhibitCallback>, opaque=0x555555605bc0) at libvirt.c:653
  #9  0x00005555555784db in daemonRunStateInit (opaque=0x555555605bc0) at
remote/remote_daemon.c:797
  #10 0x00007ffff7c5a2c2 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #11 0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #12 0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 16 (Thread 0x7fffc0926700 (LWP 31606)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555663c38,
m=m@entry=0x555555663c10) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x555555661a50) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 15 (Thread 0x7fffc1127700 (LWP 31605)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555663c38,
m=m@entry=0x555555663c10) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x55555560bdc0) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 14 (Thread 0x7fffc1928700 (LWP 31604)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555663c38,
m=m@entry=0x555555663c10) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x55555566a880) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 13 (Thread 0x7fffc2129700 (LWP 31603)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555663c38,
m=m@entry=0x555555663c10) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x555555667640) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 12 (Thread 0x7fffc292a700 (LWP 31602)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555663c38,
m=m@entry=0x555555663c10) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x555555653ae0) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 11 (Thread 0x7fffc3fff700 (LWP 31601)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x5555556060f8,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5afa4 in virThreadPoolWorker (opaque=opaque@entry=0x555555617ec0) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 10 (Thread 0x7fffd8ff9700 (LWP 31600)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x5555556060f8,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5afa4 in virThreadPoolWorker (opaque=opaque@entry=0x555555618090) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 9 (Thread 0x7fffd97fa700 (LWP 31599)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x5555556060f8,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5afa4 in virThreadPoolWorker (opaque=opaque@entry=0x5555556180e0) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 8 (Thread 0x7fffd9ffb700 (LWP 31598)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x5555556060f8,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5afa4 in virThreadPoolWorker (opaque=opaque@entry=0x555555618130) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 7 (Thread 0x7fffda7fc700 (LWP 31597)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x5555556060f8,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5afa4 in virThreadPoolWorker (opaque=opaque@entry=0x555555618260) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 6 (Thread 0x7fffdaffd700 (LWP 31596)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555606058,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x5555556182b0) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 5 (Thread 0x7fffdb7fe700 (LWP 31595)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555606058,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x555555618380) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 4 (Thread 0x7fffdbfff700 (LWP 31594)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555606058,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x555555618460) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 3 (Thread 0x7fffe8971700 (LWP 31593)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555606058,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x5555556185a0) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 2 (Thread 0x7fffe9172700 (LWP 31592)):
  #0  0x00007ffff741373c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #1  0x00007ffff7c5a4f6 in virCondWait (c=c@entry=0x555555606058,
m=m@entry=0x555555606030) at util/virthread.c:154
  #2  0x00007ffff7c5aff3 in virThreadPoolWorker (opaque=opaque@entry=0x5555556192e0) at
util/virthreadpool.c:120
  #3  0x00007ffff7c5a298 in virThreadHelper (data=<optimized out>) at
util/virthread.c:206
  #4  0x00007ffff740d58e in start_thread () from /lib64/libpthread.so.0
  #5  0x00007ffff72fc6a3 in clone () from /lib64/libc.so.6

  Thread 1 (Thread 0x7ffff61c7b80 (LWP 31587)):
  #0  0x00007ffff72f1421 in poll () from /lib64/libc.so.6
  #1  0x00007ffff7bf841b in poll (__timeout=-1, __nfds=6, __fds=<optimized out>) at
/usr/include/bits/poll2.h:46
  #2  virEventPollRunOnce () at util/vireventpoll.c:636
  #3  0x00007ffff7bf6f71 in virEventRunDefaultImpl () at util/virevent.c:322
  #4  0x00007ffff7d35f95 in virNetDaemonRun (dmn=0x555555605bc0) at
rpc/virnetdaemon.c:847
  #5  0x0000555555576e30 in main (argc=<optimized out>, argv=<optimized out>)
at remote/remote_daemon.c:1455

If I revert back to 320a1480d0dbe77ae9da08b6ce6c3ad5e2706b63 or
delete all snapshots, then it works fine.

-- 
Andrea Bolognani / Red Hat / Virtualization

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Re: [libvirt] [PATCH 00/16] snapshot refactoring (incremental backup saga)