Re: [libvirt] Quantifying libvirt errors in launching the libguestfs appliance

On Wed, Jan 13, 2016 at 10:18:42AM +0000, Richard W.M. Jones wrote: >As people may know, we frequently encounter errors caused by libvirt >when running the libguestfs appliance. > >I wanted to find out exactly how frequently these happen and classify >the errors, so I ran the 'virt-df' tool overnight 1700 times. This >tool runs several parallel qemu:///session libvirt connections both >creating a short-lived appliance guest. > >Note that I have added Cole's patch to fix https://bugzilla.redhat.com/1271183 >"XML-RPC error : Cannot write data: Transport endpoint is not connected" > >Results: > >The test failed 538 times (32% of the time), which is pretty dismal. >To be fair, virt-df is aggressive about how it launches parallel >libvirt connections. Most other virt-* tools use only a single >libvirt connection and are consequently more reliable. > >Of the failures, 518 (96%) were of the form: > > process exited while connecting to monitor: qemu: could not load kernel '/home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/kernel': Permission denied > >which is https://bugzilla.redhat.com/921135 or maybe >https://bugzilla.redhat.com/1269975. It's not clear to me if these >bugs have different causes, but if they do then potentially we're >seeing a mix of both since my test has no way to distinguish them. > It looks to me as the same problem. And as the same problem we were talking about bunch of time and, apparently, didn't get to a conclusion. For each of the kernels, libvirt labels them (with both DAC and selinux labels), then proceeds to launching qemu. If this is done parallel, the race is pretty obvious. Could you remind me why you couldn't use <seclabel model='none'/> or <seclabel relabel='no'/> or something that would mitigate this? If we cannot use this, then we need to implement the <seclabel/> element for kernel and initrd.