If that was the case I wouldn't be able to attach a volume without snapshots. But just in case I did the following test, before attaching the file, I copied all the file to a local folder and changed the path in XML with the new paths. The problem repeats: 

2016-08-22 17:14:19.795+0000: 3799: info : qemuMonitorIOProcess:423 : QEMU_MONITOR_IO_PROCESS: mon=0x7f68f800c4c0 buf={"return": "Could not open backing file: Could not open '/home/ubuntu/volumes/volume-d81f7e7e-4215-4433-aad2-50adcd31b29d.7a9ea169-a3ab-48bd-bd9d-e7585860abe4': Permission denied\r\n", "id": "libvirt-30"}

But, if I use the top backing file with same permissions (/home/ubuntu/volumes/volume-d81f7e7e-4215-4433-aad2-50adcd31b29d), the operation suceeds. Can this be a libvirt/QEMU bug?

Han, can you tell me more details about the commands you used and the versions of or libvirt/qemu? I couldn't reproduce from the steps you outlined.

On Fri, Aug 19, 2016 at 12:02 AM, Han Han <hhan@redhat.com> wrote:
Hi Erlon,
Since you say you got Permission denied error while the ACL mode is 777， I guess it is due to security system like selinux or apparmor.
As I know, selinux defaultly will disable qemu to use volume based on NFS. We need to use 'setsebool virt_use_nfs 1' to enable it.
Well, your OS is ubuntu, so I think apparmor causes the permission problem. Try to find a way to enable virt_use_nfs or disable apparmor. Then check if the problem can be reproduced.

Han

----- Original Message -----
From: "Erlon Cruz" <sombrafam@gmail.com>
To: "Han Han" <hhan@redhat.com>
Cc: "Yang Yang" <yanyang@redhat.com>, "Yuan Dan" <dyuan@redhat.com>, "Xuesong Zhang" <xuzhang@redhat.com>, libvirt-users@redhat.com
Sent: Thursday, 18 August, 2016 12:44:36 PM
Subject: Re: [libvirt-users] Attaching disks with external snapshots

Hi Han,

Any update on this?

Erlon

On Wed, Aug 17, 2016 at 9:58 AM, Erlon Cruz <sombrafam@gmail.com> wrote:

> Han,
>
> Did the same test again with libvirt debug enabled. This time I added the
> disk[1] without snaps (OK), and then added the disk[2] with snapshots. The
> libvirt log[3] (line 320) says:
>
> *Could not open
> '/opt/stack/data/nova/mnt/a3b4c6ddd9bf82edd4f726872be58d05/volume-a75b42e8-8175-4516-b724-43ee6f3d2520':
> Permission denied\r\n", "id": "libvirt-33"}]*
>
> But this file has 777 permissions. I can access it with any user I
> tried[4]. Tough its mounted over NFS, the permissions seems to be right.
>
> Erlon
>
> [1]  http://paste.openstack.org/show/556055/
> [2]  http://paste.openstack.org/show/556056/
> [3]  http://pastebin.com/CCFcKq5b
> [4]  http://paste.openstack.org/show/559018/
>
>
> On Tue, Aug 16, 2016 at 2:41 PM, Erlon Cruz <sombrafam@gmail.com> wrote:
>
>> Hi Han,
>>
>> Thanks for helping. What is the XML before the snapshot is created? Can
>> you post the XML files you used too? The files are in the links pointed by
>> the numbers [1], [2] etc.
>>
>> The versions are:
>> qemu-system-x86_64 --version
>> QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.4), Copyright
>> (c) 2003-2008 Fabrice Bellard
>>
>> libvirtd --version
>> libvirtd (libvirt) 1.3.1
>>
>>
>>
>> Erlon
>>
>> On Mon, Aug 15, 2016 at 12:56 AM, Han Han <hhan@redhat.com> wrote:
>>
>>> Hi，Erlon
>>> I was not able to reproduce your problem. It is ok to attach disk with
>>> external snapshot.
>>> I did it with following steps:
>>> # qemu-img create /tmp/vdb 100M
>>> Formatting '/tmp/vdb', fmt=raw size=104857600
>>> # virsh start snap
>>> Domain snap started
>>> # virsh snapshot-create-as snap s1 --disk-only --diskspec
>>> vdb,file=/tmp/vdb.s1 --quiesce
>>> Domain snapshot s1 created
>>> # virsh dumpxml snap|awk '/<disk/,/<\/disk/'
>>>     <disk type='file' device='disk'>
>>>       <driver name='qemu' type='qcow2' cache='none'/>
>>>       <source file='/var/lib/libvirt/images/snap.s1'/>
>>>       <backingStore type='file' index='1'>
>>>         <format type='qcow2'/>
>>>         <source file='/var/lib/libvirt/images/snap.qcow2'/>
>>>         <backingStore/>
>>>       </backingStore>
>>>       <target dev='vda' bus='virtio'/>
>>>       <alias name='virtio-disk0'/>
>>>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
>>> function='0x0'/>
>>>     </disk>
>>>     <disk type='file' device='disk'>
>>>       <driver name='qemu' type='qcow2'/>
>>>       <source file='/tmp/vdb.s1'/>
>>>       <backingStore type='file' index='1'>
>>>         <format type='raw'/>
>>>         <source file='/tmp/vdb'/>
>>>         <backingStore/>
>>>       </backingStore>
>>>       <target dev='vdb' bus='virtio'/>
>>>       <alias name='virtio-disk1'/>
>>>       <address type='pci' domain='0x0000' bus='0x00' slot='0x0c'
>>> function='0x0'/>
>>>     </disk>
>>>
>>> Save the dumped vdb's xml to /tmp/vdb.xml
>>> # virsh detach-device snap /tmp/vdb.xml
>>> Device detached successfully
>>> # virsh attach-device snap /tmp/vdb.xml
>>> Device attached successfully
>>> # virsh dumpxml snap|awk '/<disk/,/<\/disk/'
>>>     <disk type='file' device='disk'>
>>>       <driver name='qemu' type='qcow2' cache='none'/>
>>>       <source file='/var/lib/libvirt/images/snap.s1'/>
>>>       <backingStore type='file' index='1'>
>>>         <format type='qcow2'/>
>>>         <source file='/var/lib/libvirt/images/snap.qcow2'/>
>>>         <backingStore/>
>>>       </backingStore>
>>>       <target dev='vda' bus='virtio'/>
>>>       <alias name='virtio-disk0'/>
>>>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
>>> function='0x0'/>
>>>     </disk>
>>>     <disk type='file' device='disk'>
>>>       <driver name='qemu' type='qcow2'/>
>>>       <source file='/tmp/vdb.s1'/>
>>>       <backingStore type='file' index='1'>
>>>         <format type='raw'/>
>>>         <source file='/tmp/vdb'/>
>>>         <backingStore/>
>>>       </backingStore>
>>>       <target dev='vdb' bus='virtio'/>
>>>       <alias name='virtio-disk1'/>
>>>       <address type='pci' domain='0x0000' bus='0x00' slot='0x0c'
>>> function='0x0'/>
>>>     </disk>
>>>
>>> So I managed to attach disks with snapshot.
>>> I want to know what is your libvirt and qemu-kvm version?
>>> And can you provide your files: /tmp/disk.xml[1],
>>> /tmp/snap-from-disk.xml[2], /tmp/disk-with-snap.xml[4],
>>> /tmp/disk-with-snap.xml . I need more details.
>>>
>>> Han Han
>>>
>>> ----- Original Message -----
>>> From: "Erlon Cruz" <sombrafam@gmail.com>
>>> To: libvirt-users@redhat.com
>>> Sent: Friday, August 12, 2016 10:23:24 AM
>>> Subject: [libvirt-users] Attaching disks with external snapshots
>>>
>>> Hi folks,
>>>
>>> I'm trying to to attach a disk to an instance using libvirt. The problem
>>> is, this disk has external snapshots. The process tried was:
>>>
>>> 1 - Attach a disk in the domain:
>>> virsh# attach-device instance-00000006 /tmp/disk.xml[1] --live
>>>
>>> 2 - Snapshot the disk[2]:
>>> virsh# snapshot-create instance-00000006 --quiesce --xmlfile
>>> /tmp/snap-from-disk.xml[2] --disk-only
>>>
>>> 3 - Dump the domain XML and create a new disk file from it:
>>> virsh# dumpxml instance-00000006
>>> ... [3]
>>>
>>> 4 - Dettach the device and re-add it using the new disk file:
>>> virsh# detach-device instance-00000006 /tmp/disk-with-snap.xml[4]
>>> Device detached successfully
>>> virsh# attach-device instance-00000006 /tmp/disk-with-snap.xml
>>> error: Failed to attach device from /tmp/disk-with-snap.xml
>>> error: internal error: unable to execute QEMU command 'device_add':
>>> Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'
>>>
>>> Question, is this operation supported? If yes, how is the correct
>>> procedure?
>>>
>>> Erlon
>>>
>>>
>>> [1] http://paste.openstack.org/show/556055/
>>> [2] http://paste.openstack.org/show/556056/
>>> [3] http://paste.openstack.org/show/556063/
>>> [4] http://paste.openstack.org/show/556064/
>>>
>>> _______________________________________________
>>> libvirt-users mailing list
>>> libvirt-users@redhat.com
>>> https://www.redhat.com/mailman/listinfo/libvirt-users
>>>
>>
>>
>