[libvirt] [Libvirt][Sheepdog]Sheepdog pool/volume storage driver for libvirt problem
Hi Sebastian, Morita and other developers, I have studied Sheepdog pool/volume storage driver for libvirt and i have following questions. Please give me some suggestions. Thanks in advance ;-) 1, In libvirt/src/storage/storage_backend_sheepdog.c, i just find we realized create/delete/resize volume(send these parameters to let libvirtd receive and execute them). I cannot find the operations about snapshot, such as creating/deleting snapshot. I cannot also find the operations about opening/closing cache. 2, In the func create/delete/resize volume, we call func 'virCommandRun(cmd, NULL);' to run these commands, which i think the func send these parameters to daemon libvirtd and libvirtd send these parameters to Sheepdog, then Sheepdog executes them, right? 3, In func "virStorageBackendSheepdogCreateVol", it composes cmd like this "virCommandPtr cmd = virCommandNewArgList(COLLIE, "vdi", "create", vol->name, NULL);" then send create parameters. When Sheepdog receive these parameters, how Sheepdog create this volume? I find Sheepdog create a volume like this. $ qemu-img create sheepdog:Alice 256G 4, What are the essences of func "virStorageBackendSheepdogRefreshPool" and "virStorageBackendSheepdogRefreshVol"? In func "virStorageBackendSheepdogCreateVol", after craete volume operations, we call "virStorageBackendSheepdogRefreshVol". But in func "virStorageBackendSheepdogDeleteVol" and "virStorageBackendSheepdogResizeVol", we do not call "virStorageBackendSheepdogRefreshVol", why? 5, Actually, i am not clear about how QEMU receive these parameters (like create/delete/snapshot, etc) and execute the real jobs? Thanks for anyone's suggestions ;-) -- Thanks Harry Wei
On 01/29/2013 10:42 AM, harryxiyou wrote:
Hi Sebastian, Morita and other developers,
Answering what I can...
2, In the func create/delete/resize volume, we call func 'virCommandRun(cmd, NULL);' to run these commands, which i think the func send these parameters to daemon libvirtd and libvirtd send these parameters to Sheepdog, then Sheepdog executes them, right?
Not quite - this is forking and executing an actual command, where the argv[] of that command was built up by the earlier virCommand* calls on that cmd object. There is no intermediate path through libvirtd during virCommandRun. It sounds like you are not quite familiar with the libvirt flow of execution. When you type a 'virsh' command line, virsh calls an API from libvirt.so. For qemu integration, the client side uses the remote driver, where the code in src/libvirt.c ends up calling the driver in src/remote/remote_driver.c, and the API ends up bundling the arguments as an RPC call, which gets sent to the existing libvirtd process. On the libvirtd daemon side, the RPC call is received and unbundled, and fed back into a call into libvirt.so, but this time the driver in src/qemu/qemu_driver.c handles the callback, and for storage operations forwards the call onto src/storage/storage_driver.c. So, the virCommandRun() you are asking about is already being run in the context of libvirtd, and is just forking out to an external program.
3, In func "virStorageBackendSheepdogCreateVol", it composes cmd like this "virCommandPtr cmd = virCommandNewArgList(COLLIE, "vdi", "create", vol->name, NULL);" then send create parameters. When Sheepdog receive these parameters, how Sheepdog create this volume? I find Sheepdog create a volume like this. $ qemu-img create sheepdog:Alice 256G
This is using 'collie', not 'qemu-img', to create the volume. That is, the current sheepdog storage driver is NOT going through qemu, but going directly to sheepdog.
5, Actually, i am not clear about how QEMU receive these parameters (like create/delete/snapshot, etc) and execute the real jobs?
The code in src/storage is primarily for offline image manipulation, when no qemu is running and the storage volume is not in use. For online manipulation, actions like resize must go through the virDomainBlockResize(), which is handled in src/qemu/qemu_driver.c as a call into the qemu QMP monitor command for online resizing. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, Jan 30, 2013 at 2:49 AM, Eric Blake <eblake@redhat.com> wrote:
On 01/29/2013 10:42 AM, harryxiyou wrote:
Hi Sebastian, Morita and other developers,
Answering what I can...
Eric, thanks for your answers, you are awesome ;-) And i have some questions about your answers like following.
3, In func "virStorageBackendSheepdogCreateVol", it composes cmd like this "virCommandPtr cmd = virCommandNewArgList(COLLIE, "vdi", "create", vol->name, NULL);" then send create parameters. When Sheepdog receive these parameters, how Sheepdog create this volume? I find Sheepdog create a volume like this. $ qemu-img create sheepdog:Alice 256G
This is using 'collie', not 'qemu-img', to create the volume. That is, the current sheepdog storage driver is NOT going through qemu, but going directly to sheepdog.
Yup, this is using 'collie', which is going directly to sheepdog. But Morita said if QEMU do not support Sheepdog, Libvirt would not support sheepdog. I think sheepdog uses collie to create/delete/... volumes but QEMU simulates sheepdog block device, which it has to go through QEMU during runtime. Do you think so? Or you have any other ideas? -- Thanks Harry Wei
On 01/30/2013 05:59 AM, harryxiyou wrote:
On Wed, Jan 30, 2013 at 2:49 AM, Eric Blake <eblake@redhat.com> wrote:
On 01/29/2013 10:42 AM, harryxiyou wrote:
Hi Sebastian, Morita and other developers,
Answering what I can...
Eric, thanks for your answers, you are awesome ;-) And i have some questions about your answers like following.
3, In func "virStorageBackendSheepdogCreateVol", it composes cmd like this "virCommandPtr cmd = virCommandNewArgList(COLLIE, "vdi", "create", vol->name, NULL);" then send create parameters. When Sheepdog receive these parameters, how Sheepdog create this volume? I find Sheepdog create a volume like this. $ qemu-img create sheepdog:Alice 256G
This is using 'collie', not 'qemu-img', to create the volume. That is, the current sheepdog storage driver is NOT going through qemu, but going directly to sheepdog.
Yup, this is using 'collie', which is going directly to sheepdog. But Morita said if QEMU do not support Sheepdog, Libvirt would not support sheepdog. I think sheepdog uses collie to create/delete/... volumes but QEMU simulates sheepdog block device, which it has to go through QEMU during runtime. Do you think so? Or you have any other ideas?
There's two aspects to storage management in libvirt. 1. Offline management - libvirt needs to be able to create pools, volumes within those pools, and list details about those volumes (basically, making 'virsh pool-*' and 'virsh vol-*' work for sheepdog); for qemu:/// URI connections, this is done through the src/storage/storage_driver.c driver, which in turn delegates to the appropriate backend_*.c backend. For sheepdog, this will call directly into collie. 2. Online management - libvirt needs to be able to tell qemu to start a domain with a particular protocol, and then use qemu monitor commands to do further manipulations on that block device. This involves two pieces - we have to have a way to record what protocol a <disk> uses in domain XML (src/conf/domain_conf), and then we have to translate that XML into the command line arguments that qemu expects (src/qemu/qemu_command). For sheepdog, this means that we had to add a sheepdog protocol into a type='network' XML (see http://libvirt.org/formatdomain.html#elementsDisks for an example of the XML), and then translate that into an appropriate file=sheepdog:... command line snippet in src/qemu/qemu_command.c (look for VIR_DOMAIN_DISK_PROTOCOL_SHEEPDOG). If you are going to add a new protocol, you need to do both aspects - teach libvirt how to manage the protocol directly, for offline management, and teach libvirt how to invoke qemu to use the new protocol for online usage. But without at least one online user (qemu), it doesn't make sense to worry about offline management in libvirt (what good is allocating a volume if you can't make any guest populate or read from that volume). That's why we are telling you to focus on getting your new protocol accepted into qemu first. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Thu, Jan 31, 2013 at 3:05 AM, Eric Blake <eblake@redhat.com> wrote: [...]
There's two aspects to storage management in libvirt.
Eric, very thanks for your perfectly answers ;-) [...]
But without at least one online user (qemu), it doesn't make sense to worry about offline management in libvirt (what good is allocating a volume if you can't make any guest populate or read from that volume). That's why we are telling you to focus on getting your new protocol accepted into qemu first.
Actually, i also cannot quite understand why we focus on getting new protocol accepted into qemu first. Could you please expain it clearer? Thanks in advance ;-) -- Thanks Harry Wei
On 01/31/2013 04:59 AM, harryxiyou wrote:
On Thu, Jan 31, 2013 at 3:05 AM, Eric Blake <eblake@redhat.com> wrote: [...]
There's two aspects to storage management in libvirt.
Eric, very thanks for your perfectly answers ;-)
[...]
But without at least one online user (qemu), it doesn't make sense to worry about offline management in libvirt (what good is allocating a volume if you can't make any guest populate or read from that volume). That's why we are telling you to focus on getting your new protocol accepted into qemu first.
Actually, i also cannot quite understand why we focus on getting new protocol accepted into qemu first. Could you please expain it clearer? Thanks in advance ;-)
Libvirt exists to manage existing hypervisor capabilities. It makes no sense for libvirt to be able to create storage volumes for a new protocol if there are no hypervisors that are also able to support that storage format. So until qemu can target your new storage protocol, it doesn't make sense to have libvirt support it either. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Fri, Feb 1, 2013 at 1:18 AM, Eric Blake <eblake@redhat.com> wrote: [...]
Libvirt exists to manage existing hypervisor capabilities. It makes no sense for libvirt to be able to create storage volumes for a new protocol if there are no hypervisors that are also able to support that storage format. So until qemu can target your new storage protocol, it doesn't make sense to have libvirt support it either.
Thanks, i see. Eric, if you have free time, could you please give me some suggestions about following qestions, thanks in advance ;-) Morita added network disk support for Libvirt, which supported nbd, rbd, sheepdog, and other protocols. Now, i wanna test network disk support for sheepdog solely. I have following questions. 1, Before this test, whether should i build the environment of sheepdog or other projects (QEMU)? 2, How could i test network disk support of sheepdog in Libvirt? Which steps should i do for it? 3, Network disk support can support online storage management, right? -- Thanks Harry Wei
At Fri, 1 Feb 2013 09:59:52 +0800, harryxiyou wrote:
On Fri, Feb 1, 2013 at 1:18 AM, Eric Blake <eblake@redhat.com> wrote: [...]
Libvirt exists to manage existing hypervisor capabilities. It makes no sense for libvirt to be able to create storage volumes for a new protocol if there are no hypervisors that are also able to support that storage format. So until qemu can target your new storage protocol, it doesn't make sense to have libvirt support it either.
Thanks, i see. Eric, if you have free time, could you please give me some suggestions about following qestions, thanks in advance ;-)
Morita added network disk support for Libvirt, which supported nbd, rbd, sheepdog, and other protocols. Now, i wanna test network disk support for sheepdog solely. I have following questions.
1, Before this test, whether should i build the environment of sheepdog or other projects (QEMU)?
You need to set up QEMU. Sheepdog is usable only from QEMU based virtual machines.
2, How could i test network disk support of sheepdog in Libvirt? Which steps should i do for it?
1. prepare a file containing an XML domain description $ cat > sheepdog.xml <domain type='qemu'> <name>testvm</name> <memory>1048576</memory> <os> <type arch='x86_64'>hvm</type> </os> <devices> <disk type='network'> <source protocol="sheepdog" name="testvdi"/> <target dev='hda' bus='ide'/> </disk> </devices> </domain> 2. boot from testvdi with virsh $ virsh create sheepdog.xml
3, Network disk support can support online storage management, right?
If you are talking about something like volume creation and deletion, the patch doesn't contain it. Thanks, Kazutaka
On Fri, Feb 1, 2013 at 12:06 PM, MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp> wrote: [...]
1. prepare a file containing an XML domain description
$ cat > sheepdog.xml <domain type='qemu'> <name>testvm</name> <memory>1048576</memory> <os> <type arch='x86_64'>hvm</type> </os> <devices> <disk type='network'> <source protocol="sheepdog" name="testvdi"/> <target dev='hda' bus='ide'/> </disk> </devices> </domain>
2. boot from testvdi with virsh
$ virsh create sheepdog.xml
Thanks, i have found this from our Sheepdog wiki page in Github. But i also have some questions about these steps, which if you have free time, please see my last mail it has sent to you named "Test boot VMs from sheepdog volumes in Libvirt".
3, Network disk support can support online storage management, right?
If you are talking about something like volume creation and deletion, the patch doesn't contain it.
Yup, i see, thanks very much. -- Thanks Harry Wei
participants (3)
-
Eric Blake -
harryxiyou -
MORITA Kazutaka