[Libvir] Proposal for the storage API (for discussion only)
This is an outline for the storage API, for discussion only. It is mostly unimplemented, but presented as a kind of "literate patch", so you can read the API calls and how they could be implemented. I've tried to follow as far as possible operations which are needed in order to make virt-install / virt-manager work in the remote case, and also to support discovery and creation of partitions / LVs / iSCSI LUNs. The new API calls are: virStorageFileLocations - returns a list of locations which can be used for creating file-backed guests virStorageFileStat virStorageFileStatVFS virStorageCreateFile - for creating file-backed guest storage virStorageListGroups virStorageCreateVolume - for discovering and creating partitions, LVs, LUNs, &c. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
John Levon wrote:
On Thu, Oct 18, 2007 at 06:11:48PM +0100, Richard W.M. Jones wrote:
virStorageListGroups virStorageCreateVolume - for discovering and creating partitions, LVs, LUNs, &c.
So a ZFS pool would be a "Group" in this API, and a zvol a 'Volume'?
I'm afraid I don't know enough about ZFS to answer this coherently. A better way would be for me to turn this around and ask _you_ if you think the concepts that I've proposed map on to ZFS :-? Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Thu, Oct 18, 2007 at 06:11:48PM +0100, Richard W.M. Jones wrote:
This is an outline for the storage API, for discussion only. It is mostly unimplemented, but presented as a kind of "literate patch", so you can read the API calls and how they could be implemented.
I've tried to follow as far as possible operations which are needed in order to make virt-install / virt-manager work in the remote case, and also to support discovery and creation of partitions / LVs / iSCSI LUNs.
The new API calls are:
virStorageFileLocations - returns a list of locations which can be used for creating file-backed guests
virStorageFileStat virStorageFileStatVFS virStorageCreateFile - for creating file-backed guest storage
virStorageListGroups virStorageCreateVolume - for discovering and creating partitions, LVs, LUNs, &c.
I don't really like any of this. We're basically going to end up with two parallel sets of storage APIs one operating on files the other operating on blocks. The APIs for creating volumes/files are not extensible since they're taking a fixed set of parameters. The means of enumerating storage areas doesn't provide for any extensible metdata. Virt-manager may currently handle these two concepts of block & files separately, but this is only because we had no choice due to lack of any storage API. I have long wanted to clean this up to just have the concept of storage pools & volumes within it. Both sets of APIs can easily be implemented in terms of the 2 concepts of a pool (a directory or LVM VG), and a volume (a file, or LVM LV or partition) As we did with the networking APIs we should try & maintain the style of existing APIs virStoragePoolListNames(virConnectPtr, char **names) virStoragePoolLookupByName(virConnectPtr, char *name) virStoragePoolLookupByUUID(virConnectPtr, unsigned char *uuid) /* Config for a pool - eg, a directory path /var/lib/xen/images, or an NFS server+export, or an iSCSI server + target */ virStoragePoolDefine(virConnectPtr, char *xml) /* Mount the NFS server, or login to iSCSI target, activate the LVM volume group */ virStoragePoolStart(virStoragePoolPtr) /* Unmount the NFS server, or logout of iSCSI, deactive the LVM volume group */ virStoragePoolDestroy(virStoragePoolPtr) /* Remove the config from libvirtd */ virStoragePoolUndefine(virStoragePoolPtr) char *virStoragePoolDumpXML(virStoragePoolPtr); long long virStoragePoolGetSize(virStoragePoolPtr); long long virStoragePoolGetFreeSpace(virStoragePoolPtr); virStorageVolListNames(virStoragePoolPtr, char **names) virStorageVolPtr virStorageVolLookupByName(virStoragePoolPtr, char *name) virStorageVolPtr virStorageVolLookupByUUID(virStoragePoolPtr, unsigned char *uuid) /* Search storage vols to find a vlume based on a path - eg to translate a guest disk into a storage volume */ virStorageVolPtr virStorageVolLookupByPath(virConnectPtr, char *path) /* Allocate a new volume in a pool, eg create a file, add an LVM logical volume, add a disk partition */ virStorageVolPtr virStorageVolCreate(virStoragePoolPtr, const char *xml) virStorageVolDestroy(virStorageVolPtr); char *virStorageVolDumpXML(virStorageVolPtr) /* The local path suitable for defining a disk in a guest */ char *virStoragePoolGetPath(virStorageVolPtr) long long virStorageVolGetSize(virStorageVolPtr); Even if we only implement this for LVM & local directories to start with, the API is capable of representing the other various types of storage outlined previously. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
Virt-manager may currently handle these two concepts of block & files separately, but this is only because we had no choice due to lack of any storage API. I have long wanted to clean this up to just have the concept of storage pools & volumes within it.
I did notice that there was an overlap between files and volumes once I started writing down the structures required. For example: struct virStorageGroup { int magic; /* Magic / structure version. */ char *name; /* Volume group name. */ int flags; /* Flags. */ unsigned long long size; /* Total size in kilobytes. */ unsigned long long avail; /* Available (free) space in kilobytes. */ }; struct virStorageStatVFSBuffer { unsigned long bsize; /* Block size. */ unsigned long long bavail; /* Blocks available. */ }; have a kind of common structure. I kept them separate for easy of retrofitting into virt-manager, but combining them is also a possibility. Another point which I thought about but forgot to put in the original email is what happens if you have (for example) LVM volume groups and iSCSI available. Then you need to do something like: list_groups = "(vgs | vgs2xml) <combine> (iscsiadm | iscsi2xml)" where <combine> is some sort of XML-combining operation. If we combine directories (for file-backed storage) with LVM VGs, then we'll also need this operation. OK, so your points are taken on board and I'll have another look at this tomorrow. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Thu, Oct 18, 2007 at 07:30:22PM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Virt-manager may currently handle these two concepts of block & files separately, but this is only because we had no choice due to lack of any storage API. I have long wanted to clean this up to just have the concept of storage pools & volumes within it.
I did notice that there was an overlap between files and volumes once I started writing down the structures required. For example:
struct virStorageGroup { int magic; /* Magic / structure version. */ char *name; /* Volume group name. */ int flags; /* Flags. */ unsigned long long size; /* Total size in kilobytes. */ unsigned long long avail; /* Available (free) space in kilobytes. */ };
struct virStorageStatVFSBuffer { unsigned long bsize; /* Block size. */ unsigned long long bavail; /* Blocks available. */ };
have a kind of common structure. I kept them separate for easy of retrofitting into virt-manager, but combining them is also a possibility.
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools.
Another point which I thought about but forgot to put in the original email is what happens if you have (for example) LVM volume groups and iSCSI available. Then you need to do something like:
list_groups = "(vgs | vgs2xml) <combine> (iscsiadm | iscsi2xml)"
where <combine> is some sort of XML-combining operation. If we combine directories (for file-backed storage) with LVM VGs, then we'll also need this operation.
This is just one reason why I don't like the idea of simply running shell scripts in the back end. This will result in a unreliable, hard-to debug system. It may be a short term win on implementation speed, but it will be a PITA for long term maintainence. One thing you will immediately get is people writing scripts which add all kinda of custom crap to the XML destroying the benefit of libvirt which is the standardization. For any given storage backend, we know what operations we need to be able to perform & so we have a finite set of commands we need to run with known predictable arguments. We just don't need the 'flexibility' of running arbitrary shell scripts & XML filters in the backend end. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools.
No, that doesn't make sense. XML for an API is a hack. It's hard to use it without requiring an additional external library to parse the XML. It's slow. It has the facade of maintaining ABI compatibility (because it's "just strings"), but in fact has no guaranteed ABI at all. It sits very poorly with static typing, virtually guaranteeing runtime errors that you are forced to do something about. See previous discussion about capabilities, also a hack: https://www.redhat.com/archives/libvir-list/2007-March/msg00104.html Notice that all the structures in my proposal are versioned:
struct virStorageGroup { int magic; /* Magic / structure version. */ char *name; /* Volume group name. */ int flags; /* Flags. */ unsigned long long size; /* Total size in kilobytes. */ unsigned long long avail; /* Available (free) space in kilobytes. */ };
and for structures which are passed in, the caller passes the structure size which allows both forwards and backwards compatibility:
* virStorageFileStat: * @conn: pointer to the hypervisor connection * @path: pathname of file * @statbuf: pointer to caller-allocated stat buffer (OUT) * @size: caller should set this to sizeof(*statbuf). <--- NB.
This is just one reason why I don't like the idea of simply running shell scripts in the back end. This will result in a unreliable, hard-to debug system. It may be a short term win on implementation speed, but it will be a PITA for long term maintainence. One thing you will immediately get is people writing scripts which add all kinda of custom crap to the XML destroying the benefit of libvirt which is the standardization. For any
I don't understand this. Scripts can be written which add custom fields to the XML, but since libvirt will just ignore those fields I don't see any issue.
given storage backend, we know what operations we need to be able to perform & so we have a finite set of commands we need to run with known predictable arguments. We just don't need the 'flexibility' of running arbitrary shell scripts & XML filters in the backend end.
We do because we need to be able to take the output of 'vgs', 'sfdisk', 'iscsiadm', etc., the output of each being essentially the same information (a list of volume groups plus the size and free space of each), and present that information in the same format back to libvirt. In this case a common format makes perfect sense. It need not be XML, it might be CSV, but in this case XML's extensibility makes sense, along with the fact that libvirt already parses XML. (Note for people snoozing through this email, we're talking about two different uses of XML). Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Fri, Oct 19, 2007 at 09:53:27AM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools.
No, that doesn't make sense. XML for an API is a hack. It's hard to
I disagree with you. XML is perfectly suitable for descriptions, especially when you need extendability and you can't control the future range of extensions. It's not proper for 'runtime' operations, but as a way to describe complex structures I find it fills its role perfectly.
use it without requiring an additional external library to parse the XML.
not an argument, "Security is a hack it requires external libraries to implement right" , doesn't work :-)
It's slow.
For description, one time operation is absolutely not a problem. Plus libxml2 parses a 20+MBytes/s with current processors.
It has the facade of maintaining ABI compatibility (because it's "just strings"), but in fact has no guaranteed ABI at all.
You can't guarantee proper processing of something you can't describe in any case. Hard versionning with id in your stuctures can't guarantee anymore, if the code doesn't handle that version you're out. Actually with XML you can still extract the part of the structure you understand. And you can similary put ids labels in the structure allowing to introduce a radical change like the difference between a Xen or KVM definition. I still maintain that using XML for description (domains and network) was the best approach which allowed to go forward 2 years ago even if most of what we use now is very different from what was available at the time. And in front of the wide variety of storage options, I guess it's still the best choise to describe how to reach the data you want to access. Once the attach has been done, sure XML would be a bad format for operations in most case. But for description, which is what Dan suggested I totally agree with him. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel Veillard wrote:
On Fri, Oct 19, 2007 at 09:53:27AM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools. No, that doesn't make sense. XML for an API is a hack. It's hard to
I disagree with you. XML is perfectly suitable for descriptions, especially when you need extendability and you can't control the future range of extensions. It's not proper for 'runtime' operations, but as a way to describe complex structures I find it fills its role perfectly.
These are not complex structures. It's a list of volumes, and each volume has 3 or 4 attributes (name, total size, free space, and a few flags). A simple array of C structures can describe that.
use it without requiring an additional external library to parse the XML.
not an argument, "Security is a hack it requires external libraries to implement right" , doesn't work :-)
This is not a relevant comment.
It's slow.
For description, one time operation is absolutely not a problem. Plus libxml2 parses a 20+MBytes/s with current processors.
It has the facade of maintaining ABI compatibility (because it's "just strings"), but in fact has no guaranteed ABI at all.
You can't guarantee proper processing of something you can't describe in any case. Hard versionning with id in your stuctures can't guarantee anymore, if the code doesn't handle that version you're out.
I didn't explain this well, but the versioning is forwards compatible. The basic fields in the structure would still be there in future versions, and callers which did not understand the future version could still use the fields they understand. The presence of the updated version number would indicate (for future callers) that there are extra fields they can access appearing after the basic fields. Sorry for the poor original explanation of this. BTW, this is precisely analogous to using XML, because you'd always want to provide the original XML fields for old callers, but with extra fields for new callers to use. It's just a lot simpler to use C structures, and allows for type safety, and removes the error-prone runtime failures of XML (what if a basic field you thought should be there is missing from the XML?).
Actually with XML you can still extract the part of the structure you understand. And you can similary put ids labels in the structure allowing to introduce a radical change like the difference between a Xen or KVM definition.
I still maintain that using XML for description (domains and network) was the best approach which allowed to go forward 2 years ago even if most of what we use now is very different from what was available at the time.
And in front of the wide variety of storage options, I guess it's still the best choise to describe how to reach the data you want to access. Once the attach has been done, sure XML would be a bad format for operations in most case. But for description, which is what Dan suggested I totally agree with him.
Daniel
Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Fri, Oct 19, 2007 at 11:05:30AM +0100, Richard W.M. Jones wrote:
Daniel Veillard wrote:
On Fri, Oct 19, 2007 at 09:53:27AM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools. No, that doesn't make sense. XML for an API is a hack. It's hard to
I disagree with you. XML is perfectly suitable for descriptions, especially when you need extendability and you can't control the future range of extensions. It's not proper for 'runtime' operations, but as a way to describe complex structures I find it fills its role perfectly.
These are not complex structures. It's a list of volumes, and each volume has 3 or 4 attributes (name, total size, free space, and a few flags).
They are not complex /currently/ because they are missing a lot of the metadata we will need for a complete solution. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
On Fri, Oct 19, 2007 at 11:05:30AM +0100, Richard W.M. Jones wrote:
Daniel Veillard wrote:
On Fri, Oct 19, 2007 at 09:53:27AM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools. No, that doesn't make sense. XML for an API is a hack. It's hard to I disagree with you. XML is perfectly suitable for descriptions, especially when you need extendability and you can't control the future range of extensions. It's not proper for 'runtime' operations, but as a way to describe complex structures I find it fills its role perfectly. These are not complex structures. It's a list of volumes, and each volume has 3 or 4 attributes (name, total size, free space, and a few flags).
They are not complex /currently/ because they are missing a lot of the metadata we will need for a complete solution.
Such as ...? Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Fri, Oct 19, 2007 at 09:53:27AM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools.
No, that doesn't make sense. XML for an API is a hack. It's hard to use it without requiring an additional external library to parse the XML. It's slow. It has the facade of maintaining ABI compatibility (because it's "just strings"), but in fact has no guaranteed ABI at all. It sits very poorly with static typing, virtually guaranteeing runtime errors that you are forced to do something about.
The speed is irrelevant in the context we're using this. Neither creating nor getting metadata about storage volumes are performance critical methods. Parsing XML is linenoise when creating a volume in comparison to the actual time required to allocate the volume itself.
See previous discussion about capabilities, also a hack: https://www.redhat.com/archives/libvir-list/2007-March/msg00104.html
The use of XML for describing metadata about managed objects is a core aspect of libvirt. We can debate use of XML vs structures for hours on end, but the fact remains that libvirt uses XML for its metadata APIs.
This is just one reason why I don't like the idea of simply running shell scripts in the back end. This will result in a unreliable, hard-to debug system. It may be a short term win on implementation speed, but it will be a PITA for long term maintainence. One thing you will immediately get is people writing scripts which add all kinda of custom crap to the XML destroying the benefit of libvirt which is the standardization. For any
I don't understand this. Scripts can be written which add custom fields to the XML, but since libvirt will just ignore those fields I don't see any issue.
given storage backend, we know what operations we need to be able to perform & so we have a finite set of commands we need to run with known predictable arguments. We just don't need the 'flexibility' of running arbitrary shell scripts & XML filters in the backend end.
We do because we need to be able to take the output of 'vgs', 'sfdisk', 'iscsiadm', etc., the output of each being essentially the same information (a list of volume groups plus the size and free space of each), and present that information in the same format back to libvirt. In this case a common format makes perfect sense. It need not be XML, it might be CSV, but in this case XML's extensibility makes sense, along with the fact that libvirt already parses XML.
This is just pushing the parsing code out of libvirt and into an external application. I am saying that libvirt should deal with it directly, since this is the core job of the driver APIs. libvirt should call 'vgs' directly and parse its output directly. Adding extra layers in the stack which transform vgs output into some further intermediate form is not going to make this easy to debug. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
On Fri, Oct 19, 2007 at 09:53:27AM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools. No, that doesn't make sense. XML for an API is a hack. It's hard to use it without requiring an additional external library to parse the XML. It's slow. It has the facade of maintaining ABI compatibility (because it's "just strings"), but in fact has no guaranteed ABI at all. It sits very poorly with static typing, virtually guaranteeing runtime errors that you are forced to do something about.
The speed is irrelevant in the context we're using this. Neither creating nor getting metadata about storage volumes are performance critical methods. Parsing XML is linenoise when creating a volume in comparison to the actual time required to allocate the volume itself.
So speed doesn't matter. What about my other points? XML requires extra libraries and efforts to parse something that simply does not need to be parsed. It could be presented directly as structures. XML sits poorly with static typing. XML introduces possible run-time errors that have to be dealt with by callers. (eg. Basic fields which you expect to be there, but for some reason are not).
See previous discussion about capabilities, also a hack: https://www.redhat.com/archives/libvir-list/2007-March/msg00104.html
The use of XML for describing metadata about managed objects is a core aspect of libvirt. We can debate use of XML vs structures for hours on end, but the fact remains that libvirt uses XML for its metadata APIs.
XML is used in a few places: As a way to describe domains (in & out) and for capabilities (out). The original capabilities proposal didn't use XML -- I'm not proud that I folded on that one. virConnectGetCapabilities returns sprawling XML that differs in ways we don't really understand between hypervisors. We have avoided defining real meanings to much of the capabilities XML. This is not a good example to follow. If this is going to be decided by some sort of popularity contest (`libvirt uses XML already so why not use a bit more'), then how about we look at other comparable C libraries. How many of them return XML from API calls, that is when they're not returning HTML documents or the like?
This is just one reason why I don't like the idea of simply running shell scripts in the back end. This will result in a unreliable, hard-to debug system. It may be a short term win on implementation speed, but it will be a PITA for long term maintainence. One thing you will immediately get is people writing scripts which add all kinda of custom crap to the XML destroying the benefit of libvirt which is the standardization. For any I don't understand this. Scripts can be written which add custom fields to the XML, but since libvirt will just ignore those fields I don't see any issue.
given storage backend, we know what operations we need to be able to perform & so we have a finite set of commands we need to run with known predictable arguments. We just don't need the 'flexibility' of running arbitrary shell scripts & XML filters in the backend end. We do because we need to be able to take the output of 'vgs', 'sfdisk', 'iscsiadm', etc., the output of each being essentially the same information (a list of volume groups plus the size and free space of each), and present that information in the same format back to libvirt. In this case a common format makes perfect sense. It need not be XML, it might be CSV, but in this case XML's extensibility makes sense, along with the fact that libvirt already parses XML.
This is just pushing the parsing code out of libvirt and into an external application. I am saying that libvirt should deal with it directly, since this is the core job of the driver APIs. libvirt should call 'vgs' directly and parse its output directly. Adding extra layers in the stack which transform vgs output into some further intermediate form is not going to make this easy to debug.
I think this is fair enough. The important part is to make sure that the sysadmin can configure it, and it doesn't make much difference whether we use scriptlets or just have configuration options. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Fri, Oct 19, 2007 at 01:35:25PM +0100, Richard W.M. Jones wrote:
This is just one reason why I don't like the idea of simply running shell scripts in the back end. This will result in a unreliable, hard-to debug system. It may be a short term win on implementation speed, but it will be a PITA for long term maintainence. One thing you will immediately get is people writing scripts which add all kinda of custom crap to the XML destroying the benefit of libvirt which is the standardization. For any I don't understand this. Scripts can be written which add custom fields to the XML, but since libvirt will just ignore those fields I don't see any issue.
given storage backend, we know what operations we need to be able to perform & so we have a finite set of commands we need to run with known predictable arguments. We just don't need the 'flexibility' of running arbitrary shell scripts & XML filters in the backend end. We do because we need to be able to take the output of 'vgs', 'sfdisk', 'iscsiadm', etc., the output of each being essentially the same information (a list of volume groups plus the size and free space of each), and present that information in the same format back to libvirt. In this case a common format makes perfect sense. It need not be XML, it might be CSV, but in this case XML's extensibility makes sense, along with the fact that libvirt already parses XML.
This is just pushing the parsing code out of libvirt and into an external application. I am saying that libvirt should deal with it directly, since this is the core job of the driver APIs. libvirt should call 'vgs' directly and parse its output directly. Adding extra layers in the stack which transform vgs output into some further intermediate form is not going to make this easy to debug.
I think this is fair enough. The important part is to make sure that the sysadmin can configure it, and it doesn't make much difference whether we use scriptlets or just have configuration options.
It depends on the level of configuration. I don't want the sysadmin to be configuring command line args directly. For any input XML metadata to the libvirt API, we should always run a pre-determined command argv. This is important to ensure that the same storage description results in the same underlying operations no matter what machine it is run on. This ensures that applications have a consistent API, and that people debugging bug reports don't have to worry about local modifications to the API's meaning. The sysadmin can define the configure the storage pools they wish to have available via the APIs we provide directly. Now, if we want the sys-admin to be able to provide specific command line tools, for storage then I'd suggest we add an explicit 'site local' storage pool type. A 'site local' pool type would simple hand all the operations off to sysadmin configured commands. This way, if they use the built in lvm, or file, or iscsi, or block device backends we know exactly what the results are going to be. If they use the 'site local' backend we know to expect the unexpected. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
On Fri, Oct 19, 2007 at 01:35:25PM +0100, Richard W.M. Jones wrote:
I think this is fair enough. The important part is to make sure that the sysadmin can configure it, and it doesn't make much difference whether we use scriptlets or just have configuration options.
It depends on the level of configuration. I don't want the sysadmin to be configuring command line args directly. For any input XML metadata to the libvirt API, we should always run a pre-determined command argv. This is important to ensure that the same storage description results in the same underlying operations no matter what machine it is run on. This ensures that applications have a consistent API, and that people debugging bug reports don't have to worry about local modifications to the API's meaning. The sysadmin can define the configure the storage pools they wish to have available via the APIs we provide directly.
Now, if we want the sys-admin to be able to provide specific command line tools, for storage then I'd suggest we add an explicit 'site local' storage pool type. A 'site local' pool type would simple hand all the operations off to sysadmin configured commands.
This way, if they use the built in lvm, or file, or iscsi, or block device backends we know exactly what the results are going to be. If they use the 'site local' backend we know to expect the unexpected.
Sure ... What I had in mind what that the sys admin should be able to enable and disable different back-ends, but the actual command line being used would be hidden within libvirt's C code. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Fri, Oct 19, 2007 at 01:35:25PM +0100, Richard W.M. Jones wrote:
See previous discussion about capabilities, also a hack: https://www.redhat.com/archives/libvir-list/2007-March/msg00104.html
The use of XML for describing metadata about managed objects is a core aspect of libvirt. We can debate use of XML vs structures for hours on end, but the fact remains that libvirt uses XML for its metadata APIs.
XML is used in a few places: As a way to describe domains (in & out) and for capabilities (out). The original capabilities proposal didn't use XML -- I'm not proud that I folded on that one. virConnectGetCapabilities returns sprawling XML that differs in ways we don't really understand between hypervisors. We have avoided defining real meanings to much of the capabilities XML. This is not a good example to follow.
If this is going to be decided by some sort of popularity contest (`libvirt uses XML already so why not use a bit more'), then how about we look at other comparable C libraries. How many of them return XML from API calls, that is when they're not returning HTML documents or the like?
IMHO, consistent style to our APIs is important. As you say we already use XML for describing domains, virtual networks, and capabilities. Introducing a different approach just for describing storage just does not make sense to me at all, thus I want storage to be described using XML in the immediate implementation. That said I do recognise that there are some benefits to having APIs that operate in terms of statically typed structures. I am open to discussing how we could introduce new APIs equivalent to the existing create & dumpxml APIs - for *all* objects we represent. I think this should be a effort that is separate / in parallel with the addition of storage management APIs so we don't block implementation of the storage. If we look at the QEMU driver we already have structs to hold the working representation of a domain and a network. The 'test' driver also has its own structs. With some appropriate adjustments we could potentially make the QEMU structs generic enough that we'd be able to expose them via the public API. This could open up some interesting possibilities for reducing the amount of code in the internal drivers. Current we have - XM config -> XML parser - XML -> XM config parser - SEXPR -> XML parser - XML -> SEXPR parser - qemu struct -> XML formatter - XML -> qemu struct parser - test struct -> XML formatter - XML -> test struct parser As a first step, we could switch to using the 'qemu' structs inside all the drivers as the working representations. This would result in a single parser and single formatter across all drivers. We'd then just need to do a SEXPR to struct convertor & XM config to struct convertor. Once the internals were unified, we'd have a higher degree of confidience that our struct design was sufficiently flexible for all drivers. Though it would be nice to have one virt backend which wasn't styled on the QEMU device model - VMWare potentially hsa a quite different set of metadata. Anyway, if we had the internals all unified on a single struct design, then we could make the next step & expose virDomainCreateStruct(virDomainStructPtr) and virDomainStructPtr virDomainGetDescription() as equivalents for the existing virDomainCreate and virDOmainDumpXML methods - likewise for the storage, networking and capabilities APIs. Personally I'd like to do the unification of all the drivers to use the same structs internally anyway. I think it would result in more straight forward code & actually make it quicker & easier to write new backends. The vast majority of the work in backends is doing the XML parser and formatter. If we have a single intenral struct representation, adding a UserModeLinux driver could reuse nearly all of the QEMU driver code as is. All we would end is a new method to build up the ARGV for spawning the UML kernels. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Fri, 2007-10-19 at 09:53 +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Using structures in the public API is not in keeping with the rest of the libvirt APIs. We should be using XML for the main metadata description of volumes & pools.
No, that doesn't make sense. XML for an API is a hack. It's hard to use it without requiring an additional external library to parse the XML. It's slow. It has the facade of maintaining ABI compatibility (because it's "just strings"), but in fact has no guaranteed ABI at all. It sits very poorly with static typing, virtually guaranteeing runtime errors that you are forced to do something about.
See previous discussion about capabilities, also a hack: https://www.redhat.com/archives/libvir-list/2007-March/msg00104.html
I've made a similar argument in the past, but this XML API is a distinctive part of the libvrt API and IMHO it makes sense for the storage pool API to follow the same model. That's e.g. why the network API went with the XML API approach too. Cheers, Mark.
This is just one reason why I don't like the idea of simply running shell scripts in the back end. This will result in a unreliable, hard-to debug system. It may be a short term win on implementation speed, but it will be a PITA for long term maintainence. One thing you will immediately get is people writing scripts which add all kinda of custom crap to the XML destroying the benefit of libvirt which is the standardization. For any
I don't understand this. Scripts can be written which add custom fields to the XML, but since libvirt will just ignore those fields I don't see any issue.
given storage backend, we know what operations we need to be able to perform & so we have a finite set of commands we need to run with known predictable arguments. We just don't need the 'flexibility' of running arbitrary shell scripts & XML filters in the backend end.
We do because we need to be able to take the output of 'vgs', 'sfdisk', 'iscsiadm', etc., the output of each being essentially the same information (a list of volume groups plus the size and free space of each), and present that information in the same format back to libvirt. In this case a common format makes perfect sense. It need not be XML, it might be CSV, but in this case XML's extensibility makes sense, along with the fact that libvirt already parses XML.
(Note for people snoozing through this email, we're talking about two different uses of XML).
(Sorry if irrelevant...) Because "The XML should be mentioned minimum management information", I think that it is better to be only frontdev, backdev, type and a kind of the device as now. Information except it should be examined with these as materials anytime by storage API. ( May not matter, When I'm thinking about "Bugzilla Bug 329091: [5.2]The disk which the OS is in...", I thought that "storage API" was such a thing...) Shigeki Sakamoto.
participants (6)
-
Daniel P. Berrange -
Daniel Veillard -
John Levon -
Mark McLoughlin -
Richard W.M. Jones -
S.Sakamoto