[libvirt] [PATCHv2 0/2] storage backend support matrix

Hello, Here is an updated version of the previous Storage backend support matrix in the doc. As discussed with Michal, the matrix is now part of hvsupport.html with a paragraph and link from storage.html. This has now been turned into a series to fix the links in TOC of the storage page Cédric Bosdonnat (2): Fix hvsupport toc Generate status of the backend implementation in hvsupport.html .gitignore | 1 + docs/Makefile.am | 12 +++++-- docs/apibuild.py | 2 ++ docs/hvsupport.pl | 4 ++- docs/storage.html.in | 13 ++++++++ docs/storagebackendstatus.py | 63 ++++++++++++++++++++++++++++++++++++ 6 files changed, 91 insertions(+), 4 deletions(-) create mode 100644 docs/storagebackendstatus.py -- 2.22.0

--- docs/hvsupport.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hvsupport.pl b/docs/hvsupport.pl index 4f4d86fbf1..494b8a27ec 100755 --- a/docs/hvsupport.pl +++ b/docs/hvsupport.pl @@ -364,7 +364,7 @@ when it was removed is also mentioned (highlighted in EOF foreach my $grp (sort { $a cmp $b } keys %groups) { - print "<h2><a name=\"$grp\">", $groupheaders{$grp}, "</a></h2>\n"; + print "<h2><a id=\"$grp\">", $groupheaders{$grp}, "</a></h2>\n"; print <<EOF; <table class="top_table"> <thead> -- 2.22.0

Since it helps a user to know which of the storage backends support what operation, include an autogenerated matrix showing it in the docs. --- .gitignore | 1 + docs/Makefile.am | 12 +++++-- docs/apibuild.py | 2 ++ docs/hvsupport.pl | 2 ++ docs/storage.html.in | 13 ++++++++ docs/storagebackendstatus.py | 63 ++++++++++++++++++++++++++++++++++++ 6 files changed, 90 insertions(+), 3 deletions(-) create mode 100644 docs/storagebackendstatus.py diff --git a/.gitignore b/.gitignore index 82495e8692..06875abebd 100644 --- a/.gitignore +++ b/.gitignore @@ -58,6 +58,7 @@ /configure.lineno /conftest.* /docs/aclperms.htmlinc +/docs/storagebackendstatus.htmlinc /docs/apibuild.py.stamp /docs/devhelp/libvirt.devhelp /docs/hvsupport.html.in diff --git a/docs/Makefile.am b/docs/Makefile.am index 1cdb584b0b..3cf114d9e1 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -195,6 +195,8 @@ EXTRA_DIST= \ $(kbase_html_in) $(kbase_html) \ aclperms.htmlinc \ hvsupport.pl \ + storagebackendstatus.py \ + storagebackendstatus.htmlinc \ $(schema_DATA) acl_generated = aclperms.htmlinc @@ -209,7 +211,8 @@ MAINTAINERCLEANFILES = \ $(addprefix $(srcdir)/,$(devhelphtml)) \ $(addprefix $(srcdir)/,$(internals_html)) \ $(addprefix $(srcdir)/,$(kbase_html)) \ - $(srcdir)/hvsupport.html.in $(srcdir)/aclperms.htmlinc + $(srcdir)/hvsupport.html.in $(srcdir)/aclperms.htmlinc \ + $(srcdir)/storagebackendstatus.htmlinc timestamp="$(shell if test -n "$$SOURCE_DATE_EPOCH"; \ then \ @@ -228,7 +231,10 @@ admin_api: $(srcdir)/libvirt-admin-api.xml $(srcdir)/libvirt-admin-refs.xml web: $(dot_html) $(internals_html) $(kbase_html) \ html/index.html devhelp/index.html -hvsupport.html: $(srcdir)/hvsupport.html.in +hvsupport.html: $(srcdir)/hvsupport.html.in $(srcdir)/storagebackendstatus.htmlinc + +$(srcdir)/storagebackendstatus.htmlinc: $(srcdir)/storagebackendstatus.py + $(PYTHON) $< $(top_srcdir)/src >$@ $(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \ $(top_srcdir)/src/libvirt_public.syms \ @@ -256,7 +262,7 @@ MAINTAINERCLEANFILES += \ convert -rotate 90 $< $@ %.html.tmp: %.html.in site.xsl subsite.xsl page.xsl \ - $(acl_generated) + $(acl_generated) storagebackendstatus.htmlinc $(AM_V_GEN)name=`echo $@ | sed -e 's/.tmp//'`; \ dir=`dirname $@` ; \ if test "$$dir" = "."; \ diff --git a/docs/apibuild.py b/docs/apibuild.py index dbdc1c95af..31944b8176 100755 --- a/docs/apibuild.py +++ b/docs/apibuild.py @@ -2010,6 +2010,8 @@ class docBuilder: self.includes = includes + list(lxc_included_files.keys()) elif name == "libvirt-admin": self.includes = includes + list(admin_included_files.keys()) + else: + self.includes = includes self.modules = {} self.headers = {} self.idx = index() diff --git a/docs/hvsupport.pl b/docs/hvsupport.pl index 494b8a27ec..14e2336018 100755 --- a/docs/hvsupport.pl +++ b/docs/hvsupport.pl @@ -453,6 +453,8 @@ EOF } print <<EOF; +<h2><a id="storageBackend">Storage Backends</a></h2> +<div id="include" filename="storagebackendstatus.htmlinc"/> </body> </html> EOF diff --git a/docs/storage.html.in b/docs/storage.html.in index e0e4edec1e..db39f7bacf 100644 --- a/docs/storage.html.in +++ b/docs/storage.html.in @@ -826,5 +826,18 @@ <h3>Valid volume format types</h3> <p>The valid volume types are the same as for the directory pool.</p> + + <h2>Storage Pool Types implementation status</h2> + + <p> + The storage backends have different level of support of the various pool and volume actions. + See the <a href="hvsupport.html">hypervisor support</a> page to find a support matrix. + </p> + + <p> + <strong>Note:</strong> some functions like Start and Stop will not trigger an exception when + called on a backend that doesn't implement them. + </p> + </body> </html> diff --git a/docs/storagebackendstatus.py b/docs/storagebackendstatus.py new file mode 100644 index 0000000000..1a28a59b27 --- /dev/null +++ b/docs/storagebackendstatus.py @@ -0,0 +1,63 @@ +import os +import os.path +import re +import sys + +def get_allowed_functions(srcdir): + functions = [] + with open(os.path.join(srcdir, 'storage', 'storage_backend.h'), 'r') as handle: + content = ''.join(handle.readlines()) + definition = re.search('struct _virStorageBackend {([^}]+)}', content) + if definition is not None: + functions = re.findall('virStorageBackend[^ ]+ ([^;]+)', definition.group(1)) + return functions + +class Backend: + def __init__(self, name, code): + self.name = name + self.functions = [member[1:] for member in re.findall('.([^ ]+) = ', code) if member != '.type'] + +def get_backends(srcdir): + backends = [] + for root, dirs, files in os.walk(os.path.join(srcdir, 'storage')): + storage_impls = [os.path.join(root, f) for f in files if re.match('storage_backend_[^.]+.c', f)] + for impl in storage_impls: + handle = open(impl, 'r') + content = ''.join(handle.readlines()) + handle.close() + chunks = re.findall('virStorageBackend virStorageBackend([^ ]+) = {([^}]*)}', content) + backends.extend([Backend(chunk[0], chunk[1]) for chunk in chunks]) + return backends + +def main(srcdir): + functions = get_allowed_functions(srcdir) + backends = get_backends(srcdir) + + headers = '\n'.join(['<th>%s</th>' % backend.name for backend in backends]) + rows = [] + for func in functions: + cell_template = '<td style="text-align: center">%s</td>' + support = [cell_template % ('✔' if func in backend.functions else '') for backend in backends] + rows.append('\n'.join(['<tr>', '<td>%s</td>' % func] + support + ['</tr>'])) + + print('''<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> +<body> +<table class='top_table'> +<thead> +<tr> +<th> +</th> +%s +</tr> +</thead> +<tbody> +%s +</tbody> +</table> +</body> +</html>''' % (headers, '\n'.join(rows))) + +if __name__ == '__main__': + main(sys.argv[1]) -- 2.22.0

On Thu, Aug 29, 2019 at 04:03:33PM +0000, Cedric Bosdonnat wrote:
Hello,
Here is an updated version of the previous Storage backend support matrix in the doc. As discussed with Michal, the matrix is now part of hvsupport.html with a paragraph and link from storage.html.
This has now been turned into a series to fix the links in TOC of the storage page
Cédric Bosdonnat (2): Fix hvsupport toc Generate status of the backend implementation in hvsupport.html
.gitignore | 1 + docs/Makefile.am | 12 +++++-- docs/apibuild.py | 2 ++ docs/hvsupport.pl | 4 ++- docs/storage.html.in | 13 ++++++++ docs/storagebackendstatus.py | 63 ++++++++++++++++++++++++++++++++++++
I can't really say I'm a fan of implementing this via a second python script, as opposed to just putting the logic into the existing hvsupport.pl script directly. I can totally understand though if the idea of writing Perl code was not appealing :-) FWIW, as the person responsible for much of the Perl code in libvirt, I think we should make it an explicit TODO item to convert all our Perl code into Python 3 code. Given that we're likely to incorporate some new languages like Rust and/or Go code in libvirt, we definitely want to try to cull some other languages we use. Perl is the obvious one. The autotools mix of autoconf/shell/m4/make languages are the other clear candidates, to replace with meson's domain specific language. I'd also suggest our docs/ directory be changed to use either RST or Markdown, instead of XSL. RST (Sphinx) is likely a good idea, given that this would give us alignment with what QEMU is using for docs. This might in turn suggest we replace POD manpages with RST too. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Thu, Aug 29, 2019 at 06:03:46PM +0100, Daniel P. Berrangé wrote:
On Thu, Aug 29, 2019 at 04:03:33PM +0000, Cedric Bosdonnat wrote:
Hello,
Here is an updated version of the previous Storage backend support matrix in the doc. As discussed with Michal, the matrix is now part of hvsupport.html with a paragraph and link from storage.html.
This has now been turned into a series to fix the links in TOC of the storage page
Cédric Bosdonnat (2): Fix hvsupport toc Generate status of the backend implementation in hvsupport.html
.gitignore | 1 + docs/Makefile.am | 12 +++++-- docs/apibuild.py | 2 ++ docs/hvsupport.pl | 4 ++- docs/storage.html.in | 13 ++++++++ docs/storagebackendstatus.py | 63 ++++++++++++++++++++++++++++++++++++
I can't really say I'm a fan of implementing this via a second python script, as opposed to just putting the logic into the existing hvsupport.pl script directly.
I can totally understand though if the idea of writing Perl code was not appealing :-)
FWIW, as the person responsible for much of the Perl code in libvirt, I think we should make it an explicit TODO item to convert all our Perl code into Python 3 code.
Please do make it explicit by proposing a patch against the HACKING page with human-readable justification (i.e. not just ASCII art of animals)
Given that we're likely to incorporate some new languages like Rust and/or Go code in libvirt, we definitely want to try to cull some
and or or? RE: your rewrite of virt-host-validate in Go, I feel like getting that merged (even in a separate repo) is a precedent for further sunsetting in case 'or' becomes the preferred conjunction there. Jano
other languages we use. Perl is the obvious one. The autotools mix of autoconf/shell/m4/make languages are the other clear candidates, to replace with meson's domain specific language.
participants (3)
-
Cedric Bosdonnat
-
Daniel P. Berrangé
-
Ján Tomko