Currently, building the NEWS file involves using a XSLT stylesheet to extract information from the same HTML file that's used on the libvirt website. The process works, but it's quite fiddly in that it requires the source HTML to be formatted in a very precise way, and a single missing newline can mess up the resulting plain text considerably. Moreover, the XSLT stylesheet itself encodes a lot of the details of converting to plain text in a way that's not necessarily easy to understand, tweak or fix. To improve the process, move all existing entries to a new XML file that contains exactly the information we care about in a simple structured format, and start generating both the HTML and plain text versions of the release notes using XSLT stylesheets that can now afford to be almost trivial. --- With this patch applied, the generated NEWS file is actually much worse off. That's fixed by the next one. .gitignore | 1 + Makefile.am | 6 +- docs/Makefile.am | 20 ++++- docs/NEWS.xsl | 61 ++++++++++++++ docs/news.html.in | 157 ----------------------------------- docs/news.xml | 241 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ docs/news.xsl | 103 ++++++++++++++++------- 7 files changed, 395 insertions(+), 194 deletions(-) create mode 100644 docs/NEWS.xsl delete mode 100644 docs/news.html.in create mode 100644 docs/news.xml diff --git a/.gitignore b/.gitignore index 984ad07..7b71bd1 100644 --- a/.gitignore +++ b/.gitignore @@ -72,6 +72,7 @@ /docs/libvirt-lxc-*.xml /docs/libvirt-qemu-*.xml /docs/libvirt-refs.xml +/docs/news.html.in /docs/search.php /docs/todo.html.in /examples/admin/client_close diff --git a/Makefile.am b/Makefile.am index 50c358c..445e35e 100644 --- a/Makefile.am +++ b/Makefile.am @@ -46,10 +46,10 @@ EXTRA_DIST = \ pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = libvirt.pc libvirt-qemu.pc libvirt-lxc.pc libvirt-admin.pc -NEWS: $(top_srcdir)/docs/news.xsl $(top_srcdir)/docs/news.html.in +NEWS: $(srcdir)/docs/NEWS.xsl $(srcdir)/docs/news.xml $(AM_V_GEN)if [ -x $(XSLTPROC) ] ; then \ - $(XSLTPROC) --nonet $(top_srcdir)/docs/news.xsl \ - $(top_srcdir)/docs/news.html.in \ + $(XSLTPROC) --nonet $(srcdir)/docs/NEWS.xsl \ + $(srcdir)/docs/news.xml \ | perl -0777 -pe 's/\n\n+$$/\n/' \ | perl -pe 's/[ \t]+$$//' \ diff --git a/docs/Makefile.am b/docs/Makefile.am index 790c0a2..6c1eb1f 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -106,9 +106,10 @@ internals_html = $(internals_html_in:%.html.in=%.html) # Since we ship pre-built html in the tarball, we must also # ship the sources, even when those sources are themselves # generated. -# Generate hvsupport.html first, since it takes one extra step. +# Generate hvsupport.html and news.html first, since they take one extra step. dot_html_in = \ hvsupport.html.in \ + news.html.in \ $(notdir $(wildcard $(srcdir)/*.html.in)) dot_html = $(dot_html_in:%.html.in=%.html) @@ -156,7 +157,7 @@ schema_DATA = $(wildcard $(srcdir)/schemas/*.rng) EXTRA_DIST= \ apibuild.py genaclperms.pl \ - site.xsl subsite.xsl newapi.xsl news.xsl page.xsl \ + site.xsl subsite.xsl newapi.xsl NEWS.xsl news.xsl page.xsl \ hacking1.xsl hacking2.xsl wrapstring.xsl \ $(dot_html) $(dot_html_in) $(gif) $(apihtml) $(apipng) \ $(devhelphtml) $(devhelppng) $(devhelpcss) $(devhelpxsl) \ @@ -179,7 +180,7 @@ MAINTAINERCLEANFILES = \ $(addprefix $(srcdir)/,$(devhelphtml)) \ $(addprefix $(srcdir)/,$(internals_html)) \ $(addprefix $(srcdir)/,$(dot_php)) \ - $(srcdir)/hvsupport.html.in $(srcdir)/aclperms.htmlinc + $(srcdir)/hvsupport.html.in $(srcdir)/news.html.in $(srcdir)/aclperms.htmlinc all-am: web @@ -200,6 +201,19 @@ $(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \ $(AM_V_GEN)$(PERL) $(srcdir)/hvsupport.pl $(top_srcdir)/src > $@ \ || { rm $@ && exit 1; } +# xsltproc seems to add the xmlns="" attribute to random output elements: +# use sed to strip it out, as leaving it there triggers XML errors during +# further transformation steps +news.html.in: $(srcdir)/news.xsl $(srcdir)/news.xml + $(AM_V_GEN) \ + if [ -x $(XSLTPROC) ]; then \ + $(XSLTPROC) --nonet $(srcdir)/news.xsl $(srcdir)/news.xml >$@-tmp \ + || { rm -f $@-tmp; exit 1; }; \ + sed 's/ xmlns=""//g' $@-tmp >$@ \ + || { rm -f $@-tmp; exit 1; }; \ + rm -f $@-tmp; \ + fi + %.png: %.fig convert -rotate 90 $< $@ diff --git a/docs/NEWS.xsl b/docs/NEWS.xsl new file mode 100644 index 0000000..0f1ca2a --- /dev/null +++ b/docs/NEWS.xsl @@ -0,0 +1,61 @@ +<?xml version="1.0"?> +<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + <xsl:output method="text" encoding="UTF-8"/> + + <!-- This XSLT stylesheet can be applied to the XML version of the release + notes to produce a plain text document. The output document is not + formatted properly and needs to be processed further --> + + <!-- Document --> + <xsl:template match="/releases"> + <xsl:text>libvirt releases +================ +</xsl:text> + <xsl:apply-templates select="release"/> + </xsl:template> + + <!-- Release --> + <xsl:template match="release"> + <xsl:text> +# </xsl:text> + <xsl:value-of select="@version"/> + <xsl:text> (</xsl:text> + <xsl:value-of select="@date"/> + <xsl:text>) +</xsl:text> + <xsl:apply-templates select="section"/> + </xsl:template> + + <!-- Section --> + <xsl:template match="section"> + <xsl:text> +* </xsl:text> + <xsl:value-of select="@title"/> + <xsl:text> +</xsl:text> + <xsl:apply-templates select="item"/> + </xsl:template> + + <!-- Item --> + <xsl:template match="item"> + <xsl:text> +- </xsl:text> + <xsl:apply-templates select="summary"/> + <xsl:apply-templates select="description"/> + </xsl:template> + + <!-- Item summary --> + <xsl:template match="summary"> + <xsl:value-of select="normalize-space()"/> + <xsl:text> +</xsl:text> + </xsl:template> + + <!-- Item description --> + <xsl:template match="description"> + <xsl:value-of select="normalize-space()"/> + <xsl:text> +</xsl:text> + </xsl:template> + +</xsl:stylesheet> diff --git a/docs/news.html.in b/docs/news.html.in deleted file mode 100644 index e544cfa..0000000 --- a/docs/news.html.in +++ /dev/null @@ -1,157 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml"> - <head> - <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> - </head> - <body> - <h1>Releases</h1> - <p>This is the list of official releases for libvirt, along with an - overview of the changes introduced by each of them.</p> - <p>For a more fine-grained view, use the - <a href="http://libvirt.org/git/?p=libvirt.git;a=log">git log</a>. - </p> - - <h3>v3.0.0 (<i>unreleased</i>)</h3> - <ul> - <li><strong>New features</strong> - <ul> - <li>New localPtr attribute for "ip" element in network XML - </li> - <li>qemu: Support QEMU group I/O throttling<br/> - Add the capability to allow group I/O throttling via a new - domain &lt;disk&gt; &lt;iotune&gt; subelement "group_name" - to allow sharing I/O throttling quota between multiple drives - </li> - <li>nss: Introduce <code>libvirt-guest</code><br/> - New <code>libvirt-guest</code> nss module that translates libvirt - guest names into IP addresses - </li> - <li>daemon: Add support for runtime logging settings adjustment<br/> - Logging-related settings like log outputs and filters can now be - adjusted during runtime using the admin interface without the - necessity of the daemon's restart - </li> - <li>storage: Add virStorageVolInfoFlags API<br/> - Add the API to support using the VIR_STORAGE_VOL_GET_PHYSICAL - flag in order to return the host physical size in bytes - of the image container in the allocation field of the - _virStorageVolInfo structure. The --physical flag has been - added to the virsh vol-info command to access the data - </li> - </ul> - </li> - <li><strong>Improvements</strong> - <ul> - <li>perf: Add more perf statistics<br/> - Add support to get the count of branch instructions - executed, branch misses, bus cycles, stalled frontend - cpu cycles, stalled backend cpu cycles, and ref cpu - cycles by applications running on the platform - </li> - <li>conf: Display &lt;physical&gt; for volume xml<br/> - Add a display of the &lt;physical&gt; size of a disk - volume in the output of the volume XML - </li> - </ul> - </li> - <li><strong>Bug fixes</strong> - <ul> - <li>qemu: Correct GetBlockInfo values<br/> - For an active domain, correct the physical value provided for - a raw sparse file backed storage and the allocation value provided - for a qcow2 file backed storage that hasn't yet been opened on - the domain - </li> - <li>qemu: Make virtio console usable on ppc64 guests<br/> - The chardev detection code has been improved and can now handle this - configuration properly - </li> - <li>qemu: Enable mount namespace<br/> - To avoid funny races with udev relabelling devices under our hands and - to enhance security, libvirt now spawns each qemu process with its own - <code>/dev</code>.</li> - </ul> - </li> - </ul> - - <h3>v2.5.0 (2016-12-04)</h3> - <ul> - <li><strong>New features</strong> - <ul> - <li>shmem: Add support for additional models<br/> - The shmem device can now utilize QEMU's ivshmem-plain and - ivshmem-doorbell, more modern versions of ivshmem - </li> - <li>vbox: Add VirtualBox 5.1 support - </li> - <li>libssh: New transport<br/> - The new libssh transport allows one to connect to a running - libvirtd via SSH, using the libssh library; for example: - <tt>qemu+libssh://<i>server</i>/system</tt> - </li> - <li>vhost-scsi: Add support scsi_host hostdev passthrough<br/> - Add the capability to pass through a scsi_host HBA and the - associated LUNs to the guest - </li> - <li>qemu: Users can now enable debug logging for native gluster - volumes in qemu using the "gluster_debug_level" option in qemu.conf - </li> - <li>memory hotplug: Slot numbers for memory devices are now - automatically allocated and thus persistent. In addition slot numbers - can be specified without providing a base address, which simplifies - user configuration - </li> - <li>qemu: Express devices will be placed on PCIe bus by default<br/> - For machine types that use a PCI Express root bus - (e.g. x86_64/Q35 and aarch64/virt), any unaddressed PCI - device that is an Express device (all virtio-1.0 devices, - e1000e, nec-xhci, vfio assigned devices) will be placed on - an Express controller (i.e. a pcie-root-port) instead of a - legacy PCI controller (i.e. pci-bridge) with the root ports - added as needed - </li> - </ul> - </li> - <li><strong>Improvements</strong> - <ul> - <li>docs: Better documentation for migration APIs and flags - </li> - <li>vbox: Address thread safety issues - </li> - <li>virsh: Add support for passing an alternative persistent XML - to migrate command - </li> - <li>vhostuser: Allow hotplug of multiqueue devices - </li> - <li>NEWS: Switch to an improved format<br/> - List user-visible changes instead of single commits for a better - high-level overview of differences between libvirt releases - </li> - <li>website: Modernize layout and branding<br/> - The libvirt website looked very cluttered and outdated; it has now - been completely overhauled, resulting in a design that's better - organized and more pleasant to look at - </li> - </ul> - </li> - <li><strong>Bug fixes</strong> - <ul> - <li>vz: Fix migration in P2P mode - </li> - <li>Forbid newline character in names of some libvirt objects - </li> - <li>Fix compilation on macOS - </li> - </ul> - </li> - </ul> - - <p>Releases earlier than v2.5.0 detailed their changes using a different - format and as such are excluded from the list above. - You can read about those older release, starting from those made in - <a href="news-2016.html">2016</a>. - </p> - - </body> -</html> diff --git a/docs/news.xml b/docs/news.xml new file mode 100644 index 0000000..fab74b2 --- /dev/null +++ b/docs/news.xml @@ -0,0 +1,241 @@ +<?xml version="1.0"?> + +<!-- libvirt release notes + + This file will be processed to produce both HTML and plain text versions + of the release notes. + + Please keep the style consistent with existing entries as much as + possible: each item should have a short, one-sentence summary and + optionally a description where the change is explained in more detail --> + +<releases> + <release version="v3.0.0" date="unreleased"> + <section title="New features"> + <item> + <summary> + New localPtr attribute for "ip" element in network XML + </summary> + </item> + <item> + <summary> + qemu: Support QEMU group I/O throttling + </summary> + <description> + Add the capability to allow group I/O throttling via a new + domain &lt;disk&gt; &lt;iotune&gt; subelement "group_name" + to allow sharing I/O throttling quota between multiple drives + </description> + </item> + <item> + <summary> + nss: Introduce <code>libvirt-guest</code> + </summary> + <description> + New <code>libvirt-guest</code> nss module that translates libvirt + guest names into IP addresses + </description> + </item> + <item> + <summary> + daemon: Add support for runtime logging settings adjustment + </summary> + <description> + Logging-related settings like log outputs and filters can now be + adjusted during runtime using the admin interface without the + necessity of the daemon's restart + </description> + </item> + <item> + <summary> + storage: Add virStorageVolInfoFlags API + </summary> + <description> + Add the API to support using the VIR_STORAGE_VOL_GET_PHYSICAL + flag in order to return the host physical size in bytes + of the image container in the allocation field of the + _virStorageVolInfo structure. The --physical flag has been + added to the virsh vol-info command to access the data + </description> + </item> + </section> + <section title="Improvements"> + <item> + <summary> + perf: Add more perf statistics + </summary> + <description> + Add support to get the count of branch instructions + executed, branch misses, bus cycles, stalled frontend + cpu cycles, stalled backend cpu cycles, and ref cpu + cycles by applications running on the platform + </description> + </item> + <item> + <summary> + conf: Display &lt;physical&gt; for volume xml + </summary> + <description> + Add a display of the &lt;physical&gt; size of a disk + volume in the output of the volume XML + </description> + </item> + </section> + <section title="Bug fixes"> + <item> + <summary> + qemu: Correct GetBlockInfo values + </summary> + <description> + For an active domain, correct the physical value provided for + a raw sparse file backed storage and the allocation value provided + for a qcow2 file backed storage that hasn't yet been opened on + the domain + </description> + </item> + <item> + <summary> + qemu: Make virtio console usable on ppc64 guests + </summary> + <description> + The chardev detection code has been improved and can now handle this + configuration properly + </description> + </item> + <item> + <summary> + qemu: Enable mount namespace + </summary> + <description> + To avoid funny races with udev relabelling devices under our hands and + to enhance security, libvirt now spawns each qemu process with its own + <code>/dev</code>. + </description> + </item> + </section> + </release> + <release version="v2.5.0" date="2016-12-04"> + <section title="New features"> + <item> + <summary> + shmem: Add support for additional models + </summary> + <description> + The shmem device can now utilize QEMU's ivshmem-plain and + ivshmem-doorbell, more modern versions of ivshmem + </description> + </item> + <item> + <summary> + vbox: Add VirtualBox 5.1 support + </summary> + </item> + <item> + <summary> + libssh: New transport + </summary> + <description> + The new libssh transport allows one to connect to a running + libvirtd via SSH, using the libssh library; for example: + <tt>qemu+libssh://<i>server</i>/system</tt> + </description> + </item> + <item> + <summary> + vhost-scsi: Add support scsi_host hostdev passthrough + </summary> + <description> + Add the capability to pass through a scsi_host HBA and the + associated LUNs to the guest + </description> + </item> + <item> + <summary> + qemu: Users can now enable debug logging for native gluster + volumes in qemu using the "gluster_debug_level" option in qemu.conf + </summary> + </item> + <item> + <summary> + memory hotplug: Slot numbers for memory devices are now + automatically allocated and thus persistent. In addition slot numbers + can be specified without providing a base address, which simplifies + user configuration + </summary> + </item> + <item> + <summary> + qemu: Express devices will be placed on PCIe bus by default + </summary> + <description> + For machine types that use a PCI Express root bus + (e.g. x86_64/Q35 and aarch64/virt), any unaddressed PCI + device that is an Express device (all virtio-1.0 devices, + e1000e, nec-xhci, vfio assigned devices) will be placed on + an Express controller (i.e. a pcie-root-port) instead of a + legacy PCI controller (i.e. pci-bridge) with the root ports + added as needed + </description> + </item> + </section> + <section title="Improvements"> + <item> + <summary> + docs: Better documentation for migration APIs and flags + </summary> + </item> + <item> + <summary> + vbox: Address thread safety issues + </summary> + </item> + <item> + <summary> + virsh: Add support for passing an alternative persistent XML + to migrate command + </summary> + </item> + <item> + <summary> + vhostuser: Allow hotplug of multiqueue devices + </summary> + </item> + <item> + <summary> + NEWS: Switch to an improved format + </summary> + <description> + List user-visible changes instead of single commits for a better + high-level overview of differences between libvirt releases + </description> + </item> + <item> + <summary> + website: Modernize layout and branding + </summary> + <description> + The libvirt website looked very cluttered and outdated; it has now + been completely overhauled, resulting in a design that's better + organized and more pleasant to look at + </description> + </item> + </section> + <section title="Bug fixes"> + <item> + <summary> + vz: Fix migration in P2P mode + </summary> + </item> + <item> + <summary> + Forbid newline character in names of some libvirt objects + </summary> + </item> + <item> + <summary> + Fix compilation on macOS + </summary> + </item> + </section> + </release> +</releases> diff --git a/docs/news.xsl b/docs/news.xsl index 33506a9..a940026 100644 --- a/docs/news.xsl +++ b/docs/news.xsl @@ -1,50 +1,91 @@ <?xml version="1.0"?> -<xsl:stylesheet version="1.0" - xmlns:html="http://www.w3.org/1999/xhtml" - xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> - <xsl:output method="text" encoding="UTF-8"/> - - <xsl:template match="/"> - <xsl:text>libvirt releases -================ -</xsl:text> - <xsl:apply-templates select="html:html/html:body/*"/> - </xsl:template> +<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + <xsl:output method="xml" indent="yes" encoding="UTF-8"/> - <xsl:template match="html:h1"/> - <xsl:template match="html:p"/> + <!-- This XSLT stylesheet can be applied to the XML version of the release + notes to produce an HTML document suitable for further processing. + In particular, the final output will end up on the libvirt website --> - <xsl:template match="html:h3"> - <xsl:text> + <!-- Document --> + <xsl:template match="/releases"> + <xsl:text disable-output-escaping="yes">&lt;!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"&gt; </xsl:text> - <xsl:text># </xsl:text> - <xsl:apply-templates/> - <xsl:text> + <html xmlns="http://www.w3.org/1999/xhtml"> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> + </head> + <body> + <h1>Releases</h1> + <p>This is the list of official releases for libvirt, along with an + overview of the changes introduced by each of them.</p> + <p>For a more fine-grained view, use the + <a href="http://libvirt.org/git/?p=libvirt.git;a=log">git log</a>. + </p> + <xsl:apply-templates select="release"/> + <p>Releases earlier than v2.5.0 detailed their changes using a + different format and as such are excluded from the list above. + You can read about those older release, starting from those made in + <a href="news-2016.html">2016</a>. + </p> + </body> + </html> + </xsl:template> -</xsl:text> + <!-- Release --> + <xsl:template match="release"> + <h3> + <strong> + <xsl:value-of select="@version"/> + <xsl:text> (</xsl:text> + <xsl:value-of select="@date"/> + <xsl:text>)</xsl:text> + </strong> + </h3> + <ul> + <xsl:apply-templates select="section"/> + </ul> </xsl:template> - <xsl:template match="html:ul"> - <xsl:apply-templates select="html:li"/> + <!-- Section --> + <xsl:template match="section"> + <li> + <strong> + <xsl:value-of select="@title"/> + </strong> + <ul> + <xsl:apply-templates select="item"/> + </ul> + </li> </xsl:template> - <xsl:template match="html:li"> - <xsl:text> * </xsl:text> - <xsl:apply-templates select="html:strong|*/html:li"/> + <!-- Item --> + <xsl:template match="item"> + <li> + <xsl:apply-templates select="summary"/> + <xsl:apply-templates select="description"/> + </li> </xsl:template> - <xsl:template match="html:li/*/html:li"> - <xsl:text> - </xsl:text> + <!-- Item summary --> + <xsl:template match="summary"> <xsl:apply-templates/> - <xsl:text> -</xsl:text> </xsl:template> - <xsl:template match="html:strong"> + <!-- Item description --> + <xsl:template match="description"> + <br/> <xsl:apply-templates/> - <xsl:text> + </xsl:template> -</xsl:text> + <!-- Misc HTML tags, add more as they are needed --> + <xsl:template match="code|i|tt"> + <xsl:text disable-output-escaping="yes">&lt;</xsl:text> + <xsl:value-of select="name()"/> + <xsl:text disable-output-escaping="yes">&gt;</xsl:text> + <xsl:apply-templates/> + <xsl:text disable-output-escaping="yes">&lt;/</xsl:text> + <xsl:value-of select="name()"/> + <xsl:text disable-output-escaping="yes">&gt;</xsl:text> </xsl:template> </xsl:stylesheet> -- 2.7.4

On Wed, Jan 04, 2017 at 04:49:08PM +0100, Andrea Bolognani wrote: Oh, so that's why there's not print(), just sys.stdout.write()... =) I'm not going to comment on the python style, for me it's enough that it works now, it can be made more python-ish later on. This ^^ and ^^ this fails (. Do you even) syntax-check.

On Wed, Jan 04, 2017 at 04:49:09PM +0100, Andrea Bolognani wrote: "documented" doesn't necessarily mean "documented in release notes", I would like to have this clarified here. Don't be afraid of "must" here, as in RFC 2119, or just do what the previous paragraph does and say: "Update release notes when ..." Maybe this can be in its own list item.