[PATCH 0/6] docs: link fixes and improvements
*** BLURB HERE *** Peter Krempa (6): docs: remote: Replace broken link to article about 'ssh-agent' docs: Reject non-https external links kbase: Fix link in 'merging_disk_image_chains' article docs: Use relative links within the web page docs: newreposetup: Drop section about 'libvirt project server' docs: Prohibit 'external' links within the webpage docs/api_extension.rst | 4 +- docs/issue-handling.rst | 2 +- docs/kbase/backing_chains.rst | 4 +- docs/kbase/debuglogs.rst | 4 +- docs/kbase/internals/incremental-backup.rst | 6 +- docs/kbase/launch_security_sev.rst | 2 +- docs/kbase/live_full_disk_backup.rst | 2 +- docs/kbase/merging_disk_image_chains.rst | 6 +- docs/kbase/rpm-deployment.rst | 2 +- docs/kbase/s390_protected_virt.rst | 2 +- docs/kbase/systemtap.rst | 2 +- docs/meson.build | 4 ++ docs/newreposetup.rst | 34 ----------- docs/remote.rst | 2 +- docs/securityprocess.rst | 2 +- docs/testtck.rst | 2 +- docs/windows.rst | 5 +- scripts/check-html-references.py | 67 +++++++++++++++++++-- 18 files changed, 86 insertions(+), 66 deletions(-) -- 2.46.0
The documentation about remote access to libvirt was pointing to a blog post about usage of the 'ssh-agent' to avoid being asked for passwords. The blog/host is now unfortunately defunct thus the link is dead. Replace the link by the official man page of the 'ssh-agent' program. This convenietnly removes the last non-'https' link on our web. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/remote.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/remote.rst b/docs/remote.rst index ab11405d56..8fb7f1e2ad 100644 --- a/docs/remote.rst +++ b/docs/remote.rst @@ -57,7 +57,7 @@ Remote libvirt supports a range of transports: shell) <https://www.openssh.com/>`__ connection. Requires `Netcat (nc) <https://en.wikipedia.org/wiki/Netcat>`__ installed and libvirtd should be running on the remote machine. You should use some sort of ssh key management - (eg. `ssh-agent <http://mah.everybody.org/docs/ssh>`__) otherwise programs + (eg. `ssh-agent <https://man.openbsd.org/ssh-agent>`__) otherwise programs which use this transport will stop to ask for a password. ``ext`` Any external program which can make a connection to the remote machine by -- 2.46.0
Add a '--require-https' switch to 'check-html-references' helper script which will error out if any non-https external link is used from our web and use it while builidng docs. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/meson.build | 1 + scripts/check-html-references.py | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/docs/meson.build b/docs/meson.build index 53b518f987..a94f481730 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -358,6 +358,7 @@ if tests_enabled[0] python3_prog, args: [ check_html_references_prog.full_path(), + '--require-https', '--webroot', meson.project_build_root() / 'docs' ], diff --git a/scripts/check-html-references.py b/scripts/check-html-references.py index d15f28bea7..3382d838c5 100755 --- a/scripts/check-html-references.py +++ b/scripts/check-html-references.py @@ -224,6 +224,18 @@ def check_images(usedimages, imagefiles, ignoreimages): return fail +# checks that all links are accessed via https +def check_https(links): + fail = False + + for link in links: + if link.startswith('http://'): + print(f'ERROR: URI \'{link}\' uses insecure "http" protocol') + fail = True + + return fail + + parser = argparse.ArgumentParser(description='HTML reference checker') parser.add_argument('--webroot', required=True, help='path to the web root') @@ -233,6 +245,8 @@ parser.add_argument('--external', action="store_true", help='print external references instead') parser.add_argument('--ignore-images', action='append', help='paths to images that should be considered as used') +parser.add_argument('--require-https', action="store_true", + help='require secure https for external links') args = parser.parse_args() @@ -269,6 +283,13 @@ else: if check_images(usedimages, imagefiles, args.ignore_images): fail = True + if args.require_https: + if check_https(externallinks): + fail = True + + if check_https(externalimages): + fail = True + if fail: sys.exit(1) -- 2.46.0
Use the proper rST syntax for an external link. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/kbase/merging_disk_image_chains.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kbase/merging_disk_image_chains.rst b/docs/kbase/merging_disk_image_chains.rst index 2ff8b7cd76..9bff8da1af 100644 --- a/docs/kbase/merging_disk_image_chains.rst +++ b/docs/kbase/merging_disk_image_chains.rst @@ -20,7 +20,7 @@ accomplish this *without* incurring guest down time. Here's how to go about it. The same principles used in the `live full disk backup -<https://libvirt.org/kbase/live_full_disk_backup.html>` document are +<https://libvirt.org/kbase/live_full_disk_backup.html>`_ document are used here too. Reducing the disk image chain length -- 2.46.0
Replace full/external links which point to content within 'https://libvirt.org/' with relative links so that the web page works fully locally. This does not change the links in 'docs/manpages' as we want the installed man page to work from everywhere (even when the local docs are not installed) and the generated API docs which take links from the C source. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/api_extension.rst | 4 +--- docs/issue-handling.rst | 2 +- docs/kbase/backing_chains.rst | 4 ++-- docs/kbase/debuglogs.rst | 4 ++-- docs/kbase/internals/incremental-backup.rst | 6 +++--- docs/kbase/launch_security_sev.rst | 2 +- docs/kbase/live_full_disk_backup.rst | 2 +- docs/kbase/merging_disk_image_chains.rst | 6 ++---- docs/kbase/rpm-deployment.rst | 2 +- docs/kbase/s390_protected_virt.rst | 2 +- docs/kbase/systemtap.rst | 2 +- docs/securityprocess.rst | 2 +- docs/testtck.rst | 2 +- docs/windows.rst | 5 ++--- 14 files changed, 20 insertions(+), 25 deletions(-) diff --git a/docs/api_extension.rst b/docs/api_extension.rst index a42c82daca..b9f701dd11 100644 --- a/docs/api_extension.rst +++ b/docs/api_extension.rst @@ -23,9 +23,7 @@ Adding a new API to libvirt is not difficult, but there are quite a few steps. This document assumes that you are familiar with C programming and have checked out the libvirt code from the source code repository and successfully built the existing tree. Instructions on how to check -out and build the code can be found at: - -https://libvirt.org/downloads.html +out and build the code can be found at the `downloads <downloads.html>`__ page. Once you have a working development environment, the steps to create a new API are: diff --git a/docs/issue-handling.rst b/docs/issue-handling.rst index cfde53f876..b4269e01b7 100644 --- a/docs/issue-handling.rst +++ b/docs/issue-handling.rst @@ -178,5 +178,5 @@ community. .. _Untriaged issues: https://gitlab.com/libvirt/libvirt/-/issues/?sort=created_date&state=opened¬%5Blabel_name%5D%5B%5D=state%3A%3Aunconfirmed¬%5Blabel_name%5D%5B%5D=state%3A%3Aneedinfo¬%5Blabel_name%5D%5B%5D=state%3A%3Aconfirmed&first_page_size=100 .. _Unconfirmed bugs: https://gitlab.com/libvirt/libvirt/-/issues/?sort=created_date&state=opened&label_name%5B%5D=kind%3A%3Abug&label_name%5B%5D=state%3A%3Aunconfirmed&first_page_size=100 .. _Unconfirmed features: https://gitlab.com/libvirt/libvirt/-/issues/?sort=created_date&state=opened&label_name%5B%5D=kind%3A%3Aenhancement&label_name%5B%5D=state%3A%3Aunconfirmed&first_page_size=100 -.. _debug logs: https://libvirt.org/kbase/debuglogs.html +.. _debug logs: kbase/debuglogs.html .. _code of conduct: governance.html#code-of-conduct diff --git a/docs/kbase/backing_chains.rst b/docs/kbase/backing_chains.rst index 38a9a2337b..1c5e231195 100644 --- a/docs/kbase/backing_chains.rst +++ b/docs/kbase/backing_chains.rst @@ -97,8 +97,8 @@ specification can be used: </disk> This makes libvirt follow the settings as configured in the XML. Note that this -is supported only when the https://libvirt.org/formatdomaincaps.html#backingstoreinput -capability is present. +is supported only when the `backingStoreInput +<../formatdomaincaps.html#backingstoreinput>`_ capability is present. An empty ``<backingStore/>`` element signals the end of the chain. Using this will prevent libvirt or qemu from probing the backing chain. diff --git a/docs/kbase/debuglogs.rst b/docs/kbase/debuglogs.rst index dff2cfd144..f8c05b6922 100644 --- a/docs/kbase/debuglogs.rst +++ b/docs/kbase/debuglogs.rst @@ -120,8 +120,8 @@ Libvirt daemons run either in the ``system`` mode or on ``session`` (user) mode, depending on the configuration of the host and available permission levels. -The `connection URI <https://libvirt.org/uri.html>`__ influences which daemon -the client will communicate with. +The `connection URI <../uri.html>`__ influences which daemon the client will +communicate with. System daemon mode ~~~~~~~~~~~~~~~~~~ diff --git a/docs/kbase/internals/incremental-backup.rst b/docs/kbase/internals/incremental-backup.rst index 29e90092e8..af464e8178 100644 --- a/docs/kbase/internals/incremental-backup.rst +++ b/docs/kbase/internals/incremental-backup.rst @@ -16,9 +16,9 @@ this document will try to summarize them. Glossary ======== -See the knowledge base article on -`domain state capture <https://libvirt.org/kbase/domainstatecapture.html>`_ for -a deeper explanation of some of the concepts. +See the knowledge base article on `domain state capture +<../domainstatecapture.html>`_ for a deeper explanation of some of the +concepts. Checkpoint diff --git a/docs/kbase/launch_security_sev.rst b/docs/kbase/launch_security_sev.rst index 6a3740d297..cfde2c49dc 100644 --- a/docs/kbase/launch_security_sev.rst +++ b/docs/kbase/launch_security_sev.rst @@ -154,7 +154,7 @@ VM Configuration ================ SEV is enabled in the XML by specifying the -`<launchSecurity> <https://libvirt.org/formatdomain.html#launch-security>`__ +`<launchSecurity> <../formatdomain.html#launch-security>`__ element. However, specifying ``launchSecurity`` isn't enough to boot an SEV VM. Further configuration requirements are discussed below. diff --git a/docs/kbase/live_full_disk_backup.rst b/docs/kbase/live_full_disk_backup.rst index f20169e314..f9dcd2a1bd 100644 --- a/docs/kbase/live_full_disk_backup.rst +++ b/docs/kbase/live_full_disk_backup.rst @@ -10,7 +10,7 @@ Overview Live full disk backups are preferred in many scenarios, *despite* their space requirements. The following outlines an efficient method to do that using libvirt's APIs. This method involves concepts: the notion of -`backing chains <https://libvirt.org/kbase/backing_chains.html>`_, +`backing chains <backing_chains.html>`_, `QCOW2 overlays <https://www.qemu.org/docs/master/interop/live-block-operations.html#disk-image-backing-chain-notation>`_, and a special operation called "active block-commit", which allows diff --git a/docs/kbase/merging_disk_image_chains.rst b/docs/kbase/merging_disk_image_chains.rst index 9bff8da1af..40b565f74b 100644 --- a/docs/kbase/merging_disk_image_chains.rst +++ b/docs/kbase/merging_disk_image_chains.rst @@ -7,8 +7,7 @@ Merging disk image image chains Context ======= -Sometimes a `disk image chain -<https://libvirt.org/kbase/backing_chains.html>`_ can get long and +Sometimes a `disk image chain <backing_chains.html>`_ can get long and cumbersome. For the purpose of illustration, consider this smaller disk image chain:: @@ -20,8 +19,7 @@ accomplish this *without* incurring guest down time. Here's how to go about it. The same principles used in the `live full disk backup -<https://libvirt.org/kbase/live_full_disk_backup.html>`_ document are -used here too. +<live_full_disk_backup.html>`_ document are used here too. Reducing the disk image chain length ==================================== diff --git a/docs/kbase/rpm-deployment.rst b/docs/kbase/rpm-deployment.rst index 26fe1be8e6..ae2ed42eb6 100644 --- a/docs/kbase/rpm-deployment.rst +++ b/docs/kbase/rpm-deployment.rst @@ -370,7 +370,7 @@ RPM packages * libvirt-docs - A local copy of the `libvirt website <https://libvirt.org>`_ website content + A local copy of the libvirt website website content that matches the deployed version of libvirt. * libvirt-libs diff --git a/docs/kbase/s390_protected_virt.rst b/docs/kbase/s390_protected_virt.rst index a8c627931b..faacd6fc7a 100644 --- a/docs/kbase/s390_protected_virt.rst +++ b/docs/kbase/s390_protected_virt.rst @@ -128,7 +128,7 @@ As the virtio data structures of secure guests are not accessible by the host, it is necessary to use shared memory ('bounce buffers'). Since libvirt 7.6.0 the -`<launchSecurity> <https://libvirt.org/formatdomain.html#launch-security>`__ +`<launchSecurity> <../formatdomain.html#launch-security>`__ element with type ``s390-pv`` should be used on protected virtualization guests. Without ``launchSecurity`` you must enable all virtio devices to use shared buffers by configuring them with platform_iommu enabled. diff --git a/docs/kbase/systemtap.rst b/docs/kbase/systemtap.rst index 8a3acabdf7..5d4717b7aa 100644 --- a/docs/kbase/systemtap.rst +++ b/docs/kbase/systemtap.rst @@ -27,7 +27,7 @@ For libvirt before **6.7.0**, it can be configured by: ../configure --with-dtrace For libvirt **6.7.0** or later, configure it by the ``meson`` (seeing -`libvirt compiling <https://libvirt.org/compiling.html>`__): +`libvirt compiling <../compiling.html>`__): :: diff --git a/docs/securityprocess.rst b/docs/securityprocess.rst index 1f5176ec75..075679df74 100644 --- a/docs/securityprocess.rst +++ b/docs/securityprocess.rst @@ -35,7 +35,7 @@ format in the `libvirt-security-notice GIT repository <https://gitlab.com/libvirt/libvirt-security-notice>`__ and `published online <https://security.libvirt.org>`__ in text, HTML and XML formats. Security notices are published on the `libvirt-announce mailing -list <https://libvirt.org/contact.html#mailing-lists>`__ when any embargo is +list <contact.html#mailing-lists>`__ when any embargo is lifted, or as soon as triaged if already public knowledge. Security team diff --git a/docs/testtck.rst b/docs/testtck.rst index 568899dcdd..621dadf0db 100644 --- a/docs/testtck.rst +++ b/docs/testtck.rst @@ -101,7 +101,7 @@ script containing functions describing GitLab job definitions it can be utilized to run integration test suite as well. In this case, one needs to get a copy of their libvirt repository containing the changes to be tested inside the VM (either by cloning it manually or sharing the repo e.g. via -`virtiofs <https://libvirt.org/kbase/virtiofs.html>`__). Make sure that the +`virtiofs <kbase/virtiofs.html>`__). Make sure that the user which is going to execute the following has passwordless "sudo" permissions (lcitool's default "test" user does). Then it's just a matter of running diff --git a/docs/windows.rst b/docs/windows.rst index ee3caef41a..37fd8e02d9 100644 --- a/docs/windows.rst +++ b/docs/windows.rst @@ -54,9 +54,8 @@ Connecting to VMware ESX/vSphere -------------------------------- Details on the capabilities, certificates, and connection string syntax used for -connecting to VMware ESX and vSphere can be found online here: - -https://libvirt.org/drvesx.html +connecting to VMware ESX and vSphere can be found on the +`ESX driver <drvesx.html>`_ page. TLS Certificates ---------------- -- 2.46.0
Now that most things were migrated out of the old server which hosted the 'libvirt.org' web (now handles only 'https://download.libvirt.org') which no longer even hosts the cgit web interface (any link redirects to gitlab) the whole section now is obsolete. Remove it. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/newreposetup.rst | 34 ---------------------------------- 1 file changed, 34 deletions(-) diff --git a/docs/newreposetup.rst b/docs/newreposetup.rst index d5375b23ea..17f74d8659 100644 --- a/docs/newreposetup.rst +++ b/docs/newreposetup.rst @@ -129,37 +129,3 @@ required under the **Settings** page: In the master git repository create a file `.github/lockdown.yml` to restrict use of issue tracker and pull requests. - - -libvirt.org -=========== - -The `libvirt project server <https://libvirt.org>`_ hosts read-only mirrors of -all the libvirt Git repositories in the directory ``/data/git``. - -When creating a new repository the following steps are required: - -* Create repo with - :: - - $ sudo su - - # cd /data/git - # mkdir $repo.git - # cd $repo.git - # git init --bare - # touch export - # touch git-daemon-export-ok - # cd .. - # chown -R gitmirror.gitmirror $repo.git - # chmod -R g+w $repo.git - # find -type d $repo.git | xargs chmod g+s - -* Set the ``description`` and ``config`` files following other repos' example - -* Setup mirroring - :: - - $ sudo su - gitmirror - # ./newrepo.sh /data/git/$repo.git - # cd mirrors - # $HOME/sync-one.sh $repo.git -- 2.46.0
Enforce that relative links are used within the page, so that local installations don't require internet conection and/or don't redirect to the web needlessly. This is done by looking for any local link (barring exceptions) when checking links with 'check-html-references.py'. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/meson.build | 3 +++ scripts/check-html-references.py | 46 +++++++++++++++++++++++++++----- 2 files changed, 43 insertions(+), 6 deletions(-) diff --git a/docs/meson.build b/docs/meson.build index a94f481730..d7343b6665 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -359,6 +359,9 @@ if tests_enabled[0] args: [ check_html_references_prog.full_path(), '--require-https', + '--project-uri', 'https://libvirt.org', + '--project-uri-exceptions', 'docs/manpages/', + '--project-uri-exceptions', 'docs/html/', '--webroot', meson.project_build_root() / 'docs' ], diff --git a/scripts/check-html-references.py b/scripts/check-html-references.py index 3382d838c5..74299a1958 100755 --- a/scripts/check-html-references.py +++ b/scripts/check-html-references.py @@ -53,7 +53,7 @@ def get_file_list(prefix): # loads an XHTML and extracts all anchors, local and remote links for the one file -def process_file(filename): +def process_file(filename, project_uri): tree = ET.parse(filename) root = tree.getroot() docname = root.get('data-sourcedoc') @@ -65,6 +65,7 @@ def process_file(filename): anchors = [filename] targets = [] images = [] + projectlinks = [] for elem in root.findall('.//html:a', ns): target = elem.get('href') @@ -76,6 +77,10 @@ def process_file(filename): if target: if re.search('://', target): externallinks.append(target) + + if project_uri is not None and target.startswith(project_uri): + projectlinks.append((target, docname)) + elif target[0] != '#' and 'mailto:' not in target: targetfull = os.path.normpath(os.path.join(dirname, target)) @@ -106,22 +111,24 @@ def process_file(filename): imagefull = os.path.normpath(os.path.join(dirname, src)) images.append((imagefull, docname)) - return (anchors, targets, images) + return (anchors, targets, images, projectlinks) -def process_all(filelist): +def process_all(filelist, project_uri): anchors = [] targets = [] images = [] + projectlinks = [] for file in filelist: - anchor, target, image = process_file(file) + anchor, target, image, projectlink = process_file(file, project_uri) targets = targets + target anchors = anchors + anchor images = images + image + projectlinks = projectlinks + projectlink - return (targets, anchors, images) + return (targets, anchors, images, projectlinks) def check_targets(targets, anchors): @@ -236,6 +243,26 @@ def check_https(links): return fail +# checks prohibited external links to local files +def check_projectlinks(projectlinks, exceptions): + fail = False + + for (link, filename) in projectlinks: + allowed = False + + if exceptions is not None: + for exc in exceptions: + if exc in filename: + allowed = True + break + + if not allowed: + print(f'ERROR: prohibited external URI \'{link}\' to local project in \'{filename}\'') + fail = True + + return fail + + parser = argparse.ArgumentParser(description='HTML reference checker') parser.add_argument('--webroot', required=True, help='path to the web root') @@ -247,6 +274,10 @@ parser.add_argument('--ignore-images', action='append', help='paths to images that should be considered as used') parser.add_argument('--require-https', action="store_true", help='require secure https for external links') +parser.add_argument('--project-uri', + help='external prefix of the local project (e.g. https://libvirt.org; external links with that prefix are prohibited') +parser.add_argument('--project-uri-exceptions', action='append', + help='list of path prefixes excluded from the "--project-uri" checks') args = parser.parse_args() @@ -254,7 +285,7 @@ files, imagefiles = get_file_list(os.path.abspath(args.webroot)) entrypoint = os.path.join(os.path.abspath(args.webroot), args.entrypoint) -targets, anchors, usedimages = process_all(files) +targets, anchors, usedimages, projectlinks = process_all(files, args.project_uri) fail = False @@ -283,6 +314,9 @@ else: if check_images(usedimages, imagefiles, args.ignore_images): fail = True + if check_projectlinks(projectlinks, args.project_uri_exceptions): + fail = True + if args.require_https: if check_https(externallinks): fail = True -- 2.46.0
On a Tuesday in 2024, Peter Krempa wrote:
*** BLURB HERE ***
Peter Krempa (6): docs: remote: Replace broken link to article about 'ssh-agent' docs: Reject non-https external links kbase: Fix link in 'merging_disk_image_chains' article docs: Use relative links within the web page docs: newreposetup: Drop section about 'libvirt project server' docs: Prohibit 'external' links within the webpage
docs/api_extension.rst | 4 +- docs/issue-handling.rst | 2 +- docs/kbase/backing_chains.rst | 4 +- docs/kbase/debuglogs.rst | 4 +- docs/kbase/internals/incremental-backup.rst | 6 +- docs/kbase/launch_security_sev.rst | 2 +- docs/kbase/live_full_disk_backup.rst | 2 +- docs/kbase/merging_disk_image_chains.rst | 6 +- docs/kbase/rpm-deployment.rst | 2 +- docs/kbase/s390_protected_virt.rst | 2 +- docs/kbase/systemtap.rst | 2 +- docs/meson.build | 4 ++ docs/newreposetup.rst | 34 ----------- docs/remote.rst | 2 +- docs/securityprocess.rst | 2 +- docs/testtck.rst | 2 +- docs/windows.rst | 5 +- scripts/check-html-references.py | 67 +++++++++++++++++++-- 18 files changed, 86 insertions(+), 66 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano
participants (2)
-
Ján Tomko -
Peter Krempa