[libvirt] [PATCH 00/11] Integrate usage of glib into libvirt

This is a followup to a previous patch series: https://www.redhat.com/archives/libvir-list/2019-August/msg01374.html The first abort-on-oom parts of that series merged already. As well as fixing the issues mentioned last time, the glib parts now do a little more: - Demonstrate conversion of virObject to GObject - Convert to gbase64 APIs - Start to convert getopt to GOptionContext I have explicitly now confirmed we can freely mix g_malloc/malloc and g_free/free, given our min glib version. my intention with the glib code will be to focus on converting bits of code that allow us to eliminate gnulib modules. About 50% of the gnulib stuff is related to the Windows portability for sockets() and poll(). We can address this by integrate the event loop with GMainLoop, and using GIO for its GSocket APIs. The other gnulib stuff is a random bag of APIs. Some may be replaced by glib APIs, for others we can pull the gnulib fix straight into libvirt, for others the portability problems might be obsolete. Daniel P. Berrangé (11): build: probe for glib-2 library in configure build: link to glib, gobject, gio libraries remote: don't pull anonymous enums into rpc protocol structs util: use glib memory allocation functions util: use glib string allocation/formatting functions util: use glib base64 encoding/decoding APIs util: convert virIdentity class to use GObject src: convert over to use GRegex for regular exprssions virsh: convert command line parsing to use GOptionContext virt-admin: convert command line parsing to use GOptionContext virt-login-shell: convert command line parsing to use GOptionContext bootstrap.conf | 5 - configure.ac | 2 + libvirt.spec.in | 1 + m4/virt-glib.m4 | 36 ++++ mingw-libvirt.spec.in | 2 + src/Makefile.am | 5 +- src/access/viraccessdriverpolkit.c | 21 +- src/admin/admin_server.c | 3 +- src/admin_protocol-structs | 9 - src/conf/domain_event.c | 25 +-- src/conf/virsecretobj.c | 38 +--- src/internal.h | 1 + src/libvirt_private.syms | 1 - src/libxl/libxl_capabilities.c | 44 ++-- src/libxl/libxl_conf.c | 3 +- src/lxc/Makefile.inc.am | 2 + src/qemu/qemu_agent.c | 9 +- src/qemu/qemu_command.c | 5 +- src/qemu/qemu_domain.c | 8 +- src/qemu/qemu_process.c | 4 +- src/remote/Makefile.inc.am | 1 + src/remote/remote_daemon.c | 3 +- src/remote/remote_daemon_dispatch.c | 35 ++-- src/remote_protocol-structs | 9 - src/rpc/virnetserverclient.c | 57 +++--- src/rpc/virnetserverprogram.c | 13 +- src/secret/secret_driver.c | 1 - src/storage/storage_backend_rbd.c | 4 +- src/util/Makefile.inc.am | 1 + src/util/viralloc.c | 29 +-- src/util/viridentity.c | 87 ++++---- src/util/viridentity.h | 7 +- src/util/virstring.c | 40 +--- src/util/virstring.h | 2 - tests/Makefile.am | 3 +- tests/viridentitytest.c | 45 ++--- tests/virnetserverclienttest.c | 3 +- tools/Makefile.am | 1 + tools/virsh-secret.c | 17 +- tools/virsh.c | 303 +++++++++++++--------------- tools/virt-admin.c | 207 +++++++++---------- tools/virt-login-shell-helper.c | 66 ++---- 42 files changed, 495 insertions(+), 663 deletions(-) create mode 100644 m4/virt-glib.m4 -- 2.21.0

Prepare for linking with glib by probing for it at configure time. Per supported platforms target, the min glib versions on relevant distros are: RHEL-8: 2.56.1 RHEL-7: 2.50.3 Debian (Buster): 2.58.3 Debian (Stretch): 2.50.3 OpenBSD (Ports): 2.58.3 FreeBSD (Ports): 2.56.3 OpenSUSE Leap 15: 2.54.3 SLE12-SP2: 2.48.2 Ubuntu (Xenial): 2.48.0 macOS (Homebrew): 2.56.0 This suggests that a minimum glib of 2.48 is a reasonable target. This aligns with the minimum version required by qemu too. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- configure.ac | 2 ++ libvirt.spec.in | 1 + m4/virt-glib.m4 | 36 ++++++++++++++++++++++++++++++++++++ mingw-libvirt.spec.in | 2 ++ 4 files changed, 41 insertions(+) create mode 100644 m4/virt-glib.m4 diff --git a/configure.ac b/configure.ac index af8cbcdfd8..573c3d85f2 100644 --- a/configure.ac +++ b/configure.ac @@ -317,6 +317,7 @@ LIBVIRT_CHECK_DLOPEN LIBVIRT_CHECK_FIREWALLD LIBVIRT_CHECK_FIREWALLD_ZONE LIBVIRT_CHECK_FUSE +LIBVIRT_CHECK_GLIB LIBVIRT_CHECK_GLUSTER LIBVIRT_CHECK_GNUTLS LIBVIRT_CHECK_HAL @@ -1007,6 +1008,7 @@ LIBVIRT_RESULT_DLOPEN LIBVIRT_RESULT_FIREWALLD LIBVIRT_RESULT_FIREWALLD_ZONE LIBVIRT_RESULT_FUSE +LIBVIRT_RESULT_GLIB LIBVIRT_RESULT_GLUSTER LIBVIRT_RESULT_GNUTLS LIBVIRT_RESULT_HAL diff --git a/libvirt.spec.in b/libvirt.spec.in index 7f5183f341..dcad08cb5f 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -273,6 +273,7 @@ BuildRequires: systemd-units %if %{with_libxl} BuildRequires: xen-devel %endif +BuildRequires: glib2-devel >= 2.48 BuildRequires: libxml2-devel BuildRequires: libxslt BuildRequires: readline-devel diff --git a/m4/virt-glib.m4 b/m4/virt-glib.m4 new file mode 100644 index 0000000000..5a5bc19660 --- /dev/null +++ b/m4/virt-glib.m4 @@ -0,0 +1,36 @@ +dnl The glib.so library +dnl +dnl Copyright (C) 2016 Red Hat, Inc. +dnl +dnl This library is free software; you can redistribute it and/or +dnl modify it under the terms of the GNU Lesser General Public +dnl License as published by the Free Software Foundation; either +dnl version 2.1 of the License, or (at your option) any later version. +dnl +dnl This library is distributed in the hope that it will be useful, +dnl but WITHOUT ANY WARRANTY; without even the implied warranty of +dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +dnl Lesser General Public License for more details. +dnl +dnl You should have received a copy of the GNU Lesser General Public +dnl License along with this library. If not, see +dnl <http://www.gnu.org/licenses/>. +dnl + +AC_DEFUN([LIBVIRT_ARG_GLIB], [ + LIBVIRT_ARG_WITH([GLIB], [glib-2.0 location], [check]) +]) + +AC_DEFUN([LIBVIRT_CHECK_GLIB],[ + GLIB_REQUIRED=2.48.0 + + LIBVIRT_CHECK_PKG([GLIB], [glib-2.0], [$GLIB_REQUIRED]) + + if test "$with_glib" = "no" ; then + AC_MSG_ERROR([glib-2.0 >= $GLIB_REQUIRED is required for libvirt]) + fi +]) + +AC_DEFUN([LIBVIRT_RESULT_GLIB], [ + LIBVIRT_RESULT_LIB([GLIB]) +]) diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index a20c4b7d74..c29f3eeed2 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -52,6 +52,8 @@ BuildRequires: mingw32-gcc BuildRequires: mingw64-gcc BuildRequires: mingw32-binutils BuildRequires: mingw64-binutils +BuildRequires: mingw32-glib2 >= 2.48 +BuildRequires: mingw64-glib2 >= 2.48 BuildRequires: mingw32-libgpg-error BuildRequires: mingw64-libgpg-error BuildRequires: mingw32-libgcrypt -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:23PM +0100, Daniel P. Berrangé wrote:
Prepare for linking with glib by probing for it at configure time. Per supported platforms target, the min glib versions on relevant distros are:
RHEL-8: 2.56.1 RHEL-7: 2.50.3 Debian (Buster): 2.58.3 Debian (Stretch): 2.50.3 OpenBSD (Ports): 2.58.3 FreeBSD (Ports): 2.56.3 OpenSUSE Leap 15: 2.54.3 SLE12-SP2: 2.48.2 Ubuntu (Xenial): 2.48.0 macOS (Homebrew): 2.56.0
This suggests that a minimum glib of 2.48 is a reasonable target. This aligns with the minimum version required by qemu too.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- configure.ac | 2 ++ libvirt.spec.in | 1 + m4/virt-glib.m4 | 36 ++++++++++++++++++++++++++++++++++++ mingw-libvirt.spec.in | 2 ++ 4 files changed, 41 insertions(+) create mode 100644 m4/virt-glib.m4
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>

Add the main glib.h to internal.h so that all common code can use it. Historically glib allowed applications to register an alternative memory allocator, so mixing g_malloc/g_free with malloc/free was not safe. This was feature was dropped in 2.46.0 with: commit 3be6ed60aa58095691bd697344765e715a327fc1 Author: Alexander Larsson <alexl@redhat.com> Date: Sat Jun 27 18:38:42 2015 +0200 Deprecate and drop support for memory vtables Applications are still encourged to match g_malloc/g_free, but it is no longer a mandatory requirement for correctness, just stylistic. This is explicitly clarified in commit 1f24b36607bf708f037396014b2cdbc08d67b275 Author: Daniel P. Berrangé <berrange@redhat.com> Date: Thu Sep 5 14:37:54 2019 +0100 gmem: clarify that g_malloc always uses the system allocator Applications can still use custom allocators in general, but they must do this by linking to a library that replaces the core malloc/free implemenentation entirely, instead of via a glib specific call. This means that libvirt does not need to be concerned about use of g_malloc/g_free causing an ABI change in the public libary, and can avoid memory copying when talking to external libraries. This patch probes for glib, gobject and gio. Glib provides the foundation layer with a collection of data structures, helper APIs, and platform portability logic. GObject provides the object type system, built on glib. GIO builds on GObject, providing objects for various interesting tasks, most notably including DBus client and server support and portable sockets APIs, but much more too. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- m4/virt-glib.m4 | 4 ++-- src/Makefile.am | 2 ++ src/internal.h | 1 + src/lxc/Makefile.inc.am | 2 ++ src/remote/Makefile.inc.am | 1 + src/util/Makefile.inc.am | 1 + tests/Makefile.am | 3 ++- tools/Makefile.am | 1 + 8 files changed, 12 insertions(+), 3 deletions(-) diff --git a/m4/virt-glib.m4 b/m4/virt-glib.m4 index 5a5bc19660..e9992e9824 100644 --- a/m4/virt-glib.m4 +++ b/m4/virt-glib.m4 @@ -24,10 +24,10 @@ AC_DEFUN([LIBVIRT_ARG_GLIB], [ AC_DEFUN([LIBVIRT_CHECK_GLIB],[ GLIB_REQUIRED=2.48.0 - LIBVIRT_CHECK_PKG([GLIB], [glib-2.0], [$GLIB_REQUIRED]) + LIBVIRT_CHECK_PKG([GLIB], [glib-2.0 gobject-2.0 gio-2.0], [$GLIB_REQUIRED]) if test "$with_glib" = "no" ; then - AC_MSG_ERROR([glib-2.0 >= $GLIB_REQUIRED is required for libvirt]) + AC_MSG_ERROR([glib-2.0, gobject-2.0, gio-2.0 >= $GLIB_REQUIRED is required for libvirt]) fi ]) diff --git a/src/Makefile.am b/src/Makefile.am index cd955ee552..66ba710e50 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -34,6 +34,7 @@ AM_CPPFLAGS = -I../gnulib/lib \ WARN_CFLAGS += $(STRICT_FRAME_LIMIT_CFLAGS) AM_CFLAGS = $(LIBXML_CFLAGS) \ + $(GLIB_CFLAGS) \ $(WARN_CFLAGS) \ $(LOCK_CHECKING_CFLAGS) \ $(WIN32_EXTRA_CFLAGS) \ @@ -559,6 +560,7 @@ libvirt_admin_la_LIBADD += \ $(YAJL_LIBS) \ $(DEVMAPPER_LIBS) \ $(LIBXML_LIBS) \ + $(GLIB_LIBS) \ $(SSH2_LIBS) \ $(SASL_LIBS) \ $(GNUTLS_LIBS) \ diff --git a/src/internal.h b/src/internal.h index adc1e3f496..55aaf937cf 100644 --- a/src/internal.h +++ b/src/internal.h @@ -27,6 +27,7 @@ #include <stdint.h> #include <stdio.h> #include <string.h> +#include <glib.h> #if STATIC_ANALYSIS # undef NDEBUG /* Don't let a prior NDEBUG definition cause trouble. */ diff --git a/src/lxc/Makefile.inc.am b/src/lxc/Makefile.inc.am index b4d560702c..0c9618e185 100644 --- a/src/lxc/Makefile.inc.am +++ b/src/lxc/Makefile.inc.am @@ -184,6 +184,7 @@ libvirt_lxc_LDFLAGS = \ $(PIE_LDFLAGS) \ $(CAPNG_LIBS) \ $(LIBXML_LIBS) \ + $(GLIB_LIBS) \ $(NULL) libvirt_lxc_LDADD = \ libvirt.la \ @@ -200,6 +201,7 @@ libvirt_lxc_CFLAGS = \ $(PIE_CFLAGS) \ $(CAPNG_CFLAGS) \ $(LIBXML_CFLAGS) \ + $(GLIB_CFLAGS) \ $(LIBNL_CFLAGS) \ $(FUSE_CFLAGS) \ $(DBUS_CFLAGS) \ diff --git a/src/remote/Makefile.inc.am b/src/remote/Makefile.inc.am index 5a5c90a922..e02c20d47c 100644 --- a/src/remote/Makefile.inc.am +++ b/src/remote/Makefile.inc.am @@ -38,6 +38,7 @@ REMOTE_DAEMON_SOURCES = \ REMOTE_DAEMON_CFLAGS = \ $(LIBXML_CFLAGS) \ + $(GLIB_CFLAGS) \ $(GNUTLS_CFLAGS) \ $(SASL_CFLAGS) \ $(XDR_CFLAGS) \ diff --git a/src/util/Makefile.inc.am b/src/util/Makefile.inc.am index 482b657a90..a415378fe2 100644 --- a/src/util/Makefile.inc.am +++ b/src/util/Makefile.inc.am @@ -289,6 +289,7 @@ libvirt_util_la_LIBADD = \ $(DBUS_LIBS) \ $(WIN32_EXTRA_LIBS) \ $(LIBXML_LIBS) \ + $(GLIB_LIBS) \ $(SECDRIVER_LIBS) \ $(NUMACTL_LIBS) \ $(ACL_LIBS) \ diff --git a/tests/Makefile.am b/tests/Makefile.am index d88ad7f686..7b81ee88f1 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -35,6 +35,7 @@ AM_CFLAGS = \ -Dabs_srcdir="\"$(abs_srcdir)\"" \ -Dabs_top_srcdir="\"$(abs_top_srcdir)\"" \ $(LIBXML_CFLAGS) \ + $(GLIB_CFLAGS) \ $(LIBNL_CFLAGS) \ $(GNUTLS_CFLAGS) \ $(SASL_CFLAGS) \ @@ -525,7 +526,7 @@ libxlxml2domconfigtest_LDADD = $(libxl_LDADDS) $(LIBXML_LIBS) libxlmock_la_SOURCES = \ libxlmock.c -libxlmock_la_CFLAGS = $(LIBXL_CFLAGS) $(LIBXML_CFLAGS) +libxlmock_la_CFLAGS = $(LIBXL_CFLAGS) $(LIBXML_CFLAGS) $(GLIB_CFLAGS) libxlmock_la_LDFLAGS = $(MOCKLIBS_LDFLAGS) libxlmock_la_LIBADD = $(MOCKLIBS_LIBS) diff --git a/tools/Makefile.am b/tools/Makefile.am index 29fdbfe846..9489cb35db 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -36,6 +36,7 @@ AM_CFLAGS = \ $(COVERAGE_CFLAGS) \ $(PIE_CFLAGS) \ $(LIBXML_CFLAGS) \ + $(GLIB_CFLAGS) \ $(NULL) AM_LDFLAGS = \ -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:24PM +0100, Daniel P. Berrangé wrote:
Add the main glib.h to internal.h so that all common code can use it.
Historically glib allowed applications to register an alternative memory allocator, so mixing g_malloc/g_free with malloc/free was not safe.
This was feature was dropped in 2.46.0 with:
commit 3be6ed60aa58095691bd697344765e715a327fc1 Author: Alexander Larsson <alexl@redhat.com> Date: Sat Jun 27 18:38:42 2015 +0200
Deprecate and drop support for memory vtables
Applications are still encourged to match g_malloc/g_free, but it is no longer a mandatory requirement for correctness, just stylistic. This is explicitly clarified in
commit 1f24b36607bf708f037396014b2cdbc08d67b275 Author: Daniel P. Berrangé <berrange@redhat.com> Date: Thu Sep 5 14:37:54 2019 +0100
gmem: clarify that g_malloc always uses the system allocator
Applications can still use custom allocators in general, but they must do this by linking to a library that replaces the core malloc/free implemenentation entirely, instead of via a glib specific call.
This means that libvirt does not need to be concerned about use of g_malloc/g_free causing an ABI change in the public libary, and can avoid memory copying when talking to external libraries.
This patch probes for glib, gobject and gio. Glib provides the foundation layer with a collection of data structures, helper APIs, and platform portability logic. GObject provides the object type system, built on glib. GIO builds on GObject, providing objects for various interesting tasks, most notably including DBus client and server support and portable sockets APIs, but much more too.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- m4/virt-glib.m4 | 4 ++-- src/Makefile.am | 2 ++ src/internal.h | 1 + src/lxc/Makefile.inc.am | 2 ++ src/remote/Makefile.inc.am | 1 + src/util/Makefile.inc.am | 1 + tests/Makefile.am | 3 ++- tools/Makefile.am | 1 + 8 files changed, 12 insertions(+), 3 deletions(-)
I would not link to gio for now as it's not used by this series, we can introduce it later once there is a need for it. Reviewed-by: Pavel Hrdina <phrdina@redhat.com>

The VIR_TYPED_PARAM_* enum fields are defined in libvirt-common.h, not in the remote protcol, so shouldn't be part of the protocol structs output check. This avoids similar problems hitting when we add use of glib, which has other such anonymous enums. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/Makefile.am | 3 +-- src/admin_protocol-structs | 9 --------- src/remote_protocol-structs | 9 --------- 3 files changed, 1 insertion(+), 20 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 66ba710e50..9d21395892 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -241,8 +241,7 @@ PDWTAGS = \ else \ $(PERL) -0777 -n \ -e 'foreach my $$p (split m!\n*(?:$(r1)|$(r2))\n!) {' \ - -e ' if ($$p =~ /^(struct|enum) $(struct_prefix)/ ||' \ - -e ' $$p =~ /^enum \{/) {' \ + -e ' if ($$p =~ /^(struct|enum) $(struct_prefix)/) {' \ -e ' $$p =~ s!\t*/\*.*?\*/!!sg;' \ -e ' $$p =~ s!\s+\n!\n!sg;' \ -e ' $$p =~ s!\s+$$!!;' \ diff --git a/src/admin_protocol-structs b/src/admin_protocol-structs index a6c53807ee..983e6e5292 100644 --- a/src/admin_protocol-structs +++ b/src/admin_protocol-structs @@ -1,13 +1,4 @@ /* -*- c -*- */ -enum { - VIR_TYPED_PARAM_INT = 1, - VIR_TYPED_PARAM_UINT = 2, - VIR_TYPED_PARAM_LLONG = 3, - VIR_TYPED_PARAM_ULLONG = 4, - VIR_TYPED_PARAM_DOUBLE = 5, - VIR_TYPED_PARAM_BOOLEAN = 6, - VIR_TYPED_PARAM_STRING = 7, -}; struct admin_typed_param_value { int type; union { diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index eb689b3574..51606e7473 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -1,13 +1,4 @@ /* -*- c -*- */ -enum { - VIR_TYPED_PARAM_INT = 1, - VIR_TYPED_PARAM_UINT = 2, - VIR_TYPED_PARAM_LLONG = 3, - VIR_TYPED_PARAM_ULLONG = 4, - VIR_TYPED_PARAM_DOUBLE = 5, - VIR_TYPED_PARAM_BOOLEAN = 6, - VIR_TYPED_PARAM_STRING = 7, -}; struct remote_nonnull_domain { remote_nonnull_string name; remote_uuid uuid; -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:25PM +0100, Daniel P. Berrangé wrote:
The VIR_TYPED_PARAM_* enum fields are defined in libvirt-common.h, not in the remote protcol, so shouldn't be part of the protocol structs output check. This avoids similar problems hitting when we add use of glib, which has other such anonymous enums.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/Makefile.am | 3 +-- src/admin_protocol-structs | 9 --------- src/remote_protocol-structs | 9 --------- 3 files changed, 1 insertion(+), 20 deletions(-)
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>

Convert the VIR_ALLOC family of APIs with use of the g_malloc family of APIs. Use of VIR_ALLOC related functions should be incrementally phased out over time, allowing return value checks to be dropped. Use of VIR_FREE should be replaced with auto-cleanup whenever possible. We previously used the 'calloc-posix' gnulib module because mingw does not set errno to ENOMEM on failure. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/util/viralloc.c | 29 ++++++----------------------- 2 files changed, 6 insertions(+), 24 deletions(-) diff --git a/bootstrap.conf b/bootstrap.conf index 4f944a9c23..549d18c6d4 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -28,7 +28,6 @@ byteswap c-ctype c-strcase c-strcasestr -calloc-posix canonicalize-lgpl chown clock-time diff --git a/src/util/viralloc.c b/src/util/viralloc.c index 10a8d0fb73..b8ca850764 100644 --- a/src/util/viralloc.c +++ b/src/util/viralloc.c @@ -45,10 +45,7 @@ VIR_LOG_INIT("util.alloc"); int virAlloc(void *ptrptr, size_t size) { - *(void **)ptrptr = calloc(1, size); - if (*(void **)ptrptr == NULL) - abort(); - + *(void **)ptrptr = g_malloc0(size); return 0; } @@ -69,10 +66,7 @@ int virAllocN(void *ptrptr, size_t size, size_t count) { - *(void**)ptrptr = calloc(count, size); - if (*(void**)ptrptr == NULL) - abort(); - + *(void**)ptrptr = g_malloc0_n(count, size); return 0; } @@ -94,16 +88,7 @@ int virReallocN(void *ptrptr, size_t size, size_t count) { - void *tmp; - - if (xalloc_oversized(count, size)) - abort(); - - tmp = realloc(*(void**)ptrptr, size * count); - if (!tmp && ((size * count) != 0)) - abort(); - - *(void**)ptrptr = tmp; + *(void **)ptrptr = g_realloc_n(*(void**)ptrptr, size, count); return 0; } @@ -343,9 +328,7 @@ int virAllocVar(void *ptrptr, abort(); alloc_size = struct_size + (element_size * count); - *(void **)ptrptr = calloc(1, alloc_size); - if (*(void **)ptrptr == NULL) - abort(); + *(void **)ptrptr = g_malloc0(alloc_size); return 0; } @@ -362,7 +345,7 @@ void virFree(void *ptrptr) { int save_errno = errno; - free(*(void**)ptrptr); + g_free(*(void**)ptrptr); *(void**)ptrptr = NULL; errno = save_errno; } @@ -395,7 +378,7 @@ void virDispose(void *ptrptr, if (*(void**)ptrptr && count > 0) memset(*(void **)ptrptr, 0, count * element_size); - free(*(void**)ptrptr); + g_free(*(void**)ptrptr); *(void**)ptrptr = NULL; if (countptr) -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:26PM +0100, Daniel P. Berrangé wrote:
Convert the VIR_ALLOC family of APIs with use of the g_malloc family of APIs. Use of VIR_ALLOC related functions should be incrementally phased out over time, allowing return value checks to be dropped. Use of VIR_FREE should be replaced with auto-cleanup whenever possible.
We previously used the 'calloc-posix' gnulib module because mingw does not set errno to ENOMEM on failure.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/util/viralloc.c | 29 ++++++----------------------- 2 files changed, 6 insertions(+), 24 deletions(-)
\o/ Another gnulib module removed. On Friday I was trying to figure out how to incorporate gnulib into meson build system and it's not easy at all and still I did not verify that it actually works so we might need to get rid of gnulib before we can do that conversion. Reviewed-by: Pavel Hrdina <phrdina@redhat.com>

On Mon, Sep 30, 2019 at 12:02:17PM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:26PM +0100, Daniel P. Berrangé wrote:
Convert the VIR_ALLOC family of APIs with use of the g_malloc family of APIs. Use of VIR_ALLOC related functions should be incrementally phased out over time, allowing return value checks to be dropped. Use of VIR_FREE should be replaced with auto-cleanup whenever possible.
We previously used the 'calloc-posix' gnulib module because mingw does not set errno to ENOMEM on failure.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/util/viralloc.c | 29 ++++++----------------------- 2 files changed, 6 insertions(+), 24 deletions(-)
\o/ Another gnulib module removed. On Friday I was trying to figure out how to incorporate gnulib into meson build system and it's not easy at all and still I did not verify that it actually works so we might need to get rid of gnulib before we can do that conversion.
I think it is possible roughly as follows - Create a minimal configure.ac that only runs gnulib, nothing else - In meson, we have to run 'configure' using 'run_command' - Define 'custom_target' that invokes 'make' - Declare a dependancy for the result of the custom_target and add that as a pre-req for every other target The main problem when I was exploring this is that 'run_command' does not show any output from configure. So you don't see the progress messages from configure. This is not nice, but also not the end of the world since this configure script will be much smaller than what we have today, and getting ever smaller as we purge more gnulib.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Sep 30, 2019 at 11:08:09AM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 12:02:17PM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:26PM +0100, Daniel P. Berrangé wrote:
Convert the VIR_ALLOC family of APIs with use of the g_malloc family of APIs. Use of VIR_ALLOC related functions should be incrementally phased out over time, allowing return value checks to be dropped. Use of VIR_FREE should be replaced with auto-cleanup whenever possible.
We previously used the 'calloc-posix' gnulib module because mingw does not set errno to ENOMEM on failure.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/util/viralloc.c | 29 ++++++----------------------- 2 files changed, 6 insertions(+), 24 deletions(-)
\o/ Another gnulib module removed. On Friday I was trying to figure out how to incorporate gnulib into meson build system and it's not easy at all and still I did not verify that it actually works so we might need to get rid of gnulib before we can do that conversion.
I think it is possible roughly as follows
- Create a minimal configure.ac that only runs gnulib, nothing else - In meson, we have to run 'configure' using 'run_command' - Define 'custom_target' that invokes 'make' - Declare a dependancy for the result of the custom_target and add that as a pre-req for every other target
The main problem when I was exploring this is that 'run_command' does not show any output from configure. So you don't see the progress messages from configure. This is not nice, but also not the end of the world since this configure script will be much smaller than what we have today, and getting ever smaller as we purge more gnulib.
I have it in a similar way except that I don't use 'run_command' but only 'custom_target' as I don't want to pollute source dire. It actually compiles and the static library libgnu.a is generated. I just did not compile any executable that would actually use gnulib and tested it that it works. But I'll definitely try to finish it and post some RFC patches to list so we can discuss that solution and figure out what to do. Pavel

On Mon, Sep 30, 2019 at 12:18:11PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 11:08:09AM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 12:02:17PM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:26PM +0100, Daniel P. Berrangé wrote:
Convert the VIR_ALLOC family of APIs with use of the g_malloc family of APIs. Use of VIR_ALLOC related functions should be incrementally phased out over time, allowing return value checks to be dropped. Use of VIR_FREE should be replaced with auto-cleanup whenever possible.
We previously used the 'calloc-posix' gnulib module because mingw does not set errno to ENOMEM on failure.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/util/viralloc.c | 29 ++++++----------------------- 2 files changed, 6 insertions(+), 24 deletions(-)
\o/ Another gnulib module removed. On Friday I was trying to figure out how to incorporate gnulib into meson build system and it's not easy at all and still I did not verify that it actually works so we might need to get rid of gnulib before we can do that conversion.
I think it is possible roughly as follows
- Create a minimal configure.ac that only runs gnulib, nothing else - In meson, we have to run 'configure' using 'run_command' - Define 'custom_target' that invokes 'make' - Declare a dependancy for the result of the custom_target and add that as a pre-req for every other target
The main problem when I was exploring this is that 'run_command' does not show any output from configure. So you don't see the progress messages from configure. This is not nice, but also not the end of the world since this configure script will be much smaller than what we have today, and getting ever smaller as we purge more gnulib.
I have it in a similar way except that I don't use 'run_command' but only 'custom_target' as I don't want to pollute source dire. It actually compiles and the static library libgnu.a is generated. I just did not compile any executable that would actually use gnulib and tested it that it works. But I'll definitely try to finish it and post some RFC patches to list so we can discuss that solution and figure out what to do.
Looks like wiring up a custom target as a dependancy is not quite as simple as I thought. There's a recentish enhancement to support it though: https://github.com/mesonbuild/meson/pull/5103 Seems to need a fairly new meson though (>= 0.51 IIUC) Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Convert the string duplication APIs to use the g_strdup family of APIs. Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat with these semantics we have a call to strlen which costs CPU time. We previously used the 'strdup-posix' gnulib module because mingw does not set errno to ENOMEM on failure We previously used the 'strndup' gnulib module because this function does not exist on mingw. We previously used the 'vasprintf' gnulib module because of many GNU supported format specifiers not working on non-Linux platforms. glib's own equivalent standardizes on GNU format specifiers too. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 3 --- src/util/virstring.c | 19 +++++++------------ 2 files changed, 7 insertions(+), 15 deletions(-) diff --git a/bootstrap.conf b/bootstrap.conf index 549d18c6d4..b6b75f9301 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -100,8 +100,6 @@ stat-time stdarg stpcpy strchrnul -strdup-posix -strndup strerror strerror_r-posix strptime @@ -117,7 +115,6 @@ uname unsetenv useless-if-before-free usleep -vasprintf verify vc-list-files vsnprintf diff --git a/src/util/virstring.c b/src/util/virstring.c index a4cc7e9c0a..c8c888b2a0 100644 --- a/src/util/virstring.c +++ b/src/util/virstring.c @@ -730,12 +730,9 @@ virVasprintfInternal(char **strp, const char *fmt, va_list list) { - int ret; + *strp = g_strdup_vprintf(fmt, list); - if ((ret = vasprintf(strp, fmt, list)) == -1) - abort(); - - return ret; + return strlen(*strp); } int @@ -743,12 +740,12 @@ virAsprintfInternal(char **strp, const char *fmt, ...) { va_list ap; - int ret; va_start(ap, fmt); - ret = virVasprintfInternal(strp, fmt, ap); + *strp = g_strdup_vprintf(fmt, ap); va_end(ap); - return ret; + + return strlen(*strp); } /** @@ -936,8 +933,7 @@ virStrdup(char **dest, *dest = NULL; if (!src) return 0; - if (!(*dest = strdup(src))) - abort(); + *dest = g_strdup(src); return 1; } @@ -965,8 +961,7 @@ virStrndup(char **dest, return 0; if (n < 0) n = strlen(src); - if (!(*dest = strndup(src, n))) - abort(); + *dest = g_strndup(src, n); return 1; } -- 2.21.0

On Fri, Sep 27, 2019 at 18:17:27 +0100, Daniel Berrange wrote:
Convert the string duplication APIs to use the g_strdup family of APIs.
Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat with these semantics we have a call to strlen which costs CPU time.
I'd rather refactor the handful of cases which care about the formatted length and drop the strlens.

On Mon, Sep 30, 2019 at 10:37:53AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:27 +0100, Daniel Berrange wrote:
Convert the string duplication APIs to use the g_strdup family of APIs.
Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat
The actual percentage is way closer to 100%, the only function that actually uses the return value of virAsprintf is virNWFilterSnoopLeaseFileWrite. And the only other function needing adjustment is libxlDomainCleanup where we check for > 0 instead of >= 0 like in other places. Also, making virAsprintf return 0/-1 will actually fix the return value of virLogSetDefaultOutput to match its documentation. Jano
with these semantics we have a call to strlen which costs CPU time.
I'd rather refactor the handful of cases which care about the formatted length and drop the strlens.
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

On Mon, Sep 30, 2019 at 01:32:50PM +0200, Ján Tomko wrote:
On Mon, Sep 30, 2019 at 10:37:53AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:27 +0100, Daniel Berrange wrote:
Convert the string duplication APIs to use the g_strdup family of APIs.
Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat
The actual percentage is way closer to 100%, the only function that actually uses the return value of virAsprintf is virNWFilterSnoopLeaseFileWrite.
Something inthe test suite uses one of these functions, because when I did have it returning 0, the test suite never printed any output at all :-) Given the huge number of calls I didn't want to check them all for bugs.
And the only other function needing adjustment is libxlDomainCleanup where we check for > 0 instead of >= 0 like in other places.
Also, making virAsprintf return 0/-1 will actually fix the return value of virLogSetDefaultOutput to match its documentation.
Jano
with these semantics we have a call to strlen which costs CPU time.
I'd rather refactor the handful of cases which care about the formatted length and drop the strlens.
I'd certainly like to Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Fri, Sep 27, 2019 at 06:17:27PM +0100, Daniel P. Berrangé wrote:
Convert the string duplication APIs to use the g_strdup family of APIs.
Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat with these semantics we have a call to strlen which costs CPU time.
We previously used the 'strdup-posix' gnulib module because mingw does not set errno to ENOMEM on failure
We previously used the 'strndup' gnulib module because this function does not exist on mingw.
We previously used the 'vasprintf' gnulib module because of many GNU supported format specifiers not working on non-Linux platforms. glib's own equivalent standardizes on GNU format specifiers too.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 3 --- src/util/virstring.c | 19 +++++++------------ 2 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/bootstrap.conf b/bootstrap.conf index 549d18c6d4..b6b75f9301 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -100,8 +100,6 @@ stat-time stdarg stpcpy strchrnul -strdup-posix -strndup strerror strerror_r-posix strptime @@ -117,7 +115,6 @@ uname unsetenv useless-if-before-free usleep -vasprintf verify vc-list-files vsnprintf diff --git a/src/util/virstring.c b/src/util/virstring.c index a4cc7e9c0a..c8c888b2a0 100644 --- a/src/util/virstring.c +++ b/src/util/virstring.c @@ -730,12 +730,9 @@ virVasprintfInternal(char **strp, const char *fmt, va_list list) { - int ret; + *strp = g_strdup_vprintf(fmt, list);
- if ((ret = vasprintf(strp, fmt, list)) == -1) - abort(); - - return ret; + return strlen(*strp);
This will cause a SEGFAULT if strp is NULL as g_strdup_vprintf doesn't abort on failure. We can use g_vasprintf which returns length. But if we want to return only -1 or 0 and let the caller to decide on the length there are only few places to modify. src/nwfilter/nwfilter_dhcpsnoop.c:1770 src/util/virfile.c:3410 These two looks like the only cases where we actually care about the length. There are some other cases for which we would have to only tweak to comparison: src/libxl/libxl_domain.c:916: There is a function virDoubleToStr that returns the length but it's usage doesn't care about the length so we would have to change the description. Pavel

On Mon, Sep 30, 2019 at 01:35:36PM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:27PM +0100, Daniel P. Berrangé wrote:
Convert the string duplication APIs to use the g_strdup family of APIs.
Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat with these semantics we have a call to strlen which costs CPU time.
We previously used the 'strdup-posix' gnulib module because mingw does not set errno to ENOMEM on failure
We previously used the 'strndup' gnulib module because this function does not exist on mingw.
We previously used the 'vasprintf' gnulib module because of many GNU supported format specifiers not working on non-Linux platforms. glib's own equivalent standardizes on GNU format specifiers too.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 3 --- src/util/virstring.c | 19 +++++++------------ 2 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/bootstrap.conf b/bootstrap.conf index 549d18c6d4..b6b75f9301 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -100,8 +100,6 @@ stat-time stdarg stpcpy strchrnul -strdup-posix -strndup strerror strerror_r-posix strptime @@ -117,7 +115,6 @@ uname unsetenv useless-if-before-free usleep -vasprintf verify vc-list-files vsnprintf diff --git a/src/util/virstring.c b/src/util/virstring.c index a4cc7e9c0a..c8c888b2a0 100644 --- a/src/util/virstring.c +++ b/src/util/virstring.c @@ -730,12 +730,9 @@ virVasprintfInternal(char **strp, const char *fmt, va_list list) { - int ret; + *strp = g_strdup_vprintf(fmt, list);
- if ((ret = vasprintf(strp, fmt, list)) == -1) - abort(); - - return ret; + return strlen(*strp);
This will cause a SEGFAULT if strp is NULL as g_strdup_vprintf doesn't abort on failure.
We can use g_vasprintf which returns length.
Oh yes, that makes life easier.
But if we want to return only -1 or 0 and let the caller to decide on the length there are only few places to modify.
src/nwfilter/nwfilter_dhcpsnoop.c:1770 src/util/virfile.c:3410
These two looks like the only cases where we actually care about the length. There are some other cases for which we would have to only tweak to comparison:
src/libxl/libxl_domain.c:916:
There is a function virDoubleToStr that returns the length but it's usage doesn't care about the length so we would have to change the description.
There something else hiding as we break all test suite output if we return 0 Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Sep 30, 2019 at 12:42:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:35:36PM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:27PM +0100, Daniel P. Berrangé wrote:
Convert the string duplication APIs to use the g_strdup family of APIs.
Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat with these semantics we have a call to strlen which costs CPU time.
We previously used the 'strdup-posix' gnulib module because mingw does not set errno to ENOMEM on failure
We previously used the 'strndup' gnulib module because this function does not exist on mingw.
We previously used the 'vasprintf' gnulib module because of many GNU supported format specifiers not working on non-Linux platforms. glib's own equivalent standardizes on GNU format specifiers too.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 3 --- src/util/virstring.c | 19 +++++++------------ 2 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/bootstrap.conf b/bootstrap.conf index 549d18c6d4..b6b75f9301 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -100,8 +100,6 @@ stat-time stdarg stpcpy strchrnul -strdup-posix -strndup strerror strerror_r-posix strptime @@ -117,7 +115,6 @@ uname unsetenv useless-if-before-free usleep -vasprintf verify vc-list-files vsnprintf diff --git a/src/util/virstring.c b/src/util/virstring.c index a4cc7e9c0a..c8c888b2a0 100644 --- a/src/util/virstring.c +++ b/src/util/virstring.c @@ -730,12 +730,9 @@ virVasprintfInternal(char **strp, const char *fmt, va_list list) { - int ret; + *strp = g_strdup_vprintf(fmt, list);
- if ((ret = vasprintf(strp, fmt, list)) == -1) - abort(); - - return ret; + return strlen(*strp);
This will cause a SEGFAULT if strp is NULL as g_strdup_vprintf doesn't abort on failure.
We can use g_vasprintf which returns length.
Oh yes, that makes life easier.
But if we want to return only -1 or 0 and let the caller to decide on the length there are only few places to modify.
src/nwfilter/nwfilter_dhcpsnoop.c:1770 src/util/virfile.c:3410
[0]
These two looks like the only cases where we actually care about the length. There are some other cases for which we would have to only tweak to comparison:
src/libxl/libxl_domain.c:916:
There is a function virDoubleToStr that returns the length but it's usage doesn't care about the length so we would have to change the description.
There something else hiding as we break all test suite output if we return 0
Yes, that's the virFilePrintf function Pavel pointed out above [0] Jano
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

On Mon, Sep 30, 2019 at 01:35:36PM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:27PM +0100, Daniel P. Berrangé wrote:
Convert the string duplication APIs to use the g_strdup family of APIs.
Annoyingly our virVasprintf/virAsprintf functions return the character count, even though 90% of our usage doesn't need it. To retain compat with these semantics we have a call to strlen which costs CPU time.
We previously used the 'strdup-posix' gnulib module because mingw does not set errno to ENOMEM on failure
We previously used the 'strndup' gnulib module because this function does not exist on mingw.
We previously used the 'vasprintf' gnulib module because of many GNU supported format specifiers not working on non-Linux platforms. glib's own equivalent standardizes on GNU format specifiers too.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 3 --- src/util/virstring.c | 19 +++++++------------ 2 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/bootstrap.conf b/bootstrap.conf index 549d18c6d4..b6b75f9301 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -100,8 +100,6 @@ stat-time stdarg stpcpy strchrnul -strdup-posix -strndup strerror strerror_r-posix strptime @@ -117,7 +115,6 @@ uname unsetenv useless-if-before-free usleep -vasprintf verify vc-list-files vsnprintf diff --git a/src/util/virstring.c b/src/util/virstring.c index a4cc7e9c0a..c8c888b2a0 100644 --- a/src/util/virstring.c +++ b/src/util/virstring.c @@ -730,12 +730,9 @@ virVasprintfInternal(char **strp, const char *fmt, va_list list) { - int ret; + *strp = g_strdup_vprintf(fmt, list);
- if ((ret = vasprintf(strp, fmt, list)) == -1) - abort(); - - return ret; + return strlen(*strp);
This will cause a SEGFAULT if strp is NULL as g_strdup_vprintf doesn't abort on failure.
I spent a long time investigating this.... g_strdup_vprintf calls g_vasprintf() which in turn has 3 impls. 2 out of the 3 impls will abort on OOM, but one won't. The one we use on Linux is the one that won't abort. No application code that I can find ever checks the return value of g_strdup_vprintf or the output string of g_vasprintf. I eventually found a bug indicating the lack of abort on OOM is indeed considered a mistake: https://gitlab.gnome.org/GNOME/glib/issues/1622 I've thus sent a patch to force an abort on OOM: https://gitlab.gnome.org/GNOME/glib/merge_requests/1145 Thus I think from libvirt's POV we can assume this aborts on OOM, since every single other application using this does the same. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Replace use of the gnulib base64 module with glib's own base64 API family. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/conf/virsecretobj.c | 38 +++++++------------------------ src/libvirt_private.syms | 1 - src/libxl/libxl_conf.c | 3 +-- src/qemu/qemu_agent.c | 9 +++----- src/qemu/qemu_command.c | 5 ++-- src/qemu/qemu_domain.c | 8 +++---- src/secret/secret_driver.c | 1 - src/storage/storage_backend_rbd.c | 4 +--- src/util/virstring.c | 21 ----------------- src/util/virstring.h | 2 -- tools/virsh-secret.c | 17 ++++---------- 12 files changed, 22 insertions(+), 88 deletions(-) diff --git a/bootstrap.conf b/bootstrap.conf index b6b75f9301..383e19fa70 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -21,7 +21,6 @@ gnulib_modules=' accept areadlink autobuild -base64 bind bitrotate byteswap diff --git a/src/conf/virsecretobj.c b/src/conf/virsecretobj.c index 7800912bff..4fecb33cd3 100644 --- a/src/conf/virsecretobj.c +++ b/src/conf/virsecretobj.c @@ -31,7 +31,6 @@ #include "virhash.h" #include "virlog.h" #include "virstring.h" -#include "base64.h" #define VIR_FROM_THIS VIR_FROM_SECRET @@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL; if (!obj->value) return 0; - if (!(base64 = virStringEncodeBase64(obj->value, obj->value_size))) - goto cleanup; + base64 = g_base64_encode(obj->value, obj->value_size); if (virFileRewriteStr(obj->base64File, S_IRUSR | S_IWUSR, base64) < 0) - goto cleanup; - - ret = 0; + return -1; - cleanup: - VIR_FREE(base64); - return ret; + return 0; } @@ -835,8 +828,7 @@ virSecretLoadValue(virSecretObjPtr obj) { int ret = -1, fd = -1; struct stat st; - char *contents = NULL, *value = NULL; - size_t value_size; + char *contents = NULL; if ((fd = open(obj->base64File, O_RDONLY)) == -1) { if (errno == ENOENT) { @@ -861,7 +853,7 @@ virSecretLoadValue(virSecretObjPtr obj) goto cleanup; } - if (VIR_ALLOC_N(contents, st.st_size) < 0) + if (VIR_ALLOC_N(contents, st.st_size + 1) < 0) goto cleanup; if (saferead(fd, contents, st.st_size) != st.st_size) { @@ -869,29 +861,15 @@ virSecretLoadValue(virSecretObjPtr obj) obj->base64File); goto cleanup; } + contents[st.st_size] = '\0'; VIR_FORCE_CLOSE(fd); - if (!base64_decode_alloc(contents, st.st_size, &value, &value_size)) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("invalid base64 in '%s'"), - obj->base64File); - goto cleanup; - } - if (value == NULL) - goto cleanup; - - obj->value = (unsigned char *)value; - value = NULL; - obj->value_size = value_size; + obj->value = g_base64_decode(contents, &obj->value_size); ret = 0; cleanup: - if (value != NULL) { - memset(value, 0, value_size); - VIR_FREE(value); - } if (contents != NULL) { memset(contents, 0, st.st_size); VIR_FREE(contents); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 7b681fac64..3b5bb35375 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3068,7 +3068,6 @@ virSkipSpacesBackwards; virStrcpy; virStrdup; virStringBufferIsPrintable; -virStringEncodeBase64; virStringFilterChars; virStringHasCaseSuffix; virStringHasChars; diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index c76704a11d..de56567cf0 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -999,8 +999,7 @@ libxlMakeNetworkDiskSrc(virStorageSourcePtr src, char **srcstr) goto cleanup; /* RBD expects an encoded secret */ - if (!(base64secret = virStringEncodeBase64(secret, secretlen))) - goto cleanup; + base64secret = g_base64_encode(secret, secretlen); } if (!(*srcstr = libxlMakeNetworkDiskSrcStr(src, username, base64secret))) diff --git a/src/qemu/qemu_agent.c b/src/qemu/qemu_agent.c index 34e1a85d64..f1978ad6f1 100644 --- a/src/qemu/qemu_agent.c +++ b/src/qemu/qemu_agent.c @@ -39,7 +39,6 @@ #include "virtime.h" #include "virobject.h" #include "virstring.h" -#include "base64.h" #include "virenum.h" #define VIR_FROM_THIS VIR_FROM_QEMU @@ -2516,11 +2515,10 @@ qemuAgentSetUserPassword(qemuAgentPtr mon, int ret = -1; virJSONValuePtr cmd = NULL; virJSONValuePtr reply = NULL; - char *password64 = NULL; + g_autofree char *password64 = NULL; - if (!(password64 = virStringEncodeBase64((unsigned char *)password, - strlen(password)))) - goto cleanup; + password64 = g_base64_encode((unsigned char *)password, + strlen(password)); if (!(cmd = qemuAgentMakeCommand("guest-set-user-password", "b:crypted", crypted, @@ -2538,7 +2536,6 @@ qemuAgentSetUserPassword(qemuAgentPtr mon, cleanup: virJSONValueFree(cmd); virJSONValueFree(reply); - VIR_FREE(password64); return ret; } diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 77470a6037..3d9198c608 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -834,9 +834,8 @@ qemuBuildRBDSecinfoURI(virBufferPtr buf, switch ((qemuDomainSecretInfoType) secinfo->type) { case VIR_DOMAIN_SECRET_INFO_TYPE_PLAIN: - if (!(base64secret = virStringEncodeBase64(secinfo->s.plain.secret, - secinfo->s.plain.secretlen))) - return -1; + base64secret = g_base64_encode(secinfo->s.plain.secret, + secinfo->s.plain.secretlen); virBufferEscape(buf, '\\', ":", ":id=%s", secinfo->s.plain.username); virBufferEscape(buf, '\\', ":", ":key=%s:auth_supported=cephx\\;none", diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 2aa2164953..bfd49beb21 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1436,8 +1436,7 @@ qemuDomainSecretAESSetup(qemuDomainObjPrivatePtr priv, goto cleanup; /* Encode the IV and save that since qemu will need it */ - if (!(secinfo->s.aes.iv = virStringEncodeBase64(raw_iv, ivlen))) - goto cleanup; + secinfo->s.aes.iv = g_base64_encode(raw_iv, ivlen); /* Grab the unencoded secret */ if (virSecretGetSecretString(conn, seclookupdef, usageType, @@ -1454,9 +1453,8 @@ qemuDomainSecretAESSetup(qemuDomainObjPrivatePtr priv, memset(secret, 0, secretlen); /* Now encode the ciphertext and store to be passed to qemu */ - if (!(secinfo->s.aes.ciphertext = virStringEncodeBase64(ciphertext, - ciphertextlen))) - goto cleanup; + secinfo->s.aes.ciphertext = g_base64_encode(ciphertext, + ciphertextlen); ret = 0; diff --git a/src/secret/secret_driver.c b/src/secret/secret_driver.c index ed3bd3c751..13f75ee4fa 100644 --- a/src/secret/secret_driver.c +++ b/src/secret/secret_driver.c @@ -25,7 +25,6 @@ #include <unistd.h> #include "internal.h" -#include "base64.h" #include "datatypes.h" #include "driver.h" #include "virlog.h" diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c index c4781debd8..b10ca1503d 100644 --- a/src/storage/storage_backend_rbd.c +++ b/src/storage/storage_backend_rbd.c @@ -28,7 +28,6 @@ #include "storage_conf.h" #include "viralloc.h" #include "virlog.h" -#include "base64.h" #include "viruuid.h" #include "virstring.h" #include "virrandom.h" @@ -218,8 +217,7 @@ virStorageBackendRBDOpenRADOSConn(virStorageBackendRBDStatePtr ptr, &secret_value, &secret_value_size) < 0) goto cleanup; - if (!(rados_key = virStringEncodeBase64(secret_value, secret_value_size))) - goto cleanup; + rados_key = g_base64_encode(secret_value, secret_value_size); if (virStorageBackendRBDRADOSConfSet(ptr->cluster, "key", rados_key) < 0) diff --git a/src/util/virstring.c b/src/util/virstring.c index c8c888b2a0..0499904eb3 100644 --- a/src/util/virstring.c +++ b/src/util/virstring.c @@ -21,7 +21,6 @@ #include <regex.h> #include <locale.h> -#include "base64.h" #include "c-ctype.h" #include "virstring.h" #include "virthread.h" @@ -1417,26 +1416,6 @@ virStringBufferIsPrintable(const uint8_t *buf, } -/** - * virStringEncodeBase64: - * @buf: buffer of bytes to encode - * @buflen: number of bytes to encode - * - * Encodes @buf to base 64 and returns the resulting string. The caller is - * responsible for freeing the result. - */ -char * -virStringEncodeBase64(const uint8_t *buf, size_t buflen) -{ - char *ret; - - base64_encode_alloc((const char *) buf, buflen, &ret); - if (!ret) - abort(); - - return ret; -} - /** * virStringTrimOptionalNewline: * @str: the string to modify in-place diff --git a/src/util/virstring.h b/src/util/virstring.h index f537f3472e..f5cbc7f91a 100644 --- a/src/util/virstring.h +++ b/src/util/virstring.h @@ -283,8 +283,6 @@ void virStringFilterChars(char *str, const char *valid); bool virStringIsPrintable(const char *str); bool virStringBufferIsPrintable(const uint8_t *buf, size_t buflen); -char *virStringEncodeBase64(const uint8_t *buf, size_t buflen); - void virStringTrimOptionalNewline(char *str); int virStringParsePort(const char *str, diff --git a/tools/virsh-secret.c b/tools/virsh-secret.c index b34ae12bbe..48058bea05 100644 --- a/tools/virsh-secret.c +++ b/tools/virsh-secret.c @@ -22,7 +22,6 @@ #include "virsh-secret.h" #include "internal.h" -#include "base64.h" #include "virbuffer.h" #include "viralloc.h" #include "virfile.h" @@ -192,7 +191,7 @@ cmdSecretSetValue(vshControl *ctl, const vshCmd *cmd) virSecretPtr secret; size_t value_size; const char *base64 = NULL; - char *value; + unsigned char *value; int res; bool ret = false; @@ -202,16 +201,9 @@ cmdSecretSetValue(vshControl *ctl, const vshCmd *cmd) if (vshCommandOptStringReq(ctl, cmd, "base64", &base64) < 0) goto cleanup; - if (!base64_decode_alloc(base64, strlen(base64), &value, &value_size)) { - vshError(ctl, "%s", _("Invalid base64 data")); - goto cleanup; - } - if (value == NULL) { - vshError(ctl, "%s", _("Failed to allocate memory")); - goto cleanup; - } + value = g_base64_decode(base64, &value_size); - res = virSecretSetValue(secret, (unsigned char *)value, value_size, 0); + res = virSecretSetValue(secret, value, value_size, 0); memset(value, 0, value_size); VIR_FREE(value); @@ -267,8 +259,7 @@ cmdSecretGetValue(vshControl *ctl, const vshCmd *cmd) if (value == NULL) goto cleanup; - if (!(base64 = virStringEncodeBase64(value, value_size))) - goto cleanup; + base64 = g_base64_encode(value, value_size); vshPrint(ctl, "%s", base64); ret = true; -- 2.21.0

On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
Replace use of the gnulib base64 module with glib's own base64 API family.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/conf/virsecretobj.c | 38 +++++++------------------------ src/libvirt_private.syms | 1 - src/libxl/libxl_conf.c | 3 +-- src/qemu/qemu_agent.c | 9 +++----- src/qemu/qemu_command.c | 5 ++-- src/qemu/qemu_domain.c | 8 +++---- src/secret/secret_driver.c | 1 - src/storage/storage_backend_rbd.c | 4 +--- src/util/virstring.c | 21 ----------------- src/util/virstring.h | 2 -- tools/virsh-secret.c | 17 ++++---------- 12 files changed, 22 insertions(+), 88 deletions(-)
[...]
@@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL;
I'm not a fan of adding another style here. Either use VIR_FREE(), or convert all VIR_FREE to g_autofree first. I'm aware it will be unavoidable to use the glib auto pointer macro for complex types but we should at least here where it's interchangable have some kind of consistency.

On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
Replace use of the gnulib base64 module with glib's own base64 API family.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/conf/virsecretobj.c | 38 +++++++------------------------ src/libvirt_private.syms | 1 - src/libxl/libxl_conf.c | 3 +-- src/qemu/qemu_agent.c | 9 +++----- src/qemu/qemu_command.c | 5 ++-- src/qemu/qemu_domain.c | 8 +++---- src/secret/secret_driver.c | 1 - src/storage/storage_backend_rbd.c | 4 +--- src/util/virstring.c | 21 ----------------- src/util/virstring.h | 2 -- tools/virsh-secret.c | 17 ++++---------- 12 files changed, 22 insertions(+), 88 deletions(-)
[...]
@@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL;
I'm not a fan of adding another style here. Either use VIR_FREE(), or convert all VIR_FREE to g_autofree first.
I'm aware it will be unavoidable to use the glib auto pointer macro for complex types but we should at least here where it's interchangable have some kind of consistency.
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr. But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers. This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have. Pavel

On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
Replace use of the gnulib base64 module with glib's own base64 API family.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/conf/virsecretobj.c | 38 +++++++------------------------ src/libvirt_private.syms | 1 - src/libxl/libxl_conf.c | 3 +-- src/qemu/qemu_agent.c | 9 +++----- src/qemu/qemu_command.c | 5 ++-- src/qemu/qemu_domain.c | 8 +++---- src/secret/secret_driver.c | 1 - src/storage/storage_backend_rbd.c | 4 +--- src/util/virstring.c | 21 ----------------- src/util/virstring.h | 2 -- tools/virsh-secret.c | 17 ++++---------- 12 files changed, 22 insertions(+), 88 deletions(-)
[...]
@@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL;
I'm not a fan of adding another style here. Either use VIR_FREE(), or convert all VIR_FREE to g_autofree first.
I'm aware it will be unavoidable to use the glib auto pointer macro for complex types but we should at least here where it's interchangable have some kind of consistency.
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist. I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Sep 30, 2019 at 12:45:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
Replace use of the gnulib base64 module with glib's own base64 API family.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/conf/virsecretobj.c | 38 +++++++------------------------ src/libvirt_private.syms | 1 - src/libxl/libxl_conf.c | 3 +-- src/qemu/qemu_agent.c | 9 +++----- src/qemu/qemu_command.c | 5 ++-- src/qemu/qemu_domain.c | 8 +++---- src/secret/secret_driver.c | 1 - src/storage/storage_backend_rbd.c | 4 +--- src/util/virstring.c | 21 ----------------- src/util/virstring.h | 2 -- tools/virsh-secret.c | 17 ++++---------- 12 files changed, 22 insertions(+), 88 deletions(-)
[...]
@@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL;
I'm not a fan of adding another style here. Either use VIR_FREE(), or convert all VIR_FREE to g_autofree first.
I'm aware it will be unavoidable to use the glib auto pointer macro for complex types but we should at least here where it's interchangable have some kind of consistency.
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist.
I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases.
Agreed, for now let's keep all the wrappers but eventually we can remove them to make the code cleaner. Pavel

On Mon, Sep 30, 2019 at 13:56:06 +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 12:45:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
Replace use of the gnulib base64 module with glib's own base64 API family.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> ---
[..]
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist.
I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases.
I agree in case of the other infrastructure for automatic pointers as that will require more changes. In this case I don't see why we shouldn't just replace all use of VIR_AUTOFREE with g_autofree if the idea is to use g_autofree from now on.

On Mon, Sep 30, 2019 at 02:03:53PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 13:56:06 +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 12:45:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
Replace use of the gnulib base64 module with glib's own base64 API family.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> ---
[..]
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist.
I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases.
I agree in case of the other infrastructure for automatic pointers as that will require more changes.
In this case I don't see why we shouldn't just replace all use of VIR_AUTOFREE with g_autofree if the idea is to use g_autofree from now on.
Well that's 1500 uses, across 150 files, so quite a big bang conversion. It would need to be split up quite alot otherwise it will be a backport conflict magnet. Certainly we want to clean this at some point, its just a question of timing. My preference is to focus on things with functional benefit as the higher priority. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Sep 30, 2019 at 13:19:41 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 02:03:53PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 13:56:06 +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 12:45:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote: > Replace use of the gnulib base64 module with glib's own base64 API family. > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > ---
[..]
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist.
I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases.
I agree in case of the other infrastructure for automatic pointers as that will require more changes.
In this case I don't see why we shouldn't just replace all use of VIR_AUTOFREE with g_autofree if the idea is to use g_autofree from now on.
Well that's 1500 uses, across 150 files, so quite a big bang conversion. It would need to be split up quite alot otherwise it will be a backport conflict magnet. Certainly we want to clean this at some point, its just a question of timing.
My preference is to focus on things with functional benefit as the higher priority.
Then please stick with VIR_AUTOFREE for any code you plan to introduce.

On Mon, Sep 30, 2019 at 02:21:46PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 13:19:41 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 02:03:53PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 13:56:06 +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 12:45:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote: > On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote: > > Replace use of the gnulib base64 module with glib's own base64 API family. > > > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > > ---
[..]
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist.
I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases.
I agree in case of the other infrastructure for automatic pointers as that will require more changes.
In this case I don't see why we shouldn't just replace all use of VIR_AUTOFREE with g_autofree if the idea is to use g_autofree from now on.
Well that's 1500 uses, across 150 files, so quite a big bang conversion. It would need to be split up quite alot otherwise it will be a backport conflict magnet. Certainly we want to clean this at some point, its just a question of timing.
My preference is to focus on things with functional benefit as the higher priority.
Then please stick with VIR_AUTOFREE for any code you plan to introduce.
That forces an even bigger switch over at a later date. An incremental conversion is much less painful overall, even if there is a period when there are two styles in use. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 12:45:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
@@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL;
I'm not a fan of adding another style here. Either use VIR_FREE(), or convert all VIR_FREE to g_autofree first.
I'm aware it will be unavoidable to use the glib auto pointer macro for complex types but we should at least here where it's interchangable have some kind of consistency.
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist.
I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases.
Agreed, for now let's keep all the wrappers but eventually we can remove them to make the code cleaner.
Note that we should be able to use VIR_AUTOPTR() instead of g_autoptr() even for objects, by simply registering the same GLib-provided free function with our own macros. That way we could keep using VIR_AUTO* everywhere and then, when we're ready, mechanically switch everything to g_auto* in one fell swoop, without having any point in time where the two styles are coexisting. -- Andrea Bolognani / Red Hat / Virtualization

On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 12:45:56PM +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 01:41:52PM +0200, Pavel Hrdina wrote:
On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
@@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL;
I'm not a fan of adding another style here. Either use VIR_FREE(), or convert all VIR_FREE to g_autofree first.
I'm aware it will be unavoidable to use the glib auto pointer macro for complex types but we should at least here where it's interchangable have some kind of consistency.
Here I agree with Peter, for this series I would use VIR_FREE() where it's possible and only for glib objects we can use g_autoptr.
But eventually I would like to switch to g_autofree and friends in order to eliminate our specific helpers in favor of glib helpers.
This also brings a question if we should keep our wrappers for glib or use it directly. For example the string functions that we have.
Where any libvirt code just duplicates something that alrady exists, then I think there's no compelling reason to keep it, the best code is code that doesn't exist.
I don't want todo too many big bang replacements though, so I think best to deprecate existing libvirt code and phase it out incrementally in many cases.
Agreed, for now let's keep all the wrappers but eventually we can remove them to make the code cleaner.
Note that we should be able to use VIR_AUTOPTR() instead of g_autoptr() even for objects, by simply registering the same GLib-provided free function with our own macros.
That way we could keep using VIR_AUTO* everywhere and then, when we're ready, mechanically switch everything to g_auto* in one fell swoop, without having any point in time where the two styles are coexisting.
That creates an even bigger conversion later. Such big conversions cause more pain for backports, than doing an incremental conversion at appropriate times. Converting everything to g_autofree right now doesn't give style consistency as we'd still be matching VIR_ALLOC + g_autofree, so I don't see a benefit to a big conversion in 1 step. Incrementally converting VIR_ALLOC + VIR_AUTOFREE at the same time, makes more sense stylewise, as then within the scope of a single method we'd be consistent. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, 2019-09-30 at 13:41 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
Agreed, for now let's keep all the wrappers but eventually we can remove them to make the code cleaner.
Note that we should be able to use VIR_AUTOPTR() instead of g_autoptr() even for objects, by simply registering the same GLib-provided free function with our own macros.
That way we could keep using VIR_AUTO* everywhere and then, when we're ready, mechanically switch everything to g_auto* in one fell swoop, without having any point in time where the two styles are coexisting.
That creates an even bigger conversion later. Such big conversions cause more pain for backports, than doing an incremental conversion at appropriate times.
Converting everything to g_autofree right now doesn't give style consistency as we'd still be matching VIR_ALLOC + g_autofree, so I don't see a benefit to a big conversion in 1 step.
Incrementally converting VIR_ALLOC + VIR_AUTOFREE at the same time, makes more sense stylewise, as then within the scope of a single method we'd be consistent.
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether. Doing the conversion incrementally will IMHO result in dragging it for much longer, causing more pain in the long run than ripping the bandaid would. -- Andrea Bolognani / Red Hat / Virtualization

On Mon, Sep 30, 2019 at 04:05:57PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:41 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
Agreed, for now let's keep all the wrappers but eventually we can remove them to make the code cleaner.
Note that we should be able to use VIR_AUTOPTR() instead of g_autoptr() even for objects, by simply registering the same GLib-provided free function with our own macros.
That way we could keep using VIR_AUTO* everywhere and then, when we're ready, mechanically switch everything to g_auto* in one fell swoop, without having any point in time where the two styles are coexisting.
That creates an even bigger conversion later. Such big conversions cause more pain for backports, than doing an incremental conversion at appropriate times.
Converting everything to g_autofree right now doesn't give style consistency as we'd still be matching VIR_ALLOC + g_autofree, so I don't see a benefit to a big conversion in 1 step.
Incrementally converting VIR_ALLOC + VIR_AUTOFREE at the same time, makes more sense stylewise, as then within the scope of a single method we'd be consistent.
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether.
Doing the conversion incrementally will IMHO result in dragging it for much longer, causing more pain in the long run than ripping the bandaid would.
There's really not any significant real world pain from mixing the two styles. It is visually distasteful but doesn't cause any functional problems at runtime, nor complexity for maintainers. A large conversion over the whole codebase does cause very significant pain in conflicts for anyone cherry picking patches. That is just not a net win overall. I'll take visually mixed styles any day over creating patch conflicts in backports. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Sep 30, 2019 at 15:53:35 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 04:05:57PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:41 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
[...]
Incrementally converting VIR_ALLOC + VIR_AUTOFREE at the same time, makes more sense stylewise, as then within the scope of a single method we'd be consistent.
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether.
Doing the conversion incrementally will IMHO result in dragging it for much longer, causing more pain in the long run than ripping the bandaid would.
There's really not any significant real world pain from mixing the two styles. It is visually distasteful but doesn't cause any functional problems at runtime, nor complexity for maintainers. A large conversion over the whole codebase does cause very significant pain in conflicts for anyone cherry picking patches. That is just not a net win overall. I'll take visually mixed styles any day over creating patch conflicts in backports.
I don't see how. If the end-goal is to convert everything to the new form you will get into potential pain/conflicts sooner or later anyways. Or the other option is to leave it as a half-done lingering refactor and that doesn't help either.

On Mon, Sep 30, 2019 at 05:03:47PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 15:53:35 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 04:05:57PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:41 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
[...]
Incrementally converting VIR_ALLOC + VIR_AUTOFREE at the same time, makes more sense stylewise, as then within the scope of a single method we'd be consistent.
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether.
Doing the conversion incrementally will IMHO result in dragging it for much longer, causing more pain in the long run than ripping the bandaid would.
There's really not any significant real world pain from mixing the two styles. It is visually distasteful but doesn't cause any functional problems at runtime, nor complexity for maintainers. A large conversion over the whole codebase does cause very significant pain in conflicts for anyone cherry picking patches. That is just not a net win overall. I'll take visually mixed styles any day over creating patch conflicts in backports.
I don't see how. If the end-goal is to convert everything to the new form you will get into potential pain/conflicts sooner or later anyways.
If we incrementally convert methods, then when backporting a patch related to that method, we have good chance of being able to cherry-pick the small conversion patch. If we bulk convert entire file at a time, across the whole codebase, attempting to cherry-pick the conversion patches will have much higher conflict liklihood.
Or the other option is to leave it as a half-done lingering refactor and that doesn't help either.
It don't be in a half-done state forever. We can let things be converted incrementally over the next 3-6 months. At the end of say 6 months if anything is left we bulk convert it them. That gets the benefits opf incremental work without downside of stuff remaining unconverted forever. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Sep 30, 2019 at 16:10:11 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 05:03:47PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 15:53:35 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 04:05:57PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:41 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
[...]
There's really not any significant real world pain from mixing the two styles. It is visually distasteful but doesn't cause any functional problems at runtime, nor complexity for maintainers. A large conversion over the whole codebase does cause very significant pain in conflicts for anyone cherry picking patches. That is just not a net win overall. I'll take visually mixed styles any day over creating patch conflicts in backports.
I don't see how. If the end-goal is to convert everything to the new form you will get into potential pain/conflicts sooner or later anyways.
If we incrementally convert methods, then when backporting a patch related to that method, we have good chance of being able to cherry-pick the small conversion patch. If we bulk convert entire file at a time, across the whole codebase, attempting to cherry-pick the conversion patches will have much higher conflict liklihood.
I doubt that there will be any motivation to start a incremental coversion. While when adding VIR_AUTOFREE and coverting to it there was a clear benefit of simplification in doing so, converting from VIR_AUTOFREE to g_autofree has exactly 0 benefit.
Or the other option is to leave it as a half-done lingering refactor and that doesn't help either.
It don't be in a half-done state forever. We can let things be converted incrementally over the next 3-6 months. At the end of say 6 months if anything is left we bulk convert it them. That gets the benefits opf incremental work without downside of stuff remaining unconverted forever.
How is this not a big-bang conversion?

On Mon, Sep 30, 2019 at 05:24:00PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 16:10:11 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 05:03:47PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 15:53:35 +0100, Daniel Berrange wrote:
On Mon, Sep 30, 2019 at 04:05:57PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:41 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote: > On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
[...]
There's really not any significant real world pain from mixing the two styles. It is visually distasteful but doesn't cause any functional problems at runtime, nor complexity for maintainers. A large conversion over the whole codebase does cause very significant pain in conflicts for anyone cherry picking patches. That is just not a net win overall. I'll take visually mixed styles any day over creating patch conflicts in backports.
I don't see how. If the end-goal is to convert everything to the new form you will get into potential pain/conflicts sooner or later anyways.
If we incrementally convert methods, then when backporting a patch related to that method, we have good chance of being able to cherry-pick the small conversion patch. If we bulk convert entire file at a time, across the whole codebase, attempting to cherry-pick the conversion patches will have much higher conflict liklihood.
I doubt that there will be any motivation to start a incremental coversion. While when adding VIR_AUTOFREE and coverting to it there was a clear benefit of simplification in doing so, converting from VIR_AUTOFREE to g_autofree has exactly 0 benefit.
The motivation for conversion is something under our direct control as reviewers. eg we could define a coding style rule that any function which makes a call to any glib API (ie a g_XXX prefix) must be converted as a prior step.
Or the other option is to leave it as a half-done lingering refactor and that doesn't help either.
It don't be in a half-done state forever. We can let things be converted incrementally over the next 3-6 months. At the end of say 6 months if anything is left we bulk convert it them. That gets the benefits opf incremental work without downside of stuff remaining unconverted forever.
How is this not a big-bang conversion?
Most of the conversion patches will be small & targetted. Obviously it assumes that 95% of the stuff gets converted this way during the first period. So the final conversion at the end is quite small. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, 2019-09-30 at 16:34 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 05:24:00PM +0200, Peter Krempa wrote:
On Mon, Sep 30, 2019 at 16:10:11 +0100, Daniel Berrange wrote:
It don't be in a half-done state forever. We can let things be converted incrementally over the next 3-6 months. At the end of say 6 months if anything is left we bulk convert it them. That gets the benefits opf incremental work without downside of stuff remaining unconverted forever.
How is this not a big-bang conversion?
Most of the conversion patches will be small & targetted. Obviously it assumes that 95% of the stuff gets converted this way during the first period. So the final conversion at the end is quite small.
But there's no GSoC planned for the next six months! On a more serious note, I don't think this expectation is realistic, and at the end of it all the we'll just end up doing pretty much the same big-bang conversion, only with a few months of neither here nor there in between. -- Andrea Bolognani / Red Hat / Virtualization

On Mon, 2019-09-30 at 15:53 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 04:05:57PM +0200, Andrea Bolognani wrote:
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether.
Doing the conversion incrementally will IMHO result in dragging it for much longer, causing more pain in the long run than ripping the bandaid would.
There's really not any significant real world pain from mixing the two styles. It is visually distasteful but doesn't cause any functional problems at runtime, nor complexity for maintainers. A large conversion over the whole codebase does cause very significant pain in conflicts for anyone cherry picking patches. That is just not a net win overall. I'll take visually mixed styles any day over creating patch conflicts in backports.
If we allow both at the same time, then we'll keep using both going forward because there's no incentive *not* to mix the two styles; one of the stated advantages of adopting GLib, making the libvirt code base more approachable to people familiar with QEMU and other GLib-using projects, will then not be realized. Both the conversion from one style to the other and, consequently, addressing any conflict resulting from it, can be tackled in a purely mechanical fashion: a bit annoying, sure, but hardly worth keeping the conversion ongoing forever for in my opinion. -- Andrea Bolognani / Red Hat / Virtualization

On Mon, Sep 30, 2019 at 05:17:32PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 15:53 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 04:05:57PM +0200, Andrea Bolognani wrote:
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether.
Doing the conversion incrementally will IMHO result in dragging it for much longer, causing more pain in the long run than ripping the bandaid would.
There's really not any significant real world pain from mixing the two styles. It is visually distasteful but doesn't cause any functional problems at runtime, nor complexity for maintainers. A large conversion over the whole codebase does cause very significant pain in conflicts for anyone cherry picking patches. That is just not a net win overall. I'll take visually mixed styles any day over creating patch conflicts in backports.
If we allow both at the same time, then we'll keep using both going forward because there's no incentive *not* to mix the two styles; one of the stated advantages of adopting GLib, making the libvirt code base more approachable to people familiar with QEMU and other GLib-using projects, will then not be realized.
No matter what we do, Realizing the advantage of familiarity with GLib is not going to be something we win overnight. We have far too much existing code for that to happen, so this familiarity is always something that take time for us to achieve.
Both the conversion from one style to the other and, consequently, addressing any conflict resulting from it, can be tackled in a purely mechanical fashion: a bit annoying, sure, but hardly worth keeping the conversion ongoing forever for in my opinion.
I never suggested we want a conversion to run forever. It absolutely must be time limited, it just doesn't all have to be done in one giant big bang. Spreading it over several releases, with a clear final deadline strikes a balance between getting it done and mitigating pain for backporting. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 9/30/19 10:05 AM, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:41 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 30, 2019 at 02:18:17PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-30 at 13:56 +0200, Pavel Hrdina wrote:
Agreed, for now let's keep all the wrappers but eventually we can remove them to make the code cleaner. Note that we should be able to use VIR_AUTOPTR() instead of g_autoptr() even for objects, by simply registering the same GLib-provided free function with our own macros.
That way we could keep using VIR_AUTO* everywhere and then, when we're ready, mechanically switch everything to g_auto* in one fell swoop, without having any point in time where the two styles are coexisting. That creates an even bigger conversion later. Such big conversions cause more pain for backports, than doing an incremental conversion at appropriate times.
Converting everything to g_autofree right now doesn't give style consistency as we'd still be matching VIR_ALLOC + g_autofree, so I don't see a benefit to a big conversion in 1 step.
Incrementally converting VIR_ALLOC + VIR_AUTOFREE at the same time, makes more sense stylewise, as then within the scope of a single method we'd be consistent. I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether.
Don't forget that make syntax-check doesn't work properly for many downstream maintenance branches that would be backported to (it has to be disabled due to copyright date checks failing, or something like that). Of course that wouldn't be a problem until some future maintenance branch based on a release that uses glib but still has VIR_AUTO* defined. On the other hand, many *current* maintenance branches that must be targeted by backports don't even have the VIR_AUTO* macros (which were introduced in 4.6.0, but *still* aren't used in a lot of the code), so there are going to be backport headaches anyway, and they are going to be of equal pain regardless of whether we do a big bang conversion or gradual conversion. The differences in pain level will be noticed in downstream maintenance branches >= 4.6.0 and <= 5.9.0 (or whatever version).
Doing the conversion incrementally will IMHO result in dragging it for much longer, causing more pain in the long run than ripping the bandaid would.
In order to allay Andrea's fears of new usage of VIR_AUTO* that just draws out the conversion, maybe we could (temporarily, until the conversion is complete) put a commit hook in place to disallow new use of VIR_AUTO ? Or just, you know, pay attention in reviews (but of course part of the point of all of this is to eliminate the potential for human error, by depending less on humans paying attention, so... :-P) (BTW, I'm not firmly in *either* camp, although I may lean a bit more towards a gradual change (but with a *very* steep slope to minimize the period of confusion)

On Mon, 2019-09-30 at 13:13 -0400, Laine Stump wrote:
On 9/30/19 10:05 AM, Andrea Bolognani wrote:
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether.
Don't forget that make syntax-check doesn't work properly for many downstream maintenance branches that would be backported to (it has to be disabled due to copyright date checks failing, or something like that).
That's a problem for downstream to solve. By the same token, all the existing syntax-check rules are pointless because they can't be guaranteed to hold for downstream branches.
In order to allay Andrea's fears of new usage of VIR_AUTO* that just draws out the conversion, maybe we could (temporarily, until the conversion is complete) put a commit hook in place to disallow new use of VIR_AUTO ? Or just, you know, pay attention in reviews (but of course part of the point of all of this is to eliminate the potential for human error, by depending less on humans paying attention, so... :-P)
Writing a check that compares the situation before a commit and after it is not as easy as a point-in-time check. Instead of spending a non-trival amount of time implementing something like that, I'd rather spend my time dealing with the fallout of a one-time conversion.
(BTW, I'm not firmly in *either* camp, although I may lean a bit more towards a gradual change (but with a *very* steep slope to minimize the period of confusion)
That's just a big-bang conversion with extra steps! -- Andrea Bolognani / Red Hat / Virtualization

On 9/30/19 1:35 PM, Andrea Bolognani wrote:
I see your point about backports being more painful when you have a bunch of unrelated changes mixed in, but I would still prefer if we converted everything at once and at the same time introduced a suitable syntax-check rule preventing more instances of whatever function we just removed all callers of from creeping back in, or actually just dropping the function altogether. Don't forget that make syntax-check doesn't work properly for many downstream maintenance branches that would be backported to (it has to be disabled due to copyright date checks failing, or something like
On 9/30/19 10:05 AM, Andrea Bolognani wrote: that). That's a problem for downstream to solve. By the same token, all
On Mon, 2019-09-30 at 13:13 -0400, Laine Stump wrote: the existing syntax-check rules are pointless because they can't be guaranteed to hold for downstream branches.
Yeah, I'm just bitter that make syntax-check doesn't work downstream, and can't let a mention of the two in the same context go by without making a whining comment.
In order to allay Andrea's fears of new usage of VIR_AUTO* that just draws out the conversion, maybe we could (temporarily, until the conversion is complete) put a commit hook in place to disallow new use of VIR_AUTO ? Or just, you know, pay attention in reviews (but of course part of the point of all of this is to eliminate the potential for human error, by depending less on humans paying attention, so... :-P) Writing a check that compares the situation before a commit and after it is not as easy as a point-in-time check.
Not all that bad though - just examine the lines that start with +
Instead of spending a non-trival amount of time implementing something like that, I'd rather spend my time dealing with the fallout of a one-time conversion.
Without seeing concrete examples of what actually is "dealing with the fallout" in both cases, I don't want to speculate too much on which would cause more difficulty. I would say that even if we do the conversion all at once, it should be in multiple patches so that if there is some regression caused by the conversion, a git bisect will lead to a multi-hundred line commit instead of a multi-thousand line commit.
(BTW, I'm not firmly in *either* camp, although I may lean a bit more towards a gradual change (but with a *very* steep slope to minimize the period of confusion) That's just a big-bang conversion with extra steps!
Well, after all *anything* we each do before meeting a sad and lonely demise is really just a part of Heat Death of the Universe with extra steps. You need to subdivide *somewhere*.

On Tue, 2019-10-01 at 10:12 -0400, Laine Stump wrote:
On 9/30/19 1:35 PM, Andrea Bolognani wrote:
Writing a check that compares the situation before a commit and after it is not as easy as a point-in-time check.
Not all that bad though - just examine the lines that start with +
Fair enough. That wouldn't work outside of git, but I guess it would be an acceptable compromise as all our CI jobs are executed from inside a git checkout anyway.
Instead of spending a non-trival amount of time implementing something like that, I'd rather spend my time dealing with the fallout of a one-time conversion.
Without seeing concrete examples of what actually is "dealing with the fallout" in both cases, I don't want to speculate too much on which would cause more difficulty. I would say that even if we do the conversion all at once, it should be in multiple patches so that if there is some regression caused by the conversion, a git bisect will lead to a multi-hundred line commit instead of a multi-thousand line commit.
Of course when I talk about "big-bang conversion" I mean done in a single series, not a single patch! -- Andrea Bolognani / Red Hat / Virtualization

On Mon, Sep 30, 2019 at 09:02:36AM +0200, Peter Krempa wrote:
On Fri, Sep 27, 2019 at 18:17:28 +0100, Daniel Berrange wrote:
Replace use of the gnulib base64 module with glib's own base64 API family.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- bootstrap.conf | 1 - src/conf/virsecretobj.c | 38 +++++++------------------------ src/libvirt_private.syms | 1 - src/libxl/libxl_conf.c | 3 +-- src/qemu/qemu_agent.c | 9 +++----- src/qemu/qemu_command.c | 5 ++-- src/qemu/qemu_domain.c | 8 +++---- src/secret/secret_driver.c | 1 - src/storage/storage_backend_rbd.c | 4 +--- src/util/virstring.c | 21 ----------------- src/util/virstring.h | 2 -- tools/virsh-secret.c | 17 ++++---------- 12 files changed, 22 insertions(+), 88 deletions(-)
[...]
@@ -698,23 +697,17 @@ virSecretObjSaveConfig(virSecretObjPtr obj) int virSecretObjSaveData(virSecretObjPtr obj) { - char *base64 = NULL; - int ret = -1; + g_autofree char *base64 = NULL;
I'm not a fan of adding another style here. Either use VIR_FREE(), or convert all VIR_FREE to g_autofree first.
I'm aware it will be unavoidable to use the glib auto pointer macro for complex types but we should at least here where it's interchangable have some kind of consistency.
I was attempting to be consistent in matching g_new/g_free/g_autofree, but as you say, it probably makes more sense to be consistent within the scope of a single method. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Converting from virObject to GObject is reasonably straightforward, as illustrated by this patch for virIdentity In the header file - Remove typedef struct _virIdentity virIdentity - Add #define VIR_TYPE_IDENTITY virIdentity_get_type () G_DECLARE_FINAL_TYPE (virIdentity, virIdentity, VIR, IDENTITY, GObject); Which provides the typedef we just removed, and class declaration boilerplate and various other constants/macros. In the source file - Change 'virObject parent' to 'GObject parent' in the struct - Remove the virClass variable and its initializing call - Add G_DEFINE_TYPE(virIdentity, virIdentity, G_TYPE_OBJECT) which declares the instance & class constructor functions - Add an impl of the instance & class constructors wiring up the finalize method to point to our dispose impl In all files - Replace VIR_AUTOUNREF(virIdentityPtr) with g_autoptr(virIdentity) - Replace virObjectRef/Unref with g_object_ref/unref. Note the latter functions do *NOT* accept a NULL object where as libvirt's do. If you replace g_object_unref with g_clear_object it is NULL safe, but also clears the pointer. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/access/viraccessdriverpolkit.c | 21 +++---- src/admin/admin_server.c | 3 +- src/qemu/qemu_process.c | 4 +- src/remote/remote_daemon.c | 3 +- src/remote/remote_daemon_dispatch.c | 35 ++++-------- src/rpc/virnetserverclient.c | 57 ++++++++----------- src/rpc/virnetserverprogram.c | 13 +---- src/util/viridentity.c | 87 ++++++++++++++++------------- src/util/viridentity.h | 7 ++- tests/viridentitytest.c | 45 ++++++--------- tests/virnetserverclienttest.c | 3 +- 11 files changed, 122 insertions(+), 156 deletions(-) diff --git a/src/access/viraccessdriverpolkit.c b/src/access/viraccessdriverpolkit.c index e61ac6fa19..b39a28c834 100644 --- a/src/access/viraccessdriverpolkit.c +++ b/src/access/viraccessdriverpolkit.c @@ -78,8 +78,7 @@ virAccessDriverPolkitGetCaller(const char *actionid, unsigned long long *startTime, uid_t *uid) { - virIdentityPtr identity = virIdentityGetCurrent(); - int ret = -1; + g_autoptr(virIdentity) identity = virIdentityGetCurrent(); int rc; if (!identity) { @@ -90,37 +89,33 @@ virAccessDriverPolkitGetCaller(const char *actionid, } if ((rc = virIdentityGetProcessID(identity, pid)) < 0) - goto cleanup; + return -1; if (rc == 0) { virAccessError(VIR_ERR_INTERNAL_ERROR, "%s", _("No process ID available")); - goto cleanup; + return -1; } if ((rc = virIdentityGetProcessTime(identity, startTime)) < 0) - goto cleanup; + return -1; if (rc == 0) { virAccessError(VIR_ERR_INTERNAL_ERROR, "%s", _("No process start time available")); - goto cleanup; + return -1; } if ((rc = virIdentityGetUNIXUserID(identity, uid)) < 0) - goto cleanup; + return -1; if (rc == 0) { virAccessError(VIR_ERR_INTERNAL_ERROR, "%s", _("No UNIX caller UID available")); - goto cleanup; + return -1; } - ret = 0; - - cleanup: - virObjectUnref(identity); - return ret; + return 0; } diff --git a/src/admin/admin_server.c b/src/admin/admin_server.c index 248df3f795..7ae819b7b0 100644 --- a/src/admin/admin_server.c +++ b/src/admin/admin_server.c @@ -221,7 +221,7 @@ adminClientGetInfo(virNetServerClientPtr client, char *sock_addr = NULL; const char *attr = NULL; virTypedParameterPtr tmpparams = NULL; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; int rc; virCheckFlags(0, -1); @@ -310,7 +310,6 @@ adminClientGetInfo(virNetServerClientPtr client, ret = 0; cleanup: - virObjectUnref(identity); VIR_FREE(sock_addr); return ret; } diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a50cd54393..872750ff4d 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -8051,7 +8051,7 @@ qemuProcessReconnect(void *opaque) bool tryMonReconn = false; virIdentitySetCurrent(data->identity); - virObjectUnref(data->identity); + g_clear_object(&data->identity); VIR_FREE(data); qemuDomainObjRestoreJob(obj, &oldjob); @@ -8364,7 +8364,7 @@ qemuProcessReconnectHelper(virDomainObjPtr obj, virDomainObjEndAPI(&obj); virNWFilterUnlockFilterUpdates(); - virObjectUnref(data->identity); + g_clear_object(&data->identity); VIR_FREE(data); return -1; } diff --git a/src/remote/remote_daemon.c b/src/remote/remote_daemon.c index 9281e226c4..7fcaa31c49 100644 --- a/src/remote/remote_daemon.c +++ b/src/remote/remote_daemon.c @@ -826,7 +826,7 @@ handleSystemMessageFunc(DBusConnection *connection ATTRIBUTE_UNUSED, static void daemonRunStateInit(void *opaque) { virNetDaemonPtr dmn = opaque; - virIdentityPtr sysident = virIdentityGetSystem(); + g_autoptr(virIdentity) sysident = virIdentityGetSystem(); #ifdef MODULE_NAME bool mandatory = true; #else /* ! MODULE_NAME */ @@ -879,7 +879,6 @@ static void daemonRunStateInit(void *opaque) cleanup: daemonInhibitCallback(false, dmn); virObjectUnref(dmn); - virObjectUnref(sysident); virIdentitySetCurrent(NULL); } diff --git a/src/remote/remote_daemon_dispatch.c b/src/remote/remote_daemon_dispatch.c index dbd2985c38..9fc5dc3998 100644 --- a/src/remote/remote_daemon_dispatch.c +++ b/src/remote/remote_daemon_dispatch.c @@ -159,7 +159,7 @@ remoteRelayDomainEventCheckACL(virNetServerClientPtr client, virConnectPtr conn, virDomainPtr dom) { virDomainDef def; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; bool ret = false; /* For now, we just create a virDomainDef with enough contents to @@ -177,7 +177,6 @@ remoteRelayDomainEventCheckACL(virNetServerClientPtr client, cleanup: ignore_value(virIdentitySetCurrent(NULL)); - virObjectUnref(identity); return ret; } @@ -187,7 +186,7 @@ remoteRelayNetworkEventCheckACL(virNetServerClientPtr client, virConnectPtr conn, virNetworkPtr net) { virNetworkDef def; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; bool ret = false; /* For now, we just create a virNetworkDef with enough contents to @@ -204,7 +203,6 @@ remoteRelayNetworkEventCheckACL(virNetServerClientPtr client, cleanup: ignore_value(virIdentitySetCurrent(NULL)); - virObjectUnref(identity); return ret; } @@ -214,7 +212,7 @@ remoteRelayStoragePoolEventCheckACL(virNetServerClientPtr client, virStoragePoolPtr pool) { virStoragePoolDef def; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; bool ret = false; /* For now, we just create a virStoragePoolDef with enough contents to @@ -231,7 +229,6 @@ remoteRelayStoragePoolEventCheckACL(virNetServerClientPtr client, cleanup: ignore_value(virIdentitySetCurrent(NULL)); - virObjectUnref(identity); return ret; } @@ -241,7 +238,7 @@ remoteRelayNodeDeviceEventCheckACL(virNetServerClientPtr client, virNodeDevicePtr dev) { virNodeDeviceDef def; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; bool ret = false; /* For now, we just create a virNodeDeviceDef with enough contents to @@ -257,7 +254,6 @@ remoteRelayNodeDeviceEventCheckACL(virNetServerClientPtr client, cleanup: ignore_value(virIdentitySetCurrent(NULL)); - virObjectUnref(identity); return ret; } @@ -267,7 +263,7 @@ remoteRelaySecretEventCheckACL(virNetServerClientPtr client, virSecretPtr secret) { virSecretDef def; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; bool ret = false; /* For now, we just create a virSecretDef with enough contents to @@ -285,7 +281,6 @@ remoteRelaySecretEventCheckACL(virNetServerClientPtr client, cleanup: ignore_value(virIdentitySetCurrent(NULL)); - virObjectUnref(identity); return ret; } @@ -294,7 +289,7 @@ remoteRelayDomainQemuMonitorEventCheckACL(virNetServerClientPtr client, virConnectPtr conn, virDomainPtr dom) { virDomainDef def; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; bool ret = false; /* For now, we just create a virDomainDef with enough contents to @@ -311,7 +306,6 @@ remoteRelayDomainQemuMonitorEventCheckACL(virNetServerClientPtr client, cleanup: ignore_value(virIdentitySetCurrent(NULL)); - virObjectUnref(identity); return ret; } @@ -1869,7 +1863,7 @@ void remoteRelayConnectionClosedEvent(virConnectPtr conn ATTRIBUTE_UNUSED, int r static void remoteClientFreePrivateCallbacks(struct daemonClientPrivate *priv) { - virIdentityPtr sysident = virIdentityGetSystem(); + g_autoptr(virIdentity) sysident = virIdentityGetSystem(); virIdentitySetCurrent(sysident); DEREG_CB(priv->conn, priv->domainEventCallbacks, @@ -1898,7 +1892,6 @@ remoteClientFreePrivateCallbacks(struct daemonClientPrivate *priv) } virIdentitySetCurrent(NULL); - virObjectUnref(sysident); } #undef DEREG_CB @@ -1965,7 +1958,7 @@ remoteOpenConn(const char *uri, } if (preserveIdentity) { - VIR_AUTOUNREF(virIdentityPtr) ident = NULL; + g_autoptr(virIdentity) ident = NULL; if (!(ident = virIdentityGetCurrent())) return -1; @@ -2436,7 +2429,7 @@ remoteDispatchConnectSetIdentity(virNetServerPtr server ATTRIBUTE_UNUSED, int nparams = 0; int rv = -1; virConnectPtr conn = remoteGetHypervisorConn(client); - VIR_AUTOUNREF(virIdentityPtr) ident = NULL; + g_autoptr(virIdentity) ident = NULL; if (!conn) goto cleanup; @@ -3982,7 +3975,7 @@ static int remoteSASLFinish(virNetServerPtr server, virNetServerClientPtr client) { - virIdentityPtr clnt_identity = NULL; + g_autoptr(virIdentity) clnt_identity = NULL; const char *identity; struct daemonClientPrivate *priv = virNetServerClientGetPrivateData(client); int ssf; @@ -3990,7 +3983,7 @@ remoteSASLFinish(virNetServerPtr server, /* TLS or UNIX domain sockets trivially OK */ if (!virNetServerClientIsSecure(client)) { if ((ssf = virNetSASLSessionGetKeySize(priv->sasl)) < 0) - goto error; + return -1; VIR_DEBUG("negotiated an SSF of %d", ssf); if (ssf < 56) { /* 56 is good for Kerberos */ @@ -4006,7 +3999,7 @@ remoteSASLFinish(virNetServerPtr server, return -2; if (!(clnt_identity = virNetServerClientGetIdentity(client))) - goto error; + return -1; virNetServerSetClientAuthenticated(server, client); virNetServerClientSetSASLSession(client, priv->sasl); @@ -4018,14 +4011,10 @@ remoteSASLFinish(virNetServerPtr server, "client=%p auth=%d identity=%s", client, REMOTE_AUTH_SASL, identity); - virObjectUnref(clnt_identity); virObjectUnref(priv->sasl); priv->sasl = NULL; return 0; - - error: - return -1; } /* diff --git a/src/rpc/virnetserverclient.c b/src/rpc/virnetserverclient.c index 171ee636dd..8482c5c29c 100644 --- a/src/rpc/virnetserverclient.c +++ b/src/rpc/virnetserverclient.c @@ -759,13 +759,13 @@ int virNetServerClientGetUNIXIdentity(virNetServerClientPtr client, static virIdentityPtr virNetServerClientCreateIdentity(virNetServerClientPtr client) { - char *username = NULL; - char *groupname = NULL; - char *seccontext = NULL; - virIdentityPtr ret = NULL; + g_autofree char *username = NULL; + g_autofree char *groupname = NULL; + g_autofree char *seccontext = NULL; + g_autoptr(virIdentity) ret = NULL; if (!(ret = virIdentityNew())) - goto error; + return NULL; if (client->sock && virNetSocketIsLocal(client->sock)) { gid_t gid; @@ -775,59 +775,50 @@ virNetServerClientCreateIdentity(virNetServerClientPtr client) if (virNetSocketGetUNIXIdentity(client->sock, &uid, &gid, &pid, ×tamp) < 0) - goto error; + return NULL; if (!(username = virGetUserName(uid))) - goto error; + return NULL; if (virIdentitySetUserName(ret, username) < 0) - goto error; + return NULL; if (virIdentitySetUNIXUserID(ret, uid) < 0) - goto error; + return NULL; if (!(groupname = virGetGroupName(gid))) - goto error; + return NULL; if (virIdentitySetGroupName(ret, groupname) < 0) - goto error; + return NULL; if (virIdentitySetUNIXGroupID(ret, gid) < 0) - goto error; + return NULL; if (virIdentitySetProcessID(ret, pid) < 0) - goto error; + return NULL; if (virIdentitySetProcessTime(ret, timestamp) < 0) - goto error; + return NULL; } #if WITH_SASL if (client->sasl) { const char *identity = virNetSASLSessionGetIdentity(client->sasl); if (virIdentitySetSASLUserName(ret, identity) < 0) - goto error; + return NULL; } #endif if (client->tls) { const char *identity = virNetTLSSessionGetX509DName(client->tls); if (virIdentitySetX509DName(ret, identity) < 0) - goto error; + return NULL; } if (client->sock && virNetSocketGetSELinuxContext(client->sock, &seccontext) < 0) - goto error; + return NULL; if (seccontext && virIdentitySetSELinuxContext(ret, seccontext) < 0) - goto error; - - cleanup: - VIR_FREE(username); - VIR_FREE(groupname); - VIR_FREE(seccontext); - return ret; + return NULL; - error: - virObjectUnref(ret); - ret = NULL; - goto cleanup; + return g_steal_pointer(&ret); } @@ -838,7 +829,7 @@ virIdentityPtr virNetServerClientGetIdentity(virNetServerClientPtr client) if (!client->identity) client->identity = virNetServerClientCreateIdentity(client); if (client->identity) - ret = virObjectRef(client->identity); + ret = g_object_ref(client->identity); virObjectUnlock(client); return ret; } @@ -848,10 +839,10 @@ void virNetServerClientSetIdentity(virNetServerClientPtr client, virIdentityPtr identity) { virObjectLock(client); - virObjectUnref(client->identity); + g_clear_object(&client->identity); client->identity = identity; if (client->identity) - virObjectRef(client->identity); + g_object_ref(client->identity); virObjectUnlock(client); } @@ -988,7 +979,7 @@ void virNetServerClientDispose(void *obj) if (client->privateData) client->privateDataFreeFunc(client->privateData); - virObjectUnref(client->identity); + g_clear_object(&client->identity); #if WITH_SASL virObjectUnref(client->sasl); @@ -1683,7 +1674,7 @@ virNetServerClientGetInfo(virNetServerClientPtr client, goto cleanup; } - *identity = virObjectRef(client->identity); + *identity = g_object_ref(client->identity); ret = 0; cleanup: diff --git a/src/rpc/virnetserverprogram.c b/src/rpc/virnetserverprogram.c index 7ae1d2e955..cca820ca5a 100644 --- a/src/rpc/virnetserverprogram.c +++ b/src/rpc/virnetserverprogram.c @@ -370,13 +370,13 @@ virNetServerProgramDispatchCall(virNetServerProgramPtr prog, virNetServerClientPtr client, virNetMessagePtr msg) { - char *arg = NULL; - char *ret = NULL; + g_autofree char *arg = NULL; + g_autofree char *ret = NULL; int rv = -1; virNetServerProgramProcPtr dispatcher; virNetMessageError rerr; size_t i; - virIdentityPtr identity = NULL; + g_autoptr(virIdentity) identity = NULL; memset(&rerr, 0, sizeof(rerr)); @@ -484,10 +484,7 @@ virNetServerProgramDispatchCall(virNetServerProgramPtr prog, } xdr_free(dispatcher->ret_filter, ret); - VIR_FREE(arg); - VIR_FREE(ret); - virObjectUnref(identity); /* Put reply on end of tx queue to send out */ return virNetServerClientSendMessage(client, msg); @@ -496,10 +493,6 @@ virNetServerProgramDispatchCall(virNetServerProgramPtr prog, * RPC error message we can send back to the client */ rv = virNetServerProgramSendReplyError(prog, client, msg, &rerr, &msg->header); - VIR_FREE(arg); - VIR_FREE(ret); - virObjectUnref(identity); - return rv; } diff --git a/src/util/viridentity.c b/src/util/viridentity.c index 22e2644c19..467d953e17 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -43,25 +43,29 @@ VIR_LOG_INIT("util.identity"); struct _virIdentity { - virObject parent; + GObject parent; int nparams; int maxparams; virTypedParameterPtr params; }; -static virClassPtr virIdentityClass; +G_DEFINE_TYPE(virIdentity, virIdentity, G_TYPE_OBJECT) + static virThreadLocal virIdentityCurrent; -static void virIdentityDispose(void *obj); +static void virIdentityDispose(GObject *obj); -static int virIdentityOnceInit(void) +static void virIdentityCurrentCleanup(void *ident) { - if (!VIR_CLASS_NEW(virIdentity, virClassForObject())) - return -1; + if (ident) + g_object_unref(ident); +} +static int virIdentityOnceInit(void) +{ if (virThreadLocalInit(&virIdentityCurrent, - (virThreadLocalCleanup)virObjectUnref) < 0) { + virIdentityCurrentCleanup) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Cannot initialize thread local for current identity")); return -1; @@ -72,13 +76,24 @@ static int virIdentityOnceInit(void) VIR_ONCE_GLOBAL_INIT(virIdentity); +static void virIdentity_init(virIdentity *ident G_GNUC_UNUSED) +{ +} + +static void virIdentity_class_init(virIdentityClass *klass) +{ + GObjectClass *obj = G_OBJECT_CLASS(klass); + + obj->finalize = virIdentityDispose; +} + /** * virIdentityGetCurrent: * * Get the current identity associated with this thread. The * caller will own a reference to the returned identity, but * must not modify the object in any way, other than to - * release the reference when done with virObjectUnref + * release the reference when done with g_object_unref * * Returns: a reference to the current identity, or NULL */ @@ -90,7 +105,9 @@ virIdentityPtr virIdentityGetCurrent(void) return NULL; ident = virThreadLocalGet(&virIdentityCurrent); - return virObjectRef(ident); + if (ident) + g_object_ref(ident); + return ident; } @@ -105,7 +122,7 @@ virIdentityPtr virIdentityGetCurrent(void) */ int virIdentitySetCurrent(virIdentityPtr ident) { - virIdentityPtr old; + g_autoptr(virIdentity) old = NULL; if (virIdentityInitialize() < 0) return -1; @@ -113,15 +130,14 @@ int virIdentitySetCurrent(virIdentityPtr ident) old = virThreadLocalGet(&virIdentityCurrent); if (virThreadLocalSet(&virIdentityCurrent, - virObjectRef(ident)) < 0) { + ident ? g_object_ref(ident) : NULL) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Unable to set thread local identity")); - virObjectUnref(ident); + if (ident) + g_object_unref(ident); return -1; } - virObjectUnref(old); - return 0; } @@ -139,36 +155,36 @@ virIdentityPtr virIdentityGetSystem(void) VIR_AUTOFREE(char *) username = NULL; VIR_AUTOFREE(char *) groupname = NULL; unsigned long long startTime; - virIdentityPtr ret = NULL; + g_autoptr(virIdentity) ret = NULL; #if WITH_SELINUX security_context_t con; #endif if (!(ret = virIdentityNew())) - goto error; + return NULL; if (virIdentitySetProcessID(ret, getpid()) < 0) - goto error; + return NULL; if (virProcessGetStartTime(getpid(), &startTime) < 0) - goto error; + return NULL; if (startTime != 0 && virIdentitySetProcessTime(ret, startTime) < 0) - goto error; + return NULL; if (!(username = virGetUserName(geteuid()))) return ret; if (virIdentitySetUserName(ret, username) < 0) - goto error; + return NULL; if (virIdentitySetUNIXUserID(ret, getuid()) < 0) - goto error; + return NULL; if (!(groupname = virGetGroupName(getegid()))) return ret; if (virIdentitySetGroupName(ret, groupname) < 0) - goto error; + return NULL; if (virIdentitySetUNIXGroupID(ret, getgid()) < 0) - goto error; + return NULL; #if WITH_SELINUX if (is_selinux_enabled() > 0) { @@ -179,17 +195,13 @@ virIdentityPtr virIdentityGetSystem(void) } if (virIdentitySetSELinuxContext(ret, con) < 0) { freecon(con); - goto error; + return NULL; } freecon(con); } #endif - return ret; - - error: - virObjectUnref(ret); - return NULL; + return g_steal_pointer(&ret); } @@ -203,26 +215,21 @@ virIdentityPtr virIdentityGetSystem(void) */ virIdentityPtr virIdentityNew(void) { - virIdentityPtr ident; - - if (virIdentityInitialize() < 0) - return NULL; - - if (!(ident = virObjectNew(virIdentityClass))) - return NULL; - - return ident; + return VIR_IDENTITY(g_object_new(VIR_TYPE_IDENTITY, NULL)); } -static void virIdentityDispose(void *object) +static void virIdentityDispose(GObject *object) { - virIdentityPtr ident = object; + virIdentityPtr ident = VIR_IDENTITY(object); virTypedParamsFree(ident->params, ident->nparams); + + G_OBJECT_CLASS(virIdentity_parent_class)->finalize(object); } + /* * Returns: 0 if not present, 1 if present, -1 on error */ diff --git a/src/util/viridentity.h b/src/util/viridentity.h index 861ecca736..b63a53e6c4 100644 --- a/src/util/viridentity.h +++ b/src/util/viridentity.h @@ -21,9 +21,12 @@ #pragma once -#include "virobject.h" +#include "internal.h" +#include <glib-object.h> + +#define VIR_TYPE_IDENTITY virIdentity_get_type () +G_DECLARE_FINAL_TYPE (virIdentity, virIdentity, VIR, IDENTITY, GObject); -typedef struct _virIdentity virIdentity; typedef virIdentity *virIdentityPtr; virIdentityPtr virIdentityGetCurrent(void); diff --git a/tests/viridentitytest.c b/tests/viridentitytest.c index 1eadd6173a..81db049fcb 100644 --- a/tests/viridentitytest.c +++ b/tests/viridentitytest.c @@ -38,58 +38,52 @@ VIR_LOG_INIT("tests.identitytest"); static int testIdentityAttrs(const void *data ATTRIBUTE_UNUSED) { - int ret = -1; - virIdentityPtr ident; + g_autoptr(virIdentity) ident = NULL; const char *val; int rc; - if (!(ident = virIdentityNew())) - goto cleanup; + ident = virIdentityNew(); if (virIdentitySetUserName(ident, "fred") < 0) - goto cleanup; + return -1; if ((rc = virIdentityGetUserName(ident, &val)) < 0) - goto cleanup; + return -1; if (STRNEQ_NULLABLE(val, "fred") || rc != 1) { VIR_DEBUG("Expected 'fred' got '%s'", NULLSTR(val)); - goto cleanup; + return -1; } if ((rc = virIdentityGetGroupName(ident, &val)) < 0) - goto cleanup; + return -1; if (val != NULL || rc != 0) { VIR_DEBUG("Unexpected groupname attribute"); - goto cleanup; + return -1; } if (virIdentitySetUserName(ident, "joe") >= 0) { VIR_DEBUG("Unexpectedly overwrote attribute"); - goto cleanup; + return -1; } if ((rc = virIdentityGetUserName(ident, &val)) < 0) - goto cleanup; + return -1; if (STRNEQ_NULLABLE(val, "fred") || rc != 1) { VIR_DEBUG("Expected 'fred' got '%s'", NULLSTR(val)); - goto cleanup; + return -1; } - ret = 0; - cleanup: - virObjectUnref(ident); - return ret; + return 0; } static int testIdentityGetSystem(const void *data) { const char *context = data; - int ret = -1; - virIdentityPtr ident = NULL; + g_autoptr(virIdentity) ident = NULL; const char *val; int rc; @@ -97,35 +91,32 @@ static int testIdentityGetSystem(const void *data) if (context) { VIR_DEBUG("libvirt not compiled with SELinux, skipping this test"); ret = EXIT_AM_SKIP; - goto cleanup; + return -1; } #endif if (!(ident = virIdentityGetSystem())) { VIR_DEBUG("Unable to get system identity"); - goto cleanup; + return -1; } if ((rc = virIdentityGetSELinuxContext(ident, &val)) < 0) - goto cleanup; + return -1; if (context == NULL) { if (val != NULL || rc != 0) { VIR_DEBUG("Unexpected SELinux context %s", NULLSTR(val)); - goto cleanup; + return -1; } } else { if (STRNEQ_NULLABLE(val, context) || rc != 1) { VIR_DEBUG("Want SELinux context '%s' got '%s'", context, val); - goto cleanup; + return -1; } } - ret = 0; - cleanup: - virObjectUnref(ident); - return ret; + return 0; } static int testSetFakeSELinuxContext(const void *data ATTRIBUTE_UNUSED) diff --git a/tests/virnetserverclienttest.c b/tests/virnetserverclienttest.c index d094de9840..42393d7dbe 100644 --- a/tests/virnetserverclienttest.c +++ b/tests/virnetserverclienttest.c @@ -51,7 +51,7 @@ static int testIdentity(const void *opaque ATTRIBUTE_UNUSED) int ret = -1; virNetSocketPtr sock = NULL; virNetServerClientPtr client = NULL; - virIdentityPtr ident = NULL; + g_autoptr(virIdentity) ident = NULL; const char *gotUsername = NULL; uid_t gotUserID; const char *gotGroupname = NULL; @@ -141,7 +141,6 @@ static int testIdentity(const void *opaque ATTRIBUTE_UNUSED) if (client) virNetServerClientClose(client); virObjectUnref(client); - virObjectUnref(ident); VIR_FORCE_CLOSE(sv[0]); VIR_FORCE_CLOSE(sv[1]); return ret; -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:29PM +0100, Daniel P. Berrangé wrote:
Converting from virObject to GObject is reasonably straightforward, as illustrated by this patch for virIdentity
The change would be much easier to see if this patch did not contain the g_autofree changes and the removal of cleanup/error labels, which generate a lot of churn.
In the header file
- Remove
typedef struct _virIdentity virIdentity
- Add
#define VIR_TYPE_IDENTITY virIdentity_get_type () G_DECLARE_FINAL_TYPE (virIdentity, virIdentity, VIR, IDENTITY, GObject);
Which provides the typedef we just removed, and class declaration boilerplate and various other constants/macros.
In the source file
- Change 'virObject parent' to 'GObject parent' in the struct - Remove the virClass variable and its initializing call - Add
G_DEFINE_TYPE(virIdentity, virIdentity, G_TYPE_OBJECT)
which declares the instance & class constructor functions
- Add an impl of the instance & class constructors wiring up the finalize method to point to our dispose impl
In all files
- Replace VIR_AUTOUNREF(virIdentityPtr) with g_autoptr(virIdentity)
Is the idea to never mix VIR_ALLOC/VIR_AUTOFREE with g_alloc/g_auto*? If not, this step could be separated by temporarily setting virObjectUnref as the cleanup function for g_autoptr. Jano
- Replace virObjectRef/Unref with g_object_ref/unref. Note the latter functions do *NOT* accept a NULL object where as libvirt's do. If you replace g_object_unref with g_clear_object it is NULL safe, but also clears the pointer.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/access/viraccessdriverpolkit.c | 21 +++---- src/admin/admin_server.c | 3 +- src/qemu/qemu_process.c | 4 +- src/remote/remote_daemon.c | 3 +- src/remote/remote_daemon_dispatch.c | 35 ++++-------- src/rpc/virnetserverclient.c | 57 ++++++++----------- src/rpc/virnetserverprogram.c | 13 +---- src/util/viridentity.c | 87 ++++++++++++++++------------- src/util/viridentity.h | 7 ++- tests/viridentitytest.c | 45 ++++++--------- tests/virnetserverclienttest.c | 3 +- 11 files changed, 122 insertions(+), 156 deletions(-)

On Mon, Sep 30, 2019 at 03:04:38PM +0200, Ján Tomko wrote:
On Fri, Sep 27, 2019 at 06:17:29PM +0100, Daniel P. Berrangé wrote:
Converting from virObject to GObject is reasonably straightforward, as illustrated by this patch for virIdentity
The change would be much easier to see if this patch did not contain the g_autofree changes and the removal of cleanup/error labels, which generate a lot of churn.
In the header file
- Remove
typedef struct _virIdentity virIdentity
- Add
#define VIR_TYPE_IDENTITY virIdentity_get_type () G_DECLARE_FINAL_TYPE (virIdentity, virIdentity, VIR, IDENTITY, GObject);
Which provides the typedef we just removed, and class declaration boilerplate and various other constants/macros.
In the source file
- Change 'virObject parent' to 'GObject parent' in the struct - Remove the virClass variable and its initializing call - Add
G_DEFINE_TYPE(virIdentity, virIdentity, G_TYPE_OBJECT)
which declares the instance & class constructor functions
- Add an impl of the instance & class constructors wiring up the finalize method to point to our dispose impl
In all files
- Replace VIR_AUTOUNREF(virIdentityPtr) with g_autoptr(virIdentity)
Is the idea to never mix VIR_ALLOC/VIR_AUTOFREE with g_alloc/g_auto*?
Functionally there shouldn't be any problem with mixing the cleanup / allocator APIs..
If not, this step could be separated by temporarily setting virObjectUnref as the cleanup function for g_autoptr.
I'll have more of a think about separating the changes. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Fri, Sep 27, 2019 at 06:17:29PM +0100, Daniel P. Berrangé wrote:
Converting from virObject to GObject is reasonably straightforward, as illustrated by this patch for virIdentity
In the header file
- Remove
typedef struct _virIdentity virIdentity
- Add
#define VIR_TYPE_IDENTITY virIdentity_get_type () G_DECLARE_FINAL_TYPE (virIdentity, virIdentity, VIR, IDENTITY, GObject);
Which provides the typedef we just removed, and class declaration boilerplate and various other constants/macros.
In the source file
- Change 'virObject parent' to 'GObject parent' in the struct - Remove the virClass variable and its initializing call - Add
G_DEFINE_TYPE(virIdentity, virIdentity, G_TYPE_OBJECT)
which declares the instance & class constructor functions
- Add an impl of the instance & class constructors wiring up the finalize method to point to our dispose impl
In all files
- Replace VIR_AUTOUNREF(virIdentityPtr) with g_autoptr(virIdentity)
- Replace virObjectRef/Unref with g_object_ref/unref. Note the latter functions do *NOT* accept a NULL object where as libvirt's do. If you replace g_object_unref with g_clear_object it is NULL safe, but also clears the pointer.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/access/viraccessdriverpolkit.c | 21 +++---- src/admin/admin_server.c | 3 +- src/qemu/qemu_process.c | 4 +- src/remote/remote_daemon.c | 3 +- src/remote/remote_daemon_dispatch.c | 35 ++++-------- src/rpc/virnetserverclient.c | 57 ++++++++----------- src/rpc/virnetserverprogram.c | 13 +---- src/util/viridentity.c | 87 ++++++++++++++++------------- src/util/viridentity.h | 7 ++- tests/viridentitytest.c | 45 ++++++--------- tests/virnetserverclienttest.c | 3 +- 11 files changed, 122 insertions(+), 156 deletions(-)
As Jan pointed out this patch should do the minimal conversion to GObject to demonstrate the transition. Let's move the cleanup into separate patch. I'm OK with using g_autoptr for the new GObject as we don't have to define anything else for it to work and that's what we want to eventually use in our code base.
diff --git a/src/util/viridentity.c b/src/util/viridentity.c index 22e2644c19..467d953e17 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -43,25 +43,29 @@ VIR_LOG_INIT("util.identity");
struct _virIdentity { - virObject parent; + GObject parent;
int nparams; int maxparams; virTypedParameterPtr params; };
-static virClassPtr virIdentityClass; +G_DEFINE_TYPE(virIdentity, virIdentity, G_TYPE_OBJECT) + static virThreadLocal virIdentityCurrent;
-static void virIdentityDispose(void *obj); +static void virIdentityDispose(GObject *obj);
We should rename it to virIdentityFinalize as there is a difference between finalize and dispose in glib. Dispose may be called multiple times and should only remove and clear references to other objects for cyclic dependencies, finalize is called only once and should actually free any resources.
-static int virIdentityOnceInit(void) +static void virIdentityCurrentCleanup(void *ident) { - if (!VIR_CLASS_NEW(virIdentity, virClassForObject())) - return -1; + if (ident) + g_object_unref(ident); +}
+static int virIdentityOnceInit(void) +{ if (virThreadLocalInit(&virIdentityCurrent, - (virThreadLocalCleanup)virObjectUnref) < 0) { + virIdentityCurrentCleanup) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Cannot initialize thread local for current identity")); return -1; @@ -72,13 +76,24 @@ static int virIdentityOnceInit(void)
VIR_ONCE_GLOBAL_INIT(virIdentity);
+static void virIdentity_init(virIdentity *ident G_GNUC_UNUSED) +{ +} + +static void virIdentity_class_init(virIdentityClass *klass) +{ + GObjectClass *obj = G_OBJECT_CLASS(klass); + + obj->finalize = virIdentityDispose;
Here we correctly use finalize so lets match the function name to it as well. Otherwise looks good. Pavel

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/conf/domain_event.c | 25 ++++++++----------- src/libxl/libxl_capabilities.c | 44 ++++++++++++++++------------------ 2 files changed, 31 insertions(+), 38 deletions(-) diff --git a/src/conf/domain_event.c b/src/conf/domain_event.c index b33589f472..f7d1a38e46 100644 --- a/src/conf/domain_event.c +++ b/src/conf/domain_event.c @@ -22,8 +22,6 @@ #include <config.h> -#include <regex.h> - #include "domain_event.h" #include "object_event.h" #include "object_event_private.h" @@ -2009,7 +2007,7 @@ virDomainQemuMonitorEventNew(int id, * deregisters. */ struct virDomainQemuMonitorEventData { char *event; - regex_t regex; + GRegex *regex; unsigned int flags; void *opaque; virFreeCallback freecb; @@ -2241,7 +2239,7 @@ virDomainQemuMonitorEventFilter(virConnectPtr conn ATTRIBUTE_UNUSED, if (data->flags == -1) return true; if (data->flags & VIR_CONNECT_DOMAIN_QEMU_MONITOR_EVENT_REGISTER_REGEX) - return regexec(&data->regex, monitorEvent->event, 0, NULL, 0) == 0; + return g_regex_match(data->regex, monitorEvent->event, 0, NULL) == TRUE; if (data->flags & VIR_CONNECT_DOMAIN_QEMU_MONITOR_EVENT_REGISTER_NOCASE) return STRCASEEQ(monitorEvent->event, data->event); return STREQ(monitorEvent->event, data->event); @@ -2255,7 +2253,7 @@ virDomainQemuMonitorEventCleanup(void *opaque) VIR_FREE(data->event); if (data->flags & VIR_CONNECT_DOMAIN_QEMU_MONITOR_EVENT_REGISTER_REGEX) - regfree(&data->regex); + g_regex_unref(data->regex); if (data->freecb) (data->freecb)(data->opaque); VIR_FREE(data); @@ -2306,20 +2304,17 @@ virDomainQemuMonitorEventStateRegisterID(virConnectPtr conn, return -1; data->flags = flags; if (event && flags != -1) { - int rflags = REG_NOSUB; - - if (flags & VIR_CONNECT_DOMAIN_QEMU_MONITOR_EVENT_REGISTER_NOCASE) - rflags |= REG_ICASE; if (flags & VIR_CONNECT_DOMAIN_QEMU_MONITOR_EVENT_REGISTER_REGEX) { - int err = regcomp(&data->regex, event, rflags); + int cflags = 0; + g_autoptr(GError) err = NULL; - if (err) { - char error[100]; - regerror(err, &data->regex, error, sizeof(error)); + if (flags & VIR_CONNECT_DOMAIN_QEMU_MONITOR_EVENT_REGISTER_NOCASE) + cflags |= G_REGEX_CASELESS; + data->regex = g_regex_new(event, cflags, 0, &err); + if (!data->regex) { virReportError(VIR_ERR_INVALID_ARG, _("failed to compile regex '%s': %s"), - event, error); - regfree(&data->regex); + event, err->message); VIR_FREE(data); return -1; } diff --git a/src/libxl/libxl_capabilities.c b/src/libxl/libxl_capabilities.c index 73ae0b3fa1..90d9bac9c7 100644 --- a/src/libxl/libxl_capabilities.c +++ b/src/libxl/libxl_capabilities.c @@ -21,7 +21,6 @@ #include <config.h> #include <libxl.h> -#include <regex.h> #include "internal.h" #include "virlog.h" @@ -374,10 +373,10 @@ static int libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) { const libxl_version_info *ver_info; - int err; - regex_t regex; + g_autoptr(GRegex) regex = NULL; + g_autoptr(GError) err = NULL; + g_autoptr(GMatchInfo) info = NULL; char *str, *token; - regmatch_t subs[4]; char *saveptr = NULL; size_t i; @@ -398,12 +397,10 @@ libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) return -1; } - err = regcomp(®ex, XEN_CAP_REGEX, REG_EXTENDED); - if (err != 0) { - char error[100]; - regerror(err, ®ex, error, sizeof(error)); + regex = g_regex_new(XEN_CAP_REGEX, G_REGEX_EXTENDED, 0, &err); + if (!regex) { virReportError(VIR_ERR_INTERNAL_ERROR, - _("Failed to compile regex %s"), error); + _("Failed to compile regex %s"), err->message); return -1; } @@ -436,31 +433,33 @@ libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) nr_guest_archs < sizeof(guest_archs) / sizeof(guest_archs[0]) && (token = strtok_r(str, " ", &saveptr)) != NULL; str = NULL) { - if (regexec(®ex, token, sizeof(subs) / sizeof(subs[0]), - subs, 0) == 0) { - int hvm = STRPREFIX(&token[subs[1].rm_so], "hvm"); + if (g_regex_match(regex, token, 0, &info)) { + g_autofree char *modestr = g_match_info_fetch(info, 1); + g_autofree char *archstr = g_match_info_fetch(info, 2); + g_autofree char *suffixstr = g_match_info_fetch(info, 3); + int hvm = STRPREFIX(modestr, "hvm"); virArch arch; int pae = 0, nonpae = 0, ia64_be = 0; - if (STRPREFIX(&token[subs[2].rm_so], "x86_32")) { + if (STRPREFIX(archstr, "x86_32")) { arch = VIR_ARCH_I686; - if (subs[3].rm_so != -1 && - STRPREFIX(&token[subs[3].rm_so], "p")) + if (suffixstr != NULL && + STRPREFIX(suffixstr, "p")) pae = 1; else nonpae = 1; - } else if (STRPREFIX(&token[subs[2].rm_so], "x86_64")) { + } else if (STRPREFIX(archstr, "x86_64")) { arch = VIR_ARCH_X86_64; - } else if (STRPREFIX(&token[subs[2].rm_so], "ia64")) { + } else if (STRPREFIX(archstr, "ia64")) { arch = VIR_ARCH_ITANIUM; - if (subs[3].rm_so != -1 && - STRPREFIX(&token[subs[3].rm_so], "be")) + if (suffixstr != NULL && + STRPREFIX(suffixstr, "be")) ia64_be = 1; - } else if (STRPREFIX(&token[subs[2].rm_so], "powerpc64")) { + } else if (STRPREFIX(archstr, "powerpc64")) { arch = VIR_ARCH_PPC64; - } else if (STRPREFIX(&token[subs[2].rm_so], "armv7l")) { + } else if (STRPREFIX(archstr, "armv7l")) { arch = VIR_ARCH_ARMV7L; - } else if (STRPREFIX(&token[subs[2].rm_so], "aarch64")) { + } else if (STRPREFIX(archstr, "aarch64")) { arch = VIR_ARCH_AARCH64; } else { continue; @@ -515,7 +514,6 @@ libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) #endif } } - regfree(®ex); for (i = 0; i < nr_guest_archs; ++i) { virCapsGuestPtr guest; -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:30PM +0100, Daniel P. Berrangé wrote:
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/conf/domain_event.c | 25 ++++++++----------- src/libxl/libxl_capabilities.c | 44 ++++++++++++++++------------------ 2 files changed, 31 insertions(+), 38 deletions(-)
[...]
diff --git a/src/libxl/libxl_capabilities.c b/src/libxl/libxl_capabilities.c index 73ae0b3fa1..90d9bac9c7 100644 --- a/src/libxl/libxl_capabilities.c +++ b/src/libxl/libxl_capabilities.c @@ -21,7 +21,6 @@ #include <config.h>
#include <libxl.h> -#include <regex.h>
#include "internal.h" #include "virlog.h" @@ -374,10 +373,10 @@ static int libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) { const libxl_version_info *ver_info; - int err; - regex_t regex; + g_autoptr(GRegex) regex = NULL; + g_autoptr(GError) err = NULL; + g_autoptr(GMatchInfo) info = NULL; char *str, *token; - regmatch_t subs[4]; char *saveptr = NULL; size_t i;
@@ -398,12 +397,10 @@ libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) return -1; }
- err = regcomp(®ex, XEN_CAP_REGEX, REG_EXTENDED); - if (err != 0) { - char error[100]; - regerror(err, ®ex, error, sizeof(error)); + regex = g_regex_new(XEN_CAP_REGEX, G_REGEX_EXTENDED, 0, &err); + if (!regex) { virReportError(VIR_ERR_INTERNAL_ERROR, - _("Failed to compile regex %s"), error); + _("Failed to compile regex %s"), err->message); return -1; }
@@ -436,31 +433,33 @@ libxlCapsInitGuests(libxl_ctx *ctx, virCapsPtr caps) nr_guest_archs < sizeof(guest_archs) / sizeof(guest_archs[0]) && (token = strtok_r(str, " ", &saveptr)) != NULL; str = NULL) { - if (regexec(®ex, token, sizeof(subs) / sizeof(subs[0]), - subs, 0) == 0) { - int hvm = STRPREFIX(&token[subs[1].rm_so], "hvm"); + if (g_regex_match(regex, token, 0, &info)) { + g_autofree char *modestr = g_match_info_fetch(info, 1); + g_autofree char *archstr = g_match_info_fetch(info, 2); + g_autofree char *suffixstr = g_match_info_fetch(info, 3); + int hvm = STRPREFIX(modestr, "hvm"); virArch arch; int pae = 0, nonpae = 0, ia64_be = 0;
- if (STRPREFIX(&token[subs[2].rm_so], "x86_32")) { + if (STRPREFIX(archstr, "x86_32")) { arch = VIR_ARCH_I686; - if (subs[3].rm_so != -1 && - STRPREFIX(&token[subs[3].rm_so], "p")) + if (suffixstr != NULL && + STRPREFIX(suffixstr, "p"))
Just a nit-pick since you are touching it, I would go for if (suffixstr && STRPREFIX(suffixstr, "p")) pae = 1; else nonpae = 1; To comply with our syntax style, otherwise {} would be required.
pae = 1; else nonpae = 1; - } else if (STRPREFIX(&token[subs[2].rm_so], "x86_64")) { + } else if (STRPREFIX(archstr, "x86_64")) { arch = VIR_ARCH_X86_64; - } else if (STRPREFIX(&token[subs[2].rm_so], "ia64")) { + } else if (STRPREFIX(archstr, "ia64")) { arch = VIR_ARCH_ITANIUM; - if (subs[3].rm_so != -1 && - STRPREFIX(&token[subs[3].rm_so], "be")) + if (suffixstr != NULL && + STRPREFIX(suffixstr, "be")) ia64_be = 1;
Same here. Otherwise looks good. Reviewed-by: Pavel Hrdina <phrdina@redhat.com>

The GOptionContext API has the benefit over getopt_long that it will automatically handle --help output formatting. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tools/virsh.c | 303 ++++++++++++++++++++++---------------------------- 1 file changed, 135 insertions(+), 168 deletions(-) diff --git a/tools/virsh.c b/tools/virsh.c index ec20f35a77..6c469ff576 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -23,7 +23,6 @@ #include <stdarg.h> #include <unistd.h> -#include <getopt.h> #include <sys/time.h> #include <fcntl.h> #include <time.h> @@ -445,53 +444,36 @@ virshDeinit(vshControl *ctl) } /* - * Print usage + * Build help description for commands */ -static void -virshUsage(void) +static char * +virshBuildDescription(void) { const vshCmdGrp *grp; const vshCmdDef *cmd; - - fprintf(stdout, _("\n%s [options]... [<command_string>]" - "\n%s [options]... <command> [args...]\n\n" - " options:\n" - " -c | --connect=URI hypervisor connection URI\n" - " -d | --debug=NUM debug level [0-4]\n" - " -e | --escape <char> set escape sequence for console\n" - " -h | --help this help\n" - " -k | --keepalive-interval=NUM\n" - " keepalive interval in seconds, 0 for disable\n" - " -K | --keepalive-count=NUM\n" - " number of possible missed keepalive messages\n" - " -l | --log=FILE output logging to file\n" - " -q | --quiet quiet mode\n" - " -r | --readonly connect readonly\n" - " -t | --timing print timing information\n" - " -v short version\n" - " -V long version\n" - " --version[=TYPE] version, TYPE is short or long (default short)\n" - " commands (non interactive mode):\n\n"), progname, - progname); + GString *str = g_string_new("Commands (non interactive mode):\n\n"); for (grp = cmdGroups; grp->name; grp++) { - fprintf(stdout, _(" %s (help keyword '%s')\n"), - grp->name, grp->keyword); + g_string_append_printf(str, + _(" %s (help keyword '%s')\n"), + grp->name, grp->keyword); for (cmd = grp->commands; cmd->name; cmd++) { if (cmd->flags & VSH_CMD_FLAG_ALIAS) continue; - fprintf(stdout, - " %-30s %s\n", cmd->name, - _(vshCmddefGetInfo(cmd, "help"))); + g_string_append_printf(str, + " %-30s %s\n", + cmd->name, + _(vshCmddefGetInfo(cmd, "help"))); } - fprintf(stdout, "\n"); + g_string_append_printf(str, "\n"); } - fprintf(stdout, "%s", - _("\n (specify help <group> for details about the commands in the group)\n")); - fprintf(stdout, "%s", - _("\n (specify help <command> for details about the command)\n\n")); - return; + g_string_append_printf(str, + _("Specify help <group> for details about the commands in the group)\n")); + g_string_append_printf(str, + _("Specify help <command> for details about the command)\n")); + + return g_string_free(str, FALSE); } /* @@ -647,6 +629,22 @@ virshAllowedEscapeChar(char c) ('@' <= c && c <= '_'); } +static gboolean +virshVersion(const gchar *option_name G_GNUC_UNUSED, + const gchar *value, + gpointer data, + GError **error G_GNUC_UNUSED) +{ + vshControl *ctl = data; + + if (value == NULL || STRNEQ(value, "long")) + puts(VERSION); + else + virshShowVersion(ctl); + + exit(EXIT_SUCCESS); +} + /* * argv[]: virsh [options] [command] * @@ -654,152 +652,121 @@ virshAllowedEscapeChar(char c) static bool virshParseArgv(vshControl *ctl, int argc, char **argv) { - int arg, len, debug, keepalive; - size_t i; - int longindex = -1; + int debug = 0; + bool version = false; + size_t len; virshControlPtr priv = ctl->privData; - struct option opt[] = { - {"connect", required_argument, NULL, 'c'}, - {"debug", required_argument, NULL, 'd'}, - {"escape", required_argument, NULL, 'e'}, - {"help", no_argument, NULL, 'h'}, - {"keepalive-interval", required_argument, NULL, 'k'}, - {"keepalive-count", required_argument, NULL, 'K'}, - {"log", required_argument, NULL, 'l'}, - {"quiet", no_argument, NULL, 'q'}, - {"readonly", no_argument, NULL, 'r'}, - {"timing", no_argument, NULL, 't'}, - {"version", optional_argument, NULL, 'v'}, - {NULL, 0, NULL, 0} + char *logfile = NULL; + int keepalive_interval = INT_MAX; + int keepalive_count = INT_MAX; + GOptionEntry opt[] = { + { "connect", 'c', 0, + G_OPTION_ARG_STRING, &ctl->connname, + _("hypervisor connection URI"), "URI" }, + { "debug", 'd', 0, + G_OPTION_ARG_INT, &debug, + _("debug level [0-4]\n"), "LEVEL" }, + { "escape", 'e', 0, + G_OPTION_ARG_STRING, &priv->escapeChar, + _("set escape sequence for console"), "ESCAPE" }, + { "keepalive-interval", 'k', 0, + G_OPTION_ARG_INT, &keepalive_interval, + _("keepalive interval in seconds, 0 for disable"), "SECS" }, + { "keepalive-count", 'K', 0, + G_OPTION_ARG_INT, &keepalive_count, + _("number of possible missed keepalive messages"), "NUM" }, + { "log", 'l', 0, + G_OPTION_ARG_STRING, &logfile, + _("output logging to file"), "FILENAME" }, + { "quiet", 'q', 0, + G_OPTION_ARG_NONE, &ctl->quiet, + _("quite mode"), NULL }, + { "readonly", 'r', 0, + G_OPTION_ARG_NONE, &priv->readonly, + _("connect readonly"), NULL }, + { "timing", 't', 0, + G_OPTION_ARG_NONE, &ctl->timing, + _("print timing information"), NULL }, + { "version", 'v', G_OPTION_FLAG_OPTIONAL_ARG, + G_OPTION_ARG_CALLBACK, virshVersion, + _("print short version"), "[short]" }, + { "version", 'V', 0, + G_OPTION_ARG_NONE, &version, + _("print long version"), "long" }, + { NULL, 0, 0, 0, NULL, NULL, NULL }, }; + g_autoptr(GOptionContext) optctx = NULL; + GOptionGroup *optgrp; + g_autoptr(GError) error = NULL; + + optctx = g_option_context_new(_("[COMMAND [OPTION…] - libvirt shell")); + optgrp = g_option_group_new(NULL, NULL, NULL, ctl, NULL); + g_option_group_set_translation_domain(optgrp, PACKAGE); + g_option_group_add_entries(optgrp, opt); + g_option_context_set_main_group(optctx, optgrp); + g_option_context_set_strict_posix(optctx, true); + g_option_context_set_description(optctx, + virshBuildDescription()); + + if (!g_option_context_parse(optctx, &argc, &argv, &error)) { + vshError(ctl, _("option parsing failed: %s\n"), error->message); + exit(EXIT_FAILURE); + } - /* Standard (non-command) options. The leading + ensures that no - * argument reordering takes place, so that command options are - * not confused with top-level virsh options. */ - while ((arg = getopt_long(argc, argv, "+:c:d:e:hk:K:l:qrtvV", opt, &longindex)) != -1) { - switch (arg) { - case 'c': - VIR_FREE(ctl->connname); - ctl->connname = vshStrdup(ctl, optarg); - break; - case 'd': - if (virStrToLong_i(optarg, NULL, 10, &debug) < 0) { - vshError(ctl, _("option %s takes a numeric argument"), - longindex == -1 ? "-d" : "--debug"); - exit(EXIT_FAILURE); - } - if (debug < VSH_ERR_DEBUG || debug > VSH_ERR_ERROR) - vshError(ctl, _("ignoring debug level %d out of range [%d-%d]"), - debug, VSH_ERR_DEBUG, VSH_ERR_ERROR); - else - ctl->debug = debug; - break; - case 'e': - len = strlen(optarg); - - if ((len == 2 && *optarg == '^' && - virshAllowedEscapeChar(optarg[1])) || - (len == 1 && *optarg != '^')) { - priv->escapeChar = optarg; - } else { - vshError(ctl, _("Invalid string '%s' for escape sequence"), - optarg); - exit(EXIT_FAILURE); - } - break; - case 'h': - virshUsage(); - exit(EXIT_SUCCESS); - break; - case 'k': - if (virStrToLong_i(optarg, NULL, 0, &keepalive) < 0) { - vshError(ctl, - _("Invalid value for option %s"), - longindex == -1 ? "-k" : "--keepalive-interval"); - exit(EXIT_FAILURE); - } - - if (keepalive < 0) { - vshError(ctl, - _("option %s requires a positive integer argument"), - longindex == -1 ? "-k" : "--keepalive-interval"); - exit(EXIT_FAILURE); - } - ctl->keepalive_interval = keepalive; - break; - case 'K': - if (virStrToLong_i(optarg, NULL, 0, &keepalive) < 0) { - vshError(ctl, - _("Invalid value for option %s"), - longindex == -1 ? "-K" : "--keepalive-count"); - exit(EXIT_FAILURE); - } + if (debug < VSH_ERR_DEBUG || debug > VSH_ERR_ERROR) { + vshError(ctl, _("debug level %d out of range [%d-%d]"), + debug, VSH_ERR_DEBUG, VSH_ERR_ERROR); + exit(EXIT_FAILURE); + } - if (keepalive < 0) { - vshError(ctl, - _("option %s requires a positive integer argument"), - longindex == -1 ? "-K" : "--keepalive-count"); - exit(EXIT_FAILURE); - } - ctl->keepalive_count = keepalive; - break; - case 'l': - vshCloseLogFile(ctl); - ctl->logfile = vshStrdup(ctl, optarg); - vshOpenLogFile(ctl); - break; - case 'q': - ctl->quiet = true; - break; - case 't': - ctl->timing = true; - break; - case 'r': - priv->readonly = true; - break; - case 'v': - if (STRNEQ_NULLABLE(optarg, "long")) { - puts(VERSION); - exit(EXIT_SUCCESS); - } - ATTRIBUTE_FALLTHROUGH; - case 'V': - virshShowVersion(ctl); - exit(EXIT_SUCCESS); - case ':': - for (i = 0; opt[i].name != NULL; i++) { - if (opt[i].val == optopt) - break; - } - if (opt[i].name) - vshError(ctl, _("option '-%c'/'--%s' requires an argument"), - optopt, opt[i].name); - else - vshError(ctl, _("option '-%c' requires an argument"), optopt); + if (keepalive_interval != INT_MAX) { + if (keepalive_interval < 0) { + vshError(ctl, + _("keepalive interval requires a positive integer")); exit(EXIT_FAILURE); - case '?': - if (optopt) - vshError(ctl, _("unsupported option '-%c'. See --help."), optopt); - else - vshError(ctl, _("unsupported option '%s'. See --help."), argv[optind - 1]); - exit(EXIT_FAILURE); - default: - vshError(ctl, _("unknown option")); + } + ctl->keepalive_interval = keepalive_interval; + } + + if (keepalive_count != INT_MAX) { + if (keepalive_count < 0) { + vshError(ctl, + _("keepalive count requires a positive integer")); exit(EXIT_FAILURE); } - longindex = -1; + ctl->keepalive_count = keepalive_count; + } + + len = strlen(priv->escapeChar); + if (!((len == 2 && *priv->escapeChar == '^' && + virshAllowedEscapeChar(priv->escapeChar[1])) || + (len == 1 && *priv->escapeChar != '^'))) { + vshError(ctl, _("Invalid string '%s' for escape sequence"), + priv->escapeChar); + exit(EXIT_FAILURE); + } + + if (version) { + virshShowVersion(ctl); + exit(EXIT_SUCCESS); + } + + if (logfile) { + vshCloseLogFile(ctl); + ctl->logfile = logfile; + vshOpenLogFile(ctl); } - if (argc == optind) { + if (argc == 1) { ctl->imode = true; } else { /* parse command */ ctl->imode = false; - if (argc - optind == 1) { - vshDebug(ctl, VSH_ERR_INFO, "commands: \"%s\"\n", argv[optind]); - return vshCommandStringParse(ctl, argv[optind], NULL); + if (argc == 2) { + vshDebug(ctl, VSH_ERR_INFO, "commands: \"%s\"\n", argv[1]); + return vshCommandStringParse(ctl, argv[1], NULL); } else { - return vshCommandArgvParse(ctl, argc - optind, argv + optind); + return vshCommandArgvParse(ctl, argc - 1, argv + 1); } } return true; -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:31PM +0100, Daniel P. Berrangé wrote:
The GOptionContext API has the benefit over getopt_long that it will automatically handle --help output formatting.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tools/virsh.c | 303 ++++++++++++++++++++++---------------------------- 1 file changed, 135 insertions(+), 168 deletions(-)
diff --git a/tools/virsh.c b/tools/virsh.c index ec20f35a77..6c469ff576 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -23,7 +23,6 @@
#include <stdarg.h> #include <unistd.h> -#include <getopt.h> #include <sys/time.h> #include <fcntl.h> #include <time.h> @@ -445,53 +444,36 @@ virshDeinit(vshControl *ctl) }
/* - * Print usage + * Build help description for commands */ -static void -virshUsage(void) +static char * +virshBuildDescription(void) { const vshCmdGrp *grp; const vshCmdDef *cmd; - - fprintf(stdout, _("\n%s [options]... [<command_string>]" - "\n%s [options]... <command> [args...]\n\n" - " options:\n" - " -c | --connect=URI hypervisor connection URI\n" - " -d | --debug=NUM debug level [0-4]\n" - " -e | --escape <char> set escape sequence for console\n" - " -h | --help this help\n" - " -k | --keepalive-interval=NUM\n" - " keepalive interval in seconds, 0 for disable\n" - " -K | --keepalive-count=NUM\n" - " number of possible missed keepalive messages\n" - " -l | --log=FILE output logging to file\n" - " -q | --quiet quiet mode\n" - " -r | --readonly connect readonly\n" - " -t | --timing print timing information\n" - " -v short version\n" - " -V long version\n" - " --version[=TYPE] version, TYPE is short or long (default short)\n" - " commands (non interactive mode):\n\n"), progname, - progname); + GString *str = g_string_new("Commands (non interactive mode):\n\n");
for (grp = cmdGroups; grp->name; grp++) { - fprintf(stdout, _(" %s (help keyword '%s')\n"), - grp->name, grp->keyword); + g_string_append_printf(str, + _(" %s (help keyword '%s')\n"), + grp->name, grp->keyword); for (cmd = grp->commands; cmd->name; cmd++) { if (cmd->flags & VSH_CMD_FLAG_ALIAS) continue; - fprintf(stdout, - " %-30s %s\n", cmd->name, - _(vshCmddefGetInfo(cmd, "help"))); + g_string_append_printf(str, + " %-30s %s\n", + cmd->name, + _(vshCmddefGetInfo(cmd, "help"))); } - fprintf(stdout, "\n"); + g_string_append_printf(str, "\n"); }
- fprintf(stdout, "%s", - _("\n (specify help <group> for details about the commands in the group)\n")); - fprintf(stdout, "%s", - _("\n (specify help <command> for details about the command)\n\n")); - return; + g_string_append_printf(str, + _("Specify help <group> for details about the commands in the group)\n")); + g_string_append_printf(str, + _("Specify help <command> for details about the command)\n")); + + return g_string_free(str, FALSE); }
/* @@ -647,6 +629,22 @@ virshAllowedEscapeChar(char c) ('@' <= c && c <= '_'); }
+static gboolean +virshVersion(const gchar *option_name G_GNUC_UNUSED, + const gchar *value, + gpointer data, + GError **error G_GNUC_UNUSED) +{ + vshControl *ctl = data; + + if (value == NULL || STRNEQ(value, "long")) + puts(VERSION); + else + virshShowVersion(ctl); + + exit(EXIT_SUCCESS); +} + /* * argv[]: virsh [options] [command] * @@ -654,152 +652,121 @@ virshAllowedEscapeChar(char c) static bool virshParseArgv(vshControl *ctl, int argc, char **argv) { - int arg, len, debug, keepalive; - size_t i; - int longindex = -1; + int debug = 0; + bool version = false; + size_t len; virshControlPtr priv = ctl->privData; - struct option opt[] = { - {"connect", required_argument, NULL, 'c'}, - {"debug", required_argument, NULL, 'd'}, - {"escape", required_argument, NULL, 'e'}, - {"help", no_argument, NULL, 'h'}, - {"keepalive-interval", required_argument, NULL, 'k'}, - {"keepalive-count", required_argument, NULL, 'K'}, - {"log", required_argument, NULL, 'l'}, - {"quiet", no_argument, NULL, 'q'}, - {"readonly", no_argument, NULL, 'r'}, - {"timing", no_argument, NULL, 't'}, - {"version", optional_argument, NULL, 'v'}, - {NULL, 0, NULL, 0} + char *logfile = NULL; + int keepalive_interval = INT_MAX; + int keepalive_count = INT_MAX; + GOptionEntry opt[] = { + { "connect", 'c', 0, + G_OPTION_ARG_STRING, &ctl->connname, + _("hypervisor connection URI"), "URI" }, + { "debug", 'd', 0, + G_OPTION_ARG_INT, &debug, + _("debug level [0-4]\n"), "LEVEL" }, + { "escape", 'e', 0, + G_OPTION_ARG_STRING, &priv->escapeChar, + _("set escape sequence for console"), "ESCAPE" }, + { "keepalive-interval", 'k', 0, + G_OPTION_ARG_INT, &keepalive_interval, + _("keepalive interval in seconds, 0 for disable"), "SECS" }, + { "keepalive-count", 'K', 0, + G_OPTION_ARG_INT, &keepalive_count, + _("number of possible missed keepalive messages"), "NUM" }, + { "log", 'l', 0, + G_OPTION_ARG_STRING, &logfile, + _("output logging to file"), "FILENAME" }, + { "quiet", 'q', 0, + G_OPTION_ARG_NONE, &ctl->quiet, + _("quite mode"), NULL }, + { "readonly", 'r', 0, + G_OPTION_ARG_NONE, &priv->readonly, + _("connect readonly"), NULL }, + { "timing", 't', 0, + G_OPTION_ARG_NONE, &ctl->timing, + _("print timing information"), NULL }, + { "version", 'v', G_OPTION_FLAG_OPTIONAL_ARG, + G_OPTION_ARG_CALLBACK, virshVersion, + _("print short version"), "[short]" }, + { "version", 'V', 0, + G_OPTION_ARG_NONE, &version, + _("print long version"), "long" },
We should be able to have both -v and -V call virshVersion if the functions will look like this: static gboolean virshVersion(const gchar *option_name, const gchar *value, gpointer data, GError **error G_GNUC_UNUSED) { vshControl *ctl = data; if (STREQ(option_name, "-V") || STREQ_NULLABLE(value, "long")) virshShowVersion(ctl); else puts(VERSION); exit(EXIT_SUCCESS); } That way we will have only a single place where the version printing code is. Otherwise looks good to me. Pavel

On Tue, Oct 01, 2019 at 11:37:17AM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:31PM +0100, Daniel P. Berrangé wrote:
The GOptionContext API has the benefit over getopt_long that it will automatically handle --help output formatting.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tools/virsh.c | 303 ++++++++++++++++++++++---------------------------- 1 file changed, 135 insertions(+), 168 deletions(-)
diff --git a/tools/virsh.c b/tools/virsh.c index ec20f35a77..6c469ff576 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -23,7 +23,6 @@
#include <stdarg.h> #include <unistd.h> -#include <getopt.h> #include <sys/time.h> #include <fcntl.h> #include <time.h> @@ -445,53 +444,36 @@ virshDeinit(vshControl *ctl) }
+ { "version", 'v', G_OPTION_FLAG_OPTIONAL_ARG, + G_OPTION_ARG_CALLBACK, virshVersion, + _("print short version"), "[short]" }, + { "version", 'V', 0, + G_OPTION_ARG_NONE, &version, + _("print long version"), "long" },
We should be able to have both -v and -V call virshVersion if the functions will look like this:
static gboolean virshVersion(const gchar *option_name, const gchar *value, gpointer data, GError **error G_GNUC_UNUSED) { vshControl *ctl = data;
if (STREQ(option_name, "-V") || STREQ_NULLABLE(value, "long")) virshShowVersion(ctl); else puts(VERSION);
exit(EXIT_SUCCESS); }
That way we will have only a single place where the version printing code is.
Hmm, so "option_name" in the callback is the option that the user actually passed ? I was thinking it was the option name from the static array - ie it would always be "version". If you're right though, this is definitely my preference. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Tue, Oct 01, 2019 at 10:48:50AM +0100, Daniel P. Berrangé wrote:
On Tue, Oct 01, 2019 at 11:37:17AM +0200, Pavel Hrdina wrote:
On Fri, Sep 27, 2019 at 06:17:31PM +0100, Daniel P. Berrangé wrote:
The GOptionContext API has the benefit over getopt_long that it will automatically handle --help output formatting.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tools/virsh.c | 303 ++++++++++++++++++++++---------------------------- 1 file changed, 135 insertions(+), 168 deletions(-)
diff --git a/tools/virsh.c b/tools/virsh.c index ec20f35a77..6c469ff576 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -23,7 +23,6 @@
#include <stdarg.h> #include <unistd.h> -#include <getopt.h> #include <sys/time.h> #include <fcntl.h> #include <time.h> @@ -445,53 +444,36 @@ virshDeinit(vshControl *ctl) }
+ { "version", 'v', G_OPTION_FLAG_OPTIONAL_ARG, + G_OPTION_ARG_CALLBACK, virshVersion, + _("print short version"), "[short]" }, + { "version", 'V', 0, + G_OPTION_ARG_NONE, &version, + _("print long version"), "long" },
We should be able to have both -v and -V call virshVersion if the functions will look like this:
static gboolean virshVersion(const gchar *option_name, const gchar *value, gpointer data, GError **error G_GNUC_UNUSED) { vshControl *ctl = data;
if (STREQ(option_name, "-V") || STREQ_NULLABLE(value, "long")) virshShowVersion(ctl); else puts(VERSION);
exit(EXIT_SUCCESS); }
That way we will have only a single place where the version printing code is.
Hmm, so "option_name" in the callback is the option that the user actually passed ? I was thinking it was the option name from the static array - ie it would always be "version". If you're right though, this is definitely my preference.
It is the option actually used by the user, from the glib docs: option_name The name of the option being parsed. This will be either a single dash followed by a single letter (for a short name) or two dashes followed by a long option name. Pavel

The GOptionContext API has the benefit over getopt_long that it will automatically handle --help output formatting. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tools/virt-admin.c | 207 +++++++++++++++++++++------------------------ 1 file changed, 98 insertions(+), 109 deletions(-) diff --git a/tools/virt-admin.c b/tools/virt-admin.c index e549ec1f83..3b48dd549d 100644 --- a/tools/virt-admin.c +++ b/tools/virt-admin.c @@ -21,8 +21,6 @@ #include <config.h> #include "virt-admin.h" -#include <getopt.h> - #if WITH_READLINE # include <readline/readline.h> # include <readline/history.h> @@ -1212,44 +1210,34 @@ vshAdmDeinit(vshControl *ctl) /* * Print usage */ -static void -vshAdmUsage(void) +static char * +vshAdmBuildDescription(void) { const vshCmdGrp *grp; const vshCmdDef *cmd; - - fprintf(stdout, _("\n%s [options]... [<command_string>]" - "\n%s [options]... <command> [args...]\n\n" - " options:\n" - " -c | --connect=URI daemon admin connection URI\n" - " -d | --debug=NUM debug level [0-4]\n" - " -h | --help this help\n" - " -l | --log=FILE output logging to file\n" - " -q | --quiet quiet mode\n" - " -v short version\n" - " -V long version\n" - " --version[=TYPE] version, TYPE is short or long (default short)\n" - " commands (non interactive mode):\n\n"), progname, - progname); + GString *str = g_string_new("Commands (non interactive mode):\n\n"); for (grp = cmdGroups; grp->name; grp++) { - fprintf(stdout, _(" %s (help keyword '%s')\n"), - grp->name, grp->keyword); + g_string_append_printf(str, + _(" %s (help keyword '%s')\n"), + grp->name, grp->keyword); for (cmd = grp->commands; cmd->name; cmd++) { if (cmd->flags & VSH_CMD_FLAG_ALIAS) continue; - fprintf(stdout, - " %-30s %s\n", cmd->name, - _(vshCmddefGetInfo(cmd, "help"))); + g_string_append_printf(str, + " %-30s %s\n", + cmd->name, + _(vshCmddefGetInfo(cmd, "help"))); } - fprintf(stdout, "\n"); + g_string_append_printf(str, "\n"); } - fprintf(stdout, "%s", - _("\n (specify help <group> for details about the commands in the group)\n")); - fprintf(stdout, "%s", - _("\n (specify help <command> for details about the command)\n\n")); - return; + g_string_append_printf(str, + _("Specify help <group> for details about the commands in the group)\n")); + g_string_append_printf(str, + _("Specify help <command> for details about the command)\n")); + + return g_string_free(str, FALSE); } /* @@ -1275,103 +1263,104 @@ vshAdmShowVersion(vshControl *ctl ATTRIBUTE_UNUSED) vshPrint(ctl, "\n"); } +static gboolean +vshAdmVersion(const gchar *option_name G_GNUC_UNUSED, + const gchar *value, + gpointer data, + GError **error G_GNUC_UNUSED) +{ + vshControl *ctl = data; + + if (value == NULL || STRNEQ(value, "long")) + puts(VERSION); + else + vshAdmShowVersion(ctl); + + exit(EXIT_SUCCESS); +} + +/* + * argv[]: virsh [options] [command] + * + */ static bool vshAdmParseArgv(vshControl *ctl, int argc, char **argv) { - int arg, debug; - size_t i; - int longindex = -1; - struct option opt[] = { - {"connect", required_argument, NULL, 'c'}, - {"debug", required_argument, NULL, 'd'}, - {"help", no_argument, NULL, 'h'}, - {"log", required_argument, NULL, 'l'}, - {"quiet", no_argument, NULL, 'q'}, - {"version", optional_argument, NULL, 'v'}, - {NULL, 0, NULL, 0} + int debug = 0; + bool version = false; + char *logfile = NULL; + GOptionEntry opt[] = { + { "connect", 'c', 0, + G_OPTION_ARG_STRING, &ctl->connname, + _("hypervisor connection URI"), "URI" }, + { "debug", 'd', 0, + G_OPTION_ARG_INT, &debug, + _("debug level [0-4]\n"), "LEVEL" }, + { "log", 'l', 0, + G_OPTION_ARG_STRING, &logfile, + _("output logging to file"), "FILENAME" }, + { "quiet", 'q', 0, + G_OPTION_ARG_NONE, &ctl->quiet, + _("quite mode"), NULL }, + { "version", 'v', G_OPTION_FLAG_OPTIONAL_ARG, + G_OPTION_ARG_CALLBACK, vshAdmVersion, + _("print short version"), "[short]" }, + { "version", 'V', 0, + G_OPTION_ARG_NONE, &version, + _("print long version"), "long" }, + { NULL, 0, 0, 0, NULL, NULL, NULL }, }; + g_autoptr(GOptionContext) optctx = NULL; + GOptionGroup *optgrp; + g_autoptr(GError) error = NULL; + + optctx = g_option_context_new(_("[COMMAND [OPTION…] - libvirt admin shell")); + optgrp = g_option_group_new(NULL, NULL, NULL, ctl, NULL); + g_option_group_set_translation_domain(optgrp, PACKAGE); + g_option_group_add_entries(optgrp, opt); + g_option_context_set_main_group(optctx, optgrp); + g_option_context_set_strict_posix(optctx, true); + g_option_context_set_description(optctx, + vshAdmBuildDescription()); + + if (!g_option_context_parse(optctx, &argc, &argv, &error)) { + vshError(ctl, _("option parsing failed: %s\n"), error->message); + exit(EXIT_FAILURE); + } - /* Standard (non-command) options. The leading + ensures that no - * argument reordering takes place, so that command options are - * not confused with top-level virt-admin options. */ - while ((arg = getopt_long(argc, argv, "+:c:d:hl:qvV", opt, &longindex)) != -1) { - switch (arg) { - case 'c': - VIR_FREE(ctl->connname); - ctl->connname = vshStrdup(ctl, optarg); - break; - case 'd': - if (virStrToLong_i(optarg, NULL, 10, &debug) < 0) { - vshError(ctl, _("option %s takes a numeric argument"), - longindex == -1 ? "-d" : "--debug"); - exit(EXIT_FAILURE); - } - if (debug < VSH_ERR_DEBUG || debug > VSH_ERR_ERROR) - vshError(ctl, _("ignoring debug level %d out of range [%d-%d]"), - debug, VSH_ERR_DEBUG, VSH_ERR_ERROR); - else - ctl->debug = debug; - break; - case 'h': - vshAdmUsage(); - exit(EXIT_SUCCESS); - break; - case 'l': - vshCloseLogFile(ctl); - ctl->logfile = vshStrdup(ctl, optarg); - vshOpenLogFile(ctl); - break; - case 'q': - ctl->quiet = true; - break; - case 'v': - if (STRNEQ_NULLABLE(optarg, "long")) { - puts(VERSION); - exit(EXIT_SUCCESS); - } - ATTRIBUTE_FALLTHROUGH; - case 'V': - vshAdmShowVersion(ctl); - exit(EXIT_SUCCESS); - case ':': - for (i = 0; opt[i].name != NULL; i++) { - if (opt[i].val == optopt) - break; - } - if (opt[i].name) - vshError(ctl, _("option '-%c'/'--%s' requires an argument"), - optopt, opt[i].name); - else - vshError(ctl, _("option '-%c' requires an argument"), optopt); - exit(EXIT_FAILURE); - case '?': - if (optopt) - vshError(ctl, _("unsupported option '-%c'. See --help."), optopt); - else - vshError(ctl, _("unsupported option '%s'. See --help."), argv[optind - 1]); - exit(EXIT_FAILURE); - default: - vshError(ctl, _("unknown option")); - exit(EXIT_FAILURE); - } - longindex = -1; + if (debug < VSH_ERR_DEBUG || debug > VSH_ERR_ERROR) { + vshError(ctl, _("debug level %d out of range [%d-%d]"), + debug, VSH_ERR_DEBUG, VSH_ERR_ERROR); + exit(EXIT_FAILURE); + } + + if (version) { + vshAdmShowVersion(ctl); + exit(EXIT_SUCCESS); } - if (argc == optind) { + if (logfile) { + vshCloseLogFile(ctl); + ctl->logfile = logfile; + vshOpenLogFile(ctl); + } + + if (argc == 1) { ctl->imode = true; } else { /* parse command */ ctl->imode = false; - if (argc - optind == 1) { - vshDebug(ctl, VSH_ERR_INFO, "commands: \"%s\"\n", argv[optind]); - return vshCommandStringParse(ctl, argv[optind], NULL); + if (argc == 2) { + vshDebug(ctl, VSH_ERR_INFO, "commands: \"%s\"\n", argv[1]); + return vshCommandStringParse(ctl, argv[1], NULL); } else { - return vshCommandArgvParse(ctl, argc - optind, argv + optind); + return vshCommandArgvParse(ctl, argc - 1, argv + 1); } } return true; } + static const vshCmdDef vshAdmCmds[] = { VSH_CMD_CD, VSH_CMD_ECHO, -- 2.21.0

On Fri, Sep 27, 2019 at 06:17:32PM +0100, Daniel P. Berrangé wrote:
The GOptionContext API has the benefit over getopt_long that it will automatically handle --help output formatting.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tools/virt-admin.c | 207 +++++++++++++++++++++------------------------ 1 file changed, 98 insertions(+), 109 deletions(-)
Same comment about the -v -V --version, otherwise looks good. Pavel
participants (6)
-
Andrea Bolognani
-
Daniel P. Berrangé
-
Ján Tomko
-
Laine Stump
-
Pavel Hrdina
-
Peter Krempa