[libvirt] how to print size_t in LGPLv2+ program
libvirt is currently LGPLv2+, but needs to use the printf family of functions to display size_t values, even on mingw where %zu support is not built in. My understanding is that on 64-bit windows, sizeof(long)==4 but sizeof(void*)==8; and if I'm reading http://msdn.microsoft.com/en-us/library/s3f49ktz%28VS.80%29.aspx correctly, then sizeof(size_t) is also 8. Which means you _can't_ use "%lu",(unsigned long)size_t_val. And mingw also lacks support for "%llu",(unsigned long long)size_t_val. So that leaves several options: Is it worth relaxing the license on the *printf-posix family of modules to LGPLv2+ from their current LGPLv3+, or is this too big of a request? Ultimately, this is the nicest - %zu would just work. Is it worth relaxing the license of the inttostr family of modules?. But that is also LGPLv3+, and requires a more careful audit to convert all uses of "%zu",size_t_val into "%s",umaxtostr(size_t_val,buf). Or, since using umaxtostr penalizes 32-bit machines for converting to the 64-bit intermediary, maybe it's worth adding a size_t variant? That, and coreutils has a TODO to remove use of umaxtostr in favor of <inttypes.h> solutions. Finally, within existing LGPLv2+ modules, it is possible for libvirt to manually use "%"PRIuMAX,(uintmax_t)size_t_val everywhere by relying on gnulib's <inttypes.h> replacement, but that's also a painful audit to perform and maintain, and also wastes processor cycles on 32-bit machines for converting to the 64-bit intermediary. What a shame that POSIX omitted an <inttypes.h> PRIu* for size_t. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Mingw64 lacks %zu, and has the unfortunate setup where sizeof(long)==4 but sizeof(size_t)==8. Since gnulib's printf-posix module is not LGPLv2+, the best we can do is manually cast to the only portable int type known to hold size_t, and rely on gnulib's inttypes.h. * src/remote/remote_driver.c (remoteStreamPacket): Rewrite size_t formatting. * src/storage/storage_driver.c (storageWipeExtent): Likewise. --- Here's the latter option; there are many more uses of %zu, but only in DEBUG statements. I suppose it would also be easy enough to teach cfg.mk how to recognize and reject %z inside translated strings, as part of 'make syntax-check'. src/remote/remote_driver.c | 7 +++++-- src/storage/storage_driver.c | 7 ++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index cb0d8e1..d9115c8 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -34,6 +34,7 @@ #include <fcntl.h> #include <arpa/inet.h> #include <sys/wait.h> +#include <inttypes.h> /* Windows socket compatibility functions. */ #include <errno.h> @@ -8024,8 +8025,10 @@ remoteStreamPacket(virStreamPtr st, if (status == REMOTE_CONTINUE) { if (((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength) < nbytes) { - remoteError(VIR_ERR_RPC, _("data size %zu too large for payload %d"), - nbytes, ((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength)); + remoteError(VIR_ERR_RPC, + _("data size %" PRIuMAX " too large for payload %d"), + (uintmax_t) nbytes, + ((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength)); goto error; } diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index 4ebbced..3837182 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -1,7 +1,7 @@ /* * storage_driver.c: core driver for storage APIs * - * Copyright (C) 2006-2009 Red Hat, Inc. + * Copyright (C) 2006-2010 Red Hat, Inc. * Copyright (C) 2006-2008 Daniel P. Berrange * * This library is free software; you can redistribute it and/or @@ -29,6 +29,7 @@ #include <sys/stat.h> #include <sys/param.h> #include <fcntl.h> +#include <inttypes.h> #if HAVE_PWD_H # include <pwd.h> @@ -1597,9 +1598,9 @@ storageWipeExtent(virStorageVolDefPtr vol, written = safewrite(fd, writebuf, write_size); if (written < 0) { virReportSystemError(errno, - _("Failed to write %zu bytes to " + _("Failed to write %" PRIuMAX " bytes to " "storage volume with path '%s'"), - write_size, vol->target.path); + (uintmax_t) write_size, vol->target.path); goto out; } -- 1.7.2.1
On 08/17/2010 04:05 PM, Eric Blake wrote:
Mingw64 lacks %zu, and has the unfortunate setup where sizeof(long)==4 but sizeof(size_t)==8. Since gnulib's printf-posix module is not LGPLv2+, the best we can do is manually cast to the only portable int type known to hold size_t, and rely on gnulib's inttypes.h.
- _("Failed to write %zu bytes to " + _("Failed to write %" PRIuMAX " bytes to "
Hmmm. Use of PRIuMAX in a translated string produces a .pot file that refers to the special string <PRIuMAX>, and which requires the need-formatstring-macros option passed to AM_GNU_GETTEXT in configure.ac to be universally supported. If I'm reading gettext.git history correctly, need-formatstring-macros was added in gettext 0.11.4 (commit 8b45c5df), but had bugs until 0.17 (commit 4e34b2ac); which is newer than the oldest version of gettext that libvirt is currently willing to support. I'm not sure if you get a fixed gettext.m4 by using just using gnulib's gettext-h module, but using gnulib's gettext module is out of the question since it would force the use of gettext 0.18. Bruno, am I missing any details about how to properly use the gettext formatstring-macros option, and which gettext versions are supported, if I missed something in my sleuthing through gettext.git? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
[This question ought to have been sent to bug-gettext, not bug-gnulib.] Hi Eric,
Use of PRIuMAX in a translated string produces a .pot file that refers to the special string <PRIuMAX>, and which requires the need-formatstring-macros option passed to AM_GNU_GETTEXT in configure.ac to be universally supported. If I'm reading gettext.git history correctly, need-formatstring-macros was added in gettext 0.11.4 (commit 8b45c5df),
yes.
but had bugs until 0.17 (commit 4e34b2ac);
It had bugs until 0.16, fixed in versions >= 0.16.1 (released on 2006-11-27).
am I missing any details about how to properly use the gettext formatstring-macros option
You invoke AM_GNU_GETTEXT([external], [need-formatstring-macros]) and, on glibc systems, use a glibc newer than 2002.
which gettext versions are supported
Versions 0.16.1, 0.17, 0.18.1.1 support need-formatstring-macros. Bruno
On Tue, Aug 17, 2010 at 04:05:03PM -0600, Eric Blake wrote:
Mingw64 lacks %zu, and has the unfortunate setup where sizeof(long)==4 but sizeof(size_t)==8. Since gnulib's printf-posix module is not LGPLv2+, the best we can do is manually cast to the only portable int type known to hold size_t, and rely on gnulib's inttypes.h.
* src/remote/remote_driver.c (remoteStreamPacket): Rewrite size_t formatting. * src/storage/storage_driver.c (storageWipeExtent): Likewise. ---
Here's the latter option; there are many more uses of %zu, but only in DEBUG statements. I suppose it would also be easy enough to teach cfg.mk how to recognize and reject %z inside translated strings, as part of 'make syntax-check'.
src/remote/remote_driver.c | 7 +++++-- src/storage/storage_driver.c | 7 ++++--- 2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index cb0d8e1..d9115c8 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -34,6 +34,7 @@ #include <fcntl.h> #include <arpa/inet.h> #include <sys/wait.h> +#include <inttypes.h>
/* Windows socket compatibility functions. */ #include <errno.h> @@ -8024,8 +8025,10 @@ remoteStreamPacket(virStreamPtr st,
if (status == REMOTE_CONTINUE) { if (((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength) < nbytes) { - remoteError(VIR_ERR_RPC, _("data size %zu too large for payload %d"), - nbytes, ((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength)); + remoteError(VIR_ERR_RPC, + _("data size %" PRIuMAX " too large for payload %d"), + (uintmax_t) nbytes, + ((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength));
I find the PRI* stuff rather fugly. Can't we just use %llu and cast to (unsigned long long) The question of printf-posix license doesn't appear relevant since remoteError & friends all use asprintf() which is LGPLv2+ already. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
I find the PRI* stuff rather fugly. Can't we just use %llu and cast to (unsigned long long)
Unfortunately, %llu is equally non-portable to mingw. And yes, we also have some %llu encoded into translated strings, which would also need help.
The question of printf-posix license doesn't appear relevant since remoteError & friends all use asprintf() which is LGPLv2+ already.
We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not guarantee the existence of %llu nor %zu -- it only guarantees that you have the [v]asprintf wrappers around your current system's (non-)compliant printf family, so it inherits the same bugs regarding unsupported specifiers. We would have to use the vasprintf-posix module to get %zu, but that module is LGPLv3+. One other potential solution: most (all?) of our translated strings involve error messages, and are therefore already funneled through our virterror.c implementation. As long as a string containing %zu or %llu is never directly handed to printf, but is guaranteed to go through virterror.c, then it would be possible to have virterror.c do some #ifdef magic such that on all sane platforms, the %zu and %llu are used unchanged (no extra overhead required); but on broken platforms (aka mingw), %zu and %llu are translated at runtime to %lu or %I64u (the %zu translation depends on whether you are 32-bit/64-bit mingw, and the 64-bit translation relies on microsoft's non-standard format specifier). This solution would then confine the ugliness to one file, such that the rest of libvirt can use %zu and %llu at will. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Wed, Aug 18, 2010 at 07:41:16AM -0600, Eric Blake wrote:
On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
I find the PRI* stuff rather fugly. Can't we just use %llu and cast to (unsigned long long)
Unfortunately, %llu is equally non-portable to mingw. And yes, we also have some %llu encoded into translated strings, which would also need help.
The question of printf-posix license doesn't appear relevant since remoteError & friends all use asprintf() which is LGPLv2+ already.
We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not guarantee the existence of %llu nor %zu -- it only guarantees that you have the [v]asprintf wrappers around your current system's (non-)compliant printf family, so it inherits the same bugs regarding unsupported specifiers. We would have to use the vasprintf-posix module to get %zu, but that module is LGPLv3+.
Hmm, that's odd, because support for %llu in printf was one of the primary reasons we started using GNULIB in the first place. We have been relying on %llu working, throughout the code. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
[re-adding bug-gnulib for another question] On 08/18/2010 07:51 AM, Daniel P. Berrange wrote:
On Wed, Aug 18, 2010 at 07:41:16AM -0600, Eric Blake wrote:
On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
I find the PRI* stuff rather fugly. Can't we just use %llu and cast to (unsigned long long)
Unfortunately, %llu is equally non-portable to mingw. And yes, we also have some %llu encoded into translated strings, which would also need help.
The question of printf-posix license doesn't appear relevant since remoteError & friends all use asprintf() which is LGPLv2+ already.
We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not guarantee the existence of %llu nor %zu -- it only guarantees that you have the [v]asprintf wrappers around your current system's (non-)compliant printf family, so it inherits the same bugs regarding unsupported specifiers. We would have to use the vasprintf-posix module to get %zu, but that module is LGPLv3+.
Hmm, that's odd, because support for %llu in printf was one of the primary reasons we started using GNULIB in the first place. We have been relying on %llu working, throughout the code.
Bruno, is my understanding of the differences between vasprintf and vasprintf-posix correct? If so, it means that gnulib is already in the situation that the bulk of the source code to support %zu and %llu is available via the vasprintf module, but hidden behind #defines that are not supplied unless you use the LGPL vasprint-posix module. But the vasprintf-module only adds .m4 files on top of the existing mix of vasprintf files, and there is nothing about the .m4 files that must be LGPLv3+ only (since they already have a more permissive license). On the surface, it seems like I could technically copy the contents of the vasprintf-posix .m4 files (since they are already a more permissive license) and couple that with the existing LGPLv2+ vasprintf module, all without violating any licensing. Looking a bit closer, though, vasprintf-posix drags in some new LGPLv3+ dependencies, like isnand-libm and printf-frexp; but even then, if I'm willing to link with -lm on mingw and avoid %a and %Lg, it seems like I can avoid those extra dependencies. Still, I'm reluctant to bite the bullet and go with the LGPLv2+ cascade on vasprintf-posix. So maybe the solution is an intermediate module: LGPLv2+ vasprintf - bare bones, guarantees a wrapper around system printf, so %zu and %llu are unsafe because of mingw LGPLv2+ vasprintf-sizes - guarantees %zu, %llu, %ju, %tu; but not %Lg (which means splitting gl_PRINTF_SIZES_C99 into two) or %'d LGPLV3+ vasprintf-posix - guarantees full contingency of POSIX specifiers If this three-level proposal makes sense, then I can start on the work of extracting the simpler portions of vasprintf-posix into the new vasprintf-sizes. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 08/18/2010 08:30 AM, Eric Blake wrote:
Still, I'm reluctant to bite the bullet and go with the LGPLv2+ cascade on vasprintf-posix. So maybe the solution is an intermediate module:
LGPLv2+ vasprintf - bare bones, guarantees a wrapper around system printf, so %zu and %llu are unsafe because of mingw
LGPLv2+ vasprintf-sizes - guarantees %zu, %llu, %ju, %tu; but not %Lg (which means splitting gl_PRINTF_SIZES_C99 into two) or %'d
LGPLV3+ vasprintf-posix - guarantees full contingency of POSIX specifiers
If this three-level proposal makes sense, then I can start on the work of extracting the simpler portions of vasprintf-posix into the new vasprintf-sizes.
Actually, after looking into this deeper, it (happily) appears that I may have been mistaken. It looks like the gnulib vasprintf module _already_ performs printf parsing on mingw; and that as a virtue of that printf parsing, %zu and %llu are already rewritten into modifiers understood by mingw. I tested this by modifying test-vasprintf.c to try %zu and %llu on a mingw compilation, and the test still passed. The gnulib module snprintf will likewise support %zu and %llu, but only if it is in use (right now, libvirt is not using the snprintf module). However, there is no counterpart printf or sprintf, but libvirt uses those functions as well. So there is still some work to be done in libvirt to make sure all *printf() family calls eventually end up going through gnulib wrappers, to take advantage of the fact that gnulib's vasprintf _is_ sufficient to support %zu and %llu on mingw. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Wed, Aug 18, 2010 at 09:54:51AM -0600, Eric Blake wrote:
On 08/18/2010 08:30 AM, Eric Blake wrote:
Still, I'm reluctant to bite the bullet and go with the LGPLv2+ cascade on vasprintf-posix. So maybe the solution is an intermediate module:
LGPLv2+ vasprintf - bare bones, guarantees a wrapper around system printf, so %zu and %llu are unsafe because of mingw
LGPLv2+ vasprintf-sizes - guarantees %zu, %llu, %ju, %tu; but not %Lg (which means splitting gl_PRINTF_SIZES_C99 into two) or %'d
LGPLV3+ vasprintf-posix - guarantees full contingency of POSIX specifiers
If this three-level proposal makes sense, then I can start on the work of extracting the simpler portions of vasprintf-posix into the new vasprintf-sizes.
Actually, after looking into this deeper, it (happily) appears that I may have been mistaken. It looks like the gnulib vasprintf module _already_ performs printf parsing on mingw; and that as a virtue of that printf parsing, %zu and %llu are already rewritten into modifiers understood by mingw. I tested this by modifying test-vasprintf.c to try %zu and %llu on a mingw compilation, and the test still passed.
The gnulib module snprintf will likewise support %zu and %llu, but only if it is in use (right now, libvirt is not using the snprintf module).
However, there is no counterpart printf or sprintf, but libvirt uses those functions as well. So there is still some work to be done in libvirt to make sure all *printf() family calls eventually end up going through gnulib wrappers, to take advantage of the fact that gnulib's vasprintf _is_ sufficient to support %zu and %llu on mingw.
Do we actually have any places where printf/sprintf hurts ? I see nothing using %zu # find -name '*.c' | xargs grep -i printf | grep -i -v asprintf | grep -v virBuffer | grep -v gnulib | grep '%z' And just a handful of things using %ll $ find -name '*.c' | xargs grep -i printf | grep -i -v asprintf | grep -v virBuffer | grep -v gnulib | grep '%ll' ./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%lluK", vol->capacity/1024); ./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%llu", vol->capacity/1024/1024); ./src/storage/parthelper.c: printf("%s%s%d%c%s%c%s%c%llu%c%llu%c%llu%c", ./src/storage/parthelper.c: printf("%s%c%s%c%s%c%llu%c%llu%c%llu%c", ./src/storage/storage_backend_disk.c: snprintf(start, sizeof(start)-1, "%lluB", startOffset); ./src/storage/storage_backend_disk.c: snprintf(end, sizeof(end)-1, "%lluB", endOffset); ./src/storage/storage_backend_logical.c: snprintf(size, sizeof(size)-1, "%lluK", vol->capacity/1024); ./tests/qemuhelptest.c: fprintf(stderr, "Computed flags do not match: got 0x%llx, expected 0x%llx\n", ./examples/domain-events/events-c/event-test.c: printf("%s EVENT: Domain %s(%d) rtc change %lld\n", __func__, virDomainGetName(dom), All these are code modules which are Linux specific, though arguably we should kill the snprintf() in favour of virAsprintf() for sanity sake. Everything else ends up routed via vasprintf so, I think it is sufficient for libvirt to just use vasprintf and not worry about the printf/snprintf issue (at least not urgently) Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On 08/18/2010 10:07 AM, Daniel P. Berrange wrote:
Do we actually have any places where printf/sprintf hurts ?
[v]as[n]printf are already safe, thanks to the vasprintf module. snprintf is safe, but only indirectly, due to the getaddrinfo module dragging it in (if getaddrinfo is changed to not rely on snprintf, then we would lose the indirect support), but I agree that we should be using virAsprintf in that case, anyways. vsnprintf is not safe, but can easily be made safe at the same time as snprintf. [v][f]printf and [v]sprintf are not safe, with nothing in gnulib to protect them while still staying at LGPLv2+; but I agree that we can probably avoid the issues with these by converting sprintf to virAsprintf, and just being careful with [f]printf.
And just a handful of things using %ll
$ find -name '*.c' | xargs grep -i printf | grep -i -v asprintf | grep -v virBuffer | grep -v gnulib | grep '%ll' ./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%lluK", vol->capacity/1024); ./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%llu", vol->capacity/1024/1024); ./src/storage/parthelper.c: printf("%s%s%d%c%s%c%s%c%llu%c%llu%c%llu%c", ./src/storage/parthelper.c: printf("%s%c%s%c%s%c%llu%c%llu%c%llu%c", ./src/storage/storage_backend_disk.c: snprintf(start, sizeof(start)-1, "%lluB", startOffset); ./src/storage/storage_backend_disk.c: snprintf(end, sizeof(end)-1, "%lluB", endOffset); ./src/storage/storage_backend_logical.c: snprintf(size, sizeof(size)-1, "%lluK", vol->capacity/1024); ./tests/qemuhelptest.c: fprintf(stderr, "Computed flags do not match: got 0x%llx, expected 0x%llx\n", ./examples/domain-events/events-c/event-test.c: printf("%s EVENT: Domain %s(%d) rtc change %lld\n", __func__, virDomainGetName(dom),
For a more complete list of all potential problems, I used: $ git grep '\bv\?s\?f\?printf \?(' \ daemon/ tools/ src/ include/ proxy/ tests/ | wc -l 236 And except for the few you already listed above, none of them had %z or %ll issues. So fixing those few, plus converting snprintf to virAsprintf, seems like a manageable task; I'm now working on it. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Hi Eric,
after looking into this deeper, it (happily) appears that I may have been mistaken. It looks like the gnulib vasprintf module _already_ performs printf parsing on mingw; and that as a virtue of that printf parsing, %zu and %llu are already rewritten into modifiers understood by mingw.
And if this were not the case, I would recommend to you to use #include <inttypes.h> printf ("%" PRIuPTR, (uintptr_t) size_t_value); with the gnulib module 'inttypes' (LGPLv2+). If someone insists on a guarantee that printf ("%zu", size_t_value); works, he needs the 'printf-posix' module, which is not available under LGPLv2. I'm reluctant to move all of 'printf-posix', 'fprintf-posix', 'sprintf-posix' to LGPLv2+, including the auxiliary modules fseterr, isnand-nolibm, isnanl-nolibm, frexp-nolibm, frexpl-nolibm, printf-frexp, printf-frexpl, signbit, fpucw, because this is a significant piece of code, and LGPLv3+ has benefits over LGPLv2+ for us. Bruno
On Wed, Aug 18, 2010 at 07:41:16AM -0600, Eric Blake wrote:
On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
I find the PRI* stuff rather fugly. Can't we just use %llu and cast to (unsigned long long)
Unfortunately, %llu is equally non-portable to mingw. And yes, we also have some %llu encoded into translated strings, which would also need help.
The question of printf-posix license doesn't appear relevant since remoteError & friends all use asprintf() which is LGPLv2+ already.
We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not guarantee the existence of %llu nor %zu -- it only guarantees that you have the [v]asprintf wrappers around your current system's (non-)compliant printf family, so it inherits the same bugs regarding unsupported specifiers. We would have to use the vasprintf-posix module to get %zu, but that module is LGPLv3+.
I don't think this is correct. The 'vasprintf' module was added in GNULIB in 87b04f998fd3e668027074b5b5d37205d3cdfec3. This commit includes a full re-implementation of format parsing that appears independent of the host system printf() impl. $ git show 87b04f998fd3e668027074b5b5d37205d3cdfec3 | diffstat | grep lib lib/ChangeLog | 13 lib/asnprintf.c | 38 ++ lib/asprintf.c | 38 ++ lib/printf-args.c | 119 ++++++++ lib/printf-args.h | 134 +++++++++ lib/printf-parse.c | 477 ++++++++++++++++++++++++++++++++ lib/printf-parse.h | 72 ++++ lib/vasnprintf.c | 767 +++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/vasnprintf.h | 61 ++++ lib/vasprintf.c | 38 ++ lib/vasprintf.h | 64 ++++ These printf-args/parse files appear to handle long long int & %llu combinations correctly. Since mingw32 lacks any vasprintf() at all, we will be using this gnulib replacement. The vasprintf-posix seems to only be used where vasprintf() exists but is broken, thus not on mingw32 I find it strange that vasprintf is a more liberal license than the vasprintf-posix, since the former is where all the really cool code is - the latter just seems to be a few m4 macros that anyone could reimplement with ease. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On 08/18/2010 08:35 AM, Daniel P. Berrange wrote:
These printf-args/parse files appear to handle long long int & %llu combinations correctly. Since mingw32 lacks any vasprintf() at all, we will be using this gnulib replacement. The vasprintf-posix seems to only be used where vasprintf() exists but is broken, thus not on mingw32
The vasprintf module only uses the full printf parsing when told to do so by #ifdef; it prefers the much smaller approach of using snprintf to do all the work when there is no other compelling reason to use full parsing.
I find it strange that vasprintf is a more liberal license than the vasprintf-posix, since the former is where all the really cool code is - the latter just seems to be a few m4 macros that anyone could reimplement with ease.
vasprintf-posix is more than just m4 macros; it also drags in LGPLv3+ module dependencies that implement floating point parsing (such as printf-ldexp and isnand-nolibm), then does the additional checks for which #defines to enable. Among other things, it is these additional #defines that then force the compelling reason for vasnprintf.c to use full-blown parsing rather than the smaller footprint of wrapping snprintf. So our dilemma is that %zu and %llu need to be compelling reasons to turn on a subset of the full-blown parser, since mingw's snprintf lacks those, but without also dragging in the LGPLv3+ floating-point handling. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Hi Eric,
My understanding is that on 64-bit windows, sizeof(long)==4 but sizeof(void*)==8; and ... sizeof(size_t) is also 8.
Yes, correct.
Which means you _can't_ use "%lu",(unsigned long)size_t_val.
You _can_ use this. It will work as long as each of your program's data structures is less than 4 GB large. Remember that size_t values get larger than 4 GB only if you have a memory object (array) that is larger than 4 GB.
What a shame that POSIX omitted an <inttypes.h> PRIu* for size_t.
You can define it by yourself: Basically you define #if @BITSIZEOF_SIZE_T@ == 32 # define PRIuSIZE PRIu32 #endif #if @BITSIZEOF_SIZE_T@ == 64 # define PRIuSIZE PRIu64 #endif This will work with mingw's and msvc's native printf, because gnulib's <inttypes.h> replacement defines PRIu64 to "I64u", and the native printf supports %I64u directives. Note that this will not work inside gettext() arguments, though, because PRIuSIZE is not standard. For internationalized messages, you will need the workaround described in the second half of <http://www.gnu.org/software/gettext/manual/html_node/Preparing-Strings.html>. I hope one of these two alternatives works for you, so that we can avoid an LGPLv2+ cascade. Bruno
On 08/17/2010 04:33 PM, Bruno Haible wrote:
What a shame that POSIX omitted an <inttypes.h> PRIu* for size_t.
You can define it by yourself: Basically you define
#if @BITSIZEOF_SIZE_T@ == 32 # define PRIuSIZE PRIu32 #endif #if @BITSIZEOF_SIZE_T@ == 64 # define PRIuSIZE PRIu64 #endif
Note that this will not work inside gettext() arguments, though, because PRIuSIZE is not standard. For internationalized messages, you will need the workaround described in the second half of <http://www.gnu.org/software/gettext/manual/html_node/Preparing-Strings.html>.
You mean like this? char buf1[100]; sprintf (buf1, "%0" MYPRId64, number); printf (gettext ("The amount is %s\n"), buf1); No thanks - like inttostr() it requires a temporary buffer, but unlike inttostr(), it ends up being a slower three-liner due to the full overhead of a second printf() call. If gettext can't support it, then the shortest approach is the two-liner: char buf1[INT_BUFSIZE_BOUND (size_t)]; printf (_("The amount is %s\n"), sizetomax (val, buf1)); Here's where a cross-project change to GNU Coding Standards could be helpful - if we all agree that gnulib should add the macro and gettext should add the support for it at the same time, then it would be much easier for all remaining GNU projects to take advantage of a standardized name (that is, use GCS rather than POSIX as our standard that documents the portable use of PRIuSIZE among GNU projects). However, it does take us back to the minimum build tool requirement issue - any project relying on that proposed feature would necessarily have to drag in whatever new version of gettext that added the support for it.
it still requires auditing code and forbidding "%zu" in favor of "%"PRIuSIZE (or whatever other name we settle on).
You could hack GCC, clang, or even xgettext to produce a warning when it sees a 'z' size modifier in a format string.
It sounds like xgettext might be the best place to detect %zu, particularly if it already diagnoses existing uses of PRIuMAX in projects that forgot to specify need-formatstring-macros in AM_GNU_GETTEXT (and if it doesn't already diagnose translated strings that require more features than the settings requested in configure.ac, it probably should). But there's also maint.mk (or cfg.mk) that has heuristics for detecting untranslated strings, which seems like it could be pretty easily modified to also check for translated strings containing %zu. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Eric Blake wrote:
Here's where a cross-project change to GNU Coding Standards could be helpful - if we all agree that gnulib should add the macro and gettext should add the support for it at the same time, then it would be much easier for all remaining GNU projects to take advantage of a standardized name
Yes. And it will need either a POSIX change or a common GNU extension, because you will also need it implemented in glibc <http://sourceware.org/git/?p=glibc.git;a=blob;f=intl/loadmsgcat.c>. That's certainly a thing you could propose to the Austin group. Bruno
On 08/17/2010 04:33 PM, Bruno Haible wrote:
Which means you _can't_ use "%lu",(unsigned long)size_t_val.
You _can_ use this. It will work as long as each of your program's data structures is less than 4 GB large. Remember that size_t values get larger than 4 GB only if you have a memory object (array) that is larger than 4 GB.
That's a safe assumption if you are talking about sizeof(object), as it is unlikely that anyone is intentionally compiling .o files with objects that are statically allocated to be that large. But it is not a safe assumption when using size_t as the argument to hold a user-specified value matching the size of an dynamically allocated array, and goes against the GCS mantra of no arbitrary limits (maybe the user really does have a reason that they wanted a 4 gigabyte string loaded into memory on a 64-bit machine). In other words, I think your statement is true only for a subset of the valid uses of printing a size_t value. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
What a shame that POSIX omitted an <inttypes.h> PRIu* for size_t.
You can define it by yourself ...
Or use uintptr_t instead of size_t. By the definition of these types, you can be sure that uintptr_t is at least as wide as size_t. Then use printf ("%" PRIuPTR, (uintptr_t) size_t_value); - Works fine with gnulib's <inttypes.h>. - Works fine with gettext. - Doesn't need gettext or glibc versions that don't exist yet. Bruno
On 08/17/2010 05:35 PM, Bruno Haible wrote:
What a shame that POSIX omitted an <inttypes.h> PRIu* for size_t.
You can define it by yourself ...
Or use uintptr_t instead of size_t. By the definition of these types, you can be sure that uintptr_t is at least as wide as size_t. Then use
printf ("%" PRIuPTR, (uintptr_t) size_t_value);
- Works fine with gnulib's <inttypes.h>. - Works fine with gettext. - Doesn't need gettext or glibc versions that don't exist yet.
But still needs the use of need-formatstring-macros, and gettext 0.16.1 or newer (whereas libvirt is stuck on 0.14.1 at the moment). -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
[removed bug-gnulib from CC] Eric Blake wrote:
printf ("%" PRIuPTR, (uintptr_t) size_t_value);
But still needs the use of need-formatstring-macros, and gettext 0.16.1 or newer (whereas libvirt is stuck on 0.14.1 at the moment).
The uses of these formatstring macros require - at package preparation time: the AM_GNU_GETTEXT macro from gettext 0.16.1 or newer, - at build time and run time: a glibc newer than 2004-01-14 [1]. So, it requires gettext >= 0.16.1 to be installed ONLY on the machines which are used to develop and release libvirt. Bruno [1] http://sourceware.org/git/?p=glibc.git;a=commit;h=083dc54a01d3b65b9d95599e40...
participants (3)
-
Bruno Haible -
Daniel P. Berrange -
Eric Blake