4:17 p.m.
libvirt is currently LGPLv2+, but needs to use the printf family of
functions to display size_t values, even on mingw where %zu support is
not built in. My understanding is that on 64-bit windows,
sizeof(long)==4 but sizeof(void*)==8; and if I'm reading
http://msdn.microsoft.com/en-us/library/s3f49ktz%28VS.80%29.aspx
correctly, then sizeof(size_t) is also 8. Which means you _can't_ use
"%lu",(unsigned long)size_t_val. And mingw also lacks support for
"%llu",(unsigned long long)size_t_val.
So that leaves several options:
Is it worth relaxing the license on the *printf-posix family of modules
to LGPLv2+ from their current LGPLv3+, or is this too big of a request?
Ultimately, this is the nicest - %zu would just work.
Is it worth relaxing the license of the inttostr family of modules?.
But that is also LGPLv3+, and requires a more careful audit to convert
all uses of "%zu",size_t_val into "%s",umaxtostr(size_t_val,buf).
Or,
since using umaxtostr penalizes 32-bit machines for converting to the
64-bit intermediary, maybe it's worth adding a size_t variant? That,
and coreutils has a TODO to remove use of umaxtostr in favor of
<inttypes.h> solutions.
Finally, within existing LGPLv2+ modules, it is possible for libvirt to
manually use "%"PRIuMAX,(uintmax_t)size_t_val everywhere by relying on
gnulib's <inttypes.h> replacement, but that's also a painful audit to
perform and maintain, and also wastes processor cycles on 32-bit
machines for converting to the 64-bit intermediary.
What a shame that POSIX omitted an <inttypes.h> PRIu* for size_t.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
5:05 p.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
Mingw64 lacks %zu, and has the unfortunate setup where
sizeof(long)==4 but sizeof(size_t)==8. Since gnulib's
printf-posix module is not LGPLv2+, the best we can do
is manually cast to the only portable int type known to
hold size_t, and rely on gnulib's inttypes.h.
* src/remote/remote_driver.c (remoteStreamPacket): Rewrite size_t
formatting.
* src/storage/storage_driver.c (storageWipeExtent): Likewise.
---
Here's the latter option; there are many more uses of %zu, but
only in DEBUG statements. I suppose it would also be easy enough
to teach cfg.mk how to recognize and reject %z inside translated
strings, as part of 'make syntax-check'.
src/remote/remote_driver.c | 7 +++++--
src/storage/storage_driver.c | 7 ++++---
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c
index cb0d8e1..d9115c8 100644
--- a/src/remote/remote_driver.c
+++ b/src/remote/remote_driver.c
@@ -34,6 +34,7 @@
#include <fcntl.h>
#include <arpa/inet.h>
#include <sys/wait.h>
+#include <inttypes.h>
/* Windows socket compatibility functions. */
#include <errno.h>
@@ -8024,8 +8025,10 @@ remoteStreamPacket(virStreamPtr st,
if (status == REMOTE_CONTINUE) {
if (((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength) < nbytes) {
- remoteError(VIR_ERR_RPC, _("data size %zu too large for payload
%d"),
- nbytes, ((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength));
+ remoteError(VIR_ERR_RPC,
+ _("data size %" PRIuMAX " too large for payload
%d"),
+ (uintmax_t) nbytes,
+ ((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength));
goto error;
}
diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c
index 4ebbced..3837182 100644
--- a/src/storage/storage_driver.c
+++ b/src/storage/storage_driver.c
@@ -1,7 +1,7 @@
/*
* storage_driver.c: core driver for storage APIs
*
- * Copyright (C) 2006-2009 Red Hat, Inc.
+ * Copyright (C) 2006-2010 Red Hat, Inc.
* Copyright (C) 2006-2008 Daniel P. Berrange
*
* This library is free software; you can redistribute it and/or
@@ -29,6 +29,7 @@
#include <sys/stat.h>
#include <sys/param.h>
#include <fcntl.h>
+#include <inttypes.h>
#if HAVE_PWD_H
# include <pwd.h>
@@ -1597,9 +1598,9 @@ storageWipeExtent(virStorageVolDefPtr vol,
written = safewrite(fd, writebuf, write_size);
if (written < 0) {
virReportSystemError(errno,
- _("Failed to write %zu bytes to "
+ _("Failed to write %" PRIuMAX " bytes to
"
"storage volume with path '%s'"),
- write_size, vol->target.path);
+ (uintmax_t) write_size, vol->target.path);
goto out;
}
--
1.7.2.1
5:37 p.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On 08/17/2010 04:05 PM, Eric Blake wrote:
Mingw64 lacks %zu, and has the unfortunate setup where
sizeof(long)==4 but sizeof(size_t)==8. Since gnulib's
printf-posix module is not LGPLv2+, the best we can do
is manually cast to the only portable int type known to
hold size_t, and rely on gnulib's inttypes.h.
- _("Failed to write %zu bytes
to "
+ _("Failed to write %" PRIuMAX " bytes to
"
Hmmm. Use of PRIuMAX in a translated string produces a .pot file that
refers to the special string <PRIuMAX>, and which requires the
need-formatstring-macros option passed to AM_GNU_GETTEXT in configure.ac
to be universally supported. If I'm reading gettext.git history
correctly, need-formatstring-macros was added in gettext 0.11.4 (commit
8b45c5df), but had bugs until 0.17 (commit 4e34b2ac); which is newer
than the oldest version of gettext that libvirt is currently willing to
support. I'm not sure if you get a fixed gettext.m4 by using just using
gnulib's gettext-h module, but using gnulib's gettext module is out of
the question since it would force the use of gettext 0.18.
Bruno, am I missing any details about how to properly use the gettext
formatstring-macros option, and which gettext versions are supported, if
I missed something in my sleuthing through gettext.git?
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
5:53 p.m.
New subject: [libvirt] gettext and need-formatstring-macros
[This question ought to have been sent to bug-gettext, not bug-gnulib.]
Hi Eric,
Use of PRIuMAX in a translated string produces a .pot file that
refers to the special string <PRIuMAX>, and which requires the
need-formatstring-macros option passed to AM_GNU_GETTEXT in configure.ac
to be universally supported. If I'm reading gettext.git history
correctly, need-formatstring-macros was added in gettext 0.11.4 (commit
8b45c5df),
yes.
but had bugs until 0.17 (commit 4e34b2ac);
It had bugs until 0.16, fixed in versions >= 0.16.1 (released on 2006-11-27).
am I missing any details about how to properly use the gettext
formatstring-macros option
You invoke AM_GNU_GETTEXT([external], [need-formatstring-macros]) and,
on glibc systems, use a glibc newer than 2002.
which gettext versions are supported
Versions 0.16.1, 0.17, 0.18.1.1 support need-formatstring-macros.
Bruno
4:04 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On Tue, Aug 17, 2010 at 04:05:03PM -0600, Eric Blake wrote:
Mingw64 lacks %zu, and has the unfortunate setup where
sizeof(long)==4 but sizeof(size_t)==8. Since gnulib's
printf-posix module is not LGPLv2+, the best we can do
is manually cast to the only portable int type known to
hold size_t, and rely on gnulib's inttypes.h.
* src/remote/remote_driver.c (remoteStreamPacket): Rewrite size_t
formatting.
* src/storage/storage_driver.c (storageWipeExtent): Likewise.
---
Here's the latter option; there are many more uses of %zu, but
only in DEBUG statements. I suppose it would also be easy enough
to teach cfg.mk how to recognize and reject %z inside translated
strings, as part of 'make syntax-check'.
src/remote/remote_driver.c | 7 +++++--
src/storage/storage_driver.c | 7 ++++---
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c
index cb0d8e1..d9115c8 100644
--- a/src/remote/remote_driver.c
+++ b/src/remote/remote_driver.c
@@ -34,6 +34,7 @@
#include <fcntl.h>
#include <arpa/inet.h>
#include <sys/wait.h>
+#include <inttypes.h>
/* Windows socket compatibility functions. */
#include <errno.h>
@@ -8024,8 +8025,10 @@ remoteStreamPacket(virStreamPtr st,
if (status == REMOTE_CONTINUE) {
if (((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength) < nbytes) {
- remoteError(VIR_ERR_RPC, _("data size %zu too large for payload
%d"),
- nbytes, ((4 + REMOTE_MESSAGE_MAX) -
thiscall->bufferLength));
+ remoteError(VIR_ERR_RPC,
+ _("data size %" PRIuMAX " too large for payload
%d"),
+ (uintmax_t) nbytes,
+ ((4 + REMOTE_MESSAGE_MAX) - thiscall->bufferLength));
I find the PRI* stuff rather fugly. Can't we just use %llu and
cast to (unsigned long long)
The question of printf-posix license doesn't appear relevant since
remoteError & friends all use asprintf() which is LGPLv2+ already.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:41 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
I find the PRI* stuff rather fugly. Can't we just use %llu and
cast to (unsigned long long)
Unfortunately, %llu is equally non-portable to mingw. And yes, we also
have some %llu encoded into translated strings, which would also need help.
The question of printf-posix license doesn't appear relevant since
remoteError & friends all use asprintf() which is LGPLv2+ already.
We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not
guarantee the existence of %llu nor %zu -- it only guarantees that you
have the [v]asprintf wrappers around your current system's
(non-)compliant printf family, so it inherits the same bugs regarding
unsupported specifiers. We would have to use the vasprintf-posix module
to get %zu, but that module is LGPLv3+.
One other potential solution: most (all?) of our translated strings
involve error messages, and are therefore already funneled through our
virterror.c implementation. As long as a string containing %zu or %llu
is never directly handed to printf, but is guaranteed to go through
virterror.c, then it would be possible to have virterror.c do some
#ifdef magic such that on all sane platforms, the %zu and %llu are used
unchanged (no extra overhead required); but on broken platforms (aka
mingw), %zu and %llu are translated at runtime to %lu or %I64u (the %zu
translation depends on whether you are 32-bit/64-bit mingw, and the
64-bit translation relies on microsoft's non-standard format specifier).
This solution would then confine the ugliness to one file, such that
the rest of libvirt can use %zu and %llu at will.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
8:51 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On Wed, Aug 18, 2010 at 07:41:16AM -0600, Eric Blake wrote:
On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
>
> I find the PRI* stuff rather fugly. Can't we just use %llu and
> cast to (unsigned long long)
Unfortunately, %llu is equally non-portable to mingw. And yes, we also
have some %llu encoded into translated strings, which would also need help.
>
> The question of printf-posix license doesn't appear relevant since
> remoteError & friends all use asprintf() which is LGPLv2+ already.
We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not
guarantee the existence of %llu nor %zu -- it only guarantees that you
have the [v]asprintf wrappers around your current system's
(non-)compliant printf family, so it inherits the same bugs regarding
unsupported specifiers. We would have to use the vasprintf-posix module
to get %zu, but that module is LGPLv3+.
Hmm, that's odd, because support for %llu in printf was one of the
primary reasons we started using GNULIB in the first place. We have
been relying on %llu working, throughout the code.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:30 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
[re-adding bug-gnulib for another question]
On 08/18/2010 07:51 AM, Daniel P. Berrange wrote:
On Wed, Aug 18, 2010 at 07:41:16AM -0600, Eric Blake wrote:
> On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
>>
>> I find the PRI* stuff rather fugly. Can't we just use %llu and
>> cast to (unsigned long long)
>
> Unfortunately, %llu is equally non-portable to mingw. And yes, we also
> have some %llu encoded into translated strings, which would also need help.
>
>>
>> The question of printf-posix license doesn't appear relevant since
>> remoteError & friends all use asprintf() which is LGPLv2+ already.
>
> We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not
> guarantee the existence of %llu nor %zu -- it only guarantees that you
> have the [v]asprintf wrappers around your current system's
> (non-)compliant printf family, so it inherits the same bugs regarding
> unsupported specifiers. We would have to use the vasprintf-posix module
> to get %zu, but that module is LGPLv3+.
Hmm, that's odd, because support for %llu in printf was one of the
primary reasons we started using GNULIB in the first place. We have
been relying on %llu working, throughout the code.
Bruno, is my understanding of the differences between vasprintf and
vasprintf-posix correct? If so, it means that gnulib is already in the
situation that the bulk of the source code to support %zu and %llu is
available via the vasprintf module, but hidden behind #defines that are
not supplied unless you use the LGPL vasprint-posix module. But the
vasprintf-module only adds .m4 files on top of the existing mix of
vasprintf files, and there is nothing about the .m4 files that must be
LGPLv3+ only (since they already have a more permissive license).
On the surface, it seems like I could technically copy the contents of
the vasprintf-posix .m4 files (since they are already a more permissive
license) and couple that with the existing LGPLv2+ vasprintf module, all
without violating any licensing. Looking a bit closer, though,
vasprintf-posix drags in some new LGPLv3+ dependencies, like isnand-libm
and printf-frexp; but even then, if I'm willing to link with -lm on
mingw and avoid %a and %Lg, it seems like I can avoid those extra
dependencies.
Still, I'm reluctant to bite the bullet and go with the LGPLv2+ cascade
on vasprintf-posix. So maybe the solution is an intermediate module:
LGPLv2+ vasprintf - bare bones, guarantees a wrapper around system
printf, so %zu and %llu are unsafe because of mingw
LGPLv2+ vasprintf-sizes - guarantees %zu, %llu, %ju, %tu; but not %Lg
(which means splitting gl_PRINTF_SIZES_C99 into two) or %'d
LGPLV3+ vasprintf-posix - guarantees full contingency of POSIX specifiers
If this three-level proposal makes sense, then I can start on the work
of extracting the simpler portions of vasprintf-posix into the new
vasprintf-sizes.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
10:54 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On 08/18/2010 08:30 AM, Eric Blake wrote:
Still, I'm reluctant to bite the bullet and go with the LGPLv2+
cascade
on vasprintf-posix. So maybe the solution is an intermediate module:
LGPLv2+ vasprintf - bare bones, guarantees a wrapper around system
printf, so %zu and %llu are unsafe because of mingw
LGPLv2+ vasprintf-sizes - guarantees %zu, %llu, %ju, %tu; but not %Lg
(which means splitting gl_PRINTF_SIZES_C99 into two) or %'d
LGPLV3+ vasprintf-posix - guarantees full contingency of POSIX specifiers
If this three-level proposal makes sense, then I can start on the work
of extracting the simpler portions of vasprintf-posix into the new
vasprintf-sizes.
Actually, after looking into this deeper, it (happily) appears that I
may have been mistaken. It looks like the gnulib vasprintf module
_already_ performs printf parsing on mingw; and that as a virtue of that
printf parsing, %zu and %llu are already rewritten into modifiers
understood by mingw. I tested this by modifying test-vasprintf.c to try
%zu and %llu on a mingw compilation, and the test still passed.
The gnulib module snprintf will likewise support %zu and %llu, but only
if it is in use (right now, libvirt is not using the snprintf module).
However, there is no counterpart printf or sprintf, but libvirt uses
those functions as well. So there is still some work to be done in
libvirt to make sure all *printf() family calls eventually end up going
through gnulib wrappers, to take advantage of the fact that gnulib's
vasprintf _is_ sufficient to support %zu and %llu on mingw.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
11:07 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On Wed, Aug 18, 2010 at 09:54:51AM -0600, Eric Blake wrote:
On 08/18/2010 08:30 AM, Eric Blake wrote:
> Still, I'm reluctant to bite the bullet and go with the LGPLv2+ cascade
> on vasprintf-posix. So maybe the solution is an intermediate module:
>
> LGPLv2+ vasprintf - bare bones, guarantees a wrapper around system
> printf, so %zu and %llu are unsafe because of mingw
>
> LGPLv2+ vasprintf-sizes - guarantees %zu, %llu, %ju, %tu; but not %Lg
> (which means splitting gl_PRINTF_SIZES_C99 into two) or %'d
>
> LGPLV3+ vasprintf-posix - guarantees full contingency of POSIX specifiers
>
> If this three-level proposal makes sense, then I can start on the work
> of extracting the simpler portions of vasprintf-posix into the new
> vasprintf-sizes.
Actually, after looking into this deeper, it (happily) appears that I
may have been mistaken. It looks like the gnulib vasprintf module
_already_ performs printf parsing on mingw; and that as a virtue of that
printf parsing, %zu and %llu are already rewritten into modifiers
understood by mingw. I tested this by modifying test-vasprintf.c to try
%zu and %llu on a mingw compilation, and the test still passed.
The gnulib module snprintf will likewise support %zu and %llu, but only
if it is in use (right now, libvirt is not using the snprintf module).
However, there is no counterpart printf or sprintf, but libvirt uses
those functions as well. So there is still some work to be done in
libvirt to make sure all *printf() family calls eventually end up going
through gnulib wrappers, to take advantage of the fact that gnulib's
vasprintf _is_ sufficient to support %zu and %llu on mingw.
Do we actually have any places where printf/sprintf hurts ?
I see nothing using %zu
# find -name '*.c' | xargs grep -i printf | grep -i -v asprintf | grep -v
virBuffer | grep -v gnulib | grep '%z'
And just a handful of things using %ll
$ find -name '*.c' | xargs grep -i printf | grep -i -v asprintf | grep -v
virBuffer | grep -v gnulib | grep '%ll'
./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%lluK",
vol->capacity/1024);
./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%llu",
vol->capacity/1024/1024);
./src/storage/parthelper.c:
printf("%s%s%d%c%s%c%s%c%llu%c%llu%c%llu%c",
./src/storage/parthelper.c: printf("%s%c%s%c%s%c%llu%c%llu%c%llu%c",
./src/storage/storage_backend_disk.c: snprintf(start, sizeof(start)-1,
"%lluB", startOffset);
./src/storage/storage_backend_disk.c: snprintf(end, sizeof(end)-1, "%lluB",
endOffset);
./src/storage/storage_backend_logical.c: snprintf(size, sizeof(size)-1,
"%lluK", vol->capacity/1024);
./tests/qemuhelptest.c: fprintf(stderr, "Computed flags do not match: got
0x%llx, expected 0x%llx\n",
./examples/domain-events/events-c/event-test.c: printf("%s EVENT: Domain %s(%d)
rtc change %lld\n", __func__, virDomainGetName(dom),
All these are code modules which are Linux specific, though arguably we
should kill the snprintf() in favour of virAsprintf() for sanity sake.
Everything else ends up routed via vasprintf so, I think it is sufficient
for libvirt to just use vasprintf and not worry about the printf/snprintf
issue (at least not urgently)
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
12:27 p.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On 08/18/2010 10:07 AM, Daniel P. Berrange wrote:
Do we actually have any places where printf/sprintf hurts ?
[v]as[n]printf are already safe, thanks to the vasprintf module.
snprintf is safe, but only indirectly, due to the getaddrinfo module
dragging it in (if getaddrinfo is changed to not rely on snprintf, then
we would lose the indirect support), but I agree that we should be using
virAsprintf in that case, anyways.
vsnprintf is not safe, but can easily be made safe at the same time as
snprintf.
[v][f]printf and [v]sprintf are not safe, with nothing in gnulib to
protect them while still staying at LGPLv2+; but I agree that we can
probably avoid the issues with these by converting sprintf to
virAsprintf, and just being careful with [f]printf.
And just a handful of things using %ll
$ find -name '*.c' | xargs grep -i printf | grep -i -v asprintf | grep -v
virBuffer | grep -v gnulib | grep '%ll'
./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%lluK",
vol->capacity/1024);
./src/storage/storage_backend.c: snprintf(size, sizeof(size), "%llu",
vol->capacity/1024/1024);
./src/storage/parthelper.c:
printf("%s%s%d%c%s%c%s%c%llu%c%llu%c%llu%c",
./src/storage/parthelper.c:
printf("%s%c%s%c%s%c%llu%c%llu%c%llu%c",
./src/storage/storage_backend_disk.c: snprintf(start, sizeof(start)-1,
"%lluB", startOffset);
./src/storage/storage_backend_disk.c: snprintf(end, sizeof(end)-1, "%lluB",
endOffset);
./src/storage/storage_backend_logical.c: snprintf(size, sizeof(size)-1,
"%lluK", vol->capacity/1024);
./tests/qemuhelptest.c: fprintf(stderr, "Computed flags do not match: got
0x%llx, expected 0x%llx\n",
./examples/domain-events/events-c/event-test.c: printf("%s EVENT: Domain %s(%d)
rtc change %lld\n", __func__, virDomainGetName(dom),
For a more complete list of all potential problems, I used:
$ git grep '\bv\?s\?f\?printf \?(' \
daemon/ tools/ src/ include/ proxy/ tests/ | wc -l
236
And except for the few you already listed above, none of them had %z or
%ll issues. So fixing those few, plus converting snprintf to
virAsprintf, seems like a manageable task; I'm now working on it.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
10:59 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
Hi Eric,
after looking into this deeper, it (happily) appears that I
may have been mistaken. It looks like the gnulib vasprintf module
_already_ performs printf parsing on mingw; and that as a virtue of that
printf parsing, %zu and %llu are already rewritten into modifiers
understood by mingw.
And if this were not the case, I would recommend to you to use
#include <inttypes.h>
printf ("%" PRIuPTR, (uintptr_t) size_t_value);
with the gnulib module 'inttypes' (LGPLv2+).
If someone insists on a guarantee that
printf ("%zu", size_t_value);
works, he needs the 'printf-posix' module, which is not available under
LGPLv2.
I'm reluctant to move all of 'printf-posix', 'fprintf-posix',
'sprintf-posix' to LGPLv2+, including the auxiliary modules
fseterr, isnand-nolibm, isnanl-nolibm, frexp-nolibm, frexpl-nolibm,
printf-frexp, printf-frexpl, signbit, fpucw, because this is a
significant piece of code, and LGPLv3+ has benefits over LGPLv2+ for us.
Bruno
9:35 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On Wed, Aug 18, 2010 at 07:41:16AM -0600, Eric Blake wrote:
On 08/18/2010 03:04 AM, Daniel P. Berrange wrote:
>
> I find the PRI* stuff rather fugly. Can't we just use %llu and
> cast to (unsigned long long)
Unfortunately, %llu is equally non-portable to mingw. And yes, we also
have some %llu encoded into translated strings, which would also need help.
>
> The question of printf-posix license doesn't appear relevant since
> remoteError & friends all use asprintf() which is LGPLv2+ already.
We use the 'vasprintf' module, which is indeed LGPLv2+, but it does not
guarantee the existence of %llu nor %zu -- it only guarantees that you
have the [v]asprintf wrappers around your current system's
(non-)compliant printf family, so it inherits the same bugs regarding
unsupported specifiers. We would have to use the vasprintf-posix module
to get %zu, but that module is LGPLv3+.
I don't think this is correct. The 'vasprintf' module was added in
GNULIB in 87b04f998fd3e668027074b5b5d37205d3cdfec3. This commit
includes a full re-implementation of format parsing that appears
independent of the host system printf() impl.
$ git show 87b04f998fd3e668027074b5b5d37205d3cdfec3 | diffstat | grep lib
lib/ChangeLog | 13
lib/asnprintf.c | 38 ++
lib/asprintf.c | 38 ++
lib/printf-args.c | 119 ++++++++
lib/printf-args.h | 134 +++++++++
lib/printf-parse.c | 477 ++++++++++++++++++++++++++++++++
lib/printf-parse.h | 72 ++++
lib/vasnprintf.c | 767 +++++++++++++++++++++++++++++++++++++++++++++++++++++
lib/vasnprintf.h | 61 ++++
lib/vasprintf.c | 38 ++
lib/vasprintf.h | 64 ++++
These printf-args/parse files appear to handle long long int & %llu
combinations correctly. Since mingw32 lacks any vasprintf() at all,
we will be using this gnulib replacement. The vasprintf-posix seems
to only be used where vasprintf() exists but is broken, thus not on
mingw32
I find it strange that vasprintf is a more liberal license than the
vasprintf-posix, since the former is where all the really cool
code is - the latter just seems to be a few m4 macros that anyone
could reimplement with ease.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:59 a.m.
New subject: [libvirt] [RFC PATCH] build: avoid %zu in translated strings
On 08/18/2010 08:35 AM, Daniel P. Berrange wrote:
These printf-args/parse files appear to handle long long int &
%llu
combinations correctly. Since mingw32 lacks any vasprintf() at all,
we will be using this gnulib replacement. The vasprintf-posix seems
to only be used where vasprintf() exists but is broken, thus not on
mingw32
The vasprintf module only uses the full printf parsing when told to do
so by #ifdef; it prefers the much smaller approach of using snprintf to
do all the work when there is no other compelling reason to use full
parsing.
I find it strange that vasprintf is a more liberal license than the
vasprintf-posix, since the former is where all the really cool
code is - the latter just seems to be a few m4 macros that anyone
could reimplement with ease.
vasprintf-posix is more than just m4 macros; it also drags in LGPLv3+
module dependencies that implement floating point parsing (such as
printf-ldexp and isnand-nolibm), then does the additional checks for
which #defines to enable. Among other things, it is these additional
#defines that then force the compelling reason for vasnprintf.c to use
full-blown parsing rather than the smaller footprint of wrapping snprintf.
So our dilemma is that %zu and %llu need to be compelling reasons to
turn on a subset of the full-blown parser, since mingw's snprintf lacks
those, but without also dragging in the LGPLv3+ floating-point handling.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
5:33 p.m.
Hi Eric,
My understanding is that on 64-bit windows,
sizeof(long)==4 but sizeof(void*)==8; and ... sizeof(size_t) is also 8.
Yes, correct.
Which means you _can't_ use "%lu",(unsigned
long)size_t_val.
You _can_ use this. It will work as long as each of your program's data
structures is less than 4 GB large. Remember that size_t values get
larger than 4 GB only if you have a memory object (array) that is larger
than 4 GB.
What a shame that POSIX omitted an <inttypes.h> PRIu* for
size_t.
You can define it by yourself: Basically you define
#if @BITSIZEOF_SIZE_T@ == 32
# define PRIuSIZE PRIu32
#endif
#if @BITSIZEOF_SIZE_T@ == 64
# define PRIuSIZE PRIu64
#endif
This will work with mingw's and msvc's native printf, because gnulib's
<inttypes.h> replacement defines PRIu64 to "I64u", and the native printf
supports %I64u directives.
Note that this will not work inside gettext() arguments, though, because
PRIuSIZE is not standard. For internationalized messages, you will need
the workaround described in the second half of
<http://www.gnu.org/software/gettext/manual/html_node/Preparing-Strings.ht...;.
I hope one of these two alternatives works for you, so that we can avoid an
LGPLv2+ cascade.
Bruno
6:01 p.m.
On 08/17/2010 04:33 PM, Bruno Haible wrote:
> What a shame that POSIX omitted an <inttypes.h> PRIu* for
size_t.
You can define it by yourself: Basically you define
#if @BITSIZEOF_SIZE_T@ == 32
# define PRIuSIZE PRIu32
#endif
#if @BITSIZEOF_SIZE_T@ == 64
# define PRIuSIZE PRIu64
#endif
Note that this will not work inside gettext() arguments, though, because
PRIuSIZE is not standard. For internationalized messages, you will need
the workaround described in the second half of
<http://www.gnu.org/software/gettext/manual/html_node/Preparing-Strings.ht...;.
You mean like this?
char buf1[100];
sprintf (buf1, "%0" MYPRId64, number);
printf (gettext ("The amount is %s\n"), buf1);
No thanks - like inttostr() it requires a temporary buffer, but unlike
inttostr(), it ends up being a slower three-liner due to the full
overhead of a second printf() call. If gettext can't support it, then
the shortest approach is the two-liner:
char buf1[INT_BUFSIZE_BOUND (size_t)];
printf (_("The amount is %s\n"), sizetomax (val, buf1));
Here's where a cross-project change to GNU Coding Standards could be
helpful - if we all agree that gnulib should add the macro and gettext
should add the support for it at the same time, then it would be much
easier for all remaining GNU projects to take advantage of a
standardized name (that is, use GCS rather than POSIX as our standard
that documents the portable use of PRIuSIZE among GNU projects).
However, it does take us back to the minimum build tool requirement
issue - any project relying on that proposed feature would necessarily
have to drag in whatever new version of gettext that added the support
for it.
> it still requires auditing code
> and forbidding "%zu" in favor of "%"PRIuSIZE (or whatever other
name we
> settle on).
You could hack GCC, clang, or even xgettext to produce a warning when it
sees a 'z' size modifier in a format string.
It sounds like xgettext might be the best place to detect %zu,
particularly if it already diagnoses existing uses of PRIuMAX in
projects that forgot to specify need-formatstring-macros in
AM_GNU_GETTEXT (and if it doesn't already diagnose translated strings
that require more features than the settings requested in configure.ac,
it probably should). But there's also maint.mk (or cfg.mk) that has
heuristics for detecting untranslated strings, which seems like it could
be pretty easily modified to also check for translated strings
containing %zu.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
6:28 p.m.
New subject: [libvirt] PRIuSIZE
Eric Blake wrote:
Here's where a cross-project change to GNU Coding Standards could
be
helpful - if we all agree that gnulib should add the macro and gettext
should add the support for it at the same time, then it would be much
easier for all remaining GNU projects to take advantage of a
standardized name
Yes. And it will need either a POSIX change or a common GNU extension,
because you will also need it implemented in glibc
<http://sourceware.org/git/?p=glibc.git;a=blob;f=intl/loadmsgcat.c>;.
That's certainly a thing you could propose to the Austin group.
Bruno
6:21 p.m.
On 08/17/2010 04:33 PM, Bruno Haible wrote:
> Which means you _can't_ use "%lu",(unsigned
long)size_t_val.
You _can_ use this. It will work as long as each of your program's data
structures is less than 4 GB large. Remember that size_t values get
larger than 4 GB only if you have a memory object (array) that is larger
than 4 GB.
That's a safe assumption if you are talking about sizeof(object), as it
is unlikely that anyone is intentionally compiling .o files with objects
that are statically allocated to be that large. But it is not a safe
assumption when using size_t as the argument to hold a user-specified
value matching the size of an dynamically allocated array, and goes
against the GCS mantra of no arbitrary limits (maybe the user really
does have a reason that they wanted a 4 gigabyte string loaded into
memory on a 64-bit machine). In other words, I think your statement is
true only for a subset of the valid uses of printing a size_t value.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
6:35 p.m.
> What a shame that POSIX omitted an <inttypes.h> PRIu* for
size_t.
You can define it by yourself ...
Or use uintptr_t instead of size_t. By the definition of these types,
you can be sure that uintptr_t is at least as wide as size_t. Then use
printf ("%" PRIuPTR, (uintptr_t) size_t_value);
- Works fine with gnulib's <inttypes.h>.
- Works fine with gettext.
- Doesn't need gettext or glibc versions that don't exist yet.
Bruno
6:37 p.m.
On 08/17/2010 05:35 PM, Bruno Haible wrote:
>> What a shame that POSIX omitted an <inttypes.h> PRIu*
for size_t.
>
> You can define it by yourself ...
Or use uintptr_t instead of size_t. By the definition of these types,
you can be sure that uintptr_t is at least as wide as size_t. Then use
printf ("%" PRIuPTR, (uintptr_t) size_t_value);
- Works fine with gnulib's <inttypes.h>.
- Works fine with gettext.
- Doesn't need gettext or glibc versions that don't exist yet.
But still needs the use of need-formatstring-macros, and gettext 0.16.1
or newer (whereas libvirt is stuck on 0.14.1 at the moment).
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
6:53 p.m.
[removed bug-gnulib from CC]
Eric Blake wrote:
> printf ("%" PRIuPTR, (uintptr_t) size_t_value);
But still needs the use of need-formatstring-macros, and gettext 0.16.1
or newer (whereas libvirt is stuck on 0.14.1 at the moment).
The uses of these formatstring macros require
- at package preparation time: the AM_GNU_GETTEXT macro from gettext
0.16.1 or newer,
- at build time and run time: a glibc newer than 2004-01-14 [1].
So, it requires gettext >= 0.16.1 to be installed ONLY on the machines
which are used to develop and release libvirt.
Bruno
[1]
http://sourceware.org/git/?p=glibc.git;a=commit;h=083dc54a01d3b65b9d95599...
5200
days inactive
5211
days old
20 comments
3 participants
participants (3)
-
Bruno Haible -
Daniel P. Berrange -
Eric Blake