[libvirt] [PATCH] threads: add one-time initialization support
mingw lacks the counterpart to PTHREAD_MUTEX_INITIALIZER, so the best we can do is portably expose once-only runtime initialization. * src/util/threads.h (virOnceControlPtr): New opaque type. (virOnceFunc): New callback type. (virOnce): New prototype. * src/util/threads-pthread.h (virOnceControl): Declare. (VIR_ONCE_CONTROL_INITIALIZER): Define. * src/util/threads-win32.h (virOnceControl) (VIR_ONCE_CONTROL_INITIALIZER): Likewise. * src/util/threads-pthread.c (virOnce): Implement in pthreads. * src/util/threads-win32.c (virOnce): Implement in WIN32. * src/libvirt_private.syms: Export it. --- src/libvirt_private.syms | 1 + src/util/threads-pthread.c | 5 +++++ src/util/threads-pthread.h | 11 ++++++++++- src/util/threads-win32.c | 15 ++++++++++++++- src/util/threads-win32.h | 11 ++++++++++- src/util/threads.h | 21 ++++++++++++++++++++- 6 files changed, 60 insertions(+), 4 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index ba7739d..cb67861 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -866,6 +866,7 @@ virMutexInit; virMutexInitRecursive; virMutexLock; virMutexUnlock; +virOnce; virThreadCreate; virThreadID; virThreadIsSelf; diff --git a/src/util/threads-pthread.c b/src/util/threads-pthread.c index 898c4d4..82ce5c6 100644 --- a/src/util/threads-pthread.c +++ b/src/util/threads-pthread.c @@ -40,6 +40,11 @@ void virThreadOnExit(void) { } +int virOnce(virOnceControlPtr once, virOnceFunc init) +{ + return pthread_once(&once->once, init); +} + int virMutexInit(virMutexPtr m) { diff --git a/src/util/threads-pthread.h b/src/util/threads-pthread.h index b25d0c2..dcaacb7 100644 --- a/src/util/threads-pthread.h +++ b/src/util/threads-pthread.h @@ -1,7 +1,7 @@ /* * threads.c: basic thread synchronization primitives * - * Copyright (C) 2009 Red Hat, Inc. + * Copyright (C) 2009, 2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -38,3 +38,12 @@ struct virThread { struct virThreadLocal { pthread_key_t key; }; + +struct virOnceControl { + pthread_once_t once; +}; + +#define VIR_ONCE_CONTROL_INITIALIZER \ +{ \ + .once = PTHREAD_ONCE_INIT \ +} diff --git a/src/util/threads-win32.c b/src/util/threads-win32.c index 5661437..f717f98 100644 --- a/src/util/threads-win32.c +++ b/src/util/threads-win32.c @@ -1,7 +1,7 @@ /* * threads-win32.c: basic thread synchronization primitives * - * Copyright (C) 2009-2010 Red Hat, Inc. + * Copyright (C) 2009-2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -69,6 +69,19 @@ void virThreadOnExit(void) virMutexUnlock(&virThreadLocalLock); } +static BOOL CALLBACK +virOnceCallback(PINIT_ONCE once, PVOID param, PVOID *context ATTRIBUTE_UNUSED) +{ + virOnceFunc func = param; + func(); + return TRUE; +} + +int virOnce(virOnceControlPtr once, virOnceFunc init) +{ + return InitOnceExecuteOnce(&once->once, virOnceCallback, init, + NULL) ? 0 : -1; +} int virMutexInit(virMutexPtr m) { diff --git a/src/util/threads-win32.h b/src/util/threads-win32.h index bb7c455..f169460 100644 --- a/src/util/threads-win32.h +++ b/src/util/threads-win32.h @@ -1,7 +1,7 @@ /* * threads-win32.h basic thread synchronization primitives * - * Copyright (C) 2009 Red Hat, Inc. + * Copyright (C) 2009, 2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -41,3 +41,12 @@ struct virThread { struct virThreadLocal { DWORD key; }; + +struct virOnceControl { + INIT_ONCE once; +}; + +#define VIR_ONCE_CONTROL_INITIALIZER \ +{ \ + .once = INIT_ONCE_STATIC_INIT \ +} diff --git a/src/util/threads.h b/src/util/threads.h index c129301..b72610c 100644 --- a/src/util/threads.h +++ b/src/util/threads.h @@ -1,7 +1,7 @@ /* * threads.h: basic thread synchronization primitives * - * Copyright (C) 2009-2010 Red Hat, Inc. + * Copyright (C) 2009-2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -36,6 +36,10 @@ typedef virThreadLocal *virThreadLocalPtr; typedef struct virThread virThread; typedef virThread *virThreadPtr; +typedef struct virOnceControl virOnceControl; +typedef virOnceControl *virOnceControlPtr; + +typedef void (*virOnceFunc)(void); int virThreadInitialize(void) ATTRIBUTE_RETURN_CHECK; void virThreadOnExit(void); @@ -57,6 +61,21 @@ void virThreadJoin(virThreadPtr thread); int virThreadSelfID(void); int virThreadID(virThreadPtr thread); +/* Static initialization of mutexes is not possible, so we instead + * provide for guaranteed one-time initialization via a callback + * function. Usage: + * static virOnceControl once = VIR_ONCE_CONTROL_INITIALIZER; + * static void initializer(void) { ... } + * void myfunc() + * { + * if (virOnce(&once, initializer) < 0) + * goto error; + * ...now guaranteed that initializer has completed exactly once + * } + */ +int virOnce(virOnceControlPtr once, virOnceFunc init) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; + int virMutexInit(virMutexPtr m) ATTRIBUTE_RETURN_CHECK; int virMutexInitRecursive(virMutexPtr m) ATTRIBUTE_RETURN_CHECK; void virMutexDestroy(virMutexPtr m); -- 1.7.1
On 04/20/2011 04:43 PM, Eric Blake wrote:
mingw lacks the counterpart to PTHREAD_MUTEX_INITIALIZER, so the best we can do is portably expose once-only runtime initialization.
* src/util/threads.h (virOnceControlPtr): New opaque type. (virOnceFunc): New callback type. (virOnce): New prototype. * src/util/threads-pthread.h (virOnceControl): Declare. (VIR_ONCE_CONTROL_INITIALIZER): Define. * src/util/threads-win32.h (virOnceControl) (VIR_ONCE_CONTROL_INITIALIZER): Likewise. * src/util/threads-pthread.c (virOnce): Implement in pthreads. * src/util/threads-win32.c (virOnce): Implement in WIN32. * src/libvirt_private.syms: Export it. +++ b/src/util/threads-win32.h @@ -1,7 +1,7 @@ /* * threads-win32.h basic thread synchronization primitives * - * Copyright (C) 2009 Red Hat, Inc. + * Copyright (C) 2009, 2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -41,3 +41,12 @@ struct virThread { struct virThreadLocal { DWORD key; }; + +struct virOnceControl { + INIT_ONCE once; +};
Self-NAK - not all versions of Windows support INIT_ONCE yet, http://msdn.microsoft.com/en-us/library/ms684122%28v=vs.85%29.aspx states that applications must use InterlockedIncrement to set up their own critical sections instead. The idea is still doable, it just requires more work on the WIN32 side of things. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
mingw lacks the counterpart to PTHREAD_MUTEX_INITIALIZER, so the best we can do is portably expose once-only runtime initialization. * src/util/threads.h (virOnceControlPtr): New opaque type. (virOnceFunc): New callback type. (virOnce): New prototype. * src/util/threads-pthread.h (virOnceControl): Declare. (VIR_ONCE_CONTROL_INITIALIZER): Define. * src/util/threads-win32.h (virOnceControl) (VIR_ONCE_CONTROL_INITIALIZER): Likewise. * src/util/threads-pthread.c (virOnce): Implement in pthreads. * src/util/threads-win32.c (virOnce): Implement in WIN32. * src/libvirt_private.syms: Export it. --- v2: change WIN32 implementation to use lower-level primitives available to mingw and older windows, rather than higher level but newer INIT_ONCE. Meanwhile, I noticed that gnulib has an LGPLv2+ 'lock' module that provides a lot of multi-threading primitives for both pthreads, pth, and mingw; maybe we should think about rewriting threads.c in terms of gnulib and only have one implementation, rather than maintaining pthread and mingw in parallel? src/libvirt_private.syms | 1 + src/util/threads-pthread.c | 5 +++++ src/util/threads-pthread.h | 11 ++++++++++- src/util/threads-win32.c | 23 ++++++++++++++++++++++- src/util/threads-win32.h | 9 ++++++++- src/util/threads.h | 21 ++++++++++++++++++++- 6 files changed, 66 insertions(+), 4 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index ba7739d..cb67861 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -866,6 +866,7 @@ virMutexInit; virMutexInitRecursive; virMutexLock; virMutexUnlock; +virOnce; virThreadCreate; virThreadID; virThreadIsSelf; diff --git a/src/util/threads-pthread.c b/src/util/threads-pthread.c index 898c4d4..82ce5c6 100644 --- a/src/util/threads-pthread.c +++ b/src/util/threads-pthread.c @@ -40,6 +40,11 @@ void virThreadOnExit(void) { } +int virOnce(virOnceControlPtr once, virOnceFunc init) +{ + return pthread_once(&once->once, init); +} + int virMutexInit(virMutexPtr m) { diff --git a/src/util/threads-pthread.h b/src/util/threads-pthread.h index b25d0c2..dcaacb7 100644 --- a/src/util/threads-pthread.h +++ b/src/util/threads-pthread.h @@ -1,7 +1,7 @@ /* * threads.c: basic thread synchronization primitives * - * Copyright (C) 2009 Red Hat, Inc. + * Copyright (C) 2009, 2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -38,3 +38,12 @@ struct virThread { struct virThreadLocal { pthread_key_t key; }; + +struct virOnceControl { + pthread_once_t once; +}; + +#define VIR_ONCE_CONTROL_INITIALIZER \ +{ \ + .once = PTHREAD_ONCE_INIT \ +} diff --git a/src/util/threads-win32.c b/src/util/threads-win32.c index 5661437..63f699b 100644 --- a/src/util/threads-win32.c +++ b/src/util/threads-win32.c @@ -1,7 +1,7 @@ /* * threads-win32.c: basic thread synchronization primitives * - * Copyright (C) 2009-2010 Red Hat, Inc. + * Copyright (C) 2009-2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -69,6 +69,27 @@ void virThreadOnExit(void) virMutexUnlock(&virThreadLocalLock); } +int virOnce(virOnceControlPtr once, virOnceFunc func) +{ + if (!once->complete) { + if (InterlockedIncrement(&once->init) == 1) { + /* We're the first thread. */ + func(); + once->complete = 1; + } else { + /* We're a later thread. Decrement the init counter back + * to avoid overflow, then yield until the first thread + * marks that the function is complete. It is rare that + * multiple threads will be waiting here, and since each + * thread is yielding except the first, we should get out + * soon enough. */ + InterlockedDecrement(&once->init); + while (!once->complete) + Sleep(0); + } + } + return 0; +} int virMutexInit(virMutexPtr m) { diff --git a/src/util/threads-win32.h b/src/util/threads-win32.h index bb7c455..29580b8 100644 --- a/src/util/threads-win32.h +++ b/src/util/threads-win32.h @@ -1,7 +1,7 @@ /* * threads-win32.h basic thread synchronization primitives * - * Copyright (C) 2009 Red Hat, Inc. + * Copyright (C) 2009, 2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -41,3 +41,10 @@ struct virThread { struct virThreadLocal { DWORD key; }; + +struct virOnceControl { + long init; /* 0 at startup, > 0 if init has started */ + volatile long complete; /* 0 until first thread completes callback */ +}; + +#define VIR_ONCE_CONTROL_INITIALIZER { 0, 0 } diff --git a/src/util/threads.h b/src/util/threads.h index c129301..b72610c 100644 --- a/src/util/threads.h +++ b/src/util/threads.h @@ -1,7 +1,7 @@ /* * threads.h: basic thread synchronization primitives * - * Copyright (C) 2009-2010 Red Hat, Inc. + * Copyright (C) 2009-2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -36,6 +36,10 @@ typedef virThreadLocal *virThreadLocalPtr; typedef struct virThread virThread; typedef virThread *virThreadPtr; +typedef struct virOnceControl virOnceControl; +typedef virOnceControl *virOnceControlPtr; + +typedef void (*virOnceFunc)(void); int virThreadInitialize(void) ATTRIBUTE_RETURN_CHECK; void virThreadOnExit(void); @@ -57,6 +61,21 @@ void virThreadJoin(virThreadPtr thread); int virThreadSelfID(void); int virThreadID(virThreadPtr thread); +/* Static initialization of mutexes is not possible, so we instead + * provide for guaranteed one-time initialization via a callback + * function. Usage: + * static virOnceControl once = VIR_ONCE_CONTROL_INITIALIZER; + * static void initializer(void) { ... } + * void myfunc() + * { + * if (virOnce(&once, initializer) < 0) + * goto error; + * ...now guaranteed that initializer has completed exactly once + * } + */ +int virOnce(virOnceControlPtr once, virOnceFunc init) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; + int virMutexInit(virMutexPtr m) ATTRIBUTE_RETURN_CHECK; int virMutexInitRecursive(virMutexPtr m) ATTRIBUTE_RETURN_CHECK; void virMutexDestroy(virMutexPtr m); -- 1.7.4.4
2011/4/23 Eric Blake <eblake@redhat.com>:
mingw lacks the counterpart to PTHREAD_MUTEX_INITIALIZER, so the best we can do is portably expose once-only runtime initialization.
* src/util/threads.h (virOnceControlPtr): New opaque type. (virOnceFunc): New callback type. (virOnce): New prototype. * src/util/threads-pthread.h (virOnceControl): Declare. (VIR_ONCE_CONTROL_INITIALIZER): Define. * src/util/threads-win32.h (virOnceControl) (VIR_ONCE_CONTROL_INITIALIZER): Likewise. * src/util/threads-pthread.c (virOnce): Implement in pthreads. * src/util/threads-win32.c (virOnce): Implement in WIN32. * src/libvirt_private.syms: Export it. ---
v2: change WIN32 implementation to use lower-level primitives available to mingw and older windows, rather than higher level but newer INIT_ONCE.
Meanwhile, I noticed that gnulib has an LGPLv2+ 'lock' module that provides a lot of multi-threading primitives for both pthreads, pth, and mingw; maybe we should think about rewriting threads.c in terms of gnulib and only have one implementation, rather than maintaining pthread and mingw in parallel?
+int virOnce(virOnceControlPtr once, virOnceFunc func) +{ + if (!once->complete) { + if (InterlockedIncrement(&once->init) == 1) { + /* We're the first thread. */ + func(); + once->complete = 1; + } else { + /* We're a later thread. Decrement the init counter back + * to avoid overflow, then yield until the first thread + * marks that the function is complete. It is rare that + * multiple threads will be waiting here, and since each + * thread is yielding except the first, we should get out + * soon enough. */ + InterlockedDecrement(&once->init); + while (!once->complete) + Sleep(0); + } + } + return 0; +}
Compared to the gnulib lock module version, this one is less complex and I don't get why gnulib uses 3 states for the init and complete variables and an additional lock object. The only reason could be to minimize the time busy waiting in the while loop.
+ +struct virOnceControl { + long init; /* 0 at startup, > 0 if init has started */ + volatile long complete; /* 0 until first thread completes callback */ +};
MSDN docs about InterlockedIncrement suggest that init should be volatile too. ACK, with init marked volatile. Matthias
On 04/24/2011 04:14 AM, Matthias Bolte wrote:
Compared to the gnulib lock module version, this one is less complex and I don't get why gnulib uses 3 states for the init and complete variables and an additional lock object. The only reason could be to minimize the time busy waiting in the while loop.
It may also be that the gnulib module was trying to worry about cancellation points, but libvirt isn't (yet) using cancellation points. That does indeed require more complexity to handle an init callback that gets cancelled, where another thread then has to take over the one-shot initiailization.
+ +struct virOnceControl { + long init; /* 0 at startup, > 0 if init has started */ + volatile long complete; /* 0 until first thread completes callback */ +};
MSDN docs about InterlockedIncrement suggest that init should be volatile too.
ACK, with init marked volatile.
Thanks, and pushed with that modification. I guess that also means that the virObject patches should also be marking its reference-counter variable as volatile. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
participants (2)
-
Eric Blake -
Matthias Bolte