February 2013 - Devel - libvirt List Archives

[libvirt] [RFC][PATCH] Growable bitmap implementation
by Stefan Berger 04 Feb '13

04 Feb '13

Add a growable bitmap implementation building on top of the 'regular' bitmap implementation. (no doubt, this could also be done differently...) This patch also adds a test virgbitmaptest.c that is largely a copy of virbitmaptest.c with some minor modifications to test for growth of the bitmap. Signed-off-by: Stefan Berger <stefanb(a)linux.vnet.ibm.com> --- PS: This patch builds on top of a patch with a virBitmapNextClearBit implementation. --- src/Makefile.am | 1 src/libvirt_private.syms | 22 ++ src/util/virbitmap.c | 17 + src/util/virbitmap.h | 6 src/util/virgbitmap.c | 348 ++++++++++++++++++++++++++++++++ src/util/virgbitmap.h | 95 ++++++++ tests/Makefile.am | 5 tests/virgbitmaptest.c | 499 +++++++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 993 insertions(+) Index: libvirt/src/Makefile.am =================================================================== --- libvirt.orig/src/Makefile.am +++ libvirt/src/Makefile.am @@ -71,6 +71,7 @@ UTIL_SOURCES = \ util/virevent.c util/virevent.h \ util/vireventpoll.c util/vireventpoll.h \ util/virfile.c util/virfile.h \ + util/virgbitmap.c util/virgbitmap.h \ util/virhash.c util/virhash.h \ util/virhashcode.c util/virhashcode.h \ util/virhook.c util/virhook.h \ Index: libvirt/src/util/virbitmap.c =================================================================== --- libvirt.orig/src/util/virbitmap.c +++ libvirt/src/util/virbitmap.c @@ -111,6 +111,23 @@ int virBitmapCopy(virBitmapPtr dst, virB return 0; } +/* + * virBitmapCopyResize: + * @dst: Destination bitmap + * @src: Source bitmap + * + * Copy bits from source bitmap @src into destination bitmap @dst + * where the source and destination bitmaps may be of different size. + * Any bits not found in the source bitmap will remain untouched + * in the destination bitmap. + */ +void virBitmapCopyResize(virBitmapPtr dst, virBitmapPtr src) +{ + size_t tocopy = MIN(src->map_len, dst->map_len); + + memcpy(dst->map, src->map, + tocopy * sizeof(src->map[0])); +} /** * virBitmapSetBit: Index: libvirt/src/util/virbitmap.h =================================================================== --- libvirt.orig/src/util/virbitmap.h +++ libvirt/src/util/virbitmap.h @@ -49,6 +49,12 @@ void virBitmapFree(virBitmapPtr bitmap); int virBitmapCopy(virBitmapPtr dst, virBitmapPtr src); /* + * Copy bits from @src to @dst. The bitmap sizes + * need not be the same + */ +void virBitmapCopyResize(virBitmapPtr dst, virBitmapPtr src); + +/* * Set bit position @b in @bitmap */ int virBitmapSetBit(virBitmapPtr bitmap, size_t b) Index: libvirt/src/util/virgbitmap.c =================================================================== --- /dev/null +++ libvirt/src/util/virgbitmap.c @@ -0,0 +1,348 @@ +/* + * virgbitmap.c: Growable bitmap operations + * + * Copyright (C) 2013 IBM Corporation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Author: Stefan Berger <stefanb(a)linux.vnet.ibm.com> + */ + +#include <config.h> + +#include "virgbitmap.h" +#include "viralloc.h" + +struct _virGBitmap { + virBitmapPtr bitmap; +}; + +/** + * virGBitmapNew: + * @size: number of bits + * + * Allocate a growable bitmap capable of initially containing @size bits. + * + * Returns a pointer to the allocated bitmap or NULL if + * memory cannot be allocated. + */ +virGBitmapPtr virGBitmapNew(size_t size) +{ + virGBitmapPtr bitmap; + + if (VIR_ALLOC(bitmap) < 0) + return NULL; + + bitmap->bitmap = virBitmapNew(size); + + if (!bitmap->bitmap) { + virGBitmapFree(bitmap); + return NULL; + } + + return bitmap; +} + +/** + * virGBitmapFree: + * @bitmap: previously allocated bitmap + * + * Free @bitmap previously allocated by virBitmapNew. + */ +void virGBitmapFree(virGBitmapPtr bitmap) +{ + if (bitmap) { + virBitmapFree(bitmap->bitmap); + VIR_FREE(bitmap); + } +} + +/** + * virGBitmapResize: + * @bitmap: previously allocated bitmap + * @new_size: new size of the bitmap + * + * Resize the @bitmap to the given @new_size. Old bits are copied + * into the new + */ +static int virGBitmapResize(virGBitmapPtr bitmap, size_t new_size) +{ + virBitmapPtr bm; + + if (!(bm = virBitmapNew(new_size))) + return -1; + + virBitmapCopyResize(bm, bitmap->bitmap); + + virBitmapFree(bitmap->bitmap); + bitmap->bitmap = bm; + + return 0; +} + +/** + * virGBitmapSetBit: + * @bitmap: Pointer to growable bitmap + * @b: bit position to set + * + * Set bit position @b in @bitmap. Grow the @bitmap if necessary + * filling newly allocated bits with zeros. + * + * Returns 0 on if bit is successfully set, -1 on error. + */ +int virGBitmapSetBit(virGBitmapPtr bitmap, ssize_t b) +{ + if (virBitmapSize(bitmap->bitmap) <= b && + virGBitmapResize(bitmap, + ((unsigned long long)b * 120) / 100) < 0) + return -1; + + return virBitmapSetBit(bitmap->bitmap, b); +} + +/** + * virBitmapGetBit: + * @bitmap: Pointer to bitmap + * @b: bit position to get + * @result: bool pointer to receive bit setting + * + * Get setting of bit position @b in @bitmap and store in @result. + * Note: If the bitmap is not currently large enough, it will grow + * to at least the size of the bit position @b. + * + * On success, @result will contain the setting of @b and 0 is + * returned. On failure, -1 is returned and @result is unchanged. + */ +int virGBitmapGetBit(virGBitmapPtr bitmap, size_t b, bool *result) +{ + if (virBitmapSize(bitmap->bitmap) <= b) { + if (virGBitmapResize(bitmap, + ((unsigned long long)b * 120) / 100) < 0) + return -1; + *result = 0; + return 0; + } + + return virBitmapGetBit(bitmap->bitmap, b, result); +} + +/** + * virGBitmapFormat: + * @bitmap: the bitmap + * + * This function is the counterpart of virGBitmapParse. This function creates + * a human-readable string representing the bits in bitmap. + * + * See virGBitmapParse for the format of @str. + * + * Returns the string on success or NULL otherwise. Caller should call + * VIR_FREE to free the string. + */ +char *virGBitmapFormat(virGBitmapPtr bitmap) +{ + return virBitmapFormat(bitmap->bitmap); +} + +size_t virGBitmapSize(virGBitmapPtr bitmap) +{ + return virBitmapSize(bitmap->bitmap); +} + +/** + * virGBitmapSetAll: + * @bitmap: the bitmap + * + * set all bits in @bitmap. + */ +void virGBitmapSetAll(virGBitmapPtr bitmap) +{ + virBitmapSetAll(bitmap->bitmap); +} + +/** + * virGBitmapClearAll: + * @bitmap: the bitmap + * + * clear all bits in @bitmap. + */ +void virGBitmapClearAll(virGBitmapPtr bitmap) +{ + virBitmapClearAll(bitmap->bitmap); +} + +/** + * virGBitmapIsAllSet: + * @bitmap: the bitmap to check + * + * check if all bits in @bitmap are set. + */ +bool virGBitmapIsAllSet(virGBitmapPtr bitmap) +{ + return virBitmapIsAllSet(bitmap->bitmap); +} + +/** + * virGBitmapParse: + * @str: points to a string representing a human-readable bitmap + * @terminator: character separating the bitmap to parse + * @bitmap: a bitmap created from @str + * @bitmapSize: the upper limit of num of bits in created bitmap + * + * This function is the counterpart of virBitmapFormat. This function creates + * a bitmap, in which bits are set according to the content of @str. + * + * @str is a comma separated string of fields N, which means a number of bit + * to set, and ^N, which means to unset the bit, and N-M for ranges of bits + * to set. + * + * To allow parsing of bitmaps within larger strings it is possible to set + * a termination character in the argument @terminator. When the character + * in @terminator is encountered in @str, the parsing of the bitmap stops. + * Pass 0 as @terminator if it is not needed. Whitespace characters may not + * be used as terminators. + * + * Returns the number of bits set in @bitmap, or -1 in case of error. + */ +int +virGBitmapParse(const char *str, + char terminator, + virGBitmapPtr *bitmap, + size_t bitmapSize) +{ + virGBitmapPtr res; + + if (VIR_ALLOC(res) < 0) + return -1; + + if (virBitmapParse(str, terminator, &res->bitmap, bitmapSize) < 0) { + VIR_FREE(res); + return -1; + } + + *bitmap = res; + + return 0; +} + + +/** + * virGBitmapNewData: + * @data: the data + * @len: length of @data in bytes + * + * Allocate a bitmap from a chunk of data containing bits + * information + * + * Returns a pointer to the allocated bitmap or NULL if + * memory cannot be allocated. + */ +virGBitmapPtr virGBitmapNewData(void *data, int len) +{ + virGBitmapPtr res; + + if (VIR_ALLOC(res) < 0) + return NULL; + + res->bitmap = virBitmapNewData(data, len); + if (!res->bitmap) { + VIR_FREE(res); + return NULL; + } + + return res; +} + +/** + * virGBitmapToData: + * @data: the data + * @len: len of @data in byte + * + * Convert a bitmap to a chunk of data containing bits information. + * Data consists of sequential bytes, with lower bytes containing + * lower bits. + * + * Returns 0 on success, -1 otherwise. + */ +int virGBitmapToData(virGBitmapPtr bitmap, unsigned char **data, int *dataLen) +{ + return virBitmapToData(bitmap->bitmap, data, dataLen); +} + + +/** + * virGBitmapNextSetBit: + * @bitmap: the bitmap + * @pos: the position after which to search for a set bit + * + * Search the first set bit after position @pos in bitmap @bitmap. + * @pos can be -1 to search for the first set bit. Position starts + * at 0. This function will never grow the bitmap. + * + * returns the position of the found bit, or -1 if no bit found. + */ +ssize_t virGBitmapNextSetBit(virGBitmapPtr bitmap, ssize_t pos) +{ + /* never grow since we won't find set bits there */ + return virBitmapNextSetBit(bitmap->bitmap, pos); +} + +/** + * virGBitmapNextClearBit: + * @bitmap: the bitmap + * @pos: the position after which to search for a set bit + * + * Search the first clear bit after position @pos in bitmap @bitmap. + * @pos can be -1 to search for the first set bit. Position starts + * at 0. This function will grow the bitmap if the currently allocated + * bitmap has all bits set. + * + * returns the position of the found bit, or -1 if no bit found. + */ +ssize_t virGBitmapNextClearBit(virGBitmapPtr bitmap, ssize_t pos) +{ + size_t lastpos, newsize, max_bit; + ssize_t res; + + if (pos < 0) + pos = -1; + + if (pos < (ssize_t)virBitmapSize(bitmap->bitmap)) { + res = virBitmapNextClearBit(bitmap->bitmap, pos); + + if (res >= 0) + return res; + } + + max_bit = virBitmapSize(bitmap->bitmap); + + lastpos = max_bit - 1; + if (pos > lastpos) + lastpos = pos; + + newsize = (max_bit > 80) + ? ((unsigned long long)max_bit * 120) / 100 /* 20 % growth */ + : max_bit + 24; + if (virGBitmapResize(bitmap, newsize) < 0) + return -1; + + return virBitmapNextClearBit(bitmap->bitmap, lastpos); +} + +/* Return the number of bits currently set in the map. */ +size_t +virGBitmapCountBits(virGBitmapPtr bitmap) +{ + return virBitmapCountBits(bitmap->bitmap); +} Index: libvirt/src/util/virgbitmap.h =================================================================== --- /dev/null +++ libvirt/src/util/virgbitmap.h @@ -0,0 +1,95 @@ +/* + * virgbitmap.h: Growable bitmap operations + * + * Copyright (C) 2013 IBM Corporation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Derived from virbitmap.h. + * + * Author: Stefan Berger <stefanb(a)linux.vnet.ibm.com> + */ + +#ifndef __GBITMAP_H__ +# define __GBITMAP_H__ + +# include "internal.h" + +# include <sys/types.h> + +# include "virbitmap.h" + +typedef struct _virGBitmap virGBitmap; +typedef virGBitmap *virGBitmapPtr; + +/* + * Allocate a bitmap capable of initially containing @size bits. + */ +virGBitmapPtr virGBitmapNew(size_t size) ATTRIBUTE_RETURN_CHECK; + +/* + * Free previously allocated bitmap + */ +void virGBitmapFree(virGBitmapPtr bitmap); + +/* + * Set bit position @b in @bitmap + */ +int virGBitmapSetBit(virGBitmapPtr bitmap, ssize_t pos) + ATTRIBUTE_NONNULL(1); + +/* + * Get setting of bit position @b in @bitmap and store in @result + */ +int virGBitmapGetBit(virGBitmapPtr bitmap, size_t b, bool *result) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(3) ATTRIBUTE_RETURN_CHECK; + +char *virGBitmapFormat(virGBitmapPtr bitmap) + ATTRIBUTE_NONNULL(1); + +int virGBitmapParse(const char *str, + char sep, + virGBitmapPtr *bitmap, + size_t bitmapSize); + +virGBitmapPtr virGBitmapNewData(void *data, int len) ATTRIBUTE_NONNULL(1); + +int virGBitmapToData(virGBitmapPtr bitmap, unsigned char **data, int *dataLen) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + +size_t virGBitmapSize(virGBitmapPtr bitmap) + ATTRIBUTE_NONNULL(1); + +void virGBitmapSetAll(virGBitmapPtr bitmap) + ATTRIBUTE_NONNULL(1); + +void virGBitmapClearAll(virGBitmapPtr bitmap) + ATTRIBUTE_NONNULL(1); + +bool virGBitmapIsAllSet(virGBitmapPtr bitmap) + ATTRIBUTE_NONNULL(1); + + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(3); + +ssize_t virGBitmapNextSetBit(virGBitmapPtr bitmap, ssize_t pos) + ATTRIBUTE_NONNULL(1); + +ssize_t virGBitmapNextClearBit(virGBitmapPtr bitmap, ssize_t pos) + ATTRIBUTE_NONNULL(1); + +size_t virGBitmapCountBits(virGBitmapPtr bitmap) + ATTRIBUTE_NONNULL(1); + +#endif Index: libvirt/tests/Makefile.am =================================================================== --- libvirt.orig/tests/Makefile.am +++ libvirt/tests/Makefile.am @@ -96,6 +96,7 @@ test_programs = virshtest sockettest \ virtimetest viruritest virkeyfiletest \ virauthconfigtest \ virbitmaptest \ + virgbitmaptest \ virlockspacetest \ virstringtest \ virportallocatortest \ @@ -649,6 +650,10 @@ virbitmaptest_SOURCES = \ virbitmaptest.c testutils.h testutils.c virbitmaptest_LDADD = $(LDADDS) +virgbitmaptest_SOURCES = \ + virgbitmaptest.c testutils.h testutils.c +virgbitmaptest_LDADD = $(LDADDS) + jsontest_SOURCES = \ jsontest.c testutils.h testutils.c jsontest_LDADD = $(LDADDS) Index: libvirt/tests/virgbitmaptest.c =================================================================== --- /dev/null +++ libvirt/tests/virgbitmaptest.c @@ -0,0 +1,499 @@ +/* + * Copyright (C) 2012 Fujitsu. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + +#include <config.h> + +#include "testutils.h" + +#include "virgbitmap.h" + +static int test1(const void *data ATTRIBUTE_UNUSED) +{ + virGBitmapPtr bitmap; + int size; + int bit; + bool result; + int ret = -1; + + size = 1024; + bit = 100; + if (!(bitmap = virGBitmapNew(size))) + goto error; + + if (virGBitmapSetBit(bitmap, bit) < 0) + goto error; + + if (virGBitmapGetBit(bitmap, bit, &result) < 0) + goto error; + + if (!result) + goto error; + + if (virGBitmapGetBit(bitmap, bit + 1, &result) < 0) + goto error; + + if (result) + goto error; + + ret = 0; + +error: + virGBitmapFree(bitmap); + return ret; +} + +static int +testBit(virGBitmapPtr bitmap, + unsigned int start, + unsigned int end, + bool expected) +{ + int i; + bool result; + + for (i = start; i <= end; i++) { + if (virGBitmapGetBit(bitmap, i, &result) < 0) + return -1; + if (result == expected) + return 0; + } + + return -1; +} + +static int test2(const void *data ATTRIBUTE_UNUSED) +{ + const char *bitsString1 = "1-32,50,88-99,1021-1023"; + const char *bitsStringAfter = "1-32,50,88-99,1021-1023,1111"; + char *bitsString2 = NULL, *bitsString3 = NULL; + virGBitmapPtr bitmap = NULL; + int ret = -1; + int size = 1025; + + if (virGBitmapParse(bitsString1, 0, &bitmap, size) < 0) + goto error; + + if (testBit(bitmap, 1, 32, true) < 0) + goto error; + if (testBit(bitmap, 50, 50, true) < 0) + goto error; + if (testBit(bitmap, 88, 99, true) < 0) + goto error; + if (testBit(bitmap, 1021, 1023, true) < 0) + goto error; + + if (testBit(bitmap, 0, 0, false) < 0) + goto error; + if (testBit(bitmap, 33, 49, false) < 0) + goto error; + if (testBit(bitmap, 51, 87, false) < 0) + goto error; + if (testBit(bitmap, 100, 1020, false) < 0) + goto error; + + if (virGBitmapCountBits(bitmap) != 48) + goto error; + + if (!(bitsString2 = virGBitmapFormat(bitmap))) + goto error; + if (strcmp(bitsString1, bitsString2)) + goto error; + + if (virGBitmapSetBit(bitmap, 1111) < 0) + goto error; + + if (!(bitsString3 = virGBitmapFormat(bitmap))) + goto error; + if (strcmp(bitsStringAfter, bitsString3)) + goto error; + + size = 1111; + virGBitmapSetAll(bitmap); + if (testBit(bitmap, 0, size - 1, true) < 0) + goto error; + if (virGBitmapCountBits(bitmap) < size) + goto error; + + if (!virGBitmapIsAllSet(bitmap)) + goto error; + + virGBitmapClearAll(bitmap); + if (testBit(bitmap, 0, size - 1, false) < 0) + goto error; + if (virGBitmapCountBits(bitmap) != 0) + goto error; + + ret = 0; + +error: + virGBitmapFree(bitmap); + VIR_FREE(bitsString2); + VIR_FREE(bitsString3); + return ret; +} + +static int test3(const void *data ATTRIBUTE_UNUSED) +{ + virGBitmapPtr bitmap = NULL; + int ret = -1; + int size = 5; + int i; + + if ((bitmap = virGBitmapNew(size)) == NULL) + goto error; + + for (i = 0; i < size * 2; i++) + ignore_value(virGBitmapSetBit(bitmap, i)); + + if (virGBitmapSize(bitmap) < (2 * size)) + goto error; + + if (!virGBitmapIsAllSet(bitmap)) + goto error; + + ret = 0; + +error: + virGBitmapFree(bitmap); + return ret; +} + +/* test for virGBitmapNextSetBit */ +static int test4(const void *data ATTRIBUTE_UNUSED) +{ + const char *bitsString = "0, 2-4, 6-10, 12, 14-18, 20, 22, 25"; + int size = 40; + int bitsPos[] = { + 0, 2, 3, 4, 6, 7, 8, 9, 10, 12, + 14, 15, 16, 17, 18, 20, 22, 25 + }; + int npos = 18; + virGBitmapPtr bitmap = NULL; + int i, j; + + /* 1. zero set */ + + bitmap = virGBitmapNew(size); + if (!bitmap) + goto error; + + if (virGBitmapNextSetBit(bitmap, -1) >= 0) + goto error; + + virGBitmapFree(bitmap); + bitmap = NULL; + + /* 2. partial set */ + + if (virGBitmapParse(bitsString, 0, &bitmap, size) < 0) + goto error; + if (!bitmap) + goto error; + + j = 0; + i = -1; + + while (j < npos) { + i = virGBitmapNextSetBit(bitmap, i); + if (i != bitsPos[j++]) + goto error; + } + + if (virGBitmapNextSetBit(bitmap, i) > 0) + goto error; + + /* 3. full set */ + + i = -1; + virGBitmapSetAll(bitmap); + + for (j = 0; j < size; j++) { + i = virGBitmapNextSetBit(bitmap, i); + if (i != j) + goto error; + } + + if (virGBitmapNextSetBit(bitmap, i) > 0) + goto error; + + virGBitmapFree(bitmap); + return 0; + +error: + virGBitmapFree(bitmap); + return -1; +} + +/* test for virGBitmapNewData/ToData */ +static int test5(const void *v ATTRIBUTE_UNUSED) +{ + char data[] = {0x01, 0x02, 0x00, 0x00, 0x04}; + unsigned char *data2 = NULL; + int len2; + int bits[] = {0, 9, 34}; + virGBitmapPtr bitmap; + int i, j; + int ret = -1; + + bitmap = virGBitmapNewData(data, sizeof(data)); + if (!bitmap) + goto error; + + i = 0; + j = -1; + while (i < sizeof(bits)/sizeof(int) && + (j = virGBitmapNextSetBit(bitmap, j)) >= 0) { + if (j != bits[i++]) + goto error; + } + if (virGBitmapNextSetBit(bitmap, j) > 0) + goto error; + + ignore_value(virGBitmapSetBit(bitmap, 2)); + ignore_value(virGBitmapSetBit(bitmap, 15)); + + if (virGBitmapToData(bitmap, &data2, &len2) < 0) + goto error; + + if (len2 != sizeof(data) || + data2[0] != 0x05 || + data2[1] != 0x82 || + data2[2] != 0x00 || + data2[3] != 0x00 || + data2[4] != 0x04) + goto error; + + ret = 0; +error: + virGBitmapFree(bitmap); + VIR_FREE(data2); + return ret; +} + +/* test for virGBitmapFormat */ +static int test6(const void *v ATTRIBUTE_UNUSED) +{ + virGBitmapPtr bitmap = NULL; + char *str = NULL; + int size = 64; + int ret = -1; + + bitmap = virGBitmapNew(size); + if (!bitmap) + goto error; + + str = virGBitmapFormat(bitmap); + if (!str) + goto error; + + if (!STREQ(str, "")) + goto error; + + VIR_FREE(str); + + ignore_value(virGBitmapSetBit(bitmap, 0)); + str = virGBitmapFormat(bitmap); + if (!str) + goto error; + + if (!STREQ(str, "0")) + goto error; + + VIR_FREE(str); + + ignore_value(virGBitmapSetBit(bitmap, 4)); + ignore_value(virGBitmapSetBit(bitmap, 5)); + str = virGBitmapFormat(bitmap); + if (!str) + goto error; + + if (!STREQ(str, "0,4-5")) + goto error; + + VIR_FREE(str); + + ignore_value(virGBitmapSetBit(bitmap, 6)); + str = virGBitmapFormat(bitmap); + if (!str) + goto error; + + if (!STREQ(str, "0,4-6")) + goto error; + + VIR_FREE(str); + + ignore_value(virGBitmapSetBit(bitmap, 13)); + ignore_value(virGBitmapSetBit(bitmap, 14)); + ignore_value(virGBitmapSetBit(bitmap, 15)); + ignore_value(virGBitmapSetBit(bitmap, 16)); + str = virGBitmapFormat(bitmap); + if (!str) + goto error; + + if (!STREQ(str, "0,4-6,13-16")) + goto error; + + VIR_FREE(str); + + ignore_value(virGBitmapSetBit(bitmap, 62)); + ignore_value(virGBitmapSetBit(bitmap, 63)); + str = virGBitmapFormat(bitmap); + if (!str) + goto error; + + if (!STREQ(str, "0,4-6,13-16,62-63")) + goto error; + + + ret = 0; +error: + virGBitmapFree(bitmap); + VIR_FREE(str); + return ret; +} + +static int test7(const void *v ATTRIBUTE_UNUSED) +{ + virGBitmapPtr bitmap; + size_t i; + size_t maxBit[] = { + 1, 8, 31, 32, 63, 64, 95, 96, 127, 128, 159, 160 + }; + size_t nmaxBit = 12; + + for (i = 0; i < nmaxBit; i++) { + bitmap = virGBitmapNew(maxBit[i]); + if (!bitmap) + goto error; + + if (virGBitmapIsAllSet(bitmap)) + goto error; + + ignore_value(virGBitmapSetBit(bitmap, 1)); + if (virGBitmapIsAllSet(bitmap)) + goto error; + + virGBitmapSetAll(bitmap); + if (!virGBitmapIsAllSet(bitmap)) + goto error; + + virGBitmapFree(bitmap); + } + + return 0; + +error: + virGBitmapFree(bitmap); + return -1; +} + +/* test for virGBitmapNextClearBit */ +static int test8(const void *data ATTRIBUTE_UNUSED) +{ + const char *bitsString = "0, 2-4, 6-10, 12, 14-18, 20, 22, 25, 28-33, 35, " + "41-42, 45-46"; + int size = 48; + int noBitsPos[] = { + 1, 5, 11, 13, 19, 21, 23, 24, + 26, 27, 34, 36, 37, 38, 39, 40, + 43, 44, 47 + }; + int npos = 19; + virGBitmapPtr bitmap = NULL; + int i, j; + + /* 1. zero set */ + + bitmap = virGBitmapNew(size); + if (!bitmap) + goto error; + + if (virGBitmapNextClearBit(bitmap, -1) != 0) + goto error; + + virGBitmapFree(bitmap); + bitmap = NULL; + + /* 2. partial set */ + + if (virGBitmapParse(bitsString, 0, &bitmap, size) < 0) + goto error; + if (!bitmap) + goto error; + + j = 0; + i = -1; + + while (j < npos) { + i = virGBitmapNextClearBit(bitmap, i); + if (i != noBitsPos[j++]) + goto error; + } + + if (virGBitmapNextSetBit(bitmap, i) > 0) + goto error; + + /* 3. full set */ + + i = -1; + virGBitmapSetAll(bitmap); + + for (j = 0; j < size; j++) { + i = virGBitmapNextSetBit(bitmap, i); + if (i != j) + goto error; + } + + if (virGBitmapNextClearBit(bitmap, i) != i + 1) + goto error; + + virGBitmapFree(bitmap); + return 0; + +error: + virGBitmapFree(bitmap); + return -1; +} + +static int +mymain(void) +{ + int ret = 0; + + if (virtTestRun("test1", 1, test1, NULL) < 0) + ret = -1; + if (virtTestRun("test2", 1, test2, NULL) < 0) + ret = -1; + if (virtTestRun("test3", 1, test3, NULL) < 0) + ret = -1; + if (virtTestRun("test4", 1, test4, NULL) < 0) + ret = -1; + if (virtTestRun("test5", 1, test5, NULL) < 0) + ret = -1; + if (virtTestRun("test6", 1, test6, NULL) < 0) + ret = -1; + if (virtTestRun("test7", 1, test7, NULL) < 0) + ret = -1; + if (virtTestRun("test8", 1, test8, NULL) < 0) + ret = -1; + + return ret; +} + +VIRT_TEST_MAIN(mymain) Index: libvirt/src/libvirt_private.syms =================================================================== --- libvirt.orig/src/libvirt_private.syms +++ libvirt/src/libvirt_private.syms @@ -9,6 +9,7 @@ virBitmapClearAll; virBitmapClearBit; virBitmapCopy; +virBitmapCopyResize; virBitmapCountBits; virBitmapEqual; virBitmapFormat; @@ -1390,6 +1391,27 @@ virFileWrapperFdFree; virFileWrapperFdNew; +# virbitmap.h +virGBitmapClearAll; +virGBitmapClearBit; +virGBitmapCopy; +virGBitmapCountBits; +virGBitmapFormat; +virGBitmapFree; +virGBitmapGetBit; +virGBitmapIsAllSet; +virGBitmapNew; +virGBitmapNewData; +virGBitmapNextClearBit; +virGBitmapNextSetBit; +virGBitmapParse; +virGBitmapSetAll; +virGBitmapSetBit; +virGBitmapSize; +virGBitmapString; +virGBitmapToData; + + # virinitctl.h virInitctlSetRunLevel;

3 2

[libvirt] [PATCH] complete virterror->virerror name change
by Serge Hallyn 04 Feb '13

04 Feb '13

Without these two string changes in generator.py, the virGetLastError wrapper does not get created in /usr/share/pyshared/libvirt.py. Noticed when running tests with virt-install. Signed-off-by: Serge Hallyn <serge.hallyn(a)ubuntu.com> --- python/generator.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/generator.py b/python/generator.py index 5d27f66..71ca883 100755 --- a/python/generator.py +++ b/python/generator.py @@ -123,7 +123,7 @@ class docParser(xml.sax.handler.ContentHandler): self.function_return_field = attrs['field'] elif tag == 'enum': if (attrs['file'] == "libvirt" or - attrs['file'] == "virterror"): + attrs['file'] == "virerror"): enum(attrs['type'],attrs['name'],attrs['value']) elif attrs['file'] == "libvirt-lxc": lxc_enum(attrs['type'],attrs['name'],attrs['value']) @@ -137,7 +137,7 @@ class docParser(xml.sax.handler.ContentHandler): if self.function != None: if (self.function_module == "libvirt" or self.function_module == "virevent" or - self.function_module == "virterror"): + self.function_module == "virerror"): function(self.function, self.function_descr, self.function_return, self.function_args, self.function_file, self.function_module, -- 1.8.0

4 6

[libvirt] [PATCH 0/9] Glue domain and storage
by Osier Yang 04 Feb '13

04 Feb '13

This is the 4th part to implement NPIV migration support [1]. Part 1: https://www.redhat.com/archives/libvir-list/2013-January/msg00859.html Part 2: (Already ACKed by Michal) https://www.redhat.com/archives/libvir-list/2013-January/msg00859.html Part 3: https://www.redhat.com/archives/libvir-list/2013-January/msg01012.html To achieve the NPIV migration support, Part 2, Part 3 and this set are necessary, Part 1 is optional. However, this set can be reviewd independantly. ============ This introduces new XMLs to specify the disk source with storage like <disk type='volume' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source pool="$pool_name" volume='$vol_name'/> <seclabel relabel='no'/> </source> <target dev='vdb' bus='virtio'/> </disk> And before domain starting, the source represented by storage is translated into the real underlying source. Diff with RFC: * The XMLs are more simpler - only using pool name and volume name to specify disk source. * Support network pool (rbd, and sheepdog) * Support startupPolicy for volume type disk * Support seclabels for volume type disk * Fix bugs on disk source formating Osier Yang (9): rng: Add definition for network disk source Introduce new XMLs to specify disk source using libvirt storage New files to glue domain and storage together Implement translateDiskSourcePool qemu: Translate the source when starting domain Support startupPolicy for 'volume' disk conf: Fix bugs on disk source formating Support seclabels for volume type disk Support network pool for volume disk docs/formatdomain.html.in | 49 +++-- docs/schemas/domaincommon.rng | 111 +++++--- src/Makefile.am | 3 +- src/conf/domain_conf.c | 278 ++++++++++++++------ src/conf/domain_conf.h | 9 + src/conf/domain_storage.c | 44 +++ src/conf/domain_storage.h | 37 +++ src/libvirt_private.syms | 5 + src/qemu/qemu_driver.c | 7 + src/storage/storage_driver.c | 144 ++++++++++ .../qemuxml2argv-disk-source-pool.xml | 33 +++ tests/qemuxml2xmltest.c | 1 + 12 files changed, 573 insertions(+), 148 deletions(-) create mode 100644 src/conf/domain_storage.c create mode 100644 src/conf/domain_storage.h create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-source-pool.xml [1] https://www.redhat.com/archives/libvir-list/2012-November/msg00826.html Osier

3 29

[libvirt] [PATCH 00/17] virsh: Rework string option retrieval
by Peter Krempa 04 Feb '13

04 Feb '13

This patchset fixes retrieval of string arguments in virsh to report sane errors in all the places it's used. A new string option retrieval function is added that supports error reporting and changes appropriate places to use it. Along with that this series cleans up a few places with old code and bad style. *** BLURB HERE *** Peter Krempa (17): virsh-snapshot: Fix coding style and remove dead code virsh-volume: Rename helper function makeCloneXML to vshMakeCloneXML virsh-pool: Rename helper function buildPoolXML to vshBuildPoolXML virsh: Add helper to request string arguments with error reporting virsh-pool: Update pool commands to use vshCommandOptStringReq virsh-volume: Update volume commands to use vshCommandOptStringReq virsh-domain-monitor: Update domain commands to use vshCommandOptStringReq virsh-domain: Update domain commands to use vshCommandOptStringReq virsh-host: Update host commands to use vshCommandOptStringReq virsh-network: Update network commands to use vshCommandOptStringReq virsh-interface: Update interface commands to use vshCommandOptStringReq virsh-nodedev: Update node device commands to use vshCommandOptStringReq virsh-nodedev: Refactor error paths, error messages and whitespace virsh-snapshot: Refactor cmdSnapshotDumpXML virsh-snapshot: Update snapshot commands to use vshCommandOptStringReq virsh-nwfilter: Update nwfilter commands to use vshCommandOptStringReq virsh-secret: Refactor error paths tools/virsh-domain-monitor.c | 44 +++---- tools/virsh-domain.c | 299 +++++++++++++++++-------------------------- tools/virsh-host.c | 9 +- tools/virsh-interface.c | 8 +- tools/virsh-network.c | 19 +-- tools/virsh-nodedev.c | 44 ++++--- tools/virsh-nwfilter.c | 4 +- tools/virsh-pool.c | 56 ++++---- tools/virsh-secret.c | 30 +++-- tools/virsh-snapshot.c | 55 +++----- tools/virsh-volume.c | 73 +++++------ tools/virsh.c | 51 ++++++++ tools/virsh.h | 4 + 13 files changed, 320 insertions(+), 376 deletions(-) -- 1.8.1.1

2 44

[libvirt] RFC: Admin interface for connection monitoring
by Martin Kletzander 04 Feb '13

04 Feb '13

Hello everyone. Trying to create new interface for the libvirt daemon, mainly for connection monitoring, I came across few things I'd like to propose/discuss. This interface will be used to manage active connections of the daemon (list/kill) and other until-now impossible things (e.g. changing logging filters on-the-fly). Connection to this interface will be: - available even when no normal/priority workers are available, - distinguished (or completely separated) from other daemon connections, - (should be) more restricted than other connections. Having the connection available with all the worker threads being blocked/deadlocked creates the possibility of for example listing the connections even when all APIs block. Such manipulations mustn't be possible over the standard connections and that's why the connections should be distinguished. The connection restriction is hard to check, but the permissions for the socket should be less or equal to the rw socket with having no authentication only when rw socket has no authentication. My idea is following: - new 'admin_protocol.x' with specification of new program, - default connection limitation to this interface to only root-owned socket with 700 (authentication optional, see above), so only root on the host can utilize it, - configurable number of admin workers, but with 1 as a default, - splitting usable parts of virsh and reusing them to create virsh-admin binary. More controversial ideas I'd like to hear somebody else's opinion (even more than for the other things): - create a new separate library (libvirt-admin.so) to be used in the client, this library will have it's own virAdminConnect{,Auth}Open(), probably without an URI parameter (or with, but disabled for now, in case we want to enable remote connections later) - it is IMHO pointless to add a driver type for this, but it should definitely be configurable at compile time as well as in config (admin_workers = 0, maybe), - keep the connection handling on the current code and instead of duplicating much of it (so there will be a new service, much like the read-only socket, but for admin API only), - create a new pool of workers just for admin connections (similarly to the priority workers pool), with the dispatching being recognized by the protocol it is created in (remote, qemu, lxc protocols will go to normal/priority workers, whether admin protocol will be dispatched to the admin workers) and of course impossible from other connections than the admin one, which leads me to the next point; - we can either add next flag for admin connection (similarly to the readonly one) or make it future-friendly and change the readonly bool into a virConnectionType enum (for example). I must admit, I'm probably doing something wrong or we have old documentation. Commands like 'make -C src rpcgen' don't work and I don't understand why admin_protocol-structs isn't automatically generated, but I've made it work for now. Have a nice day, Martin

2 1

[libvirt] How to connect to console of domain on PowerPC?
by Yin Olivia-R63875 04 Feb '13

04 Feb '13

Hi, I tried to use libvirt to run KVM/QEMU on Freescale PowerPC platforms. So far there's only one serial device (spapr-vty) defined in QEMU to work as console for IBM PSeries platform. There's no serial device support in QEMU for Freescale PowerPC (ePAPR). libvirt/src/qemu/qemu_command.c /* This function generates the correct '-device' string for character * devices of each architecture. */ char * qemuBuildChrDeviceStr(virDomainChrDefPtr serial, virBitmapPtr qemuCaps, char *os_arch, char *machine) { virBuffer cmd = VIR_BUFFER_INITIALIZER; if (STREQ(os_arch, "ppc64") && STREQ(machine, "pseries")) { if (serial->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_SERIAL && serial->source.type == VIR_DOMAIN_CHR_TYPE_PTY && serial->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_SPAPRVIO) { virBufferAsprintf(&cmd, "spapr-vty,chardev=char%s", serial->info.alias); if (qemuBuildDeviceAddressStr(&cmd, &serial->info, qemuCaps) < 0) goto error; } } else virBufferAsprintf(&cmd, "isa-serial,chardev=char%s,id=%s", serial->info.alias, serial->info.alias); if (virBufferError(&cmd)) { virReportOOMError(); goto error; } return virBufferContentAndReset(&cmd); error: virBufferFreeAndReset(&cmd); return NULL; } We usually connect guest with telnet. For instance, /usr/bin/qemu-system-ppc -name demo -M ppce500v2 -enable-kvm -m 256 -nographic -kernel /media/ram/uImage -initrd /media/ram/ramdisk -append "root=/dev/ram rw console=ttyS0,115200" -serial tcp::4445,server Then to run 'telnet 10.193.20.xxx 4445' could connect the guest. The temporary workaround is not add '-device' string after '-serial' option. diff -Nur libvirt-0.10.1.orig/src/qemu/qemu_command.c libvirt-0.10.1/src/qemu/qemu_command.c --- libvirt-0.10.1.orig/src/qemu/qemu_command.c 2012-08-30 15:35:18.000000000 +0530 +++ libvirt-0.10.1/src/qemu/qemu_command.c 2012-10-05 17:19:32.060368755 +0530 @@ -5501,13 +5501,15 @@ virCommandAddArg(cmd, devstr); VIR_FREE(devstr); - virCommandAddArg(cmd, "-device"); - if (!(devstr = qemuBuildChrDeviceStr(serial, qemuCaps, + if (!STREQ(def->os.arch, "ppc")) { + virCommandAddArg(cmd, "-device"); + if (!(devstr = qemuBuildChrDeviceStr(serial, + qemuCaps, def->os.arch, def->os.machine))) - goto error; - virCommandAddArg(cmd, devstr); - VIR_FREE(devstr); + goto error; + virCommandAddArg(cmd, devstr); + VIR_FREE(devstr); + } } else { virCommandAddArg(cmd, "-serial"); if (!(devstr = qemuBuildChrArgStr(&serial->source, NULL))) Applying the above patch to libvirt, all the other domain control commands could work except 'virsh console domain'. # cat >demo.args <<EOF > /usr/bin/qemu-system-ppc -name demo -M ppce500v2 -enable-kvm -m 256 > -nographic -kernel /media/ram/uImage -initrd /media/ram/ramdisk > -append "root=/dev/ram rw console=ttyS0,115200" -serial > tcp::4445,server -net nic EOF # vi demo.args /usr/bin/qemu-system-ppc -name demo -M ppce500v2 -enable-kvm -m 256 -nographic -kernel /media/ram/uImage -initrd /media/ram/ramdisk -append "root=/dev/ram rw console=ttyS0,115200" -serial tcp::4445,server -net nic # virsh domxml-from-native qemu-argv demo.args >demo.xml # vi demo.xml <domain type='kvm'> <name>demo</name> <uuid>985d7154-83c8-0763-cbac-ecd159eee8a6</uuid> <memory unit='KiB'>262144</memory> <currentMemory unit='KiB'>262144</currentMemory> <vcpu placement='static'>1</vcpu> <os> <type arch='ppc' machine='ppce500v2'>hvm</type> <kernel>/media/ram/uImage</kernel> <initrd>/media/ram/ramdisk</initrd> <cmdline>root=/dev/ram rw console=ttyS0,115200</cmdline> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-ppc</emulator> <serial type='tcp'> <source mode='bind' host='' service='4445'/> <protocol type='raw'/> <target port='0'/> </serial> <console type='tcp'> <source mode='bind' host='' service='4445'/> <protocol type='raw'/> <target type='serial' port='0'/> </console> <memballoon model='virtio'/> </devices> </domain> # virsh -c qemu:///system define demo.xml # virsh -c qemu:///system start demo But it seemed that can't connect to the console. # virsh -c qemu:///system console demo Connected to domain test Escape character is ^] error: internal error character device (null) is not using a PTY I tried also use '-serial pty' option, /usr/bin/qemu-system-ppc -name test -M ppce500v2 -enable-kvm -m 256 -nographic -kernel /media/ram/uImage -initrd /media/ram/ramdisk -append "root=/dev/ram rw console=ttyS0,115200" -serial pty Then there's no other output after the below message: # virsh -c qemu:///system console demo Connected to domain test Escape character is ^] It seemed not libvirt group issue. I also tried the LXC. It could connect to the console if <init>/bin/sh/</init> instead of <init>/sbin/init</init> Best Regards, Olivia

1 0

[libvirt] [PATCH] rpc: Fix crash on error paths of message dispatching
by Peter Krempa 03 Feb '13

03 Feb '13

When reading and dispatching of a message failed the message was freed but wasn't removed from the message queue. After that when the connection was about to be closed the pointer for the message was still present in the queue and it was passed to virNetMessageFree which tried to call the callback function from an uninitialized pointer. This patch removes the message from the queue before it's freed. * rpc/virnetserverclient.c: virNetServerClientDispatchRead: - avoid use after free of RPC messages --- src/rpc/virnetserverclient.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/rpc/virnetserverclient.c b/src/rpc/virnetserverclient.c index af0560e..446e1e9 100644 --- a/src/rpc/virnetserverclient.c +++ b/src/rpc/virnetserverclient.c @@ -987,6 +987,7 @@ readmore: /* Decode the header so we can use it for routing decisions */ if (virNetMessageDecodeHeader(msg) < 0) { + virNetMessageQueueServe(&client->rx); virNetMessageFree(msg); client->wantClose = true; return; @@ -996,6 +997,7 @@ readmore: * file descriptors */ if (msg->header.type == VIR_NET_CALL_WITH_FDS && virNetMessageDecodeNumFDs(msg) < 0) { + virNetMessageQueueServe(&client->rx); virNetMessageFree(msg); client->wantClose = true; return; /* Error */ @@ -1005,6 +1007,7 @@ readmore: for (i = msg->donefds ; i < msg->nfds ; i++) { int rv; if ((rv = virNetSocketRecvFD(client->sock, &(msg->fds[i]))) < 0) { + virNetMessageQueueServe(&client->rx); virNetMessageFree(msg); client->wantClose = true; return; -- 1.8.1.1

5 9

[libvirt] [Libvirt]How to get libvirt version after git clone?
by harryxiyou 02 Feb '13

02 Feb '13

Hi all, After i clone libvirt git repo, how i could get this libvirt version exactly? $ git clone git://libvirt.org/libvirt.git $ git branch -a * master remotes/origin/HEAD -> origin/master remotes/origin/master remotes/origin/v0.10.2-maint remotes/origin/v0.9.11-maint remotes/origin/v0.9.6-maint I can just see up stuffs but exact current version. Could anyone give me some suggestions? Thanks in advance. -- Thanks Harry Wei

3 3

[libvirt] [Libvirt]Snapshot operations in Libvirt
by harryxiyou 02 Feb '13

02 Feb '13

Hi all, I see our libvirt snapshot XML form like following, which this XML is to create a disk snapshot of just vda on a qemu domain with two disks. <domainsnapshot> <description>Snapshot of OS install and updates</description> <disks> <disk name='/path/to/old'> <source file='/path/to/new'/> </disk> <disk name='vdb' snapshot='no'/> </disks> </domainsnapshot> My understanding about up XML is like this: it creates a snapshot for '/path/to/old', which '/path/to/old' would be the read-only backing file to the new active file '/path/to/new'. I also have some questions about current libvirt snapshot operations like following. 1, Is my up understanding right? Or maybe you have any other suggestions. 2, I wonder whether libvirt *NOW* support start a snapshot and then mount this new storage block, which is based on the snapshot. Please give me some real examples (like Sheepdog volumes) if you can. -- Thanks Harry Wei

2 1

[libvirt] [PATCH] hyperv: Avoid Coverity warning about resource leak in hypervOpen
by Matthias Bolte 02 Feb '13

02 Feb '13

Coverity tagged the esxhypervPrivate pointer a potentially leaked because of the conditional free call. But this is a false positive, there is not actual leak. Avoid the conditial code to make this more obvious to Coverity. --- This patch is meant as a replacement for this patch https://www.redhat.com/archives/libvir-list/2013-January/msg00542.html src/hyperv/hyperv_driver.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c index 601a85a..0d2eaae 100644 --- a/src/hyperv/hyperv_driver.c +++ b/src/hyperv/hyperv_driver.c @@ -200,14 +200,11 @@ hypervOpen(virConnectPtr conn, virConnectAuthPtr auth, unsigned int flags) } conn->privateData = priv; - + priv = NULL; result = VIR_DRV_OPEN_SUCCESS; cleanup: - if (result == VIR_DRV_OPEN_ERROR) { - hypervFreePrivate(&priv); - } - + hypervFreePrivate(&priv); VIR_FREE(username); VIR_FREE(password); hypervFreeObject(priv, (hypervObject *)computerSystem); -- 1.7.9.5

2 3