- Devel - libvirt List Archives

[libvirt] [PATCH v2] esx: Add the actual product version to the error message.
by Richard W.M. Jones 08 Sep '14

08 Sep '14

Ugh, printing the wrong field in the second error message. Try again with that fixed. Rich.

2 2

[libvirt] [PATCH] esx: Add the actual product version to the error message.
by Richard W.M. Jones 08 Sep '14

08 Sep '14

This is just an old patch I had from when I was trying to use the 'esx://' URI scheme to access a vCenter resource (which turns out to use 'vpx://' instead, which was the problem I was having). It's useful for debugging purposes to print the actual product version from the server in the error message. Rich.

2 2

[libvirt] [PATCH] qemu: panic device: check for invalid address type
by Erik Skultety 08 Sep '14

08 Sep '14

qemu now checks for invalid address type for a panic device, which is currently implemented only to use ISA address type, thus rejecting any other options, except for leaving XML attributes blank, in that case, defaults are used (this behaviour remains the same from earlier verions). --- src/qemu/qemu_command.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 1ca98fb..2611799 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -9418,12 +9418,16 @@ qemuBuildCommandLine(virConnectPtr conn, if (def->panic) { if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PANIC)) { - if (def->panic->info.addr.isa.iobase > 0) { + if (def->panic->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_ISA) { virCommandAddArg(cmd, "-device"); virCommandAddArgFormat(cmd, "pvpanic,ioport=%d", def->panic->info.addr.isa.iobase); - } else { + } else if (def->panic->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) { virCommandAddArgList(cmd, "-device", "pvpanic", NULL); + } else { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("panic device only works with ISA address type")); + goto error; } } else { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", -- 1.9.3

2 2

[libvirt] [PATCH] qemu: Propagate QEMU errors during incoming migrations
by Jiri Denemark 08 Sep '14

08 Sep '14

When QEMU fails during incoming migration after we successfully started it (i.e., during Perform or Finish phase), we report a rather unhelpful message Unable to read from monitor: Connection reset by peer We already have a code that takes error messages from QEMU's error output but we disable it once QEMU successfully starts. This patch postpones this until the end of Finish phase during incoming migration so that we can report a much better error message: internal error: early end of file from monitor: possible problem: Unknown savevm section or instance '0000:00:05.0/virtio-balloon' 0 load of migration failed https://bugzilla.redhat.com/show_bug.cgi?id=1090093 Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_migration.c | 2 ++ src/qemu/qemu_monitor.c | 7 ++++++- src/qemu/qemu_process.c | 6 ++++-- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index f2b421e..64f0d8d 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -4689,6 +4689,8 @@ qemuMigrationFinish(virQEMUDriverPtr driver, cleanup: virPortAllocatorRelease(driver->migrationPorts, port); if (vm) { + if (priv->mon) + qemuMonitorSetDomainLog(priv->mon, -1); VIR_FREE(priv->origname); virObjectUnlock(vm); } diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index d96b1b8..3bb6cb9 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -631,8 +631,11 @@ qemuMonitorIO(int watch, int fd, int events, void *opaque) error = true; } else { if (events & VIR_EVENT_HANDLE_WRITABLE) { - if (qemuMonitorIOWrite(mon) < 0) + if (qemuMonitorIOWrite(mon) < 0) { error = true; + if (errno == ECONNRESET) + hangup = true; + } events &= ~VIR_EVENT_HANDLE_WRITABLE; } @@ -642,6 +645,8 @@ qemuMonitorIO(int watch, int fd, int events, void *opaque) events &= ~VIR_EVENT_HANDLE_READABLE; if (got < 0) { error = true; + if (errno == ECONNRESET) + hangup = true; } else if (got == 0) { eof = true; } else { diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 38ed3fe..b1d8a32 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4379,8 +4379,10 @@ int qemuProcessStart(virConnectPtr conn, goto cleanup; } - /* unset reporting errors from qemu log */ - qemuMonitorSetDomainLog(priv->mon, -1); + /* Keep watching qemu log for errors during incoming migration, otherwise + * unset reporting errors from qemu log. */ + if (!migrateFrom) + qemuMonitorSetDomainLog(priv->mon, -1); virCommandFree(cmd); VIR_FORCE_CLOSE(logfile); -- 2.1.0

2 2

[libvirt] [PATCH 0/5] qemu: Fix job handling in snapshot code
by Peter Krempa 08 Sep '14

08 Sep '14

The snapshot code had some issues with job handling. This is an attempt to do it right. Jincheng Miao (1): qemu: snapshot: Acquire job earlier on snapshot revert/delete Peter Krempa (4): qemu: Rename DEFAULT_JOB_MASK to QEMU_DEFAULT_JOB_MASK qemu: snapshot: Fix job handling when creating snapshots qemu: snapshot: Fix snapshot function header formatting qemu: snapshot: Simplify error paths src/qemu/qemu_domain.c | 2 +- src/qemu/qemu_domain.h | 2 +- src/qemu/qemu_driver.c | 317 +++++++++++++++++++++------------------------- src/qemu/qemu_migration.c | 6 +- 4 files changed, 151 insertions(+), 176 deletions(-) -- 2.0.2

2 11

Re: [libvirt] [Qemu-devel] IO accounting overhaul
by Markus Armbruster 08 Sep '14

08 Sep '14

Cc'ing libvirt following Stefan's lead. Benoît Canet <benoit.canet(a)irqsave.net> writes: > Hi, > > I collected some items of a cloud provider wishlist regarding I/O accouting. Feedback from real power-users, lovely! > In a cloud I/O accouting can have 3 purpose: billing, helping the customers > and doing metrology to help the cloud provider seeks hidden costs. > > I'll cover the two former topic in this mail because they are the most important > business wize. > > 1) prefered place to collect billing IO accounting data: > -------------------------------------------------------- > For billing purpose the collected data must be as close as possible to what the > customer would see by using iostats in his vm. Good point. > The first conclusion we can draw is that the choice of collecting IO accouting > data used for billing in the block devices models is right. Slightly rephrasing: doing I/O accounting in the block device models is right for billing. There may be other uses for I/O accounting, with different preferences. For instance, data on how exactly guest I/O gets translated to host I/O as it flows through the nodes in the block graph could be useful. Doesn't diminish the need for accurate billing information, of course. > 2) what to do with occurences of rare events: > --------------------------------------------- > > Another point is that QEMU developpers agree that they don't know which policy > to apply to some I/O accounting events. > Must QEMU discard invalid I/O write IO or account them as done ? > Must QEMU count a failed read I/O as done ? > > When discusting this with a cloud provider the following appears: > these decisions > are really specific to each cloud provider and QEMU should not implement them. Good point, consistent with the old advice to avoid baking policy into inappropriately low levels of the stack. > The right thing to do is to add accouting counters to collect these events. > > Moreover these rare events are precious troubleshooting data so it's > an additional > reason not to toss them. Another good point. > 3) list of block I/O accouting metrics wished for billing and helping > the customers > ----------------------------------------------------------------------------------- > > Basic I/O accouting data will end up making the customers bills. > Extra I/O accouting informations would be a precious help for the cloud provider > to implement a monitoring panel like Amazon Cloudwatch. These are the first two from your list of three purposes, i.e. the ones you promised to cover here. > Here is the list of counters and statitics I would like to help > implement in QEMU. > > This is the most important part of the mail and the one I would like > the community > review the most. > > Once this list is settled I would proceed to implement the required > infrastructure > in QEMU before using it in the device models. For context, let me recap how I/O accounting works now. The BlockDriverState abstract data type (short: BDS) can hold the following accounting data: uint64_t nr_bytes[BDRV_MAX_IOTYPE]; uint64_t nr_ops[BDRV_MAX_IOTYPE]; uint64_t total_time_ns[BDRV_MAX_IOTYPE]; uint64_t wr_highest_sector; where BDRV_MAX_IOTYPE enumerates read, write, flush. wr_highest_sector is a high watermark updated by the block layer as it writes sectors. The other three are *not* touched by the block layer. Instead, the block layer provides a pair of functions for device models to update them: void bdrv_acct_start(BlockDriverState *bs, BlockAcctCookie *cookie, int64_t bytes, enum BlockAcctType type); void bdrv_acct_done(BlockDriverState *bs, BlockAcctCookie *cookie); bdrv_acct_start() initializes cookie for a read, write, or flush operation of a certain size. The size of a flush is always zero. bdrv_acct_done() adds the operations to the BDS's accounting data. total_time_ns is incremented by the time between _start() and _done(). You may call _start() without calling _done(). That's a feature. Device models use it to avoid accounting some requests. Device models are not supposed to mess with cookie directly, only through these two functions. Some device models implement accounting, some don't. The ones that do don't agree on how to count invalid guest requests (the ones not passed to block layer) and failed requests (passed to block layer and failed there). It's a mess in part caused by us never writing down what exactly device models are expected to do. Accounting data is used by "query-blockstats", and nothing else. Corollary: even though every BDS holds accounting data, only the ones in "top" BDSes ever get used. This is a common block layer blemish, and we're working on cleaning it up. If a device model doesn't implement accounting, query-blockstats lies. Fortunately, its lies are pretty transparent (everything's zero) as long as you don't do things like connecting a backend to a device model that doesn't implement accounting after disconnecting it from a device model that does. Still, I'd welcome a more honest QMP interface. For me, this accounting data belongs to the device model, not the BDS. Naturally, the block device models should use common infrastructure. I guess they use the block layer only because it's obvious infrastructure they share. Clumsy design. > /* volume of data transfered by the IOs */ > read_bytes > write_bytes This is nr_bytes[BDRV_ACCT_READ] and nr_bytes[BDRV_ACCT_WRITE]. nr_bytes[BDRV_ACCT_FLUSH] is always zero. Should this count only actual I/O, i.e. accumulated size of successful operations? > /* operation count */ > read_ios > write_ios > flush_ios > > /* how many invalid IOs the guest submit */ > invalid_read_ios > invalid_write_ios > invalid_flush_ios > > /* how many io error happened */ > read_ios_error > write_ios_error > flush_ios_error This is nr_ops[BDRV_ACCT_READ], nr_ops[BDRV_ACCT_WRITE], nr_ops[BDRV_ACCT_FLUSH] split up into successful, invalid and failed. > /* account the time passed doing IOs */ > total_read_time > total_write_time > total_flush_time This is total_time_ns[BDRV_ACCT_READ], total_time_ns[BDRV_ACCT_WRITE], total_time_ns[BDRV_ACCT_FLUSH]. I guess this should count both successful and failed I/O. Could throw in invalid, too, but it's probably too quick to matter. Please specify the unit clearly. Both total_FOO_time_ns or total_FOO_ns would work for me. > /* since when the volume is iddle */ > qvolume_iddleness_time "idle" The obvious way to maintain this information with the current could would be saving the value of get_clock() in bdrv_acct_done(). > /* the following would compute latecies for slices of 1 seconds then toss the > * result and start a new slice. A weighted sumation of the instant latencies > * could help to implement this. > */ > 1s_read_average_latency > 1s_write_average_latency > 1s_flush_average_latency > > /* the former three numbers could be used to further compute a 1 > minute slice value */ > 1m_read_average_latency > 1m_write_average_latency > 1m_flush_average_latency > > /* the former three numbers could be used to further compute a 1 hours > slice value */ > 1h_read_average_latency > 1h_write_average_latency > 1h_flush_average_latency This is something like "what we added to total_FOO_time in the last completed 1s / 1m / 1h time slice divided by the number of additions". Just another way to accumulate the same raw data, thus no worries. > /* 1 second average number of requests in flight */ > 1s_read_queue_depth > 1s_write_queue_depth > > /* 1 minute average number of requests in flight */ > 1m_read_queue_depth > 1m_write_queue_depth > > /* 1 hours average number of requests in flight */ > 1h_read_queue_depth > 1h_write_queue_depth I guess this involves counting bdrv_acct_start() and bdrv_acct_done(). The "you need not call bdrv_acct_done()" feature may get in the way. Solvable. Permit me a short detour into the other use for I/O accounting I mentioned: data on how exactly guest I/O gets translated to host I/O as it flows through the nodes in the block graph. Do you think this would be pretty much the same data, just collected at different points? > 4) Making this happen > ------------------------- > > Outscale want to make these IO stat happen and gave me the go to do whatever > grunt is required to do so. > That said we could collaborate on some part of the work. Cool! A quick stab at tasks: * QMP interface, either a compatible extension of query-blockstats or a new one. * Rough idea on how to do the shared infrastructure. * Implement (can be split up into several tasks if desired)

3 10

[libvirt] [PATCH-RFC] qemu: Add network bandwidth setting for ethernet interfaces
by Anirban Chakraborty 07 Sep '14

07 Sep '14

ethernet interfaces in libvirt currently do not support bandwidth setting. For example, following xml file for an interface will not apply these settings to corresponding qdiscs. ---- <interface type="ethernet"> <mac address="02:36:1d:18:2a:e4"/> <model type="virtio"/> <script path=""/> <target dev="tap361d182a-e4"/> <bandwidth> <inbound average="984" peak="1024" burst="64"/> <outbound average="2000" peak="2048" burst="128"/> </bandwidth> </interface> ----- This patch fixes the behavior. Please review it and if it appears ok, please apply. thanks, Anirban Chakraborty Signed-off-by: Anirban Chakraborty <abchak(a)juniper.net> --- src/qemu/qemu_command.c | 5 +++++ src/qemu/qemu_hotplug.c | 3 +++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 2184caa..258c6a7 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -7251,6 +7251,11 @@ qemuBuildInterfaceCommandLine(virCommandPtr cmd, if (tapfd[0] < 0) goto cleanup; } + /* Configure network bandwidth for ethernet type network interfaces */ + if (actualType == VIR_DOMAIN_NET_TYPE_ETHERNET) + if (virNetDevBandwidthSet(net->ifname, + virDomainNetGetActualBandwidth(net), false) < 0) + goto cleanup; if ((actualType == VIR_DOMAIN_NET_TYPE_NETWORK || actualType == VIR_DOMAIN_NET_TYPE_BRIDGE || diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index a364c52..aeb53c5 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -940,6 +940,9 @@ int qemuDomainAttachNetDevice(virConnectPtr conn, if (qemuOpenVhostNet(vm->def, net, priv->qemuCaps, vhostfd, &vhostfdSize) < 0) goto cleanup; } else if (actualType == VIR_DOMAIN_NET_TYPE_ETHERNET) { + if (virNetDevBandwidthSet(net->ifname, + virDomainNetGetActualBandwidth(net), false) < 0) + goto cleanup; vhostfdSize = 1; if (VIR_ALLOC(vhostfd) < 0) goto cleanup; -- 1.8.2.3

3 6

[libvirt] [PATCH v2] maint: update to latest gnulib
by Eric Blake 07 Sep '14

07 Sep '14

The usual portability fixes; and this includes a fix that adds a new syntax check for double semicolons (commit 28de556 fixed some, but gnulib found a better check). * .gnulib: Update to latest. * src/xenconfig/xen_common.c (xenFormatConfigCommon): Fix offender. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- v1 was here: https://www.redhat.com/archives/libvir-list/2014-September/msg00230.html Pushing under the gnulib maintenance rule; v1 was acked, and all that differs is that gnulib now does the syntax check. .gnulib | 2 +- src/xenconfig/xen_common.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.gnulib b/.gnulib index 2d28074..9565c3b 160000 --- a/.gnulib +++ b/.gnulib @@ -1 +1 @@ -Subproject commit 2d280742a9e30088aa169f53353765d5daafe4c0 +Subproject commit 9565c3be73eb6d76b7b42a21d68d2e00a62abb6d diff --git a/src/xenconfig/xen_common.c b/src/xenconfig/xen_common.c index accd25f..bab67de 100644 --- a/src/xenconfig/xen_common.c +++ b/src/xenconfig/xen_common.c @@ -2034,7 +2034,7 @@ xenFormatConfigCommon(virConfPtr conf, return -1; if (xenFormatCPUFeatures(conf, def, xendConfigVersion) < 0) - return -1;; + return -1; if (xenFormatCDROM(conf, def, xendConfigVersion) < 0) return -1; -- 1.9.3

1 0

Re: [libvirt] [PATCH] openvz: fixed an config file parsing error
by Hongbin Lu 07 Sep '14

07 Sep '14

Resend the email below. On Mon, Sep 1, 2014 at 8:28 PM, Hongbin Lu <hongbin034(a)gmail.com> wrote: > The OpenVZ driver reported an error on parsing some OpenVZ config > parameters (e.g. diskspace). This issue is due to the driver made > two incorrect assumptions about the value of the parameters: > 1. Assume paramaeter is just a number (e.g. 1024). > 2. Assume the number is an integer. > Actually, an OpenVZ config parameter may consists of a scalar and > a unit, and the scalar is not necessary an integer (e.g. 2.2G). > This patch is for fixing this issue. > --- > src/openvz/openvz_conf.c | 99 > ++++++++++++++++++++++++++++++++++++++++++++- > src/openvz/openvz_conf.h | 6 +++ > 2 files changed, 102 insertions(+), 3 deletions(-) > > diff --git a/src/openvz/openvz_conf.c b/src/openvz/openvz_conf.c > index 856c9f5..7dba925 100644 > --- a/src/openvz/openvz_conf.c > +++ b/src/openvz/openvz_conf.c > @@ -127,6 +127,42 @@ int openvzExtractVersion(struct openvz_driver *driver) > } > > > +static int > +openvzParseParamUnit(char* str, openvzParamUnit *unit) > +{ > + int len = strlen(str); > + if (len == 0) > + return -1; > + > + switch (str[len - 1]) { > + case 'G': > + case 'g': > + str[len - 1] = '\0'; > + if (unit != NULL) > + *unit = OPENVZ_PARAM_UNIT_GIGABYTE; > + break; > + case 'M': > + case 'm': > + str[len - 1] = '\0'; > + if (unit != NULL) > + *unit = OPENVZ_PARAM_UNIT_MEGABYTE; > + break; > + case 'K': > + case 'k': > + str[len - 1] = '\0'; > + if (unit != NULL) > + *unit = OPENVZ_PARAM_UNIT_KILOBYTE; > + break; > + case 'P': > + case 'p': > + str[len - 1] = '\0'; > + break; > + } > + > + return 0; > +} > + > + > /* Parse config values of the form barrier:limit into barrier and limit */ > static int > openvzParseBarrierLimit(const char* value, > @@ -146,7 +182,10 @@ openvzParseBarrierLimit(const char* value, > goto error; > } else { > if (barrier != NULL) { > - if (virStrToLong_ull(token, NULL, 10, barrier)) > + if (openvzParseParamUnit(token, NULL) < 0) > + goto error; > + > + if (virStrToLong_ull(token, NULL, 10, barrier) < 0) > goto error; > } > } > @@ -155,7 +194,10 @@ openvzParseBarrierLimit(const char* value, > goto error; > } else { > if (limit != NULL) { > - if (virStrToLong_ull(token, NULL, 10, limit)) > + if (openvzParseParamUnit(token, NULL) < 0) > + goto error; > + > + if (virStrToLong_ull(token, NULL, 10, limit) < 0) > goto error; > } > } > @@ -166,6 +208,57 @@ openvzParseBarrierLimit(const char* value, > } > > > +static int > +openvzParseBarrierLimit2(const char* value, > + unsigned long long *barrier, > + unsigned long long *limit, > + openvzParamUnit targetunit) > +{ > + char *token; > + char *saveptr = NULL; > + char *str; > + double dbarrier, dlimit; > + openvzParamUnit unit = OPENVZ_PARAM_UNIT_KILOBYTE; > + int ret = -1; > + > + if (VIR_STRDUP(str, value) < 0) > + goto error; > + > + token = strtok_r(str, ":", &saveptr); > + if (token == NULL) > + goto error; > + > + if (barrier != NULL) { > + if (openvzParseParamUnit(token, &unit) < 0) > + goto error; > + > + if (virStrToDouble(token, NULL, &dbarrier) < 0) > + goto error; > + > + *barrier = (unsigned long long)(dbarrier * unit / targetunit); > + } > + > + token = strtok_r(NULL, ":", &saveptr); > + if (token == NULL) > + goto error; > + > + if (limit != NULL) { > + if (openvzParseParamUnit(token, &unit) < 0) > + goto error; > + > + if (virStrToDouble(token, NULL, &dlimit) < 0) > + goto error; > + > + *limit = (unsigned long long)(dlimit * unit / targetunit); > + } > + > + ret = 0; > + error: > + VIR_FREE(str); > + return ret; > +} > + > + > virCapsPtr openvzCapsInit(void) > { > virCapsPtr caps; > @@ -396,7 +489,7 @@ openvzReadFSConf(virDomainDefPtr def, > param = "DISKSPACE"; > ret = openvzReadVPSConfigParam(veid, param, &temp); > if (ret > 0) { > - if (openvzParseBarrierLimit(temp, &barrier, &limit)) { > + if (openvzParseBarrierLimit2(temp, &barrier, &limit, > OPENVZ_PARAM_UNIT_KILOBYTE)) { > virReportError(VIR_ERR_INTERNAL_ERROR, > _("Could not read '%s' from config for > container %d"), > param, veid); > diff --git a/src/openvz/openvz_conf.h b/src/openvz/openvz_conf.h > index a7de7d2..788dae9 100644 > --- a/src/openvz/openvz_conf.h > +++ b/src/openvz/openvz_conf.h > @@ -41,6 +41,12 @@ > > # define VZCTL_BRIDGE_MIN_VERSION ((3 * 1000 * 1000) + (0 * 1000) + 22 + > 1) > > +typedef enum { > + OPENVZ_PARAM_UNIT_KILOBYTE = 1, > + OPENVZ_PARAM_UNIT_MEGABYTE = 1024, > + OPENVZ_PARAM_UNIT_GIGABYTE = 1024 * 1024, > +} openvzParamUnit; > + > struct openvz_driver { > virMutex lock; > > -- > 1.7.1 > >

2 3

[libvirt] [PATCH v2 0/2] parallels: use parallels SDK instead of prlctl tool
by Dmitry Guryanov 06 Sep '14

06 Sep '14

This patchset begins reworking of parallels driver. We have published Opensource version of parallels SDK (under LGPL license), so libvirt can link with it. Dmitry Guryanov (2): parallels: build with parallels SDK parallels: login to parallels SDK configure.ac | 11 +- po/POTFILES.in | 1 + src/Makefile.am | 8 +- src/parallels/parallels_driver.c | 40 ++++++- src/parallels/parallels_sdk.c | 234 +++++++++++++++++++++++++++++++++++++++ src/parallels/parallels_sdk.h | 30 +++++ src/parallels/parallels_utils.h | 3 + 7 files changed, 318 insertions(+), 9 deletions(-) create mode 100644 src/parallels/parallels_sdk.c create mode 100644 src/parallels/parallels_sdk.h -- 1.9.3

2 5