May 2013 - Devel - libvirt List Archives

[libvirt] [PATCH] docs: Fix closing tag in snapshot documentation
by Christophe Fergeau 01 May '13

01 May '13

Commit cc6d19f3 added text containing "<code>snapshot<code>" to formatsnapshot.html.in. The closing tag is missing '/' which causes the documentation to misrender. --- Pushed under the trivial rule. docs/formatsnapshot.html.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/formatsnapshot.html.in b/docs/formatsnapshot.html.in index b1e259f..2e830f8 100644 --- a/docs/formatsnapshot.html.in +++ b/docs/formatsnapshot.html.in @@ -147,7 +147,7 @@ devices</a> specified for the domain at the time of the snapshot. The attribute <code>snapshot</code> is optional, and the possible values are the same as the - <code>snapshot<code> attribute for + <code>snapshot</code> attribute for <a href="formatdomain.html#elementsDisks">disk devices</a> (<code>no</code>, <code>internal</code>, or <code>external</code>). Some hypervisors like ESX -- 1.8.1.4

1 0

[libvirt] Entering freeze for libvirt-1.0.5
by Daniel Veillard 01 May '13

01 May '13

So as discussed earlier this week, I have tagged the release candidate 1 of libvirt-1.0.5 in git and pushed the tarball and associated rpms to the FTP at: ftp://libvirt.org/libvirt/ We may still push some of the patches needed by Laine to finish the serie started yesterday, but at this point we should focuse our commits on bug fixes and documentation. I will make an rc2 at the beginning of next week and hopefully can push the final version on May 2nd. This passed my own small usage tests, but please give it a try. As usual some feedback on portability issues are really welcome too :-) thanks ! Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

9 16

[libvirt] Should we keep the original vnc behavior even if the underlying qemu changes?
by Guan Nan Ren 01 May '13

01 May '13

Hi Everybody, Since qemu v1.1.0 upstream commit 8cf364, qemu implemented shared-flag handling the default qemu vnc server behavior from mode 'ignore' to 'allow-exclusive'. The direct effect is as follows for libvirt user: vncviwer client: before: all client could share desktop without any option after: all client need -Shared options for desktop sharing. One client without -Shared option breaks others's connection virt-viewer: before: virt-viewer could support multiple desktop sharing after: only one active connection If we need to keep the original vnc behavior at libvirt level, the '-vnc :1,share=ignore' need to add into qemu commandline. Any ideas about this? Reference RFB Protocol about shared-flag Shared-flag is non-zero (true) if the server should try to share the desktop by leaving other clients connected, zero (false) if it should give exclusive access to this client by disconnecting all other clients From qemu doc: Set display sharing policy. 'allow-exclusive' allows clients to ask for exclusive access. As suggested by the rfb spec this is implemented by dropping other connections. Connecting multiple clients in parallel requires all clients asking for a shared session (vncviewer: -shared switch). This is the default. 'force-shared' disables exclusive client access. Useful for shared desktop sessions, where you don't want someone forgetting specify -shared disconnect everybody else. 'ignore' completely ignores the shared flag and allows everybody connect unconditionally. Doesn't conform to the rfb spec but is traditional qemu behavior. Guannan

2 1

[libvirt] Bug Reports
by Dennis Zou (yunzou) 01 May '13

01 May '13

1.Lost soapAction in http request header When I tried to retrieve the property of storage.perDatastoreUsage under specify virtual machine and got a invalidProperty error but other properties(e.g. summary.storage, guest) under the same virtual machine are ok. Then I write a same program use vsphere web service sdk in java and catch the http package by wireshark. By comparing, I found that we lost soapAction field in http request header, it seems to like this: SOAPAction: "urn:vim25/5.0". I checked the vsphere development document and found some key description as following: When a client application connects to a Web service running on an vSphere server (ESX/ESXi or vCenter Server system), the server detects the version of the API that was used to develop the client and makes available only those operations supported by the client. Client applications convey information about the API version used in the SOAP messages that they send to a vSphere server. These SOAP messages include a versionID in the soapAction attribute. The details are handled transparently by the SOAP toolkit and the client proxy code. The server adjusts its behavior based on the client's version information, exposing the API version that the client supports to the client. If you are developing a client application that must support multiple server versions at the same time (ESX/ESXi 5.0 and ESX/ESXi 3.x, for example), you must obtain information about the API versions that are supported on the server and provide logic in your code to use or not use features, based upon the version information. Finally, I added the soapAction field into http request header and got a correct result. So I added some lines in function esxVI_Context_Connect() at src/esx/esx_vi.c as following: if (virAsprintf(&soapAction, "SOAPAction: \"urn:vim25/%s\"", ctx->service->about->apiVersion) < 0) { virReportOOMError(); goto cleanup; } ctx->curl->headers = curl_slist_append(ctx->curl->headers, soapAction); 2.cat not deserialize the String(maybe other primary types are included) which as a list when I tried to retrieve the property of config.network.pnic under hostsystem and got an error in virsh.log which goes like as following: 2013-04-22 01:57:42.692+0000: 29640: error : esxVI_String_Deserialize:1233 : internal error Wrong XML element type 3 Then I debug my program and find that bug lays where deserialize resourcePoolSchedulerDisallowedReason which is a member of PhysicalNic and has a string[] type. I catch the soap message by wireshark and find that this xml element has no child ,it seems like as following: <resourcePoolSchedulerDisallowedReason>hardwareUnsupported</resourcePoolSchedulerDisallowedReason But the libvirt program still parses it as if it has children, I think this is key point that leads to this bug. Now I fix my program by setting this member as ignore in esx_vi_generator.input. Maybe you want to investigate and fix this bug. Regards, Dennis

2 1

[libvirt] libvirt-snmp on Ubuntu
by george john 01 May '13

01 May '13

Hi, Are there packages of "libvirt-snmp" for Ubuntu? Or instructions to compile libvirt-snmp on Ubuntu? Looks like the instructions given at http://wiki.libvirt.org/page/Libvirt-snmp are for RedHat. Thank you, George

1 0

[libvirt] [sandbox PATCH] Add a option --rpm if rpm autodetection fail
by Michael Scherer 01 May '13

01 May '13

https://bugzilla.redhat.com/show_bug.cgi?id=954187 If someone use a custom unit file for the sandbox, the rpm autodetection fail with a exception. Now, this will show a error message, asking to use --rpm and use this rpm if autodetection fail. --- bin/virt-sandbox-service | 46 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/bin/virt-sandbox-service b/bin/virt-sandbox-service index 3b78512..a076397 100755 --- a/bin/virt-sandbox-service +++ b/bin/virt-sandbox-service @@ -363,10 +363,11 @@ class SystemdContainer(Container): DEFAULT_UNIT = "/etc/systemd/system/%s_sandbox.service" - def __init__(self, name=None, uri = "lxc:///", path = Container.DEFAULT_PATH, config=None, create=False): + def __init__(self, name=None, uri = "lxc:///", path = Container.DEFAULT_PATH, config=None, create=False, rpm=None): Container.__init__(self, name, uri, path, config, create) self.copy = False self.unit_file_list = [] + self.rpm = rpm if create: self.config = LibvirtSandbox.ConfigServiceSystemd.new(name) else: @@ -496,8 +497,18 @@ WantedBy=%(TARGET)s self.ts = rpm.ts() + nb_rpm = 0 for u, src in self.unit_file_list: - self.extract_rpm_for_unit(src) + rpm_name = self.get_rpm_for_unit(src) + if rpm_name: + self.extract_rpm(rpm_name) + nb_rpm += 1 + + if nb_rpm == 0: + if self.rpm: + self.extract_rpm(self.rpm) + else: + raise ValueError([_("Cannot autodetect the rpm for unit files, please use --rpm")]) def split_filename(self, filename): if filename[-4:] == '.rpm': @@ -521,12 +532,21 @@ WantedBy=%(TARGET)s name = filename[epochIndex + 1:verIndex] return name, ver, rel, epoch, arch - def extract_rpm_for_unit(self, unitfile): + def get_rpm_for_unit(self, unitfile): mi = self.ts.dbMatch(rpm.RPMTAG_BASENAMES, unitfile) try: h = mi.next(); except exceptions.StopIteration: - raise ValueError([_("Cannot find package containing %s") % unitfile]) + return None + return h['name'] + + + def extract_rpm(self, rpm_name): + mi = self.ts.dbMatch('name', rpm_name) + try: + h = mi.next(); + except exceptions.StopIteration: + raise ValueError([_("Cannot find package named %s") % rpm_name]) for fentry in h.fiFromHeader(): fname = fentry[0] @@ -765,11 +785,14 @@ def create(args): if len(args.command) == 0 and len(args.unitfiles) == 0: raise ValueError([_("You must specify a command or a unit file")]) + if args.rpm and len(args.unitfiles) != 1: + raise ValueError([_("Option --rpm cannot be used without a unit file")]) + if len(args.command) > 0: container = GenericContainer(name = args.name, uri=args.uri, create = True) container.set_command(args.command) else: - container = SystemdContainer(name = args.name, uri=args.uri, create = True) + container = SystemdContainer(name = args.name, uri=args.uri, create = True, rpm = args.rpm) container.set_copy(args.copy) container.set_unit_file_list(args.unitfiles) for net in args.network: @@ -972,6 +995,15 @@ class SetNet(argparse.Action): nets = [values] setattr(namespace, self.dest, nets) +class CheckRpm(argparse.Action): + def __call__(self, parser, namespace, value, option_string=None): + nb_rpm = len(rpm.TransactionSet().dbMatch('name', value)) + if nb_rpm == 0: + raise OSError(_("Cannot find %s rpm") % value) + elif nb_rpm > 1: + raise OSError(_("%s rpm is installed more than once") % value) + setattr(namespace, self.dest, value) + def requires_name(parser): parser.add_argument("name", help=_("name of the sandbox container")) @@ -1015,6 +1047,10 @@ def gen_create_args(subparser): action=CheckUnit, dest="unitfiles", default=[], help=_("Systemd Unit file to run within the systemd sandbox container. Commands cannot be specified with unit files.")) + parser.add_argument("-r", "--rpm", + action=CheckRpm, + dest="rpm", default=None, + help=_("Rpm configuration to use in the container. Default: autodetected from unit files, unless if using a custom unit file.")) parser.add_argument("--username", dest="username", help=_("Specify the username for the container. Default: UID username.")) parser.add_argument("-U", "--uid", dest="uid", -- 1.8.2.1

2 2

[libvirt] [PATCHv2] rpc: message related sizes enlarged
by Viktor Mihajlovski 01 May '13

01 May '13

From: Daniel Hansel <daniel.hansel(a)linux.vnet.ibm.com> We have seen an issue on s390x platform where domain XMLs larger than 1MB were used. The define command was finished successfully. The dumpxml command was not successful (i.e. could not encode message payload). Enlarged message related sizes (e.g. maximum string size, message size, etc.) to handle larger system configurations used on s390x platform. To improve handling of the RPC message size the allocation during encode process is changed to a dynamic one (i.e. starting with 64kB initial size and increasing that size in steps up to 16MB if the payload data is larger). Signed-off-by: Daniel Hansel <daniel.hansel(a)linux.vnet.ibm.com> Signed-off-by: Viktor Mihajlovski <mihajlov(a)linux.vnet.ibm.com> --- src/libvirt.c | 4 ++++ src/remote/remote_protocol.x | 6 +++--- src/rpc/virnetmessage.c | 46 ++++++++++++++++++++++++++++++++++-------- src/rpc/virnetmessage.h | 3 ++- src/rpc/virnetprotocol.x | 16 +++++++++++---- tests/virnetmessagetest.c | 2 +- 6 files changed, 60 insertions(+), 17 deletions(-) diff --git a/src/libvirt.c b/src/libvirt.c index 2daa0b4..e6dd315 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -7784,6 +7784,8 @@ error: * For your program to be able to work reliably over a remote * connection you should split large requests to <= 65536 bytes. * However, with 0.9.13 this RPC limit has been raised to 1M byte. + * Starting with version 1.0.5 the RPC limit has been raised again. + * Now large requests up to 16M byte are supported. * * Returns: 0 in case of success or -1 in case of failure. */ @@ -7934,6 +7936,8 @@ error: * For your program to be able to work reliably over a remote * connection you should split large requests to <= 65536 bytes. * However, with 0.9.13 this RPC limit has been raised to 1M byte. + * Starting with version 1.0.5 the RPC limit has been raised again. + * Now large requests up to 16M byte are supported. * * Returns: 0 in case of success or -1 in case of failure. */ diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 512ba2e..f61d10c 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -65,7 +65,7 @@ * This is an arbitrary limit designed to stop the decoder from trying * to allocate unbounded amounts of memory when fed with a bad message. */ -const REMOTE_STRING_MAX = 1048576; +const REMOTE_STRING_MAX = 4194304; /* A long string, which may NOT be NULL. */ typedef string remote_nonnull_string<REMOTE_STRING_MAX>; @@ -160,13 +160,13 @@ const REMOTE_DOMAIN_SNAPSHOT_LIST_NAMES_MAX = 1024; * Note applications need to be aware of this limit and issue multiple * requests for large amounts of data. */ -const REMOTE_DOMAIN_BLOCK_PEEK_BUFFER_MAX = 1048576; +const REMOTE_DOMAIN_BLOCK_PEEK_BUFFER_MAX = 4194304; /* Maximum length of a memory peek buffer message. * Note applications need to be aware of this limit and issue multiple * requests for large amounts of data. */ -const REMOTE_DOMAIN_MEMORY_PEEK_BUFFER_MAX = 1048576; +const REMOTE_DOMAIN_MEMORY_PEEK_BUFFER_MAX = 4194304; /* * Maximum length of a security label list. diff --git a/src/rpc/virnetmessage.c b/src/rpc/virnetmessage.c index 647fef7..b2c6e5b 100644 --- a/src/rpc/virnetmessage.c +++ b/src/rpc/virnetmessage.c @@ -221,7 +221,7 @@ int virNetMessageEncodeHeader(virNetMessagePtr msg) int ret = -1; unsigned int len = 0; - msg->bufferLength = VIR_NET_MESSAGE_MAX + VIR_NET_MESSAGE_LEN_MAX; + msg->bufferLength = VIR_NET_MESSAGE_INITIAL + VIR_NET_MESSAGE_LEN_MAX; if (VIR_REALLOC_N(msg->buffer, msg->bufferLength) < 0) { virReportOOMError(); return ret; @@ -351,9 +351,27 @@ int virNetMessageEncodePayload(virNetMessagePtr msg, xdrmem_create(&xdr, msg->buffer + msg->bufferOffset, msg->bufferLength - msg->bufferOffset, XDR_ENCODE); - if (!(*filter)(&xdr, data)) { - virReportError(VIR_ERR_RPC, "%s", _("Unable to encode message payload")); - goto error; + /* Try to encode the payload. If the buffer is too small increase it. */ + while (!(*filter)(&xdr, data)) { + if ((msg->bufferLength - VIR_NET_MESSAGE_LEN_MAX) * 4 > VIR_NET_MESSAGE_MAX) { + virReportError(VIR_ERR_RPC, "%s", _("Unable to encode message payload")); + goto error; + } + + xdr_destroy(&xdr); + + msg->bufferLength = (msg->bufferLength - VIR_NET_MESSAGE_LEN_MAX) * 4 + + VIR_NET_MESSAGE_LEN_MAX; + + if (VIR_REALLOC_N(msg->buffer, msg->bufferLength) < 0) { + virReportOOMError(); + goto error; + } + + xdrmem_create(&xdr, msg->buffer + msg->bufferOffset, + msg->bufferLength - msg->bufferOffset, XDR_ENCODE); + + VIR_DEBUG("Increased message buffer length = %zu", msg->bufferLength); } /* Get the length stored in buffer. */ @@ -415,11 +433,23 @@ int virNetMessageEncodePayloadRaw(virNetMessagePtr msg, XDR xdr; unsigned int msglen; + /* If the message buffer is too small for the payload increase it accordingly. */ if ((msg->bufferLength - msg->bufferOffset) < len) { - virReportError(VIR_ERR_RPC, - _("Stream data too long to send (%zu bytes needed, %zu bytes available)"), - len, (msg->bufferLength - msg->bufferOffset)); - return -1; + if ((msg->bufferOffset + len) > VIR_NET_MESSAGE_MAX) { + virReportError(VIR_ERR_RPC, + _("Stream data too long to send (%zu bytes needed, %zu bytes available)"), + len, (VIR_NET_MESSAGE_MAX - msg->bufferOffset)); + return -1; + } + + msg->bufferLength = msg->bufferOffset + len; + + if (VIR_REALLOC_N(msg->buffer, msg->bufferLength) < 0) { + virReportOOMError(); + return -1; + } + + VIR_DEBUG("Increased message buffer length = %zu", msg->bufferLength); } memcpy(msg->buffer + msg->bufferOffset, data, len); diff --git a/src/rpc/virnetmessage.h b/src/rpc/virnetmessage.h index dfa1c6c..c94dddc 100644 --- a/src/rpc/virnetmessage.h +++ b/src/rpc/virnetmessage.h @@ -34,7 +34,8 @@ typedef void (*virNetMessageFreeCallback)(virNetMessagePtr msg, void *opaque); struct _virNetMessage { bool tracked; - char *buffer; /* Typically VIR_NET_MESSAGE_MAX + VIR_NET_MESSAGE_LEN_MAX */ + char *buffer; /* Initially VIR_NET_MESSAGE_INITIAL + VIR_NET_MESSAGE_LEN_MAX */ + /* Maximum VIR_NET_MESSAGE_MAX + VIR_NET_MESSAGE_LEN_MAX */ size_t bufferLength; size_t bufferOffset; diff --git a/src/rpc/virnetprotocol.x b/src/rpc/virnetprotocol.x index eb2e81d..131e40b 100644 --- a/src/rpc/virnetprotocol.x +++ b/src/rpc/virnetprotocol.x @@ -44,23 +44,31 @@ /*----- Data types. -----*/ +/* Initial message size. + * When the message payload is larger this initial size will be + * quadrupled until the maximum total message size is reached. + */ +const VIR_NET_MESSAGE_INITIAL = 65536; + /* Maximum total message size (serialised). */ -const VIR_NET_MESSAGE_MAX = 4194304; +const VIR_NET_MESSAGE_MAX = 16777216; /* Size of struct virNetMessageHeader (serialised)*/ const VIR_NET_MESSAGE_HEADER_MAX = 24; /* Size of message payload */ -const VIR_NET_MESSAGE_PAYLOAD_MAX = 4194280; +const VIR_NET_MESSAGE_PAYLOAD_MAX = 16777192; -/* Size of message length field. Not counted in VIR_NET_MESSAGE_MAX */ +/* Size of message length field. Not counted in VIR_NET_MESSAGE_MAX + * and VIR_NET_MESSAGE_INITIAL. + */ const VIR_NET_MESSAGE_LEN_MAX = 4; /* Length of long, but not unbounded, strings. * This is an arbitrary limit designed to stop the decoder from trying * to allocate unbounded amounts of memory when fed with a bad message. */ -const VIR_NET_MESSAGE_STRING_MAX = 1048576; +const VIR_NET_MESSAGE_STRING_MAX = 4194304; /* Limit on number of File Descriptors allowed to be * passed per message diff --git a/tests/virnetmessagetest.c b/tests/virnetmessagetest.c index e58fd1f..607e0bd 100644 --- a/tests/virnetmessagetest.c +++ b/tests/virnetmessagetest.c @@ -47,7 +47,7 @@ static int testMessageHeaderEncode(const void *args ATTRIBUTE_UNUSED) }; /* According to doc to virNetMessageEncodeHeader(&msg): * msg->buffer will be this long */ - unsigned long msg_buf_size = VIR_NET_MESSAGE_MAX + VIR_NET_MESSAGE_LEN_MAX; + unsigned long msg_buf_size = VIR_NET_MESSAGE_INITIAL + VIR_NET_MESSAGE_LEN_MAX; int ret = -1; if (!msg) { -- 1.7.9.5

3 2

[libvirt] [PATCH v2 0/5] Support NBD volumes with LXC containers
by Daniel P. Berrange 01 May '13

01 May '13

A second version of: https://www.redhat.com/archives/libvir-list/2013-March/msg00941.html Aside from the addressing the previous round of feedbac, this adds use of the new '--format' flag to qemu-nbd. This is introduced due to CVE-2013-1922. The upshot is that this code won't work with any qemu-nbd binary lacking the --format flag. This is intentionale.

2 12

[libvirt] [PATCH] qemu: fix failure to start with spice graphics and no tls
by Laine Stump 01 May '13

01 May '13

Commit eca3fdf inadvertantly caused a failure to start for any domain with the following in its config: <graphics type='spice' autoport='yes'/> The problem is that when tlsPort == 0 and defaultMode == "any" (which is the default for defaultMode), this would be flagged in the code as "needTLSPort", and if there was then no spice tls config, the new error+fail would happen. This patch checks for the case of defaultMode == "any", and in that case simply doesn't allocate a TLS port (since that's probably not what the user wanted, and it would have failed later anyway.). It does leave the error in place for cases when the user specifically asked to use tls in one way or another, though. --- src/qemu/qemu_process.c | 41 ++++++++++++++++++++++++++--------------- 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index e75c8c9..58f64b7 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -3294,23 +3294,34 @@ qemuProcessSPICEAllocatePorts(virQEMUDriverPtr driver, if (needTLSPort || graphics->data.spice.tlsPort == -1) { if (!cfg->spiceTLS) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("Auto allocation of spice TLS port requested " - "but spice TLS is disabled in qemu.conf")); - goto error; - } - - if (virPortAllocatorAcquire(driver->remotePorts, &tlsPort) < 0) - goto error; + /* log an error and fail if tls was specifically + * requested, or simply ignore (don't allocate a port) + * if we're here due to "defaultMode='any'" + * (aka unspecified). + */ + if ((graphics->data.spice.tlsPort == -1) || + (graphics->data.spice.defaultMode + == VIR_DOMAIN_GRAPHICS_SPICE_CHANNEL_MODE_SECURE)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Auto allocation of spice TLS port requested " + "but spice TLS is disabled in qemu.conf")); + goto error; + } + } else { + /* cfg->spiceTLS *is* in place, so it makes sense to + * allocate a port. + */ + if (virPortAllocatorAcquire(driver->remotePorts, &tlsPort) < 0) + goto error; - if (tlsPort == 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Unable to find an unused port for SPICE TLS")); - virPortAllocatorRelease(driver->remotePorts, port); - goto error; + if (tlsPort == 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to find an unused port for SPICE TLS")); + virPortAllocatorRelease(driver->remotePorts, port); + goto error; + } + graphics->data.spice.tlsPort = tlsPort; } - - graphics->data.spice.tlsPort = tlsPort; } return 0; -- 1.7.11.7

2 2