- Devel - libvirt List Archives

[libvirt] [PATCH v2 0/7] Add support for setting QoS
by Michal Privoznik 21 Jul '11

21 Jul '11

This patch series add support for setting traffic shaping and policing on both domain's interface and network's virtual bridge. Basically, this is done via 'tc' from iproute2 package. For shaping is HTB used, for policing we need u32 match selector. Both should be available in RHEL-6 kernel. How this works: On an virtual interface which has limits defined a root qdisc are replaced. Ingress root for outbound traffic shaping and egress for inbound. Basically, in inbound traffic policing is applied, on outbound shaping. New qdiscs are set to limit the traffic to rate set in XML. For shaping it is possible to set the size of buffer. Accepted values for rate, peak and burst have same format as 'tc' command and are documented. Supported devices are VIR_DOMAIN_NET_TYPE_NETWORK, VIR_DOMAIN_NET_TYPE_BRIDGE and VIR_DOMAIN_NET_TYPE_DIRECT. diff to v1: -rebase to current HEAD -add support for macvtap devices Michal Privoznik (7): bandwidth: Define schema and create documentation bandwidth: Declare internal structures bandwidth: Add format parsing functions bandwidth: Create format functions bandwitdh: Implement functions to enable and disable QoS bandwidth: Add test cases for network bandwidth: Add domain schema test suite configure.ac | 4 + docs/formatdomain.html.in | 32 ++ docs/formatnetwork.html.in | 30 ++ docs/schemas/domain.rng | 50 +++ docs/schemas/network.rng | 50 +++ src/conf/domain_conf.c | 6 + src/conf/domain_conf.h | 1 + src/conf/network_conf.c | 8 + src/conf/network_conf.h | 1 + src/libvirt_private.syms | 6 + src/network/bridge_driver.c | 12 + src/qemu/qemu_command.c | 10 +- src/util/macvtap.c | 12 +- src/util/macvtap.h | 5 +- src/util/network.c | 517 +++++++++++++++++++++++++ src/util/network.h | 28 ++ tests/domainschemadata/domain-bandwidth.xml | 72 ++++ tests/networkxml2xmlin/bandwidth-network.xml | 16 + tests/networkxml2xmlout/bandwidth-network.xml | 16 + tests/networkxml2xmltest.c | 1 + 20 files changed, 873 insertions(+), 4 deletions(-) create mode 100644 tests/domainschemadata/domain-bandwidth.xml create mode 100644 tests/networkxml2xmlin/bandwidth-network.xml create mode 100644 tests/networkxml2xmlout/bandwidth-network.xml -- 1.7.5.rc3

6 19

[libvirt] [PATCH 0/7] Add support for setting QoS
by Michal Privoznik 21 Jul '11

21 Jul '11

This patch series add support for setting traffic shaping and policing on both domain's interface and network's virtual bridge. Basically, this is done via 'tc' from iproute2 package. For shaping is HTB used, for policing we need u32 match selector. Both should be available in RHEL-6 kernel. How this works: On an virtual interface which has limits defined a root qdisc are replaced. Ingress root for outbound traffic shaping and egress for inbound. Basically, in inbound traffic policing is applied, on outbound shaping. New qdiscs are set to limit the traffic to rate set in XML. For shaping it is possible to set the size of buffer. Accepted values for rate, peak and burst are integer numbers. Units are kilobytes per second for rate or kilobytes for size. Supported devices are VIR_DOMAIN_NET_TYPE_NETWORK, VIR_DOMAIN_NET_TYPE_BRIDGE and VIR_DOMAIN_NET_TYPE_DIRECT. diff to v2: -Jirka's review included diff to v1: -rebase to current HEAD -add support for macvtap devices Michal Privoznik (7): bandwidth: Define schema and create documentation bandwidth: Declare internal structures bandwidth: Add parsing functions bandwidth: Create format functions bandwitdh: Implement functions to enable and disable QoS bandwidth: Add test cases for network bandwidth: Add domain schema test suite configure.ac | 4 + docs/formatdomain.html.in | 33 +++ docs/formatnetwork.html.in | 30 +++ docs/schemas/bandwidth.rng | 52 ++++ docs/schemas/domain.rng | 4 + docs/schemas/network.rng | 4 + src/conf/domain_conf.c | 6 + src/conf/domain_conf.h | 1 + src/conf/network_conf.c | 8 + src/conf/network_conf.h | 1 + src/libvirt_private.syms | 4 + src/network/bridge_driver.c | 12 + src/qemu/qemu_command.c | 10 +- src/util/macvtap.c | 12 +- src/util/macvtap.h | 5 +- src/util/network.c | 345 +++++++++++++++++++++++++ src/util/network.h | 23 ++ tests/domainschemadata/domain-bandwidth.xml | 72 +++++ tests/networkxml2xmlin/bandwidth-network.xml | 16 ++ tests/networkxml2xmlout/bandwidth-network.xml | 16 ++ tests/networkxml2xmltest.c | 1 + 21 files changed, 655 insertions(+), 4 deletions(-) create mode 100644 docs/schemas/bandwidth.rng create mode 100644 tests/domainschemadata/domain-bandwidth.xml create mode 100644 tests/networkxml2xmlin/bandwidth-network.xml create mode 100644 tests/networkxml2xmlout/bandwidth-network.xml -- 1.7.5.rc3

5 24

[libvirt] Schedule for release of 0.9.4
by Daniel Veillard 21 Jul '11

21 Jul '11

I should have posted this beginning of this week... Basically I would like to freeze for 0.9.4 next Monday, which means we need to finish ACK'ing and pushing the number of patch sets which are well advanced already. Then try to get a final 0.9.4 release on Aug 1 or 2. So let's make sure we have updated patches on the list and let's review the patches and apply them ! This includes: - virtual switch support - support for cpu bandwith (Wen can you provide a patchset including latest feedback ?) - New virDomainBlockPull API (Adam can you provide a new patchset too ?) - and various other patch(set)s floating around And if you sent a patch which got no feedback, please raise it again, thanks everybody ! Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

2 2

[libvirt] [PATCH] maint: fix typos on guaranteed
by Eric Blake 20 Jul '11

20 Jul '11

* src/conf/domain_event.c (virDomainEventDispatch): Fix typo. * src/internal.h (ATTRIBUTE_FMT_PRINTF): Likewise. * src/libvirt.c (virStreamEventUpdateCallback): Likewise. * src/remote/remote_driver.c (doRemoteOpen): Likewise. * src/storage/storage_backend_fs.c (virStorageBackendProbeTarget): Likewise. * src/util/virterror.c (virConnCopyLastError, virCopyLastError): Likewise. * src/xen/xend_internal.h (xend_wait_for_devices): Likewise. --- Pushing under the trivial rule. src/conf/domain_event.c | 2 +- src/internal.h | 2 +- src/libvirt.c | 2 +- src/remote/remote_driver.c | 2 +- src/storage/storage_backend_fs.c | 2 +- src/util/virterror.c | 4 ++-- src/xen/xend_internal.h | 2 +- 7 files changed, 8 insertions(+), 8 deletions(-) diff --git a/src/conf/domain_event.c b/src/conf/domain_event.c index c435484..d43331f 100644 --- a/src/conf/domain_event.c +++ b/src/conf/domain_event.c @@ -1069,7 +1069,7 @@ void virDomainEventDispatch(virDomainEventPtr event, { int i; /* Cache this now, since we may be dropping the lock, - and have more callbacks added. We're guarenteed not + and have more callbacks added. We're guaranteed not to have any removed */ int cbCount = callbacks->count; diff --git a/src/internal.h b/src/internal.h index 5747a90..1997031 100644 --- a/src/internal.h +++ b/src/internal.h @@ -140,7 +140,7 @@ * Macro used to check printf like functions, if compiling * with gcc. * - * We use gnulib which guarentees we always have GNU style + * We use gnulib which guarantees we always have GNU style * printf format specifiers even on broken Win32 platforms * hence we have to force 'gnu_printf' for new GCC */ diff --git a/src/libvirt.c b/src/libvirt.c index 2f5241a..16a9a02 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -13390,7 +13390,7 @@ error: * Changes the set of events to monitor for a stream. This allows * for event notification to be changed without having to * unregister & register the callback completely. This method - * is guarenteed to succeed if a callback is already registered + * is guaranteed to succeed if a callback is already registered * * Returns 0 on success, -1 if no callback is registered */ diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 6ba58ed..82b938c 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -584,7 +584,7 @@ doRemoteOpen (virConnectPtr conn, if (!(priv->client = virNetClientNewExternal(cmd_argv))) goto failed; - /* Do not set 'is_secure' flag since we can't guarentee + /* Do not set 'is_secure' flag since we can't guarantee * an external program is secure, and this flag must be * pessimistic */ } break; diff --git a/src/storage/storage_backend_fs.c b/src/storage/storage_backend_fs.c index 8d6f76d..1656215 100644 --- a/src/storage/storage_backend_fs.c +++ b/src/storage/storage_backend_fs.c @@ -143,7 +143,7 @@ virStorageBackendProbeTarget(virStorageVolTargetPtr target, } /* XXX ideally we'd fill in secret UUID here - * but we cannot guarentee 'conn' is non-NULL + * but we cannot guarantee 'conn' is non-NULL * at this point in time :-( So we only fill * in secrets when someone first queries a vol */ diff --git a/src/util/virterror.c b/src/util/virterror.c index 75058f3..0c7698a 100644 --- a/src/util/virterror.c +++ b/src/util/virterror.c @@ -327,7 +327,7 @@ int virCopyLastError(virErrorPtr to) { virErrorPtr err = virLastErrorObject(); - /* We can't guarentee caller has initialized it to zero */ + /* We can't guarantee caller has initialized it to zero */ memset(to, 0, sizeof(*to)); if (err) virCopyError(err, to); @@ -464,7 +464,7 @@ virConnGetLastError(virConnectPtr conn) int virConnCopyLastError(virConnectPtr conn, virErrorPtr to) { - /* We can't guarentee caller has initialized it to zero */ + /* We can't guarantee caller has initialized it to zero */ memset(to, 0, sizeof(*to)); if (conn == NULL) diff --git a/src/xen/xend_internal.h b/src/xen/xend_internal.h index c4ed9ba..4a0cd69 100644 --- a/src/xen/xend_internal.h +++ b/src/xen/xend_internal.h @@ -36,7 +36,7 @@ xenDaemonOpen_unix(virConnectPtr conn, const char *path); * \return 0 for success; -1 (with errno) on error * * xen_create() returns after a domain has been allocated including - * its memory. This does not guarentee, though, that the devices + * its memory. This does not guarantee, though, that the devices * have come up properly. For instance, if you create a VBD with an * invalid filename, the error won't occur until after this function * returns. -- 1.7.4.4

1 0

[libvirt] [PATCH] rpc: Pass through DISPLAY so ssh can launch askpass
by Cole Robinson 20 Jul '11

20 Jul '11

Though we prefer users to have SSH keys setup, virt-manager users still depend on remote SSH connections to launch a password dialog. This fixes launch ssh-askpass Fix suggested by danpb --- src/rpc/virnetsocket.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c index 4403fc4..6605aa3 100644 --- a/src/rpc/virnetsocket.c +++ b/src/rpc/virnetsocket.c @@ -604,6 +604,7 @@ int virNetSocketNewConnectSSH(const char *nodename, virCommandAddEnvPassCommon(cmd); virCommandAddEnvPass(cmd, "SSH_AUTH_SOCK"); virCommandAddEnvPass(cmd, "SSH_ASKPASS"); + virCommandAddEnvPass(cmd, "DISPLAY"); virCommandClearCaps(cmd); if (service) -- 1.7.4.4

3 3

[libvirt] remote driver RPC API usage breaks launching ssh askpass
by Cole Robinson 20 Jul '11

20 Jul '11

This commit broke launching SSH askpass from virt-manager: http://libvirt.org/git/?p=libvirt.git;a=commit;h=c1b226447781ba8367606f760d… Using upstream virt-manager, running python src/virt-manager.py and trying to connect to a remote SSH connection without SSH keys causes a segfault: #0 virNetSocketRead (sock=0xc, buf=0x7f746c006458 "", len=4) at rpc/virnetsocket.c:969 #1 0x00007f7477ea7374 in virNetClientIOReadMessage (client=0x7f746c006400) at rpc/virnetclient.c:712 #2 virNetClientIOHandleInput (client=0x7f746c006400) at rpc/virnetclient.c:731 #3 0x00007f7477ea7ae0 in virNetClientIncomingEvent (sock=0x7f746c002040, events=<value optimized out>, opaque=0x7f746c006400) at rpc/virnetclient.c:1120 #4 0x00007f74782332ce in libvirt_virEventInvokeHandleCallback ( self=<value optimized out>, args=<value optimized out>) at libvirt-override.c:2997 #5 0x0000003075ee965b in call_function (f=<value optimized out>, throwflag=<value optimized out>) at /usr/src/debug/Python-2.7/Python/ceval.c:4071 #6 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at /usr/src/debug/Python-2.7/Python/ceval.c:2721 ... If you apply the following patch to virt-manager: $ cat a diff --git a/src/virt-manager.py.in b/src/virt-manager.py.in index cf0a20e..4d3f1c6 100755 --- a/src/virt-manager.py.in +++ b/src/virt-manager.py.in @@ -357,8 +357,8 @@ def main(): "disabling DBus service") # Hook libvirt events into glib main loop - import virtManager.libvirtglib - virtManager.libvirtglib.register_event_impl() + #import virtManager.libvirtglib + #virtManager.libvirtglib.register_event_impl() # At this point we're either starting a brand new controlling instance, # or the dbus comms to existing instance has failed rerun make, and reproduce the error, libvirt raises an exception from virConnectOpenAuth: libvirtError: Cannot recv data: Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). : Connection reset by peer I haven't poked at it yet, putting it here incase anyone else has ideas. - Cole

2 2

[libvirt] [PATCH] udev: Don't try to dump DMI on non-intel archs
by Michal Privoznik 20 Jul '11

20 Jul '11

DMI is Intel & Intel-compatible specific. Don't try to dump information on non-compatible architectures, which results only in error message in logs. --- NB: libsmbios is exclusively for x86_64 ia64 %{ix86} src/node_device/node_device_udev.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/src/node_device/node_device_udev.c b/src/node_device/node_device_udev.c index a6b5b2e..2c5d016 100644 --- a/src/node_device/node_device_udev.c +++ b/src/node_device/node_device_udev.c @@ -1477,6 +1477,8 @@ out: } +/* DMI is intel-compatible specific */ +#if defined(__x86_64__) || defined(__i386__) || defined(__amd64__) static void udevGetDMIData(union _virNodeDevCapData *data) { @@ -1549,6 +1551,7 @@ out: } return; } +#endif static int udevSetupSystemDev(void) @@ -1573,7 +1576,9 @@ static int udevSetupSystemDev(void) goto out; } +#if defined(__x86_64__) || defined(__i386__) || defined(__amd64__) udevGetDMIData(&def->caps->data); +#endif dev = virNodeDeviceAssignDef(&driverState->devs, def); if (dev == NULL) { -- 1.7.5.rc3

2 10

[libvirt] RFC New virDomainBlockPull API family to libvirt
by Adam Litke 20 Jul '11

20 Jul '11

Unfortunately, after committing the blockPull API to libvirt, the qemu community decided to change the API somewhat and we needed to revert the libvirt implementation. Now that the qemu API is settling out again, I would like to propose an updated libvirt public API which has required only a minor set of changes to sync back up to the qemu API. Summary of changes: - Qemu dropped incremental streaming so remove libvirt incremental BlockPull() API - Rename virDomainBlockPullAll() to virDomainBlockPull() - Changes required to qemu monitor handlers for changed command names Currently, qemu block device streaming completely flattens a disk image's backing file chain. Consider the following chain: A->B->C where C is a leaf image that is backed by B and B is backed by A. The current disk streaming command will produce an independent image C with no backing file. Future versions of qemu block streaming may support an option to specify a new base image from the current chain. For example: stream --backing_file B C would pull all blocks that are only in A to produce the chain: B->C (thus eliminating a dependency on A but maintaining B as a backing image. Do we want to create room in the BlockPull API to support this advanced usage in the future? If so, then a new parameter must be added to BlockPull: const char *backing_path. Even if we extend the API in this manner, the initial implementation will not support it because qemu will not support it immediately, and libvirt is missing core functionality to support it (no public enumeration of the disk backing file chain). Thoughts? -- To help speed the provisioning process for large domains, new QED disks are created with backing to a template image. These disks are configured with copy on read such that blocks that are read from the backing file are copied to the new disk. This reduces I/O over a potentially costly path to the backing image. In such a configuration, there is a desire to remove the dependency on the backing image as the domain runs. To accomplish this, qemu will provide an interface to perform sequential copy on read operations during normal VM operation. Once all data has been copied, the disk image's link to the backing file is removed. The virDomainBlockPull API family brings this functionality to libvirt. virDomainBlockPull() instructs the hypervisor to stream the entire device in the background. Progress of this operation can be checked with the function virDomainBlockPullInfo(). An ongoing stream can be cancelled with virDomainBlockPullAbort(). An event (VIR_DOMAIN_EVENT_ID_BLOCK_PULL) will be emitted when a disk has been fully populated or if a BlockPull() operation was terminated due to an error. This event is useful to avoid polling on virDomainBlockPullInfo() for completion and could also be used by the security driver to revoke access to the backing file when it is no longer needed. /* * BlockPull API */ /* An iterator for initiating and monitoring block pull operations */ typedef unsigned long long virDomainBlockPullCursor; typedef struct _virDomainBlockPullInfo virDomainBlockPullInfo; struct _virDomainBlockPullInfo { /* * The following fields provide an indication of block pull progress. @cur * indicates the current position and will be between 0 and @end. @end is * the final cursor position for this operation and represents completion. * To approximate progress, divide @cur by @end. */ virDomainBlockPullCursor cur; virDomainBlockPullCursor end; }; typedef virDomainBlockPullInfo *virDomainBlockPullInfoPtr; /** * virDomainBlockPull: * @dom: pointer to domain object * @path: Fully-qualified filename of disk * @flags: currently unused, for future extension * * Populate a disk image with data from its backing image. Once all data from * its backing image has been pulled, the disk no longer depends on a backing * image. This function pulls data for the entire device in the background. * Progress of the operation can be checked with virDomainGetBlockPullInfo() and * the operation can be aborted with virDomainBlockPullAbort(). When finished, * an asynchronous event is raised to indicate the final status. * * Returns 0 if the operation has started, -1 on failure. */ int virDomainBlockPull(virDomainPtr dom, const char *path, unsigned int flags); /** * virDomainBlockPullAbort: * @dom: pointer to domain object * @path: fully-qualified filename of disk * @flags: currently unused, for future extension * * Cancel a pull operation previously started by virDomainBlockPullAll(). * * Returns -1 in case of failure, 0 when successful. */ int virDomainBlockPullAbort(virDomainPtr dom, const char *path, unsigned int flags); /** * virDomainGetBlockPullInfo: * @dom: pointer to domain object * @path: fully-qualified filename of disk * @info: pointer to a virDomainBlockPullInfo structure * @flags: currently unused, for future extension * * Request progress information on a block pull operation that has been started * with virDomainBlockPull(). If an operation is active for the given * parameters, @info will be updated with the current progress. * * Returns -1 in case of failure, 0 when successful. */ int virDomainGetBlockPullInfo(virDomainPtr dom, const char *path, virDomainBlockPullInfoPtr info, unsigned int flags); /** * virConnectDomainEventBlockPullStatus: * * The final status of a virDomainBlockPull() operation */ typedef enum { VIR_DOMAIN_BLOCK_PULL_COMPLETED = 0, VIR_DOMAIN_BLOCK_PULL_FAILED = 1, } virConnectDomainEventBlockPullStatus; /** * virConnectDomainEventBlockPullCallback: * @conn: connection object * @dom: domain on which the event occurred * @path: fully-qualified filename of the affected disk * @status: final status of the operation (virConnectDomainEventBlockPullStatus * * The callback signature to use when registering for an event of type * VIR_DOMAIN_EVENT_ID_BLOCK_PULL with virConnectDomainEventRegisterAny() */ typedef void (*virConnectDomainEventBlockPullCallback)(virConnectPtr conn, virDomainPtr dom, const char *path, int status, void *opaque); -- Adam Litke IBM Linux Technology Center

5 21

[libvirt] [PATCH 0/3] Fixes to key usage/purpose checking
by Daniel P. Berrange 20 Jul '11

20 Jul '11

We were not correctly checking key usage/purpose as per RFC recommendations. We should have been treated unavailable info as a non-fatal condition, and should have honoured the criticality field

2 6

Re: [libvirt] Build libvirt 0.9.3 failed
by zhang xintao 20 Jul '11

20 Jul '11

config.log 38094 configure:54859:checking for LIBNL 38995 configure:54924:result: no 38096 configure:54924:error: libnl-devel >= 1.1 is required for macvtap support I try to build libvirt 0.9.3 -without macvtap support but failed same error message. can you help me ? On Wed, Jul 20, 2011 at 2:02 PM, zhang xintao <zhang.kvm(a)gmail.com> wrote: > This is th config.log > > I try to build libvirt 0.9.3 -without macvtap support but failed > > same error message. > > can you help me ? > > On Wed, Jul 20, 2011 at 12:52 PM, Eric Blake <eblake(a)redhat.com> wrote: >> On 07/19/2011 10:42 PM, zhang xintao wrote: >>> >>> I try to build libvirt 0.9.3 but failed >>> ========================== >>> Host OS：Ubuntu server 11.04 >>> ========================== >>> configure:error libnl-devel>= 1.1 is required for macvtap support >>> >>> libnl-dev 1.1-6 is already installed >>> >>> who can help me? >> >> What does config.log say in the areas where it was checking for libnl >> support? Meanwhile, are you okay building without macvtap support? >> >> -- >> Eric Blake eblake(a)redhat.com +1-801-349-2682 >> Libvirt virtualization library http://libvirt.org >> >

2 1