- Devel - libvirt List Archives

[libvirt] Entering freeze for 1.0.1 tomorrow
by Daniel Veillard 10 Dec '12

10 Dec '12

I'm just back from vacations without any internet access, and didn't really had time to go though the list mail yet. Is there any serious reson to not enter freeze for 1.0.1 tomorrow ? If not I will make the rc1 candidate tomorrow, thanks ! Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 2

[libvirt] ANNOUNCE: Release of libvirt-sandbox version 0.1.1
by Daniel P. Berrange 10 Dec '12

10 Dec '12

I pleased to announce the a new public release of libvirt-sandbox, version 0.1.1, is now available for download ftp://libvirt.org/libvirt/sandbox/ The packages are GPG signed with Key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF (4096R) The libvirt-sandbox package provides an API layer on top of libvirt-gobject which facilitates the cration of application sandboxes using virtualization technology. An application sandbox is a virtual machine or container that runs a single application binary, directly from the host OS filesystem. In other words there is no separate guest operating system install to build or manager. At this point in time libvirt-sandbox can create sandboxes using either LXC or KVM, and should in theory be extendable to any libvirt driver. This release has focused entirely on improving the virt-sandbox-service tool Changed in this release: - Fix typos in POD docs for some classes - Only depend on libvirt-daemon-{kvm,qemu,lxc}, not full libvirt RPM. - Switch to YUM for extracting package file list - Bind mount whole of /var rather than only some subdirs - Validate unit files exist before creating sandbox - Fixes to population of files in /etc and /var - Finish 'clone' command for copying sandboxes - Populate /etc/machine-id file - Fix systemd dependancies for bulk start/stop of containers - Symlink container journal directory into host filesystem - Rename sandbox.target to multi-user.target - Fix attachment to running containers Thanks to Dan Walsh for basically doing all the work in this release Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

1 0

[libvirt] [PATCH 0/4]use newer video device by -device in qemu commandline
by Guannan Ren 10 Dec '12

10 Dec '12

This patch set aims to use qemu VGA device with -device option to replace -vga option. The mapping is as follows. '-device VGA' maps to '-vga std' '-device cirrus-vga' maps to '-vga cirrus' '-device qxl-vga' maps to '-vga qxl' (there is also '-device qxl' for secondary devices) '-device vmware-svga' maps to '-vga vmware' Through this change, guests will be able to use any available PCI slot number rather than fixed 0x2 slot for primary video device. For backwards compatibility, if a certain qemu version does not support one of the above video device arguments to -device or even -device itself, the -vga will still be used. Guannan Ren(4) [PATCH 1/4] qemu: add qemu various VGA devices Caps flag [PATCH 2/4] conf: add optional attribte primary to video <model> [PATCH 3/4] qemu: use newer -device video device in qemu commandline [PATCH 4/4] doc: add attribute primary doc to video model element docs/formatdomain.html.in | 6 +- src/conf/domain_conf.c | 27 +++- src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 10 +- src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_command.c | 183 +++++++++++++++++++--------- tests/qemuhelptest.c | 48 ++++++-- .../qemuxml2argvdata/qemuxml2argv-graphics-spice-qxl-vga.args | 5 +- tests/qemuxml2argvtest.c | 12 +- 9 files changed, 220 insertions(+), 76 deletions(-)

2 7

[libvirt] [PATCHv3 0/2] S390: Adding support for SCLP Console
by Viktor Mihajlovski 10 Dec '12

10 Dec '12

The S390 architecture comes with a native console type (SCLP console) which is now also supported by current QEMU. This series is enabling libvirt to configure S390 domains with SCLP consoles. The domain XML has to be extended for the new console target types 'sclp' and 'sclplm' (line mode = dumb). As usual the QEMU driver must do capability probing in order to find out whether SCLP is supported and format the QEMU command line for the new console type. V2/V3 Changes: Rebased to current master, resolving conflicts. J.B. Joret (2): S390: Add SCLP console front end support S390: Enable SCLP Console in QEMU driver docs/formatdomain.html.in | 19 ++++++- docs/schemas/domaincommon.rng | 2 + src/conf/domain_conf.c | 4 +- src/conf/domain_conf.h | 2 + src/qemu/qemu_capabilities.c | 3 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 59 ++++++++++++++++++++++ .../qemuxml2argv-console-sclp.args | 8 +++ .../qemuxml2argvdata/qemuxml2argv-console-sclp.xml | 24 +++++++++ tests/qemuxml2argvtest.c | 3 ++ 10 files changed, 123 insertions(+), 2 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-console-sclp.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-console-sclp.xml -- 1.7.12.4

1 2

[libvirt] [PATCH 4/4] virsh: Add cpu-getmode and cpu-setmode command
by Ichikawa, Ken 10 Dec '12

10 Dec '12

This patch adds new virsh commands 'cpu-getmode' and 'cpu-setmode' using virDomainGetCPUMode and virDomainSetCPUMode. 'cpu-getmode' allows to get cpu mode of a running domain or persistent configuration. 'cpu-setmode' allows to change cpu mode of a domain but affects only persistent configuration. Signed-off-by: Ken ICHIKAWA <ichikawa.ken(a)jp.fujitsu.com> --- tools/virsh-domain.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 19 +++++++++ 2 files changed, 128 insertions(+) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 96e62fc..b4ea137 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -5371,6 +5371,113 @@ no_memory: } /* + * "cpu-getmode" command + */ +static const vshCmdInfo info_cpu_getmode[] = { + {"help", N_("show domain cpu mode")}, + {"desc", N_("Show cpu mode of a domain")}, + {NULL, NULL} +}; + +static const vshCmdOptDef opts_cpu_getmode[] = { + {"domain", VSH_OT_DATA, VSH_OFLAG_REQ, N_("domain name, id or uuid")}, + {"live", VSH_OT_BOOL, 0, N_("get running state")}, + {"config", VSH_OT_BOOL, 0, N_("get persistent configuration")}, + {"current", VSH_OT_BOOL, 0, N_("get current state configuration")}, + {NULL, 0, 0, NULL} +}; + +static bool +cmdCPUGetMode(vshControl *ctl, const vshCmd *cmd) +{ + virDomainPtr dom; + bool config = vshCommandOptBool(cmd, "config"); + bool live = vshCommandOptBool(cmd, "live"); + bool current = vshCommandOptBool(cmd, "current"); + char *mode = NULL; + unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT; + bool ret = false; + + if (current + config + live > 1) { + vshError(ctl, "%s", + _("--config, --live, and --current are mutually exclusive")); + return false; + } + + if (config) + flags = VIR_DOMAIN_AFFECT_CONFIG; + if (live) + flags = VIR_DOMAIN_AFFECT_LIVE; + + if (!(dom = vshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (!(mode = virDomainGetCPUMode(dom, flags))) { + vshError(ctl, _("Unable to get cpu mode")); + goto cleanup; + } + + vshPrint(ctl, "%s\n", mode); + + VIR_FREE(mode); + ret = true; + + cleanup: + virDomainFree(dom); + return ret; +} + +/* + * "cpu-setmode" command + */ +static const vshCmdInfo info_cpu_setmode[] = { + {"help", N_("set domain cpu mode")}, + {"desc", N_("Modify cpu mode of a domain")}, + {NULL, NULL} +}; + +static const vshCmdOptDef opts_cpu_setmode[] = { + {"domain", VSH_OT_DATA, VSH_OFLAG_REQ, N_("domain name, id or uuid")}, + {"mode", VSH_OT_DATA, VSH_OFLAG_REQ, + N_("cpu mode, one of custom, host-model and host-passthrough")}, + {NULL, 0, 0, NULL} +}; + +static bool +cmdCPUSetMode(vshControl *ctl, const vshCmd *cmd) +{ + virDomainPtr dom; + const char *mode = NULL; + bool ret = false; + + if (!(dom = vshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (vshCommandOptString(cmd, "mode", &mode) < 0) { + vshError(ctl, _("Unable to parse mode.")); + goto cleanup; + } + + if (!mode) { + vshError(ctl, _("CPU mode is not specified.")); + goto cleanup; + } + + if (virDomainSetCPUMode(dom, mode, 0) < 0) { + vshError(ctl, _("Unable to change cpu mode")); + goto cleanup; + } + + vshPrint(ctl, _("CPU mode updated successfully\n")); + + ret = true; + + cleanup: + virDomainFree(dom); + return ret; +} + +/* * "cpu-stats" command */ static const vshCmdInfo info_cpu_stats[] = { @@ -8507,6 +8614,8 @@ const vshCmdDef domManagementCmds[] = { #endif {"cpu-baseline", cmdCPUBaseline, opts_cpu_baseline, info_cpu_baseline, 0}, {"cpu-compare", cmdCPUCompare, opts_cpu_compare, info_cpu_compare, 0}, + {"cpu-getmode", cmdCPUGetMode, opts_cpu_getmode, info_cpu_getmode, 0}, + {"cpu-setmode", cmdCPUSetMode, opts_cpu_setmode, info_cpu_setmode, 0}, {"cpu-stats", cmdCPUStats, opts_cpu_stats, info_cpu_stats, 0}, {"create", cmdCreate, opts_create, info_create, 0}, {"define", cmdDefine, opts_define, info_define, 0}, diff --git a/tools/virsh.pod b/tools/virsh.pod index b0e7064..109e256 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -999,6 +999,25 @@ except that it does some error checking. The editor used can be supplied by the C<$VISUAL> or C<$EDITOR> environment variables, and defaults to C<vi>. +=item B<cpu-getmode> I<domain> [[I<--config>] | [I<--live>] | [I<--current>]] + +Get a domain's cpu mode, corresponding to the B<mode> attribute of B<cpu> +element of the domain XML. If cpu mode is not specified in the domain XML, +'custom' is displayed as a default. + +If I<--live> is specified, affect a running guest. +If I<--config> is specified, affect the next boot of a persistent guest. +If I<--current> is specified, affect the current guest state. +All flags are exclusive. If no flag is specified, behavior is the same as +I<--current>. + +=item B<cpu-setmode> I<domain> I<mode> + +Set a domain's cpu mode, corresponding to the B<mode> attribute of B<cpu> +element of the domain XML. I<mode> can be one of 'custom', 'host-model' +and 'host-passthrough'. Modification affects the next boot of a persistent +guest. + =item B<managedsave> I<domain> [I<--bypass-cache>] [{I<--running> | I<--paused>}] [I<--verbose>] -- 1.7.11.7

1 0

[libvirt] [PATCH 3/4] remote: Add support for virDomainGetCPUMode and virDomainSetCPUMode
by Ichikawa, Ken 10 Dec '12

10 Dec '12

This patch adds support for new APIs virDomainGetCPUMode and virDomainSetCPUMode into remote protocol driver. Signed-off-by: Ken ICHIKAWA <ichikawa.ken(a)jp.fujitsu.com> --- src/remote/remote_driver.c | 2 ++ src/remote/remote_protocol.x | 19 ++++++++++++++++++- src/remote_protocol-structs | 14 ++++++++++++++ 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 5cc7e32..b2e61b3 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -6153,6 +6153,8 @@ static virDriver remote_driver = { .nodeGetMemoryParameters = remoteNodeGetMemoryParameters, /* 0.10.2 */ .nodeGetCPUMap = remoteNodeGetCPUMap, /* 1.0.0 */ .domainFSTrim = remoteDomainFSTrim, /* 1.0.1 */ + .domainSetCPUMode = remoteDomainSetCPUMode, /* 1.0.1 */ + .domainGetCPUMode = remoteDomainGetCPUMode, /* 1.0.1 */ }; static virNetworkDriver network_driver = { diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index bdad9f0..943d9a0 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2696,6 +2696,21 @@ struct remote_domain_fstrim_args { unsigned int flags; }; +struct remote_domain_set_cpu_mode_args { + remote_nonnull_domain dom; + remote_nonnull_string cpuMode; + unsigned int flags; +}; + +struct remote_domain_get_cpu_mode_args { + remote_nonnull_domain dom; + unsigned int flags; +}; + +struct remote_domain_get_cpu_mode_ret { + remote_nonnull_string cpuMode; +}; + /*----- Protocol. -----*/ /* Define the program number, protocol version and procedure numbers here. */ @@ -3042,7 +3057,9 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_PMSUSPEND_DISK = 292, /* autogen autogen */ REMOTE_PROC_NODE_GET_CPU_MAP = 293, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_FSTRIM = 294, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SEND_PROCESS_SIGNAL = 295 /* autogen autogen */ + REMOTE_PROC_DOMAIN_SEND_PROCESS_SIGNAL = 295, /* autogen autogen */ + REMOTE_PROC_DOMAIN_SET_CPU_MODE = 296, /* autogen autogen */ + REMOTE_PROC_DOMAIN_GET_CPU_MODE = 297 /* autogen autogen */ /* * Notice how the entries are grouped in sets of 10 ? diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index e7d05b8..08e6a18 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -2151,6 +2151,18 @@ struct remote_domain_fstrim_args { uint64_t minimum; u_int flags; }; +struct remote_domain_set_cpu_mode_args { + remote_nonnull_domain dom; + remote_nonnull_string cpuMode; + u_int flags; +}; +struct remote_domain_get_cpu_mode_args { + remote_nonnull_domain dom; + u_int flags; +}; +struct remote_domain_get_cpu_mode_ret { + remote_nonnull_string cpuMode; +}; enum remote_procedure { REMOTE_PROC_OPEN = 1, REMOTE_PROC_CLOSE = 2, @@ -2447,4 +2459,6 @@ enum remote_procedure { REMOTE_PROC_NODE_GET_CPU_MAP = 293, REMOTE_PROC_DOMAIN_FSTRIM = 294, REMOTE_PROC_DOMAIN_SEND_PROCESS_SIGNAL = 295, + REMOTE_PROC_DOMAIN_SET_CPU_MODE = 296, + REMOTE_PROC_DOMAIN_GET_CPU_MODE = 297, }; -- 1.7.11.7

1 0

[libvirt] [PATCH 2/4] qemu: Add support for virDomainGetCPUMode and virDomainSetCPUMode
by Ichikawa, Ken 10 Dec '12

10 Dec '12

This patch adds support for virDomainGetCPUMode and virDomainSetCPUMode into qemu driver. Signed-off-by: Ken ICHIKAWA <ichikawa.ken(a)jp.fujitsu.com> --- src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 130 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 499ab3b..8988efa 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -207,6 +207,7 @@ virCPUDefFormatBuf; virCPUDefFree; virCPUDefFreeModel; virCPUDefParseXML; +virCPUModeTypeFromString; virCPUModeTypeToString; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e099c5c..f86ec1f 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -14871,6 +14871,133 @@ cleanup: return ret; } +static int +qemuDomainSetCPUMode(virDomainPtr dom, + const char *cpuMode, + unsigned int flags) +{ + virQEMUDriverPtr driver = dom->conn->privateData; + virDomainObjPtr vm = NULL; + virDomainDefPtr persistentDef = NULL; + int mode = -1; + int ret = -1; + + virCheckFlags(0, -1); + + qemuDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + qemuDriverUnlock(driver); + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + virReportError(VIR_ERR_NO_DOMAIN, + _("no domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if ((mode = virCPUModeTypeFromString(cpuMode)) < 0){ + virReportError(VIR_ERR_OPERATION_FAILED, + _("Invalid cpu mode '%s'"), cpuMode); + goto cleanup; + } + + /* get persistent configuration */ + if (!vm->persistent) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cannot change persistent config of a " + "transient domain")); + goto cleanup; + } + if (!(persistentDef = virDomainObjGetPersistentDef(driver->caps, vm))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Get persistent config failed")); + goto cleanup; + } + + if (!persistentDef->cpu) { + if (VIR_ALLOC(persistentDef->cpu) < 0) + goto no_memory; + persistentDef->cpu->type = VIR_CPU_TYPE_GUEST; + } + + persistentDef->cpu->mode = mode; + + if (virDomainSaveConfig(driver->configDir, persistentDef) < 0) + goto cleanup; + + ret = 0; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +no_memory: + virReportOOMError(); + goto cleanup; +} + +static char * +qemuDomainGetCPUMode(virDomainPtr dom, unsigned int flags) +{ + virQEMUDriverPtr driver = dom->conn->privateData; + virDomainObjPtr vm = NULL; + virDomainDefPtr def = NULL; + const char *cpuMode = NULL; + char *ret = NULL; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, NULL); + + qemuDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + qemuDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + virReportError(VIR_ERR_NO_DOMAIN, + _("no domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (virDomainLiveConfigHelperMethod(driver->caps, vm, &flags, &def) < 0) + goto cleanup; + + /* use correct domain definition according to flags */ + if (flags & VIR_DOMAIN_AFFECT_LIVE) + def = vm->def; + + /* If there is no cpu element, return default cpu mode. */ + if (!def->cpu) { + if (!(cpuMode = virCPUModeTypeToString(VIR_CPU_MODE_CUSTOM))){ + virReportError(VIR_ERR_INTERNAL_ERROR, + _("unknown cpu mode in domain definition '%d'"), + VIR_CPU_MODE_CUSTOM); + goto cleanup; + } + if (!(ret = strdup(cpuMode))) + goto no_memory; + goto cleanup; + } + + if (!(cpuMode = virCPUModeTypeToString(def->cpu->mode))){ + virReportError(VIR_ERR_INTERNAL_ERROR, + _("unknown cpu mode in domain definition '%d'"), + def->cpu->mode); + goto cleanup; + } + if (!(ret = strdup(cpuMode))) + goto no_memory; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +no_memory: + virReportOOMError(); + goto cleanup; +} + static virDriver qemuDriver = { .no = VIR_DRV_QEMU, .name = QEMU_DRIVER_NAME, @@ -15045,6 +15172,8 @@ static virDriver qemuDriver = { .nodeSetMemoryParameters = nodeSetMemoryParameters, /* 0.10.2 */ .nodeGetCPUMap = nodeGetCPUMap, /* 1.0.0 */ .domainFSTrim = qemuDomainFSTrim, /* 1.0.1 */ + .domainSetCPUMode = qemuDomainSetCPUMode, /* 1.0.1 */ + .domainGetCPUMode = qemuDomainGetCPUMode, /* 1.0.1 */ }; -- 1.7.11.7

1 0

[libvirt] [PATCH 1/4] API: Introduce new public APIs virDomainGetCPUMode and virDomainSetCPUMode
by Ichikawa, Ken 10 Dec '12

10 Dec '12

This patch adds new public APIs virDomainGetCPUMode and virDomainSetCPUMode to get/set cpu mode of a domain. These APIs are useful for users who want to get/set cpu mode easily without messing around with domain XML by themselves. Signed-off-by: Ken ICHIKAWA <ichikawa.ken(a)jp.fujitsu.com> --- include/libvirt/libvirt.h.in | 9 ++++ src/driver.h | 11 +++++ src/libvirt.c | 112 +++++++++++++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 2 + 4 files changed, 134 insertions(+) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index 17804ca..3e5fbc3 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -1739,6 +1739,15 @@ virDomainGetMetadata(virDomainPtr domain, const char *uri, unsigned int flags); +int +virDomainSetCPUMode(virDomainPtr domain, + const char *cpuMode, + unsigned int flags); + +char * +virDomainGetCPUMode(virDomainPtr domain, + unsigned int flags); + /* * XML domain description */ diff --git a/src/driver.h b/src/driver.h index 64d652f..1fc8668 100644 --- a/src/driver.h +++ b/src/driver.h @@ -915,6 +915,15 @@ typedef int unsigned long long minimum, unsigned int flags); +typedef int + (*virDrvDomainSetCPUMode)(virDomainPtr dom, + const char *cpuMode, + unsigned int flags); + +typedef char * + (*virDrvDomainGetCPUMode)(virDomainPtr dom, + unsigned int flags); + /** * _virDriver: * @@ -1107,6 +1116,8 @@ struct _virDriver { virDrvNodeGetCPUMap nodeGetCPUMap; virDrvDomainFSTrim domainFSTrim; virDrvDomainSendProcessSignal domainSendProcessSignal; + virDrvDomainSetCPUMode domainSetCPUMode; + virDrvDomainGetCPUMode domainGetCPUMode; }; typedef int diff --git a/src/libvirt.c b/src/libvirt.c index 6a7a817..9545f36 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -9616,6 +9616,118 @@ error: } /** + * virDomainSetCPUMode: + * @domain: a domain object + * @cpuMode: cpu mode, one of 'custom', 'host-model' and 'host-passthrough' + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Change cpu mode of a domain to @cpuMode. + * @cpuMode means mode attribute of cpu element of the domain XML. + * This function does not affect a running domain state but affects + * the next boot of a persistent domain. + * + * Returns 0 on success, -1 in case of failure. + */ +int +virDomainSetCPUMode(virDomainPtr domain, + const char *cpuMode, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "cpuMode=%s, flags=%x", cpuMode, flags); + + virResetLastError(); + + if (!VIR_IS_CONNECTED_DOMAIN(domain)) { + virLibDomainError(VIR_ERR_INVALID_DOMAIN, __FUNCTION__); + goto error; + } + + conn = domain->conn; + + if (conn->flags & VIR_CONNECT_RO) { + virLibDomainError(VIR_ERR_OPERATION_DENIED, __FUNCTION__); + goto error; + } + + if (conn->driver->domainSetCPUMode) { + int ret; + ret = conn->driver->domainSetCPUMode(domain, cpuMode, flags); + if (ret < 0) + goto error; + return ret; + } + + virLibConnError(VIR_ERR_NO_SUPPORT, __FUNCTION__); + +error: + virDispatchError(domain->conn); + return -1; +} + +/** + * virDomainGetCPUMode: + * @domain: a domain object + * @flags: bitwise-OR of virDomainModificationImpact + * + * Get cpu mode of a domain. The cpu mode is specified in mode + * attribute of cpu element of the domain XML. + * If there is no cpu element or mode attribute, this function + * returns 'custom' because 'custom' mode is default. + * + * @flags controls whether the live domain or persistent + * configuration will be queried. If VIR_AFFECT_LIVE is set, this + * will query a running domain state. If VIR_AFFECT_CONFIG is set, + * this will query a persistent configuration. + * If neither flag is specified or VIR_AFFECT_CURRENT is set, this + * will query a running domain state for active domains or a persistent + * configuration for inactive domains. + * These flags are mutually exclusive. + * + * Returns cpu mode string on success (caller must free), + * or NULL in case of failure. + */ +char * +virDomainGetCPUMode(virDomainPtr domain, unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "flags=%x", flags); + + virResetLastError(); + + if (!VIR_IS_CONNECTED_DOMAIN(domain)) { + virLibDomainError(VIR_ERR_INVALID_DOMAIN, __FUNCTION__); + goto error; + } + + if ((flags & VIR_DOMAIN_AFFECT_LIVE) && + (flags & VIR_DOMAIN_AFFECT_CONFIG)) { + virReportInvalidArg(flags, + _("flags 'affect live' and 'affect config' in %s are mutually exclusive"), + __FUNCTION__); + goto error; + } + + conn = domain->conn; + + if (conn->driver->domainGetCPUMode) { + char *ret; + ret = conn->driver->domainGetCPUMode(domain, flags); + if (!ret) + goto error; + return ret; + } + + virLibConnError(VIR_ERR_NO_SUPPORT, __FUNCTION__); + +error: + virDispatchError(domain->conn); + return NULL; +} + +/** * virNodeGetSecurityModel: * @conn: a connection object * @secmodel: pointer to a virSecurityModel structure diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index e3d63d3..8fc407d 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -578,6 +578,8 @@ LIBVIRT_1.0.1 { global: virDomainFSTrim; virDomainSendProcessSignal; + virDomainSetCPUMode; + virDomainGetCPUMode; } LIBVIRT_1.0.0; # .... define new API here using predicted next version number .... -- 1.7.11.7

1 0

[libvirt] [PATCH v15] support offline migration
by liguang 10 Dec '12

10 Dec '12

original migration did not aware of offline case, so, try to support offline migration quietly (did not disturb original migration) by pass VIR_MIGRATE_OFFLINE flag to migration APIs if only the domain is really inactive, and migration process will not puzzled by domain offline and exit unexpectedly. these changes did not take care of disk images the domain required, for them could be transferred by other APIs as suggested, then VIR_MIGRATE_OFFLINE must not combined with VIR_MIGRATE_NON_SHARED_*. and you must do a persistent migration at same time, do "virsh migrate --offline --persistent ...". Signed-off-by: liguang <lig.fnst(a)cn.fujitsu.com> --- include/libvirt/libvirt.h.in | 1 + src/libvirt.c | 4 + src/qemu/qemu_driver.c | 16 +++--- src/qemu/qemu_migration.c | 140 +++++++++++++++++++++++++++--------------- src/qemu/qemu_migration.h | 9 ++- tools/virsh-domain.c | 5 ++ tools/virsh.pod | 5 +- 7 files changed, 117 insertions(+), 63 deletions(-) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index 49a361a..ea625b3 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -1092,6 +1092,7 @@ typedef enum { * whole migration process; this will be used automatically * when supported */ VIR_MIGRATE_UNSAFE = (1 << 9), /* force migration even if it is considered unsafe */ + VIR_MIGRATE_OFFLINE = (1 << 10), /* offline migrate */ } virDomainMigrateFlags; /* Domain migration. */ diff --git a/src/libvirt.c b/src/libvirt.c index bdb1dc6..6d749d9 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -4827,6 +4827,10 @@ virDomainMigrateVersion3(virDomainPtr domain, if (uri_out) uri = uri_out; /* Did domainMigratePrepare3 change URI? */ + if (flags & VIR_MIGRATE_OFFLINE) { + cancelled = 0; + goto finish; + } /* Perform the migration. The driver isn't supposed to return * until the migration is complete. The src VM should remain * running, but in paused state until the destination can diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 595c452..1ba1665 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -9625,7 +9625,7 @@ qemudDomainMigratePrepareTunnel(virConnectPtr dconn, ret = qemuMigrationPrepareTunnel(driver, dconn, NULL, 0, NULL, NULL, /* No cookies in v2 */ - st, dname, dom_xml); + st, dname, dom_xml, flags); cleanup: qemuDriverUnlock(driver); @@ -9685,7 +9685,7 @@ qemudDomainMigratePrepare2(virConnectPtr dconn, ret = qemuMigrationPrepareDirect(driver, dconn, NULL, 0, NULL, NULL, /* No cookies */ uri_in, uri_out, - dname, dom_xml); + dname, dom_xml, flags); cleanup: qemuDriverUnlock(driver); @@ -9827,7 +9827,7 @@ qemuDomainMigrateBegin3(virDomainPtr domain, asyncJob = QEMU_ASYNC_JOB_NONE; } - if (!virDomainObjIsActive(vm)) { + if (!virDomainObjIsActive(vm) && !(flags & VIR_MIGRATE_OFFLINE)) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("domain is not running")); goto endjob; @@ -9836,9 +9836,9 @@ qemuDomainMigrateBegin3(virDomainPtr domain, /* Check if there is any ejected media. * We don't want to require them on the destination. */ - - if (qemuDomainCheckEjectableMedia(driver, vm, asyncJob) < 0) - goto endjob; + if (!(flags & VIR_MIGRATE_OFFLINE) && + qemuDomainCheckEjectableMedia(driver, vm, asyncJob) < 0) + goto endjob; if (!(xml = qemuMigrationBegin(driver, vm, xmlin, dname, cookieout, cookieoutlen, @@ -9922,7 +9922,7 @@ qemuDomainMigratePrepare3(virConnectPtr dconn, cookiein, cookieinlen, cookieout, cookieoutlen, uri_in, uri_out, - dname, dom_xml); + dname, dom_xml, flags); cleanup: qemuDriverUnlock(driver); @@ -9967,7 +9967,7 @@ qemuDomainMigratePrepareTunnel3(virConnectPtr dconn, ret = qemuMigrationPrepareTunnel(driver, dconn, cookiein, cookieinlen, cookieout, cookieoutlen, - st, dname, dom_xml); + st, dname, dom_xml, flags); qemuDriverUnlock(driver); cleanup: diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index d52ec59..53171df 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1442,6 +1442,24 @@ char *qemuMigrationBegin(struct qemud_driver *driver, QEMU_MIGRATION_COOKIE_LOCKSTATE) < 0) goto cleanup; + if (flags & VIR_MIGRATE_OFFLINE) { + if (flags & (VIR_MIGRATE_NON_SHARED_DISK | + VIR_MIGRATE_NON_SHARED_INC)) { + virReportError(VIR_ERR_OPERATION_INVALID, + "%s", + _("offline migration cannot handle " + "non-shared storage")); + goto cleanup; + } + if (!(flags & VIR_MIGRATE_PERSIST_DEST)) { + virReportError(VIR_ERR_OPERATION_INVALID, + "%s", + _("offline migration must be specified with " + "the persistent flag set")); + goto cleanup; + } + } + if (xmlin) { if (!(def = virDomainDefParseString(driver->caps, xmlin, QEMU_EXPECTED_VIRT_TYPES, @@ -1499,7 +1517,8 @@ qemuMigrationPrepareAny(struct qemud_driver *driver, const char *dname, const char *dom_xml, const char *migrateFrom, - virStreamPtr st) + virStreamPtr st, + unsigned long flags) { virDomainDefPtr def = NULL; virDomainObjPtr vm = NULL; @@ -1609,15 +1628,18 @@ qemuMigrationPrepareAny(struct qemud_driver *driver, /* Start the QEMU daemon, with the same command-line arguments plus * -incoming $migrateFrom */ - if (qemuProcessStart(dconn, driver, vm, migrateFrom, dataFD[0], NULL, NULL, - VIR_NETDEV_VPORT_PROFILE_OP_MIGRATE_IN_START, - VIR_QEMU_PROCESS_START_PAUSED | - VIR_QEMU_PROCESS_START_AUTODESROY) < 0) { - virDomainAuditStart(vm, "migrated", false); - /* Note that we don't set an error here because qemuProcessStart - * should have already done that. - */ - goto endjob; + if (!(flags & VIR_MIGRATE_OFFLINE)) { + if (qemuProcessStart(dconn, driver, vm, migrateFrom, dataFD[0], + NULL, NULL, + VIR_NETDEV_VPORT_PROFILE_OP_MIGRATE_IN_START, + VIR_QEMU_PROCESS_START_PAUSED | + VIR_QEMU_PROCESS_START_AUTODESROY) < 0) { + virDomainAuditStart(vm, "migrated", false); + /* Note that we don't set an error here because qemuProcessStart + * should have already done that. + */ + goto endjob; + } } if (tunnel) { @@ -1625,7 +1647,8 @@ qemuMigrationPrepareAny(struct qemud_driver *driver, virReportSystemError(errno, "%s", _("cannot pass pipe for tunnelled migration")); virDomainAuditStart(vm, "migrated", false); - qemuProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED, 0); + if (!(flags & VIR_MIGRATE_OFFLINE)) + qemuProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED, 0); goto endjob; } dataFD[1] = -1; /* 'st' owns the FD now & will close it */ @@ -1640,13 +1663,15 @@ qemuMigrationPrepareAny(struct qemud_driver *driver, VIR_DEBUG("Received no lockstate"); } - if (qemuMigrationBakeCookie(mig, driver, vm, cookieout, cookieoutlen, - QEMU_MIGRATION_COOKIE_GRAPHICS) < 0) { - /* We could tear down the whole guest here, but - * cookie data is (so far) non-critical, so that - * seems a little harsh. We'll just warn for now. - */ - VIR_WARN("Unable to encode migration cookie"); + if (!(flags & VIR_MIGRATE_OFFLINE)) { + if (qemuMigrationBakeCookie(mig, driver, vm, cookieout, cookieoutlen, + QEMU_MIGRATION_COOKIE_GRAPHICS) < 0) { + /* We could tear down the whole guest here, but + * cookie data is (so far) non-critical, so that + * seems a little harsh. We'll just warn for now. + */ + VIR_WARN("Unable to encode migration cookie"); + } } if (qemuDomainCleanupAdd(vm, qemuMigrationPrepareCleanup) < 0) @@ -1708,7 +1733,8 @@ qemuMigrationPrepareTunnel(struct qemud_driver *driver, int *cookieoutlen, virStreamPtr st, const char *dname, - const char *dom_xml) + const char *dom_xml, + unsigned long flags) { int ret; @@ -1722,7 +1748,7 @@ qemuMigrationPrepareTunnel(struct qemud_driver *driver, */ ret = qemuMigrationPrepareAny(driver, dconn, cookiein, cookieinlen, cookieout, cookieoutlen, dname, dom_xml, - "stdio", st); + "stdio", st, flags); return ret; } @@ -1737,7 +1763,8 @@ qemuMigrationPrepareDirect(struct qemud_driver *driver, const char *uri_in, char **uri_out, const char *dname, - const char *dom_xml) + const char *dom_xml, + unsigned long flags) { static int port = 0; int this_port; @@ -1833,7 +1860,7 @@ qemuMigrationPrepareDirect(struct qemud_driver *driver, ret = qemuMigrationPrepareAny(driver, dconn, cookiein, cookieinlen, cookieout, cookieoutlen, dname, dom_xml, - migrateFrom, NULL); + migrateFrom, NULL, flags); cleanup: VIR_FREE(hostname); if (ret != 0) @@ -2675,7 +2702,9 @@ static int doPeer2PeerMigrate3(struct qemud_driver *driver, uri, &uri_out, flags, dname, resource, dom_xml); qemuDomainObjExitRemoteWithDriver(driver, vm); } + VIR_FREE(dom_xml); + if (ret == -1) goto cleanup; @@ -2858,7 +2887,7 @@ static int doPeer2PeerMigrate(struct qemud_driver *driver, } /* domain may have been stopped while we were talking to remote daemon */ - if (!virDomainObjIsActive(vm)) { + if (!virDomainObjIsActive(vm) && !(flags & VIR_MIGRATE_OFFLINE)) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("guest unexpectedly quit")); goto cleanup; @@ -2921,7 +2950,7 @@ qemuMigrationPerformJob(struct qemud_driver *driver, if (qemuMigrationJobStart(driver, vm, QEMU_ASYNC_JOB_MIGRATION_OUT) < 0) goto cleanup; - if (!virDomainObjIsActive(vm)) { + if (!virDomainObjIsActive(vm) && !(flags & VIR_MIGRATE_OFFLINE)) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("domain is not running")); goto endjob; @@ -3245,26 +3274,27 @@ qemuMigrationFinish(struct qemud_driver *driver, * object, but if no, clean up the empty qemu process. */ if (retcode == 0) { - if (!virDomainObjIsActive(vm)) { + if (!virDomainObjIsActive(vm) && !(flags & VIR_MIGRATE_OFFLINE)) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("guest unexpectedly quit")); goto endjob; } - if (qemuMigrationVPAssociatePortProfiles(vm->def) < 0) { - qemuProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED, - VIR_QEMU_PROCESS_STOP_MIGRATED); - virDomainAuditStop(vm, "failed"); - event = virDomainEventNewFromObj(vm, - VIR_DOMAIN_EVENT_STOPPED, - VIR_DOMAIN_EVENT_STOPPED_FAILED); - goto endjob; + if (!(flags & VIR_MIGRATE_OFFLINE)) { + if (qemuMigrationVPAssociatePortProfiles(vm->def) < 0) { + qemuProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED, + VIR_QEMU_PROCESS_STOP_MIGRATED); + virDomainAuditStop(vm, "failed"); + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_STOPPED, + VIR_DOMAIN_EVENT_STOPPED_FAILED); + goto endjob; + } + if (mig->network) + if (qemuDomainMigrateOPDRelocate(driver, vm, mig) < 0) + VIR_WARN("unable to provide network data for relocation"); } - if (mig->network) - if (qemuDomainMigrateOPDRelocate(driver, vm, mig) < 0) - VIR_WARN("unable to provide network data for relocation"); - if (flags & VIR_MIGRATE_PERSIST_DEST) { virDomainDefPtr vmdef; if (vm->persistent) @@ -3312,7 +3342,7 @@ qemuMigrationFinish(struct qemud_driver *driver, event = NULL; } - if (!(flags & VIR_MIGRATE_PAUSED)) { + if (!(flags & VIR_MIGRATE_PAUSED) && !(flags & VIR_MIGRATE_OFFLINE)) { /* run 'cont' on the destination, which allows migration on qemu * >= 0.10.6 to work properly. This isn't strictly necessary on * older qemu's, but it also doesn't hurt anything there @@ -3350,20 +3380,26 @@ qemuMigrationFinish(struct qemud_driver *driver, dom = virGetDomain(dconn, vm->def->name, vm->def->uuid); - event = virDomainEventNewFromObj(vm, - VIR_DOMAIN_EVENT_RESUMED, - VIR_DOMAIN_EVENT_RESUMED_MIGRATED); - if (virDomainObjGetState(vm, NULL) == VIR_DOMAIN_PAUSED) { - virDomainObjSetState(vm, VIR_DOMAIN_PAUSED, VIR_DOMAIN_PAUSED_USER); - if (event) - qemuDomainEventQueue(driver, event); + if (!(flags & VIR_MIGRATE_OFFLINE)) { event = virDomainEventNewFromObj(vm, - VIR_DOMAIN_EVENT_SUSPENDED, - VIR_DOMAIN_EVENT_SUSPENDED_PAUSED); + VIR_DOMAIN_EVENT_RESUMED, + VIR_DOMAIN_EVENT_RESUMED_MIGRATED); + if (virDomainObjGetState(vm, NULL) == VIR_DOMAIN_PAUSED) { + virDomainObjSetState(vm, VIR_DOMAIN_PAUSED, + VIR_DOMAIN_PAUSED_USER); + if (event) + qemuDomainEventQueue(driver, event); + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_SUSPENDED, + VIR_DOMAIN_EVENT_SUSPENDED_PAUSED); + } } - if (virDomainSaveStatus(driver->caps, driver->stateDir, vm) < 0) { - VIR_WARN("Failed to save status on vm %s", vm->def->name); - goto endjob; + + if (virDomainObjIsActive(vm)) { + if (virDomainSaveStatus(driver->caps, driver->stateDir, vm) < 0) { + VIR_WARN("Failed to save status on vm %s", vm->def->name); + goto endjob; + } } /* Guest is successfully running, so cancel previous auto destroy */ @@ -3430,6 +3466,9 @@ int qemuMigrationConfirm(struct qemud_driver *driver, if (!(mig = qemuMigrationEatCookie(driver, vm, cookiein, cookieinlen, 0))) return -1; + if (flags & VIR_MIGRATE_OFFLINE) + goto done; + /* Did the migration go as planned? If yes, kill off the * domain object, but if no, resume CPUs */ @@ -3465,6 +3504,7 @@ int qemuMigrationConfirm(struct qemud_driver *driver, } } +done: qemuMigrationCookieFree(mig); rv = 0; diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index 7a2269a..f2dc5aa 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -36,7 +36,8 @@ VIR_MIGRATE_NON_SHARED_DISK | \ VIR_MIGRATE_NON_SHARED_INC | \ VIR_MIGRATE_CHANGE_PROTECTION | \ - VIR_MIGRATE_UNSAFE) + VIR_MIGRATE_UNSAFE | \ + VIR_MIGRATE_OFFLINE) enum qemuMigrationJobPhase { QEMU_MIGRATION_PHASE_NONE = 0, @@ -97,7 +98,8 @@ int qemuMigrationPrepareTunnel(struct qemud_driver *driver, int *cookieoutlen, virStreamPtr st, const char *dname, - const char *dom_xml); + const char *dom_xml, + unsigned long flags); int qemuMigrationPrepareDirect(struct qemud_driver *driver, virConnectPtr dconn, @@ -108,7 +110,8 @@ int qemuMigrationPrepareDirect(struct qemud_driver *driver, const char *uri_in, char **uri_out, const char *dname, - const char *dom_xml); + const char *dom_xml, + unsigned long flags); int qemuMigrationPerform(struct qemud_driver *driver, virConnectPtr conn, diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index cc47383..5d18bdf 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -6661,6 +6661,7 @@ static const vshCmdInfo info_migrate[] = { static const vshCmdOptDef opts_migrate[] = { {"live", VSH_OT_BOOL, 0, N_("live migration")}, + {"offline", VSH_OT_BOOL, 0, N_("offline (domain's inactive) migration")}, {"p2p", VSH_OT_BOOL, 0, N_("peer-2-peer migration")}, {"direct", VSH_OT_BOOL, 0, N_("direct migration")}, {"tunneled", VSH_OT_ALIAS, 0, "tunnelled"}, @@ -6746,6 +6747,10 @@ doMigrate(void *opaque) if (vshCommandOptBool(cmd, "unsafe")) flags |= VIR_MIGRATE_UNSAFE; + if (vshCommandOptBool(cmd, "offline")) { + flags |= VIR_MIGRATE_OFFLINE; + } + if (xmlfile && virFileReadAll(xmlfile, 8192, &xml) < 0) { vshError(ctl, _("file '%s' doesn't exist"), xmlfile); diff --git a/tools/virsh.pod b/tools/virsh.pod index 29be39e..b3ef64e 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1026,13 +1026,14 @@ I<--total> for only the total stats, I<start> for only the per-cpu stats of the CPUs from I<start>, I<count> for only I<count> CPUs' stats. -=item B<migrate> [I<--live>] [I<--direct>] [I<--p2p> [I<--tunnelled>]] +=item B<migrate> [I<--live>] [I<--offline>] [I<--direct>] [I<--p2p> [I<--tunnelled>]] [I<--persistent>] [I<--undefinesource>] [I<--suspend>] [I<--copy-storage-all>] [I<--copy-storage-inc>] [I<--change-protection>] [I<--unsafe>] [I<--verbose>] I<domain> I<desturi> [I<migrateuri>] [I<dname>] [I<--timeout> B<seconds>] [I<--xml> B<file>] -Migrate domain to another host. Add I<--live> for live migration; I<--p2p> +Migrate domain to another host. Add I<--live> for live migration; +I<--offline> for offline (domain's inactive) migration; <--p2p> for peer-2-peer migration; I<--direct> for direct migration; or I<--tunnelled> for tunnelled migration. I<--persistent> leaves the domain persistent on destination host, I<--undefinesource> undefines the domain on the source host, -- 1.7.1

3 6

[libvirt] [PATCHv3 00/17] new VIR_(APPEND|INSERT|DELETE)_ELEMENT macros
by Laine Stump 08 Dec '12

08 Dec '12

DO NOT BE SCARED OFF BY THE PATCH COUNT!! Most of these are very short, and they all follow the same pattern; beyond that, 01/17 and 02/17 are the only ones that are really time-sensitive. The first patch in this series adds new macros to insert/append/delete elements from the arrays of variously-sized structs that are commonly used in libvirt. The intent is to eliminate the arithmetic mistakes associated with calling memmove() directly, and shorten application code. A full description of the added macros and their functions is included in 01/17. Patches 02..17 convert various bits of hand-coded array manipulation in libvirt to use the new macros. They can be taken now, later, or never (although I'd like to have at least 02/17 taken now, since other similar functions will be added in an upcoming patch series, and I want them to all use the same logic). Mostly I went through all of those conversions to see just how useful/usable the new macros were (and they led to a couple of rewrites). I will be posting another series momentarily that has a dependency on 01/17.

2 21