[libvirt] Add persistent XML for cpu tunables
These are pretty rough patches, something like a draft, I beleive there must be many problems, please review it heavily, :-) New XML: <cputune> <shares>2048</shares> <vcpupin vcpu='0' cpuset='0-4,^1'/> <vcpupin vcpu='1' cpuset='1,3'/> <vcpupin vcpu='2' cpuset='0,2'/> </cputune> "shares" is to define the the proportional weighted cpu share for the domain. "vcpupin" is to define the cpu affinities of vcpus, it will not be displayed if one doesn't specify it explicitly in XML or set the cpu affinites for vcpu via "vcpupin", means there will be no vcpupin element in domain XML by default, and the constraints are: - Error if one specify entries more than the count of maxvcpus. - Error when one specify entries for same vcpu. - Error if value of attribute "vcpu" is more than count of "maxvcpus - 1". Attribute "cpuset" works same as "cpuset" of element "vcpu", reuse the codes for parsing and formating value of "cpuset" of element "vcpu". NB, the idea to add persistent XML for "cpushares" is from "Nikunj A. Dadhania": https://www.redhat.com/archives/libvir-list/2011-January/msg01183.html I rebased it and include it together in this patch series. [PATCH 1/6] cputune: new tests for testing cputune xml [PATCH 2/6] cputune: support cputune for xend driver [PATCH 3/6] cputune: support cputune for qemu driver [PATCH 4/6] cputune: support cputune xml for LXC driver [PATCH 5/6] cputune: support cputune xml config [PATCH 6/6] cputune: Add docs xml schema for cputune xml
--- tests/qemuxml2argvdata/qemuxml2argv-cputune.args | 4 ++ tests/qemuxml2argvdata/qemuxml2argv-cputune.xml | 33 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 1 + tests/qemuxml2xmltest.c | 1 + 4 files changed, 39 insertions(+), 0 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cputune.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cputune.xml diff --git a/tests/qemuxml2argvdata/qemuxml2argv-cputune.args b/tests/qemuxml2argvdata/qemuxml2argv-cputune.args new file mode 100644 index 0000000..c41cec0 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-cputune.args @@ -0,0 +1,4 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test /usr/bin/qemu -S -M \ +pc -m 214 -smp 2 -name QEMUGuest1 -nographic \ +-monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c \ +-hda /dev/HostVG/QEMUGuest1 -net none -serial none -parallel none -usb diff --git a/tests/qemuxml2argvdata/qemuxml2argv-cputune.xml b/tests/qemuxml2argvdata/qemuxml2argv-cputune.xml new file mode 100644 index 0000000..76173ec --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-cputune.xml @@ -0,0 +1,33 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219136</memory> + <currentMemory>219136</currentMemory> + <vcpu>2</vcpu> + <cputune> + <shares>2048</shares> + <vcpupin vcpu='0' cpuset='0'/> + <vcpupin vcpu='1' cpuset='1'/> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <cpu> + <topology sockets='2' cores='1' threads='1'/> + </cpu> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' unit='0'/> + </disk> + <controller type='ide' index='0'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index d2864ef..ffa93bf 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -490,6 +490,7 @@ mymain(int argc, char **argv) DO_TEST("memtune", false, QEMU_CAPS_NAME); DO_TEST("blkiotune", false, QEMU_CAPS_NAME); + DO_TEST("cputune", false, QEMU_CAPS_NAME); free(driver.stateDir); virCapabilitiesFree(driver.caps); diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 67e721b..3ee94fc 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -184,6 +184,7 @@ mymain(int argc, char **argv) DO_TEST("encrypted-disk"); DO_TEST("memtune"); DO_TEST("blkiotune"); + DO_TEST("cputune"); DO_TEST("smp"); -- 1.7.4
On Sun, 27 Feb 2011 22:39:16 +0800, Osier Yang <jyang@redhat.com> wrote:
--- tests/qemuxml2argvdata/qemuxml2argv-cputune.args | 4 ++ tests/qemuxml2argvdata/qemuxml2argv-cputune.xml | 33 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 1 + tests/qemuxml2xmltest.c | 1 + 4 files changed, 39 insertions(+), 0 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cputune.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cputune.xml
diff --git a/tests/qemuxml2argvdata/qemuxml2argv-cputune.args b/tests/qemuxml2argvdata/qemuxml2argv-cputune.args new file mode 100644 index 0000000..c41cec0 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-cputune.args @@ -0,0 +1,4 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test /usr/bin/qemu -S -M \ +pc -m 214 -smp 2 -name QEMUGuest1 -nographic \ +-monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c \ +-hda /dev/HostVG/QEMUGuest1 -net none -serial none -parallel none -usb diff --git a/tests/qemuxml2argvdata/qemuxml2argv-cputune.xml b/tests/qemuxml2argvdata/qemuxml2argv-cputune.xml new file mode 100644 index 0000000..76173ec --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-cputune.xml @@ -0,0 +1,33 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219136</memory> + <currentMemory>219136</currentMemory> + <vcpu>2</vcpu> + <cputune> + <shares>2048</shares> + <vcpupin vcpu='0' cpuset='0'/> + <vcpupin vcpu='1' cpuset='1'/> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <cpu> + <topology sockets='2' cores='1' threads='1'/> + </cpu> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' unit='0'/> + </disk> + <controller type='ide' index='0'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index d2864ef..ffa93bf 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -490,6 +490,7 @@ mymain(int argc, char **argv)
DO_TEST("memtune", false, QEMU_CAPS_NAME); DO_TEST("blkiotune", false, QEMU_CAPS_NAME); + DO_TEST("cputune", false, QEMU_CAPS_NAME);
free(driver.stateDir); virCapabilitiesFree(driver.caps); diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 67e721b..3ee94fc 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -184,6 +184,7 @@ mymain(int argc, char **argv) DO_TEST("encrypted-disk"); DO_TEST("memtune"); DO_TEST("blkiotune"); + DO_TEST("cputune");
DO_TEST("smp");
Reviewed-by: "Nikunj A. Dadhania" <nikunj@linux.vnet.ibm.com> Regards Nikunj
Didn't test it, and not sure if it's the correct way to add cputune xml for xend driver, and besides, seems "xm driver" and "xen hypervisor" also support vcpu affinity, do we need to add support for them too? Any suggestion on supporting cputune for xen is appreciated. :-) --- src/xen/xend_internal.c | 29 +++++++++++++++++++++++++---- 1 files changed, 25 insertions(+), 4 deletions(-) diff --git a/src/xen/xend_internal.c b/src/xen/xend_internal.c index bfaed65..852153b 100644 --- a/src/xen/xend_internal.c +++ b/src/xen/xend_internal.c @@ -2233,11 +2233,12 @@ xenDaemonDomainSetVcpusFlags(virDomainPtr domain, unsigned int vcpus, */ int xenDaemonDomainPinVcpu(virDomainPtr domain, unsigned int vcpu, - unsigned char *cpumap, int maplen) + unsigned char *cpumap, int maplen) { char buf[VIR_UUID_BUFLEN], mapstr[sizeof(cpumap_t) * 64]; - int i, j; + int i, j, ret; xenUnifiedPrivatePtr priv; + virDomainDefPtr def = NULL; if ((domain == NULL) || (domain->conn == NULL) || (domain->name == NULL) || (cpumap == NULL) || (maplen < 1) || (maplen > (int)sizeof(cpumap_t))) { @@ -2265,8 +2266,28 @@ xenDaemonDomainPinVcpu(virDomainPtr domain, unsigned int vcpu, mapstr[strlen(mapstr) - 1] = 0; snprintf(buf, sizeof(buf), "%d", vcpu); - return(xend_op(domain->conn, domain->name, "op", "pincpu", "vcpu", buf, - "cpumap", mapstr, NULL)); + ret = xend_op(domain->conn, domain->name, "op", "pincpu", "vcpu", buf, + "cpumap", mapstr, NULL); + + if (!(def = xenDaemonDomainFetch(domain->conn, + domain->id, + domain->name, + NULL))) + goto cleanup; + + if (ret == 0) { + if (virDomainVcpupinAdd(def, cpumap, maplen, vcpu) < 0) { + virXendError(VIR_ERR_INTERNAL_ERROR, + "%s", _("failed to add vcpupin xml entry")); + return (-1); + } + } + + return ret; + +cleanup: + virDomainDefFree(def); + return -1; } /** -- 1.7.4
When domain startup, setting cpu affinity and cpu shares according to the cputune xml specified in domain xml. Modify "qemudDomainPinVcpu" to update domain config for vcpupin, and modify "qemuSetSchedulerParameters" to update domain config for cpu shares. * src/qemu/qemu_cgroup.c * src/qemu/qemu_driver.c * src/qemu/qemu_process.c --- src/qemu/qemu_cgroup.c | 15 +++++++++ src/qemu/qemu_driver.c | 9 +++++ src/qemu/qemu_process.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+), 0 deletions(-) diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index b39b5e1..f578f9e 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -340,6 +340,21 @@ int qemuSetupCgroup(struct qemud_driver *driver, vm->def->name); } + if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPU)) { + if (vm->def->cputune.shares != 0) { + rc = virCgroupSetCpuShares(cgroup, vm->def->cputune.shares); + if(rc != 0) { + virReportSystemError(-rc, + _("Unable to set io cpu shares for domain %s"), + vm->def->name); + goto cleanup; + } + } + } else { + qemuReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("CPU tuning is not available on this host")); + } + done: virCgroupFree(&cgroup); return 0; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 1a7bec9..55c3918 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2752,6 +2752,13 @@ qemudDomainPinVcpu(virDomainPtr dom, "%s", _("cpu affinity is not supported")); goto cleanup; } + + if (virDomainVcpupinAdd(vm->def, cpumap, maplen, vcpu) < 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("failed to update or add vcpupin xml")); + goto cleanup; + } + ret = 0; cleanup: @@ -4718,6 +4725,8 @@ static int qemuSetSchedulerParameters(virDomainPtr dom, _("unable to set cpu shares tunable")); goto cleanup; } + + vm->def->cputune.shares = params[i].value.ul; } else { qemuReportError(VIR_ERR_INVALID_ARG, _("Invalid parameter `%s'"), param->field); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 7879165..a650d07 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1432,6 +1432,82 @@ qemuProcessGetPCIMemballoonVendorProduct(virDomainMemballoonDefPtr def, } +/* Set CPU affinites for vcpus if vcpupin xml provided. */ +static int +qemuProcessSetVcpuAffinites(virConnectPtr conn, + virDomainObjPtr vm) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDefPtr def = vm->def; + virNodeInfo nodeinfo; + pid_t vcpupid; + unsigned char *cpumask; + int vcpu, cpumaplen, hostcpus, maxcpu; + + if (virNodeGetInfo(conn, &nodeinfo) != 0) { + return -1; + } + + if (!def->cputune.nvcpupin) + return 0; + + if (priv->vcpupids == NULL) { + qemuReportError(VIR_ERR_NO_SUPPORT, + "%s", _("cpu affinity is not supported")); + return -1; + } + + hostcpus = VIR_NODEINFO_MAXCPUS(nodeinfo); + cpumaplen = VIR_CPU_MAPLEN(hostcpus); + maxcpu = cpumaplen * 8; + + if (maxcpu > hostcpus) + maxcpu = hostcpus; + + for (vcpu = 0; vcpu < def->cputune.nvcpupin; vcpu++) { + if (vcpu != def->cputune.vcpupin[vcpu]->vcpuid) + continue; + + int i; + unsigned char *cpumap = NULL; + + if (VIR_ALLOC_VAR(cpumap, char, cpumaplen) < 0) { + virReportOOMError(); + return -1; + } + + /* Initialize cpumap to all 0s. */ + for (i = 0; i < cpumaplen; i++) + cpumap[i] = 0; + + cpumask = (unsigned char *)def->cputune.vcpupin[vcpu]->cpumask; + vcpupid = priv->vcpupids[vcpu]; + + /* Convert cpumask to bitmap here. */ + for (i = 0; i < VIR_DOMAIN_CPUMASK_LEN; i++) { + int cur = 0; + int mod = 0; + + if (i) { + cur = i / 8; + mod = i % 8; + } + + if (cpumask[i]) + cpumap[cur] |= 1 << mod; + } + + if (virProcessInfoSetAffinity(vcpupid, + cpumap, + cpumaplen, + maxcpu) < 0) { + return -1; + } + } + + return 0; +} + /* * This entire method assumes that PCI devices in 'info pci' * match ordering of devices specified on the command line @@ -2187,6 +2263,10 @@ int qemuProcessStart(virConnectPtr conn, if (qemuProcessDetectVcpuPIDs(driver, vm) < 0) goto cleanup; + VIR_DEBUG0("Setting VCPU affinities"); + if (qemuProcessSetVcpuAffinites(conn, vm) < 0) + goto cleanup; + VIR_DEBUG0("Setting any required VM passwords"); if (qemuProcessInitPasswords(conn, driver, vm, qemuCaps) < 0) goto cleanup; -- 1.7.4
On Sun, 27 Feb 2011 22:39:18 +0800, Osier Yang <jyang@redhat.com> wrote:
When domain startup, setting cpu affinity and cpu shares according to the cputune xml specified in domain xml.
Modify "qemudDomainPinVcpu" to update domain config for vcpupin, and modify "qemuSetSchedulerParameters" to update domain config for cpu shares.
* src/qemu/qemu_cgroup.c * src/qemu/qemu_driver.c * src/qemu/qemu_process.c --- src/qemu/qemu_cgroup.c | 15 +++++++++ src/qemu/qemu_driver.c | 9 +++++ src/qemu/qemu_process.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+), 0 deletions(-)
diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index b39b5e1..f578f9e 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -340,6 +340,21 @@ int qemuSetupCgroup(struct qemud_driver *driver, vm->def->name); }
+ if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPU)) { + if (vm->def->cputune.shares != 0) {
Osier, we will need patch 5/6 before this to compile, some patch reordering required here. Regards Nikunj
于 2011年03月03日 12:53, Nikunj A. Dadhania 写道:
On Sun, 27 Feb 2011 22:39:18 +0800, Osier Yang<jyang@redhat.com> wrote:
When domain startup, setting cpu affinity and cpu shares according to the cputune xml specified in domain xml.
Modify "qemudDomainPinVcpu" to update domain config for vcpupin, and modify "qemuSetSchedulerParameters" to update domain config for cpu shares.
* src/qemu/qemu_cgroup.c * src/qemu/qemu_driver.c * src/qemu/qemu_process.c --- src/qemu/qemu_cgroup.c | 15 +++++++++ src/qemu/qemu_driver.c | 9 +++++ src/qemu/qemu_process.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+), 0 deletions(-)
diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index b39b5e1..f578f9e 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -340,6 +340,21 @@ int qemuSetupCgroup(struct qemud_driver *driver, vm->def->name); }
+ if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPU)) { + if (vm->def->cputune.shares != 0) {
Osier, we will need patch 5/6 before this to compile, some patch reordering required here.
hum, yes, thanks for the reminding. Regards Osier
LXC driver doesn't support vcpu affinity yet, so just need to modify it to support cpu shares. * src/lxc/lxc_controller.c (Setup cpu shares for domain) * src/lxc/lxc_driver.c (Update domain config when "lxcSetSchedulerParameters" is invoked) --- src/lxc/lxc_controller.c | 10 ++++++++++ src/lxc/lxc_driver.c | 2 ++ 2 files changed, 12 insertions(+), 0 deletions(-) diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index d2b113c..1837beb 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -115,6 +115,16 @@ static int lxcSetContainerResources(virDomainDefPtr def) } } + if (def->cputune.shares) { + rc = virCgroupSetCpuShares(cgroup, def->cputune.shares); + if (rc != 0) { + virReportSystemError(-rc, + _("Unable to set cpu shares for domain %s"), + def->name); + goto cleanup; + } + } + rc = virCgroupSetMemory(cgroup, def->mem.max_balloon); if (rc != 0) { virReportSystemError(-rc, diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 70e74d5..77f5f82 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -2321,6 +2321,8 @@ static int lxcSetSchedulerParameters(virDomainPtr domain, params[i].value.ul); goto cleanup; } + + vm->def->cputune.shares = params[i].value.ul; } ret = 0; -- 1.7.4
cputune XML: <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> </cputune> * src/conf/domain_conf.h (New struct for vcputune and vcpupin, declarations for new internal helper functions for cputune support) * src/conf/domain_conf.c (Update "virDomainDefParseXML" to parse, and "declaration" to build cputune xml, implementations for new internal helper functions). * src/libvirt_private.syms (Add new internal functions, and also "virAllocVar", as VIR_ALLOC_VAR is used in the patch series) --- src/conf/domain_conf.c | 278 ++++++++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 27 +++++ src/libvirt_private.syms | 4 + 3 files changed, 309 insertions(+), 0 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index b97c1f0..698f738 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -848,6 +848,22 @@ virDomainClockDefClear(virDomainClockDefPtr def) VIR_FREE(def->timers); } +static void +virDomainVcpupinDefFree(virDomainVcpupinDefPtr *def, + int nvcpupin) +{ + int i; + + if (!def || !nvcpupin) + return; + + for(i = 0; i < nvcpupin; i++) { + VIR_FREE(def[i]); + } + + VIR_FREE(def); +} + void virDomainDefFree(virDomainDefPtr def) { unsigned int i; @@ -936,6 +952,8 @@ void virDomainDefFree(virDomainDefPtr def) virCPUDefFree(def->cpu); + virDomainVcpupinDefFree(def->cputune.vcpupin, def->cputune.nvcpupin); + virSysinfoDefFree(def->sysinfo); if (def->namespaceData && def->ns.free) @@ -3522,6 +3540,196 @@ error: goto cleanup; } +/* Check if vcpupin with same vcpuid already exists. + * Return 1 if exists, 0 if not. */ +int +virDomainVcpupinIsDuplicate(virDomainVcpupinDefPtr *def, + int nvcpupin, + int vcpu) +{ + int i; + + if (!def || !nvcpupin) + return 0; + + for (i = 0; i < nvcpupin; i++) { + if (def[i]->vcpuid == vcpu) + return 1; + } + + return 0; +} + +virDomainVcpupinDefPtr +virDomainVcpupinFindByVcpu(virDomainVcpupinDefPtr *def, + int nvcpupin, + int vcpu) +{ + int i; + + if (!def || !nvcpupin) + return NULL; + + for (i = 0; i < nvcpupin; i++) { + if (def[i]->vcpuid == vcpu) + return def[i]; + } + + return NULL; +} + +int +virDomainVcpupinAdd(virDomainDefPtr def, + unsigned char *cpumap, + int maplen, + int vcpu) +{ + virDomainVcpupinDefPtr *vcpupin_list = NULL; + virDomainVcpupinDefPtr vcpupin = NULL; + char *cpumask = NULL; + int i; + + if (VIR_ALLOC_N(cpumask, VIR_DOMAIN_CPUMASK_LEN) < 0) { + virReportOOMError(); + goto cleanup; + } + + /* Reset cpumask to all 0s. */ + for (i = 0; i < VIR_DOMAIN_CPUMASK_LEN; i++) + cpumask[i] = 0; + + /* Convert bitmap (cpumap) to cpumask, which is byte map? */ + for (i = 0; i < maplen; i++) { + int cur; + + for (cur = 0; cur < 8; cur++) { + if (cpumap[i] & (1 << cur)) + cpumask[i * 8 + cur] = 1; + } + } + + /* No vcpupin exists yet. */ + if (!def->cputune.nvcpupin) { + if (VIR_ALLOC(vcpupin) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (VIR_ALLOC(vcpupin_list) < 0) { + virReportOOMError(); + VIR_FREE(vcpupin); + goto cleanup; + } + + vcpupin->vcpuid = vcpu; + vcpupin->cpumask = cpumask; + vcpupin_list[def->cputune.nvcpupin++] = vcpupin; + + def->cputune.vcpupin = vcpupin_list; + } else { + vcpupin_list = def->cputune.vcpupin; + + if (virDomainVcpupinIsDuplicate(vcpupin_list, + def->cputune.nvcpupin, + vcpu)) { + vcpupin = virDomainVcpupinFindByVcpu(vcpupin_list, + def->cputune.nvcpupin, + vcpu); + vcpupin->vcpuid = vcpu; + vcpupin->cpumask = cpumask; + } else { + if (VIR_ALLOC(vcpupin) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (VIR_REALLOC_N(vcpupin_list, def->cputune.nvcpupin + 1) < 0) { + virReportOOMError(); + VIR_FREE(vcpupin); + goto cleanup; + } + + vcpupin->vcpuid = vcpu; + vcpupin->cpumask = cpumask; + vcpupin_list[def->cputune.nvcpupin++] = vcpupin; + } + } + + return 0; + +cleanup: + VIR_FREE(cpumask); + return -1; +} + +/* Parse the XML definition for a vcpupin */ +static virDomainVcpupinDefPtr +virDomainVcpupinDefParseXML(const xmlNodePtr node, + xmlXPathContextPtr ctxt, + int maxvcpus, + int flags ATTRIBUTE_UNUSED) +{ + virDomainVcpupinDefPtr def; + xmlNodePtr oldnode = ctxt->node; + unsigned int vcpuid; + char *tmp = NULL; + int ret; + + if (VIR_ALLOC(def) < 0) { + virReportOOMError(); + return NULL; + } + + ctxt->node = node; + + ret = virXPathUInt("string(./@vcpu)", ctxt, &vcpuid); + if (ret == -2) { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("vcpu id must be an unsigned integer")); + goto error; + } else if (ret == -1) { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("can't parse vcpupin node")); + goto error; + } + + if (vcpuid >= maxvcpus) { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("vcpu id must be less than maxvcpus")); + goto error; + } + + def->vcpuid = vcpuid; + + tmp = virXMLPropString(node, "cpuset"); + + if (tmp) { + char *set = tmp; + int cpumasklen = VIR_DOMAIN_CPUMASK_LEN; + + if (VIR_ALLOC_N(def->cpumask, cpumasklen) < 0) { + virReportOOMError(); + goto error; + } + if (virDomainCpuSetParse((const char **)&set, + 0, def->cpumask, + cpumasklen) < 0) + goto error; + VIR_FREE(tmp); + } else { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("missing cpuset for vcpupin")); + goto error; + } + +cleanup: + ctxt->node = oldnode; + return def; + +error: + VIR_FREE(def); + goto cleanup; +} /* Parse the XML definition for a clock timer */ static virDomainTimerDefPtr @@ -5244,6 +5452,46 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps, VIR_FREE(tmp); } + /* Extract cpu tunables. */ + if (virXPathULong("string(./cputune/shares[1])", ctxt, + &def->cputune.shares) < 0) + def->cputune.shares = 0; + + if ((n = virXPathNodeSet("./cputune/vcpupin", ctxt, &nodes)) < 0) { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("cannot extract vcpupin nodes")); + goto error; + } + + if (n && VIR_ALLOC_N(def->cputune.vcpupin, n) < 0) + goto no_memory; + + if (n > def->maxvcpus) { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("vcpupin nodes must be less than maxvcpus")); + goto error; + } + + for (i = 0 ; i < n ; i++) { + virDomainVcpupinDefPtr vcpupin = NULL; + vcpupin = virDomainVcpupinDefParseXML(nodes[i], ctxt, def->maxvcpus, 0); + + if (!vcpupin) + goto error; + + if (virDomainVcpupinIsDuplicate(def->cputune.vcpupin, + def->cputune.nvcpupin, + vcpupin->vcpuid)) { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("duplicate vcpupin for same vcpu")); + VIR_FREE(vcpupin); + goto error; + } + + def->cputune.vcpupin[def->cputune.nvcpupin++] = vcpupin; + } + VIR_FREE(nodes); + n = virXPathNodeSet("./features/*", ctxt, &nodes); if (n < 0) goto error; @@ -7708,6 +7956,36 @@ char *virDomainDefFormat(virDomainDefPtr def, virBufferVSprintf(&buf, " current='%u'", def->vcpus); virBufferVSprintf(&buf, ">%u</vcpu>\n", def->maxvcpus); + if (def->cputune.shares || def->cputune.vcpupin) + virBufferAddLit(&buf, " <cputune>\n"); + + if (def->cputune.shares) + virBufferVSprintf(&buf, " <shares>%lu</shares>\n", + def->cputune.shares); + if (def->cputune.vcpupin) { + int i; + for (i = 0; i < def->cputune.nvcpupin; i++) { + virBufferVSprintf(&buf, " <vcpupin vcpu='%u' ", + def->cputune.vcpupin[i]->vcpuid); + + char *cpumask = NULL; + cpumask = virDomainCpuSetFormat(def->cputune.vcpupin[i]->cpumask, + VIR_DOMAIN_CPUMASK_LEN); + + if (cpumask == NULL) { + virDomainReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("failed to format cpuset for vcpupin")); + goto cleanup; + } + + virBufferVSprintf(&buf, "cpuset='%s'/>\n", cpumask); + VIR_FREE(cpumask); + } + } + + if (def->cputune.shares || def->cputune.vcpupin) + virBufferAddLit(&buf, " </cputune>\n"); + if (def->sysinfo) virDomainSysinfoDefFormat(&buf, def->sysinfo); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 30aeccc..53467cd 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1032,6 +1032,21 @@ int virDomainSnapshotObjUnref(virDomainSnapshotObjPtr snapshot); int virDomainSnapshotHasChildren(virDomainSnapshotObjPtr snap, virDomainSnapshotObjListPtr snapshots); +typedef struct _virDomainVcpupinDef virDomainVcpupinDef; +typedef virDomainVcpupinDef *virDomainVcpupinDefPtr; +struct _virDomainVcpupinDef { + int vcpuid; + char *cpumask; +}; + +int virDomainVcpupinIsDuplicate(virDomainVcpupinDefPtr *def, + int nvcpupin, + int vcpu); + +virDomainVcpupinDefPtr virDomainVcpupinFindByVcpu(virDomainVcpupinDefPtr *def, + int nvcpupin, + int vcpu); + /* Guest VM main configuration */ typedef struct _virDomainDef virDomainDef; typedef virDomainDef *virDomainDefPtr; @@ -1060,6 +1075,12 @@ struct _virDomainDef { int cpumasklen; char *cpumask; + struct { + unsigned long shares; + int nvcpupin; + virDomainVcpupinDefPtr *vcpupin; + } cputune; + /* These 3 are based on virDomainLifeCycleAction enum flags */ int onReboot; int onPoweroff; @@ -1196,6 +1217,12 @@ int virDomainDeviceVirtioSerialAddressIsValid(virDomainDeviceVirtioSerialAddress int virDomainDeviceInfoIsSet(virDomainDeviceInfoPtr info); void virDomainDeviceInfoClear(virDomainDeviceInfoPtr info); void virDomainDefClearPCIAddresses(virDomainDefPtr def); + +int virDomainVcpupinAdd(virDomainDefPtr def, + unsigned char *cpumap, + int maplen, + int vcpu); + void virDomainDefClearDeviceAliases(virDomainDefPtr def); typedef int (*virDomainDeviceInfoCallback)(virDomainDefPtr def, diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 66917ca..10b97ce 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -315,6 +315,9 @@ virDomainTimerTickpolicyTypeFromString; virDomainTimerTickpolicyTypeToString; virDomainTimerTrackTypeFromString; virDomainTimerTrackTypeToString; +virDomainVcpupinAdd; +virDomainVcpupinFindByVcpu; +virDomainVcpupinIsDuplicate; virDomainVideoDefFree; virDomainVideoDefaultRAM; virDomainVideoDefaultType; @@ -563,6 +566,7 @@ virVMOperationTypeToString; # memory.h virAlloc; virAllocN; +virAllocVar; virExpandN; virFree; virReallocN; -- 1.7.4
* docs/schemas/domain.rng (New definition for cputune xml) * docs/formatdomain.html.in (Docs for cputune xml) --- docs/formatdomain.html.in | 21 +++++++++++++++++++++ docs/schemas/domain.rng | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 0 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 84b1cab..7f8493b 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -288,6 +288,13 @@ <min_guarantee>65536</min_guarantee> </memtune> <vcpu cpuset="1-4,^3,6" current="1">2</vcpu> + <cputune> + <vcpupin vcpu="0" cpuset="1-4,^2"> + <vcpupin vcpu="1" cpuset="0,1"> + <vcpupin vcpu="2" cpuset="2,3"> + <vcpupin vcpu="3" cpuset="0,4"> + <cpushares>2048</cpushares> + </cputune> ...</pre> <dl> @@ -345,6 +352,20 @@ be used to specify whether fewer than the maximum number of virtual CPUs should be enabled. </dd> + <dt><code>cputune</code></dt> + <dd> The optional <code>cputune</code> element provides details + regarding the cpu tunable parameters for the domain.</dd> + <dt><code>vcpupin</code></dt> + <dd> The optional <code>vcpupin</code> specifies which of host physical + CPUS the domain VCPU will be pinned to. If this is ommited, each VCPU + pinned to all the physical CPUS by default. It contains two required + attributes, the attribute <code>vcpu</vcpu> specifies vcpu id, and the + attribute <code>cpuset</code> is same as attribute <code>cpuset</code> + of element <code>vcpu</code>. NB, Only qemu driver supports</dd> + <dt><code>cpushares</code></dt> + <dd> The optional <code>cpushares</code> specifies the proportional + weighted share for the domain. If this is ommited, it defaults + to the OS provided defaults.</dd> </dl> <h3><a name="elementsCPU">CPU model and topology</a></h3> diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng index 8b215f3..e4430d0 100644 --- a/docs/schemas/domain.rng +++ b/docs/schemas/domain.rng @@ -363,6 +363,27 @@ <ref name="countCPU"/> </element> </optional> + + <!-- All the cpu related tunables would go in the cputune --> + <optional> + <element name="cputune"> + <optional> + <element name="shares"> + <ref name="cpushares"/> + </element> + </optional> + <zeroOrMore> + <element name="vcpupin"> + <attribute name="vcpu"> + <ref name="vcpuid"/> + </attribute> + <attribute name="cpuset"> + <ref name="cpuset"/> + </attribute> + </element> + </zeroOrMore> + </element> + </optional> </interleave> </define> <define name="clock"> @@ -2193,6 +2214,17 @@ <param name="minInclusive">1</param> </data> </define> + <define name="vcpuid"> + <data type="unsignedShort"> + <param name="pattern">[0-9]+</param> + </data> + </define> + <define name="cpushares"> + <data type="unsignedInt"> + <param name="pattern">[0-9]+</param> + <param name="maxInclusive">262144</param> + </data> + </define> <define name="hostName"> <data type="string"> <param name="pattern">[a-zA-Z0-9\.\-]+</param> -- 1.7.4
On Sun, 27 Feb 2011 22:39:15 +0800, Osier Yang <jyang@redhat.com> wrote:
These are pretty rough patches, something like a draft, I beleive there must be many problems, please review it heavily, :-)
New XML: <cputune> <shares>2048</shares> <vcpupin vcpu='0' cpuset='0-4,^1'/> <vcpupin vcpu='1' cpuset='1,3'/> <vcpupin vcpu='2' cpuset='0,2'/> </cputune>
"shares" is to define the the proportional weighted cpu share for the domain.
"vcpupin" is to define the cpu affinities of vcpus, it will not be displayed if one doesn't specify it explicitly in XML or set the cpu affinites for vcpu via "vcpupin", means there will be no vcpupin element in domain XML by default, and the constraints are:
- Error if one specify entries more than the count of maxvcpus. - Error when one specify entries for same vcpu. - Error if value of attribute "vcpu" is more than count of "maxvcpus - 1".
Attribute "cpuset" works same as "cpuset" of element "vcpu", reuse the codes for parsing and formating value of "cpuset" of element "vcpu".
NB, the idea to add persistent XML for "cpushares" is from "Nikunj A. Dadhania": https://www.redhat.com/archives/libvir-list/2011-January/msg01183.html
Thanks for picking the "cpushares" in this series. Regards Nikunj
Could anyone help review this series, Dan and Eric? :-) Thanks. Regards Osier 于 2011年02月27日 22:39, Osier Yang 写道:
These are pretty rough patches, something like a draft, I beleive there must be many problems, please review it heavily, :-)
New XML: <cputune> <shares>2048</shares> <vcpupin vcpu='0' cpuset='0-4,^1'/> <vcpupin vcpu='1' cpuset='1,3'/> <vcpupin vcpu='2' cpuset='0,2'/> </cputune>
"shares" is to define the the proportional weighted cpu share for the domain.
"vcpupin" is to define the cpu affinities of vcpus, it will not be displayed if one doesn't specify it explicitly in XML or set the cpu affinites for vcpu via "vcpupin", means there will be no vcpupin element in domain XML by default, and the constraints are:
- Error if one specify entries more than the count of maxvcpus. - Error when one specify entries for same vcpu. - Error if value of attribute "vcpu" is more than count of "maxvcpus - 1".
Attribute "cpuset" works same as "cpuset" of element "vcpu", reuse the codes for parsing and formating value of "cpuset" of element "vcpu".
NB, the idea to add persistent XML for "cpushares" is from "Nikunj A. Dadhania": https://www.redhat.com/archives/libvir-list/2011-January/msg01183.html
I rebased it and include it together in this patch series.
[PATCH 1/6] cputune: new tests for testing cputune xml [PATCH 2/6] cputune: support cputune for xend driver [PATCH 3/6] cputune: support cputune for qemu driver [PATCH 4/6] cputune: support cputune xml for LXC driver [PATCH 5/6] cputune: support cputune xml config [PATCH 6/6] cputune: Add docs xml schema for cputune xml
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
participants (2)
-
Nikunj A. Dadhania -
Osier Yang