9:14 a.m.
1) Currently the NULL virDomainPtr (meaning Domain0) is only specified in
the 3 virDomain*Memory functions (GetMax, SetMax, Set). The other
virDomain* functions need the actual virDomainPtr for Domain0. What is the reason for this
Memory-specific
option? Is it applicable to virtual CPUs?
2) For "xm vcpu-list" equivalent function (suggested virDomainGetVcpus) I
propose to return in a new structure all the information provided by
Xen Daemon. I am trying the following addition in libvirt.h :
typedef struct _virVcpuInfo virVcpuInfo;
struct _virVcpuInfo {
int number; /* virtual CPU number */
unsigned char state; /* 'r'=running, 'b'=blocked,
'p'=offline
*/
unsigned long long cpuTime; /* CPU time used, in nanoseconds */
int cpu; /* last real CPU number allocated */
char cpumap[128]; /* affinity map of real CPUs which can be
allocated */
};
typedef virVcpuInfo *virVcpuInfoPtr;
For cpumap, the xm command displays a string "x,y-z..." more compact than
the list of all possible CPUs "x y y+1 y+2...z..." provided by xend. But
the routines for this reformatting are not easy, so I propose to keep the
xend format in virCpuInfo. What is your opinion?
3) For "xm vcpu-pin" equivalent function (suggested virDomainPinVcpu) I propose
a CPU list argument in same format as virCpuInfo cpumap. Do
you agree?
Veuillez répondre à veillard(a)redhat.com
Pour : michel.ponceau(a)bull.net
cc : libvir-list(a)redhat.com
Objet : Re: [Libvir] Proposal for a new Libvirt function: change number of virtual
CPUs
On Fri, Jun 23, 2006 at 05:48:40PM +0200, michel.ponceau(a)bull.net wrote:
For our administration, we need to dynamically reduce the number of
virtual
CPUs in each domain, in same way as Xen command "xm
vcpu-set". It seems
that no Libvirt function is currently provided for that purpose. I
successfully tried some simple changes in libvirt.c an xend_internal.c,
mainly the following:
snprintf(buf, sizeof(buf), "%d", vcpus);
return xend_op(domain->conn, domain->name, "op",
"set_vcpus",
"vcpus",
buf, NULL);
We suggest to add this new function "virDomainSetVcpus" to Libvirt.
Hum, let's see. We expose the number of virtual CPUs in the info
structure
so yes that's something which is part of what we expose. So that would be
/**
* virDomainSetVcpus:
* @domain: a domain object or NULL
* @nvcpus: the new number of virtual CPUs for this domain
*
* Dynamically change the number of virtual CPUs used by the domain.
* Note that this call may fail if the underlying virtualization
hypervisor
* does not support it or if growing the number is arbitrary limited.
* This function requires priviledged access to the hypervisor.
*
* Returns 0 in case of success and -1 in case of failure.
*/
int virDomainSetVcpus(virDomainPtr domain, unsigned int nvcpus) {
}
I have not tried the hypervisor and xen store accesses for this
function.
Do you think they would be necessary?
It would be preferable to go though Xend first at least. I don't think
xenstore is a good idea at this point. Now for the direct hypervisor, I'm
uncertain, it could be a good idea to provide redundancy over xend, I
assume it should not be hard to do that at the hypervisor level, the risk
is introducing incoherence between what xend thinks and what the
hypervisor
atually implement :-)
I am also working on two other functions to reproduce "xm
vcpu-pin" and
"xm
vcpu-list" from Libvirt.
vcpu-list might be a bit subtle as an API design level. Better not
allocate
the array in libvirt, but have the user pass an array. Also how much
information
would you extract per CPU, just on/off ?
Daniel
--
Daniel Veillard | Red Hat http://redhat.com/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
9:22 a.m.
On Mon, Jun 26, 2006 at 04:14:59PM +0200, michel.ponceau(a)bull.net wrote:
2) For "xm vcpu-list" equivalent function (suggested
virDomainGetVcpus) I propose to return in a new structure all the information provided by
Xen Daemon. I am trying the following addition in libvirt.h :
typedef struct _virVcpuInfo virVcpuInfo;
struct _virVcpuInfo {
int number; /* virtual CPU number */
unsigned char state; /* 'r'=running, 'b'=blocked,
'p'=offline
*/
unsigned long long cpuTime; /* CPU time used, in nanoseconds */
int cpu; /* last real CPU number allocated */
char cpumap[128]; /* affinity map of real CPUs which can be
allocated */
};
typedef virVcpuInfo *virVcpuInfoPtr;
For cpumap, the xm command displays a string "x,y-z..." more compact than
the list of all possible CPUs "x y y+1 y+2...z..." provided by xend. But
the routines for this reformatting are not easy, so I propose to keep the
xend format in virCpuInfo. What is your opinion?
That is a rather unpleasent format to expose to applications - each application
will be forced to write their own (buggy) parser to extract useful information
from this string. It really needs to be sructured in a format which can be
directly interpreted, without requiring parsing.
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
9:58 a.m.
On Mon, Jun 26, 2006 at 03:22:15PM +0100, Daniel P. Berrange wrote:
On Mon, Jun 26, 2006 at 04:14:59PM +0200, michel.ponceau(a)bull.net
wrote:
> 2) For "xm vcpu-list" equivalent function (suggested virDomainGetVcpus) I
propose to return in a new structure all the information provided by
> Xen Daemon. I am trying the following addition in libvirt.h :
> typedef struct _virVcpuInfo virVcpuInfo;
> struct _virVcpuInfo {
> int number; /* virtual CPU number */
> unsigned char state; /* 'r'=running, 'b'=blocked,
'p'=offline
> */
> unsigned long long cpuTime; /* CPU time used, in nanoseconds */
> int cpu; /* last real CPU number allocated */
> char cpumap[128]; /* affinity map of real CPUs which can be
> allocated */
> };
> typedef virVcpuInfo *virVcpuInfoPtr;
>
> For cpumap, the xm command displays a string "x,y-z..." more compact than
> the list of all possible CPUs "x y y+1 y+2...z..." provided by xend. But
> the routines for this reformatting are not easy, so I propose to keep the
> xend format in virCpuInfo. What is your opinion?
That is a rather unpleasent format to expose to applications - each application
will be forced to write their own (buggy) parser to extract useful information
from this string. It really needs to be sructured in a format which can be
directly interpreted, without requiring parsing.
Hum, I missed that. Agreed, do not use a string, we should use an array.
The virVcpuInfo structure will be allocated by the client, make sure it is
extensible to whatever the need might be in the future. Say you have a
box with 1024 CPU, ideally the client should be able to call libvirt and
specify any combination of those. Also let's check what is the low level
hypervisor interface there.
Daniel
--
Daniel Veillard | Red Hat http://redhat.com/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
9:43 a.m.
On Mon, Jun 26, 2006 at 04:14:59PM +0200, michel.ponceau(a)bull.net wrote:
1) Currently the NULL virDomainPtr (meaning Domain0) is only
specified in
the 3 virDomain*Memory functions (GetMax, SetMax, Set). The other
virDomain* functions need the actual virDomainPtr for Domain0. What is the reason for
this Memory-specific
option?
I think it's problem. We should always require a virDomainPtr, I actually
think the shortcut is not implemented.
I wonder if it would be better to just remove it that would be cleaner,
however when think about other kind of virtualization, where the main
os would not be represented as a domain, but still with memory management,
in that case that should be kept.
Is it applicable to virtual CPUs?
Hum, similary, what if the main OS CPU usage could be set up even if
not mapped as a real domain, maybe this should be kept for the vCPU APIs
too, yes, a bit messy but more generic.
2) For "xm vcpu-list" equivalent function (suggested
virDomainGetVcpus) I propose to return in a new structure all the information provided by
Xen Daemon. I am trying the following addition in libvirt.h :
Remember to change libvirt.h.in as libvirt.h is generated by configure.
typedef struct _virVcpuInfo virVcpuInfo;
struct _virVcpuInfo {
int number; /* virtual CPU number */
unsigned int rather
unsigned char state; /* 'r'=running,
'b'=blocked, 'p'=offline
*/
Not too fond of using a letter there. I would prefer a number and a clean enum
unsigned long long cpuTime; /* CPU time used, in nanoseconds */
int cpu; /* last real CPU number allocated */
That's a bit fishy to me, how do you expect to use that information ?
char cpumap[128]; /* affinity map of real CPUs which
can be
allocated */
so we know we will never have libvirt used for a virtualization platform
which has no more than 256 physical CPU to offer ? Remember I don't want that
API to be limited to Xen and to me if we put this in the API this must be
future proof. I would at least grow the CPU map to 256 entries at the API
level. Oh and use unsigned char to go 0-255.
};
one last comment about packing the structure: it may be intersting to
reorganize fields in that arry to help,
int, int, long long, char, char[] looks better to me than the current
organization.
Another note is about access, do you really want to get those just from
xend or also from the hypervisor ? The hypervisor might be way cheaper to
reach and more accurate. Hopefully that structure still make sense with
the associated hypervisor informations could you look up the associated
hypervisor call and double check ?
typedef virVcpuInfo *virVcpuInfoPtr;
For cpumap, the xm command displays a string "x,y-z..." more compact than
the list of all possible CPUs "x y y+1 y+2...z..." provided by xend. But
the routines for this reformatting are not easy, so I propose to keep the
xend format in virCpuInfo. What is your opinion?
I never played with mapping (well I don't have anymore SMP box to play
locally), could you explain, I'm not sure I follow :-) . If what xm does
make sense, I don't see why introducing a new way is an improvement, but
I can hardly juge myself.
3) For "xm vcpu-pin" equivalent function (suggested
virDomainPinVcpu) I propose a CPU list argument in same format as virCpuInfo cpumap. Do
you agree?
I have the same worries about the hard constraints put on the number of
CPUs at the API level.
Daniel
--
Daniel Veillard | Red Hat http://redhat.com/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
11:20 a.m.
On Mon, Jun 26, 2006 at 10:43:24AM -0400, Daniel Veillard wrote:
On Mon, Jun 26, 2006 at 04:14:59PM +0200, michel.ponceau(a)bull.net
wrote:
> 1) Currently the NULL virDomainPtr (meaning Domain0) is only specified in
> the 3 virDomain*Memory functions (GetMax, SetMax, Set). The other
> virDomain* functions need the actual virDomainPtr for Domain0. What is the reason
for this Memory-specific
> option?
I think it's problem. We should always require a virDomainPtr, I actually
think the shortcut is not implemented.
I wonder if it would be better to just remove it that would be cleaner,
however when think about other kind of virtualization, where the main
os would not be represented as a domain, but still with memory management,
in that case that should be kept.
> Is it applicable to virtual CPUs?
Hum, similary, what if the main OS CPU usage could be set up even if
not mapped as a real domain, maybe this should be kept for the vCPU APIs
too, yes, a bit messy but more generic.
> 2) For "xm vcpu-list" equivalent function (suggested virDomainGetVcpus) I
propose to return in a new structure all the information provided by
> Xen Daemon. I am trying the following addition in libvirt.h :
Remember to change libvirt.h.in as libvirt.h is generated by configure.
> typedef struct _virVcpuInfo virVcpuInfo;
> struct _virVcpuInfo {
> int number; /* virtual CPU number */
unsigned int rather
> unsigned char state; /* 'r'=running, 'b'=blocked,
'p'=offline
> */
Not too fond of using a letter there. I would prefer a number and a clean enum
> unsigned long long cpuTime; /* CPU time used, in nanoseconds */
> int cpu; /* last real CPU number allocated */
That's a bit fishy to me, how do you expect to use that information ?
I guess that's to allow the 'CPU' column in the following output to be
generated
# xm vcpu-list
Name ID VCPU CPU State Time(s) CPU Affinity
Domain-0 0 0 0 -b- 636.8 any cpu
Domain-0 0 1 1 -b- 396.8 any cpu
Domain-0 0 2 2 r-- 319.5 any cpu
Domain-0 0 3 3 -b- 369.4 any cpu
xperf1 1 0 3 -b- 545.6 any cpu
xperf2 3 0 2 -b- 20.0 any cpu
xperf2 3 1 3 -b- 18.2 any cpu
xperf2 3 2 0 -b- 18.6 any cpu
xperf2 3 3 1 -b- 17.8 any cpu
> char cpumap[128]; /* affinity map of real CPUs
which can be
> allocated */
so we know we will never have libvirt used for a virtualization platform
which has no more than 256 physical CPU to offer ? Remember I don't want that
API to be limited to Xen and to me if we put this in the API this must be
future proof. I would at least grow the CPU map to 256 entries at the API
level. Oh and use unsigned char to go 0-255.
I expect we really need a bit-set in that field, one bit per CPU that the
VCPU can be scheduled on. One thought though, is do we want to mix up
configuration data (VCPU/VPU affinity) with performance metrics (state,
runtime, last cpu) ?
Might it make more sense to have a single API for retrieving the entire
VCPU <-> VPU mapping for all CPUs in a single call - basically return a
2-d matrix with VCPU in one axis and CPU in the other access, and the
elements are basically a bit 1/0
Another note is about access, do you really want to get those just
from
xend or also from the hypervisor ? The hypervisor might be way cheaper to
reach and more accurate. Hopefully that structure still make sense with
the associated hypervisor informations could you look up the associated
hypervisor call and double check ?
We definitely want to get this info via the hypervisor if its available,
since the performance hit in making the multiple RPC calls will be far
too great to make this usable otherwise.
> 3) For "xm vcpu-pin" equivalent function (suggested
virDomainPinVcpu) I propose a CPU list argument in same format as virCpuInfo cpumap. Do
> you agree?
I have the same worries about the hard constraints put on the number of
CPUs at the API level.
Whatever we do, should match up in its data structure with the API for
retrieving CPU pining
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
6537
days inactive
6537
days old
4 comments
3 participants
participants (3)
-
Daniel P. Berrange -
Daniel Veillard -
michel.ponceau＠bull.net