On 06/02/2015 07:56 AM, Qiao, Liyong wrote: > Hi Eric > Thanks for replying the mail, replied in lines. > >> +virdomainMigrateGetParameters(virDomainPtr domain, >> + int *level, >> + int *threads, >> + int *dthreads, >> + int flags) >> + > I'd rather we used virTypedParameters, to make it easier to use the same API to pass ALL future possible tuning parameters, rather than just hard-coding to only the parameters of this one feature. > > Okay ,that sound good, but if virTypedParameters can be passed to qemu_driver such as qemu_monitor_json.c qemu_monitor_text.c ? [Your quoting style makes it very hard to distinguish original text from added text. Please consider changing your outgoing mail process to ensure that you add another level of quoting to all previous text so that your added text is the only unquoted text. Also, configure your mailer to wrap long lines.]

Use existing API for a guide - for example, virDomainSetBlockIoTune takes virTypedParamters, as well as defines a specific set of parameters that it will understand. The set of parameters can be enlarged without needing a new API (good for backporting features without requiring a .so version bump), but for any given libvirt version, the set of features understood is finite and limited to what you can translate to the QMP call. So qemu_driver.c would take the virTypedParameters, reject the ones it does not understand, and convert the ones it does understand into the 'int level, int threads, int dthreads' parameters used in qemu_monitor_json.c where you drive the actual QMP command with fixed parameters and types.

> If we think that it is worth always turning on both compression styles simultaneously, then reuse the bit. Otherwise, we need a different bit, and users can choose which (or both) of the two compression styles as desired. > > +1 for reuse compressed, and we can set compress-level, compress-threads, compress-dthreads by virdomainMigrateSetParameters(maybe some new virsh command--- migrate-setparameter) > But how can we passing these parameter when we using `virsh migrate `, is there any parameter we can use in 'virsh migrate' command ? > Can you point me out ? The underlying API already has a form that takes virTypedParameters (see virDomainMigrate3()), so your first task is to figure out how to extend the API to expose new typed parameters for your new migration tunings. Once that is done, then you can worry about how to tweak the 'virsh migrate' client to pass in those new parameters.

Eric, Thank you for reply. Is it necessary to expose these two APIs to user application? + virdomainMigrateSetCapabilities + virdomainMigrateGetCapabilities For qemu , the migration capabilities are "xbzrle", "rdma-pin-all", "auto-converge", "zero-blocks" and "compress".

On 2015年06月04日 20:01, Jiri Denemark wrote: > On Wed, Jun 03, 2015 at 15:13:17 +0000, Feng, Shaohe wrote: >> Eric, >> Thank you for reply. >> >> Is it necessary to expose these two APIs to user application? >> + virdomainMigrateSetCapabilities >> + virdomainMigrateGetCapabilities >> >> For qemu , the migration capabilities are "xbzrle", "rdma-pin-all", >> "auto-converge", >> "zero-blocks" and "compress". > No, these capabilities are either turned on/off based on flags passed to > virDomainMigrate3 API. I think we need to call qemuMonitorGetMigrationCapability when

qemuMigrationSetCompression if the source/dest node doesn't support 'compress' capability, we need to stop migration if flags contain VIR_MIGRATE_COMPRESSED (currently it stands for xbzrle, and Eric's previous mail suggest we share this flag with 'compress')

-----Original Message----- From: Qiao, Liyong Sent: Wednesday, June 3, 2015 7:49 AM To: Eric Blake; Feng, Shaohe; libvir-list(a)redhat.com Cc: Bhandaru, Malini K; Ding, Jian-feng; Chylinski, Arek; Koniszewski, Pawel; Li, Liang Z; berrange(a)redhat.com; veillard(a)redhat.com Subject: Re: [RFC] libvirt support multi-thread compression for live migration On 2015年06月03日 01:02, Eric Blake wrote: > On 06/02/2015 07:56 AM, Qiao, Liyong wrote: >> Hi Eric >> Thanks for replying the mail, replied in lines. >> >>> +virdomainMigrateGetParameters(virDomainPtr domain, >>> + int *level, >>> + int *threads, >>> + int *dthreads, >>> + int flags) >>> + >> I'd rather we used virTypedParameters, to make it easier to use the same API to pass ALL future possible tuning parameters, rather than just hard-coding to only the parameters of this one feature. >> >> Okay ,that sound good, but if virTypedParameters can be passed to qemu_driver such as qemu_monitor_json.c qemu_monitor_text.c ? > [Your quoting style makes it very hard to distinguish original text > from added text. Please consider changing your outgoing mail process > to ensure that you add another level of quoting to all previous text > so that your added text is the only unquoted text. Also, configure > your mailer to wrap long lines.] hi Eric, sorry about the inconvenient mail client, I switch outlook to thunder bird, hopes it will be better. > Use existing API for a guide - for example, virDomainSetBlockIoTune > takes virTypedParamters, as well as defines a specific set of > parameters that it will understand. The set of parameters can be > enlarged without needing a new API (good for backporting features > without requiring a .so version bump), but for any given libvirt > version, the set of features understood is finite and limited to what > you can translate to the QMP call. So qemu_driver.c would take the > virTypedParameters, reject the ones it does not understand, and > convert the ones it does understand into the 'int level, int threads, > int dthreads' parameters used in qemu_monitor_json.c where you drive > the actual QMP command with fixed parameters and types. ah, that helps, thanks for kindly supporting, we will think a bit more to how implement this API (taken virTypedParamters as parameter) > > >> If we think that it is worth always turning on both compression styles simultaneously, then reuse the bit. Otherwise, we need a different bit, and users can choose which (or both) of the two compression styles as desired. >> >> +1 for reuse compressed, and we can set compress-level,

>>> Consider this scenario : one of the host/hypervisor are high cpu/memory usage, >>> Cloud Tool, such as Openstack can make a strategy that do not compression >>> by multi-thread, for high cpu usage, they just want to use "xbzrle". >>> >>> Is this scenario reasonable?

>> +compress-threads, compress-dthreads by >> +virdomainMigrateSetParameters(maybe some new virsh command--- >> +migrate-setparameter) >> But how can we passing these parameter when we using `virsh migrate `, is there any parameter we can use in 'virsh migrate' command ? >> Can you point me out ? > The underlying API already has a form that takes virTypedParameters > (see virDomainMigrate3()), so your first task is to figure out how to > extend the API to expose new typed parameters for your new migration tunings. > Once that is done, then you can worry about how to tweak the 'virsh > migrate' client to pass in those new parameters. > -- BR, Eli(Li Yong)Qiao

On Tue, Jun 02, 2015 at 11:02:27 -0600, Eric Blake wrote: > On 06/02/2015 07:56 AM, Qiao, Liyong wrote: >> Hi Eric >> Thanks for replying the mail, replied in lines. >> >>> +virdomainMigrateGetParameters(virDomainPtr domain, >>> + int *level, >>> + int *threads, >>> + int *dthreads, >>> + int flags) >>> + >> I'd rather we used virTypedParameters, to make it easier to use the same API to pass ALL future possible tuning parameters, rather than just hard-coding to only the parameters of this one feature. >> >> Okay ,that sound good, but if virTypedParameters can be passed to qemu_driver such as qemu_monitor_json.c qemu_monitor_text.c ? > [Your quoting style makes it very hard to distinguish original text from > added text. Please consider changing your outgoing mail process to > ensure that you add another level of quoting to all previous text so > that your added text is the only unquoted text. Also, configure your > mailer to wrap long lines.] > > Use existing API for a guide - for example, virDomainSetBlockIoTune > takes virTypedParamters, as well as defines a specific set of parameters > that it will understand. And do we actually need to changed these migration parameters on the fly when migration is already running. I can imagine we would need to do so for some parameters but multithreaded compression sounds like something that needs to be configured when migration starts and there's nothing to be tuned on the fly. If this is the case, I think we should just add these new parameters to virDomainMigrate3 instead of requiring another API to be called to actually configure multithreaded compression. The separate API makes it a bit harder to repeat migration (which previously failed) with a different parameters (e.g., without multithreaded compression).

Jirka