Re: [libvirt] TSC frequency configuration & invtsc migration (was Re: [PATCH 4/4] kvm: Allow migration with invtsc)

On Thu, Jan 05, 2017 at 08:48:32AM -0200, Marcelo Tosatti wrote: > On Wed, Jan 04, 2017 at 11:36:31PM -0200, Eduardo Habkost wrote: > > I am considering two possible scenarios below. You probably have > > another scenario in mind, so it would be useful if you could > > describe it so we can consider possible solutions. > > > > > > Scenario 1: > > > > You have two hosts: A and B, both with TSC scaling. They have > > different frequencies. The VM can be started on either one of > > them, and can be migrated to either one depending on the policy > > of management software. > > > > Maybe B is a backup host, the VM is expected to run most times on > > host A, and we want to use the TSC frequency from host A every > > time. Maybe it's the opposite and we want to use the frequency of > > B. Maybe we want to use the lowest frequency of both, maybe the > > highest one. We (QEMU developers) can recommend policy to libvirt > > developers, but a given QEMU process doesn't have information to > > decide what's best. > > I can't see any practical scenario here, you will always want > to start with TSC frequency of the host where the VM was started. > > If i am mistaken, please describe a practical case. > > (If a practical scenario comes up, and there is a use-case > for setting the TSC frequency on startup, lets say > a Windows VM which fails to boot if the TSC frequency > is too high, then it should be supported... But the > only known scenario to me, applying to 99.999% of cases, > is to expose the TSC frequency where the guest booted at). I don't have any specific case: my point is that I can't tell what's the best frequency if I don't know where the hosts are expected to be migrated to. You claim that using the starting host frequency is the best option on the vast majority of cases. Maybe it's true, and it would be a good default. The only problem is that this default affects migratability:

> > Scenario 2: > > > > Host A has TSC scaling, host B doesn't have TSC scaling. We want > > to be able to start the VM on host A, and migrate to B. In this > > case, the only possible solution is to use B's frequency when > > starting the VM. The QEMU process doesn't have enough information > > to make that decision. > > That is a good point. But again, its a special case and > should be supported by -cpu xxx,tsc-frequency=zzzz. > > However, for the vast majority of 99.999% cases, the issue > can be handled entirely in QEMU, without libvirt involvement, > and without adding extra steps to the management software. I agree it should cover most cases. The only problem here is that it can break migration in unexpected ways. Then my point is: assuming that libvirt will prefer to require explicit TSC frequency configuration to enable invtsc migration (instead of getting unpredictable migration compatibility), is the added complexity to migration code worth the effort, if choosing an explicit frequency is safer and more predictable? I believe this is where we disagree.