Messages in this thread | | | Date | Mon, 13 Nov 2017 11:46:34 +0100 | From | Peter Zijlstra <> | Subject | Re: [PATCH v4 2/4] KVM: X86: Add paravirt remote TLB flush |
| |
On Mon, Nov 13, 2017 at 04:26:57PM +0800, Wanpeng Li wrote: > 2017-11-13 16:04 GMT+08:00 Peter Zijlstra <peterz@infradead.org>:
> > So if at this point a vCPU gets preempted we'll still spin-wait for it, > > which is sub-optimal. > > > > I think we can come up with something to get around that 'problem' if > > indeed it is a problem. But we can easily do that as follow up patches. > > Just let me know if you think its worth spending more time on. > > You can post your idea, it is always smart. :) Then we can evaluate > the complexity and gains.
I'm not sure I have a fully baked idea just yet, but the general idea would be something like:
- switch (back) to a dedicated TLB invalidate IPI
- introduce KVM_VCPU_IPI_PENDING
- change flush_tlb_others() into something like:
for_each_cpu(cpu, flushmask) { src = &per_cpu(steal_time, cpu); state = READ_ONCE(src->preempted); do { if (state & KVM_VCPU_PREEMPTED) { if (try_cmpxchg(&src->preempted, &state, state | KVM_VCPU_SHOULD_FLUSH)) { __cpumask_clear_cpu(cpu, flushmask); break; } } } while (!try_cmpxchg(&src->preempted, &state, state | KVM_VCPU_IPI_PENDING)); }
apic->send_IPI_mask(flushmask, CALL_TLB_VECTOR);
for_each_cpu(cpu, flushmask) { src = &per_cpu(steal_time, cpu); smp_cond_load_acquire(&src->preempted, !(VAL & KVM_VCPU_IPI_PENDING); }
- have the TLB invalidate handler do something like:
state = READ_ONCE(src->preempted); if (!(state & KVM_VCPU_IPI_PENDING)) return;
local_flush_tlb();
do { } while (!try_cmpxchg(&src->preempted, &state, state & ~KVM_VCPU_IPI_PENDING));
- then at VMEXIT time do something like:
state = READ_ONCE(src->preempted); do { if (!(state & KVM_VCPU_IPI_PENDING)) break; } while (!try_cmpxchg(&src->preempted, state, (state & ~KVM_VCPU_IPI_PENDING) | KVM_VCPU_SHOULD_FLUSH));
and clear any possible pending TLB_VECTOR in the guest state to avoid raising that IPI spuriously on enter again.
This way the preemption will clear the IPI_PENDING and the flush_others() wait loop will terminate.
| |