Messages in this thread Patch in this message | | | Date | Mon, 7 Nov 2022 12:00:44 +0000 | Subject | Re: [External] Re: [v2 0/6] KVM: arm64: implement vcpu_is_preempted check | From | Usama Arif <> |
| |
On 06/11/2022 16:35, Marc Zyngier wrote: > On Fri, 04 Nov 2022 06:20:59 +0000, > Usama Arif <usama.arif@bytedance.com> wrote: >> >> This patchset adds support for vcpu_is_preempted in arm64, which >> allows the guest to check if a vcpu was scheduled out, which is >> useful to know incase it was holding a lock. vcpu_is_preempted can >> be used to improve performance in locking (see owner_on_cpu usage in >> mutex_spin_on_owner, mutex_can_spin_on_owner, rtmutex_spin_on_owner >> and osq_lock) and scheduling (see available_idle_cpu which is used >> in several places in kernel/sched/fair.c for e.g. in wake_affine to >> determine which CPU can run soonest): > > [...] > >> pvcy shows a smaller overall improvement (50%) compared to >> vcpu_is_preempted (277%). Host side flamegraph analysis shows that >> ~60% of the host time when using pvcy is spent in kvm_handle_wfx, >> compared with ~1.5% when using vcpu_is_preempted, hence >> vcpu_is_preempted shows a larger improvement. > > And have you worked out *why* we spend so much time handling WFE? > > M.
Its from the following change in pvcy patchset:
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index e778eefcf214..915644816a85 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -118,7 +118,12 @@ static int kvm_handle_wfx(struct kvm_vcpu *vcpu) }
if (esr & ESR_ELx_WFx_ISS_WFE) { - kvm_vcpu_on_spin(vcpu, vcpu_mode_priv(vcpu)); + int state; + while ((state = kvm_pvcy_check_state(vcpu)) == 0) + schedule(); + + if (state == -1) + kvm_vcpu_on_spin(vcpu, vcpu_mode_priv(vcpu)); } else { if (esr & ESR_ELx_WFx_ISS_WFxT) vcpu_set_flag(vcpu, IN_WFIT);
If my understanding is correct of the pvcy changes, whenever pvcy returns an unchanged vcpu state, we would schedule to another vcpu. And its the constant scheduling where the time is spent. I guess the affects are much higher when the lock contention is very high. This can be seem from the pvcy host side flamegraph as well with (~67% of the time spent in the schedule() call in kvm_handle_wfx), For reference, I have put the graph at: https://uarif1.github.io/pvlock/perf_host_pvcy_nmi.svg
Thanks, Usama
>
| |