lkml.org 
[lkml]   [2022]   [Jun]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH] srcu: Reduce blocking agressiveness of expedited grace periods further
On Tue, 28 Jun 2022 10:17:24 +0100,
Neeraj Upadhyay <quic_neeraju@quicinc.com> wrote:
>
>
>
> On 6/28/2022 2:32 PM, Marc Zyngier wrote:
> > On Mon, 27 Jun 2022 13:37:06 +0100,
> > Neeraj Upadhyay <quic_neeraju@quicinc.com> wrote:
> >>
> >> Commit 640a7d37c3f4 ("srcu: Block less aggressively for expedited
> >> grace periods") highlights a problem where aggressively blocking
> >> SRCU expedited grace periods, as was introduced in commit
> >> 282d8998e997 ("srcu: Prevent expedited GPs and blocking readers
> >> from consuming CPU"), introduces ~2 minutes delay to the overall
> >> ~3.5 minutes boot time, when starting VMs with "-bios QEMU_EFI.fd"
> >> cmdline on qemu, which results in very high rate of memslots
> >> add/remove, which causes > ~6000 synchronize_srcu() calls for
> >> kvm->srcu SRCU instance.
> >>
> >> Below table captures the experiments done by Zhangfei Gao, Shameer,
> >> to measure the boottime impact with various values of non-sleeping
> >> per phase counts, with HZ_250 and preemption enabled:
> >>
> >> +──────────────────────────+────────────────+
> >> | SRCU_MAX_NODELAY_PHASE | Boot time (s) |
> >> +──────────────────────────+────────────────+
> >> | 100 | 30.053 |
> >> | 150 | 25.151 |
> >> | 200 | 20.704 |
> >> | 250 | 15.748 |
> >> | 500 | 11.401 |
> >> | 1000 | 11.443 |
> >> | 10000 | 11.258 |
> >> | 1000000 | 11.154 |
> >> +──────────────────────────+────────────────+
> >>
> >> Analysis on the experiment results showed improved boot time
> >> with non blocking delays close to one jiffy duration. This
> >> was also seen when number of per-phase iterations were scaled
> >> to one jiffy.
> >>
> >> So, this change scales per-grace-period phase number of non-sleeping
> >> polls, soiuch that, non-sleeping polls are done for one jiffy. In addition
> >> to this, srcu_get_delay() call in srcu_gp_end(), which is used to calculate
> >> the delay used for scheduling callbacks, is replaced with the check for
> >> expedited grace period. This is done, to schedule cbs for completed expedited
> >> grace periods immediately, which results in improved boot time seen in
> >> experiments.
> >>
> >> In addition to the changes to default per phase delays, this change
> >> adds 3 new kernel parameters - srcutree.srcu_max_nodelay,
> >> srcutree.srcu_max_nodelay_phase, srcutree.srcu_retry_check_delay.
> >> This allows users to configure the srcu grace period scanning delays,
> >> depending on their system configuration requirements.
> >>
> >> Signed-off-by: Neeraj Upadhyay <quic_neeraju@quicinc.com>
> >
> > I've given this a go on one of my test platforms (the one I noticed
> > the issue on the first place), and found that the initial part of the
> > EFI boot under KVM (pointlessly wiping the emulated flash) went down
> > to 1m7s from 3m50s (HZ=250).
> >
> > Clearly a massive improvement, but still a far cry from the original
> > ~40s (yes, this box is utter crap -- which is why I use it).
>
> Do you see any improvement by using "srcutree.srcu_max_nodelay=1000"
> bootarg, on top of this patch?

Yup, this brings it back to 43s on a quick test run, which is close
enough to what I had before.

How does a random user come up with such a value though?

Thanks,

M.

--
Without deviation from the norm, progress is not possible.

\
 
 \ /
  Last update: 2022-06-28 11:32    [W:0.345 / U:0.032 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site