lkml.org 
[lkml]   [2023]   [Feb]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH v4 0/9] Parallel CPU bringup for x86_64
From


On 01/02/2022 20:53, David Woodhouse wrote:
> Doing the INIT/SIPI/SIPI in parallel for all APs and *then* waiting for
> them shaves about 80% off the AP bringup time on a 96-thread 2-socket
> Skylake box (EC2 c5.metal) — from about 500ms to 100ms.
>
> There are more wins to be had with further parallelisation, but this is
> the simple part.
>

Hi,

We are interested in reducing the boot time of servers (with kexec), and
smpboot takes up a significant amount of time while booting. When
testing the patch series (rebased to v6.1) on a server with 128 CPUs
split across 2 NUMA nodes, it brought down the smpboot time from ~700ms
to 100ms. Adding another cpuhp state for do_wait_cpu_initialized to make
sure cpu_init is reached (as done in v1 of the series + using the
cpu_finishup_mask) brought it down further to ~30ms.

I just wanted to check what was needed to progress the patch series
further for review? There weren't any comments on v4 of the patch so I
couldn't figure out what more is needed. I think its quite useful to
have this working so would be really glad help in anything needed to
restart the review.

Thanks!
Usama



> v2: Cut it back to just INIT/SIPI/SIPI in parallel for now, nothing more
> v3: Clean up x2apic patch, add MTRR optimisation, lock topology update
> in preparation for more parallelisation.
> v4: Fixes to the real mode parallelisation patch spotted by SeanC, to
> avoid scribbling on initial_gs in common_cpu_up(), and to allow all
> 24 bits of the physical X2APIC ID to be used. That patch still needs
> a Signed-off-by from its original author, who once claimed not to
> remember writing it at all. But now we've fixed it, hopefully he'll
> admit it now :)
>
> David Woodhouse (8):
> x86/apic/x2apic: Fix parallel handling of cluster_mask
> cpu/hotplug: Move idle_thread_get() to <linux/smpboot.h>
> cpu/hotplug: Add dynamic parallel bringup states before CPUHP_BRINGUP_CPU
> x86/smpboot: Reference count on smpboot_setup_warm_reset_vector()
> x86/smpboot: Split up native_cpu_up into separate phases and document them
> x86/smpboot: Send INIT/SIPI/SIPI to secondary CPUs in parallel
> x86/mtrr: Avoid repeated save of MTRRs on boot-time CPU bringup
> x86/smpboot: Serialize topology updates for secondary bringup
>
> Thomas Gleixner (1):
> x86/smpboot: Support parallel startup of secondary CPUs
>
> [dwoodhou@i7 linux-2.6]$ git diff --stat v5.17-rc2..share/parallel-5.17-part1
> arch/x86/include/asm/realmode.h | 3 +
> arch/x86/include/asm/smp.h | 13 +-
> arch/x86/include/asm/topology.h | 2 -
> arch/x86/kernel/acpi/sleep.c | 1 +
> arch/x86/kernel/apic/apic.c | 2 +-
> arch/x86/kernel/apic/x2apic_cluster.c | 108 ++++++-----
> arch/x86/kernel/cpu/common.c | 6 +-
> arch/x86/kernel/cpu/mtrr/mtrr.c | 9 +
> arch/x86/kernel/head_64.S | 73 ++++++++
> arch/x86/kernel/smpboot.c | 325 ++++++++++++++++++++++++----------
> arch/x86/realmode/init.c | 3 +
> arch/x86/realmode/rm/trampoline_64.S | 14 ++
> arch/x86/xen/smp_pv.c | 4 +-
> include/linux/cpuhotplug.h | 2 +
> include/linux/smpboot.h | 7 +
> kernel/cpu.c | 27 ++-
> kernel/smpboot.c | 2 +-
> kernel/smpboot.h | 2 -
> 18 files changed, 442 insertions(+), 161 deletions(-)
>
>
>
>

\
 
 \ /
  Last update: 2023-03-27 00:06    [W:0.114 / U:0.112 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site