[lkml]   [2021]   [Dec]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
Patch in this message
SubjectRe: [PATCH v3 0/9] Parallel CPU bringup for x86_64
On Tue, 2021-12-28 at 12:34 +0100, Paul Menzel wrote:
> Same on the ASUS F2A85-M PRO with AMD A6-6400K. Without serial console,
> the messages below are printed below to the monitor after nine seconds.
> [ 1.078879] smp: Bringing up secondary CPUs ...
> [ 1.080950] x86: Booting SMP configuration:
> Please find the serial log attached.

Thanks for testing. That looks like the same triple-fault on bringup
that we have been seeing, and that I reproduced without my patches
using kexec all the way back to a 5.0 kernel.

Out of interest, are you also able to reproduce it with kexec and
without the parallel bringup?

And with that patch I sent Tom in
to expand the bitlock exclusion and stop the bringup being truly in
parallel at all?

Or tbe one in
which makes it do nothing except prepare all the CPUs before bringing
them up one at a time?

My current theory (not that I've spent that much time thinking about it
in the last week) is that there's something about the existing CPU
bringup, possibly a CPU bug or something special about the AMD CPUs,
which is triggered by just making it a little bit *faster*, which is
why bringing them up from kexec (especially in qemu) can cause it too?

Tom seemed to find that it was in load_TR_desc(), so if you could try
this hack on a machine that doesn't magically wink out of existence on
a triplefault before even flushing its serial output, that would be
much appreciated...

diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h
index ab97b22ac04a..cc6590712ff4 100644
--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -8,7 +8,7 @@
#include <asm/fixmap.h>
#include <asm/irq_vectors.h>
#include <asm/cpu_entry_area.h>
+#include <asm/io.h>
#include <linux/debug_locks.h>
#include <linux/smp.h>
#include <linux/percpu.h>
@@ -265,11 +265,16 @@ static inline void native_load_tr_desc(void)
* If the current GDT is the read-only fixmap, swap to the original
* writeable version. Swap back at the end.
+ outb('d', 0x3f8);
if (gdt.address == (unsigned long)fixmap_gdt) {
+ outb('e', 0x3f8);
restore = 1;
+ outb('f', 0x3f8);
+ outb('g', 0x3f8);
asm volatile("ltr %w0"::"q" (GDT_ENTRY_TSS*8));
+ outb('h', 0x3f8);
if (restore)
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 0083464de5e3..5bc8f30c3283 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1716,7 +1716,9 @@ void identify_secondary_cpu(struct cpuinfo_x86 *c)
+outb('A', 0x3f8);
+outb('B', 0x3f8);
@@ -1957,6 +1959,7 @@ static inline void tss_setup_io_bitmap(struct tss_struct *tss)
tss->io_bitmap.mapall[IO_BITMAP_LONGS] = ~0UL;
+#include <asm/realmode.h>

* Setup everything needed to handle exceptions from the IDT, including the IST
@@ -1969,16 +1972,24 @@ void cpu_init_exception_handling(void)

/* paranoid_entry() gets the CPU number from the GDT */
+ outb('\n', 0x3f8);
+ outb('0' + cpu / 100, 0x3f8);
+ outb('0' + (cpu % 100) / 10, 0x3f8);
+ outb('0' + (cpu % 10), 0x3f8);
/* IST vectors need TSS to be set up. */
+ outb('a', 0x3f8);
set_tss_desc(cpu, &get_cpu_entry_area(cpu)->tss.x86_tss);
+ outb('b', 0x3f8);
+ outb('c', 0x3f8);

/* Finally load the IDT */
+ outb('z', 0x3f8);

/*[unhandled content-type:application/pkcs7-signature]
 \ /
  Last update: 2021-12-28 15:21    [W:0.115 / U:3.172 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site