Messages in this thread | | | Date | Wed, 13 Oct 2021 23:54:51 +0200 | From | Peter Zijlstra <> | Subject | Re: [PATCH 9/9] bpf,x86: Respect X86_FEATURE_RETPOLINE* |
| |
On Wed, Oct 13, 2021 at 02:06:05PM -0700, Josh Poimboeuf wrote: > On Wed, Oct 13, 2021 at 02:22:26PM +0200, Peter Zijlstra wrote: > > Current BPF codegen doesn't respect X86_FEATURE_RETPOLINE* flags and > > unconditionally emits a thunk call, this is sub-optimal and doesn't > > match the regular, compiler generated, code. > > > > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> > > --- > > arch/x86/net/bpf_jit_comp.c | 18 +++++++++++------- > > 1 file changed, 11 insertions(+), 7 deletions(-) > > > > --- a/arch/x86/net/bpf_jit_comp.c > > +++ b/arch/x86/net/bpf_jit_comp.c > > @@ -2123,14 +2123,18 @@ static int emit_fallback_jump(u8 **pprog > > int err = 0; > > > > #ifdef CONFIG_RETPOLINE > > - /* Note that this assumes the the compiler uses external > > - * thunks for indirect calls. Both clang and GCC use the same > > - * naming convention for external thunks. > > - */ > > - err = emit_jump(&prog, __x86_indirect_thunk_rdx, prog); > > -#else > > - EMIT2(0xFF, 0xE2); /* jmp rdx */ > > + if (cpu_feature_enabled(X86_FEATURE_RETPOLINE)) { > > + if (cpu_feature_enabled(X86_FEATURE_RETPOLINE_AMD)) { > > + /* The AMD retpoline can be easily emitted inline. */ > > + EMIT3(0x0F, 0xAE, 0xE8); /* lfence */ > > + EMIT2(0xFF, 0xE2); /* jmp rdx */ > > + } else { > > + /* Call the retpoline thunk */ > > + err = emit_jump(&prog, __x86_indirect_thunk_rdx, prog); > > + } > > + } else > > #endif > > + EMIT2(0xFF, 0xE2); /* jmp rdx */ > > But the rest of eBPF JIT just emits retpolines unconditionally > regardless of feature, for example see RETPOLINE_RCX_BPF_JIT(). So I'm > thinking this should probably be consistent with that (or that with > this).
Argh, I grepped for __x86_indirect_thunk, and missed they're writing retpolines themselves. Bah.
Yes, that needs cleaning up. I'll go prod at that tomorrow.
| |