lkml.org 
[lkml]   [2020]   [May]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    SubjectRe: [PATCH] bpf: Tweak BPF jump table optimizations for objtool compatibility
    On Tue, May 05, 2020 at 03:28:23PM -0500, Josh Poimboeuf wrote:
    > On Tue, May 05, 2020 at 12:53:20PM -0700, Alexei Starovoitov wrote:
    > > On Tue, May 05, 2020 at 01:11:08PM -0500, Josh Poimboeuf wrote:
    > > > On Tue, May 05, 2020 at 10:43:00AM -0700, Alexei Starovoitov wrote:
    > > > > > Or, if you want to minimize the patch's impact on other arches, and keep
    > > > > > the current patch the way it is (with bug fixed and changed patch
    > > > > > description), that's fine too. I can change the patch description
    > > > > > accordingly.
    > > > > >
    > > > > > Or if you want me to measure the performance impact of the +40% code
    > > > > > growth, and *then* decide what to do, that's also fine. But you'd need
    > > > > > to tell me what tests to run.
    > > > >
    > > > > I'd like to minimize the risk and avoid code churn,
    > > > > so how about we step back and debug it first?
    > > > > Which version of gcc are you using and what .config?
    > > > > I've tried:
    > > > > Linux version 5.7.0-rc2 (gcc version 10.0.1 20200505 (prerelease) (GCC)
    > > > > CONFIG_UNWINDER_ORC=y
    > > > > # CONFIG_RETPOLINE is not set
    > > > >
    > > > > and objtool didn't complain.
    > > > > I would like to reproduce it first before making any changes.
    > > >
    > > > Revert
    > > >
    > > > 3193c0836f20 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")
    > > >
    > > > and compile with retpolines off (and either ORC or FP, doesn't matter).
    > > >
    > > > I'm using GCC 9.3.1:
    > > >
    > > > kernel/bpf/core.o: warning: objtool: ___bpf_prog_run()+0x8dc: sibling call from callable instruction with modified stack frame
    > > >
    > > > That's the original issue described in that commit.
    > >
    > > I see something different.
    > > With gcc 8, 9, and 10 and CCONFIG_UNWINDER_FRAME_POINTER=y
    > > I see:
    > > kernel/bpf/core.o: warning: objtool: ___bpf_prog_run()+0x4837: call without frame pointer save/setup
    > > and sure enough assembly code for ___bpf_prog_run does not countain frame setup
    > > though -fno-omit-frame-pointer flag was passed at command line.
    > > Then I did:
    > > static u64 /*__no_fgcse*/ ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
    > > and the assembly had proper frame, but objtool wasn't happy:
    > > kernel/bpf/core.o: warning: objtool: ___bpf_prog_run()+0x480a: sibling call from callable instruction with modified stack frame
    > >
    > > gcc 6.3 doesn't have objtool warning with and without -fno-gcse.
    > >
    > > Looks like we have two issues here.
    > > First gcc 8, 9 and 10 have a severe bug with __attribute__((optimize("")))
    > > In this particular case passing -fno-gcse somehow overruled -fno-omit-frame-pointer
    > > which is serious issue. powerpc is using __nostackprotector. I don't understand
    > > how it can keep working with newer gcc-s. May be got lucky.
    > > Plenty of other projects use various __attribute__((optimize("")))
    > > they all have to double check that their vesion of GCC produces correct code.
    > > Can somebody reach out to gcc folks for explanation?
    >
    > Right. I've mentioned this several times now. That's why my patch
    > reverts 3193c0836f20. I don't see any other way around it. The GCC
    > manual even says this attribute should not be used in production code.

    What you mentioned in commit log is:
    "It doesn't append options to the command-line arguments. Instead
    it starts from a blank slate. And according to recent GCC documentation
    it's not recommended for production use."

    I don't think anyone could have guessed from such description that it kills
    -fno-omit-frame-pointer but it doesn't reduce optimization level to -O0
    and it doesn't kill -D, -m, -I, -std= and other flags.

    As far as workaround I prefer the following:
    From 94bbc27c5a70d78846a5cb675df4cf8732883564 Mon Sep 17 00:00:00 2001
    From: Alexei Starovoitov <ast@kernel.org>
    Date: Tue, 5 May 2020 16:52:41 -0700
    Subject: [PATCH] bpf,objtool: tweak interpreter compilation flags to help objtool

    tbd

    Fixes: 3193c0836f20 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")
    Reported-by: Randy Dunlap <rdunlap@infradead.org>
    Reported-by: Arnd Bergmann <arnd@arndb.de>
    Signed-off-by: Alexei Starovoitov <ast@kernel.org>
    ---
    include/linux/compiler-gcc.h | 2 +-
    1 file changed, 1 insertion(+), 1 deletion(-)

    diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
    index d7ee4c6bad48..05104c3cc033 100644
    --- a/include/linux/compiler-gcc.h
    +++ b/include/linux/compiler-gcc.h
    @@ -171,4 +171,4 @@
    #define __diag_GCC_8(s)
    #endif

    -#define __no_fgcse __attribute__((optimize("-fno-gcse")))
    +#define __no_fgcse __attribute__((optimize("-fno-gcse,-fno-omit-frame-pointer")))
    --
    2.23.0
    I've tested it with gcc 8,9,10 and clang 11 with FP=y and with ORC=y.
    All works.
    I think it's safer to go with frame pointers even for ORC=y considering
    all the pain this issue had caused. Even if objtool gets confused again
    in the future __bpf_prog_run() will have frame pointers and kernel stack
    unwinding can fall back from ORC to FP for that frame.
    wdyt?

    \
     
     \ /
      Last update: 2020-05-06 02:00    [W:2.949 / U:0.204 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site