lkml.org 
[lkml]   [2020]   [Sep]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [PATCH v2 09/28] kbuild: add support for Clang LTO
    On Fri, Sep 4, 2020 at 5:31 AM Sami Tolvanen <samitolvanen@google.com> wrote:
    >
    > This change adds build system support for Clang's Link Time
    > Optimization (LTO). With -flto, instead of ELF object files, Clang
    > produces LLVM bitcode, which is compiled into native code at link
    > time, allowing the final binary to be optimized globally. For more
    > details, see:
    >
    > https://llvm.org/docs/LinkTimeOptimization.html
    >
    > The Kconfig option CONFIG_LTO_CLANG is implemented as a choice,
    > which defaults to LTO being disabled. To use LTO, the architecture
    > must select ARCH_SUPPORTS_LTO_CLANG and support:
    >
    > - compiling with Clang,
    > - compiling inline assembly with Clang's integrated assembler,
    > - and linking with LLD.
    >
    > While using full LTO results in the best runtime performance, the
    > compilation is not scalable in time or memory. CONFIG_THINLTO
    > enables ThinLTO, which allows parallel optimization and faster
    > incremental builds. ThinLTO is used by default if the architecture
    > also selects ARCH_SUPPORTS_THINLTO:
    >
    > https://clang.llvm.org/docs/ThinLTO.html
    >
    > To enable LTO, LLVM tools must be used to handle bitcode files. The
    > easiest way is to pass the LLVM=1 option to make:
    >
    > $ make LLVM=1 defconfig
    > $ scripts/config -e LTO_CLANG
    > $ make LLVM=1
    >
    > Alternatively, at least the following LLVM tools must be used:
    >
    > CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm
    >
    > To prepare for LTO support with other compilers, common parts are
    > gated behind the CONFIG_LTO option, and LTO can be disabled for
    > specific files by filtering out CC_FLAGS_LTO.
    >
    > Note that support for DYNAMIC_FTRACE and MODVERSIONS are added in
    > follow-up patches.
    >
    > Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
    > ---
    > Makefile | 18 +++++++-
    > arch/Kconfig | 68 +++++++++++++++++++++++++++++++
    > include/asm-generic/vmlinux.lds.h | 11 +++--
    > scripts/Makefile.build | 9 +++-
    > scripts/Makefile.modfinal | 9 +++-
    > scripts/Makefile.modpost | 24 ++++++++++-
    > scripts/link-vmlinux.sh | 32 +++++++++++----
    > 7 files changed, 154 insertions(+), 17 deletions(-)



    > #define TEXT_MAIN .text
    > diff --git a/scripts/Makefile.build b/scripts/Makefile.build
    > index 6ecf30c70ced..a5f4b5d407e6 100644
    > --- a/scripts/Makefile.build
    > +++ b/scripts/Makefile.build
    > @@ -111,7 +111,7 @@ endif
    > # ---------------------------------------------------------------------------
    >
    > quiet_cmd_cc_s_c = CC $(quiet_modtag) $@
    > - cmd_cc_s_c = $(CC) $(filter-out $(DEBUG_CFLAGS), $(c_flags)) $(DISABLE_LTO) -fverbose-asm -S -o $@ $<
    > + cmd_cc_s_c = $(CC) $(filter-out $(DEBUG_CFLAGS) $(CC_FLAGS_LTO), $(c_flags)) -fverbose-asm -S -o $@ $<
    >
    > $(obj)/%.s: $(src)/%.c FORCE
    > $(call if_changed_dep,cc_s_c)
    > @@ -428,8 +428,15 @@ $(obj)/lib.a: $(lib-y) FORCE
    > # Do not replace $(filter %.o,^) with $(real-prereqs). When a single object
    > # module is turned into a multi object module, $^ will contain header file
    > # dependencies recorded in the .*.cmd file.
    > +ifdef CONFIG_LTO_CLANG
    > +quiet_cmd_link_multi-m = AR [M] $@
    > +cmd_link_multi-m = \
    > + rm -f $@; \
    > + $(AR) rcsTP$(KBUILD_ARFLAGS) $@ $(filter %.o,$^)


    KBUILD_ARFLAGS no longer exists in the mainline.
    (commit 13dc8c029cabf52ba95f60c56eb104d4d95d5889)




    > +else
    > quiet_cmd_link_multi-m = LD [M] $@
    > cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(filter %.o,$^)
    > +endif
    >
    > $(multi-used-m): FORCE
    > $(call if_changed,link_multi-m)
    > diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
    > index 411c1e600e7d..1005b147abd0 100644
    > --- a/scripts/Makefile.modfinal
    > +++ b/scripts/Makefile.modfinal
    > @@ -6,6 +6,7 @@
    > PHONY := __modfinal
    > __modfinal:
    >
    > +include $(objtree)/include/config/auto.conf
    > include $(srctree)/scripts/Kbuild.include
    >
    > # for c_flags
    > @@ -29,6 +30,12 @@ quiet_cmd_cc_o_c = CC [M] $@
    >
    > ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink)
    >
    > +ifdef CONFIG_LTO_CLANG
    > +# With CONFIG_LTO_CLANG, reuse the object file we compiled for modpost to
    > +# avoid a second slow LTO link
    > +prelink-ext := .lto
    > +endif
    > +
    > quiet_cmd_ld_ko_o = LD [M] $@
    > cmd_ld_ko_o = \
    > $(LD) -r $(KBUILD_LDFLAGS) \
    > @@ -37,7 +44,7 @@ quiet_cmd_ld_ko_o = LD [M] $@
    > -o $@ $(filter %.o, $^); \
    > $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true)
    >
    > -$(modules): %.ko: %.o %.mod.o $(KBUILD_LDS_MODULE) FORCE
    > +$(modules): %.ko: %$(prelink-ext).o %.mod.o $(KBUILD_LDS_MODULE) FORCE
    > +$(call if_changed,ld_ko_o)
    >
    > targets += $(modules) $(modules:.ko=.mod.o)
    > diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
    > index f54b6ac37ac2..a70f1f7da6aa 100644
    > --- a/scripts/Makefile.modpost
    > +++ b/scripts/Makefile.modpost
    > @@ -102,12 +102,32 @@ $(input-symdump):
    > @echo >&2 'WARNING: Symbol version dump "$@" is missing.'
    > @echo >&2 ' Modules may not have dependencies or modversions.'
    >
    > +ifdef CONFIG_LTO_CLANG
    > +# With CONFIG_LTO_CLANG, .o files might be LLVM bitcode,

    or, .o files might be even thin archives.

    For example,

    $ file net/ipv6/netfilter/nf_defrag_ipv6.o
    net/ipv6/netfilter/nf_defrag_ipv6.o: thin archive with 6 symbol entries


    Now we have 3 possibilities for .o files:

    - ELF (real .o)
    - LLVM bitcode (.bc)
    - Thin archive (.a)


    Let me discuss how to proceed with this...





    --
    Best Regards
    Masahiro Yamada

    \
     
     \ /
      Last update: 2020-09-07 17:32    [W:3.231 / U:0.384 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site