Messages in this thread | | | Date | Fri, 17 Jun 2022 13:05:53 -0700 | From | Fangrui Song <> | Subject | Re: [PATCH] kbuild: pass jobserver to cmd_ld_vmlinux.o |
| |
On 2022-06-18, Masahiro Yamada wrote: >(+LLVM list, Fangrui Song)
Thanks for tagging me. I'll clarify some stuff.
>On Fri, Jun 17, 2022 at 7:41 PM Sedat Dilek <sedat.dilek@gmail.com> wrote: >> >> On Fri, Jun 17, 2022 at 12:35 PM Sedat Dilek <sedat.dilek@gmail.com> wrote: >> > >> > On Fri, Jun 17, 2022 at 12:53 AM Sedat Dilek <sedat.dilek@gmail.com> wrote: >> > > >> > > On Thu, Jun 16, 2022 at 4:09 PM Sedat Dilek <sedat.dilek@gmail.com> wrote: >> > > > >> > > > On Thu, Jun 16, 2022 at 12:45 PM Jiri Slaby <jslaby@suse.cz> wrote: >> > > > > >> > > > > Until the link-vmlinux.sh split (cf. the commit below), the linker was >> > > > > run with jobserver set in MAKEFLAGS. After the split, the command in >> > > > > Makefile.vmlinux_o is not prefixed by "+" anymore, so this information >> > > > > is lost. >> > > > > >> > > > > Restore it as linkers working in parallel (esp. the LTO ones) make a use >> > > > > of i > >Hi Jiri, > >Please let me clarify first. > >Here, is it OK to assume you are talking about Clang LTO >instead of GCC LTO because the latter is not upstreamed ? > > > > > >I tested this patch but I did not see any performance change for Clang LTO. > > >[1] CONFIG_CLANG_LTO_FULL > > lld always runs sequential. > It never runs in parallel even if you pass -j option to Make
"lld always runs sequential" is not accurate. There are a number of parallel linker passes. ld.lld --threads= defaults to llvm::hardware_concurrency (similar to https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency, but uses sched_getaffinity to compute the number of available cores).
"lld always runs sequential" is only correct only when --threads=1 is specified or the system only provides one thread to the lld process.
I think people may be more interested in LTO parallelism here. Regular LTO (sometimes called full LTO when there is mixed-thin-and-regular LTO) supports limited parallelism which applies to code generation, but not IR-level optimization. (IR-level optimization has many interprocedural optimizations passes. Splitting will make LTO less effective. Code generation is per function, so parallelism does not regress optimization.)
> >[2] CONFIG_CLANG_LTO_THIN > > lld always runs in parallel even if you do not pass -j option > > In my machine, lld always allocated 12 threads. > This is irrespective of the Make parallelisms. > > > > >One more thing, if a program wants to participate in >Make's jobserver, it must parse MAKEFLAGS, and extract >file descriptors to be used to communicate to the jobserver. > >As a code example in the kernel tree, >scripts/jobserver-exec parses "MAKEFLAGS" and "--jobserver". > > >I grepped the lld source code, but it does not contain >"MAKEFLAGS" or "jobserver".
>masahiro@oscar:~/ref/lld$ git remote show origin >* remote origin > Fetch URL: https://github.com/llvm-mirror/lld.git > Push URL: https://github.com/llvm-mirror/lld.git > HEAD branch: master > Remote branches: > master tracked > release_36 tracked > release_37 tracked > release_38 tracked > release_39 tracked > release_40 tracked > release_50 tracked > release_60 tracked > release_70 tracked > release_80 tracked > release_90 tracked > Local branch configured for 'git pull': > master merges with remote master > Local ref configured for 'git push': > master pushes to master (up to date) >masahiro@oscar:~/ref/lld$ git grep MAKEFLAGS >masahiro@oscar:~/ref/lld$ git grep jobserver > > >So, in my research, LLD does not seem to support the jobserver.
Correct. lld does not support GNU make's jobserver. On the other hand, I don't think the jobserver implementation supports flexible "give this target N hardware concurrency". A heavy link target does not necessarily get more resources than a quick target.
If a make target knows how many hardware concurrency it gets, we can pass --threads= to lld. LTO easily takes 95+% link time, so LTO parallelism may needs a dedicated setting. lld has --thinlto-jobs=.
> > > >If you are talking about GCC LTO, yes, the code >tries to parse "--jobserver-auth=" from the MAKEFLAGS >environment variable. [1] > >[1]: https://github.com/gcc-mirror/gcc/blob/releases/gcc-12.1.0/gcc/lto-wrapper.cc#L1341 > > >But, as you may know, GCC LTO works in a different way, >at least, we cannot do it before modpost. > > >-- >Best Regards >Masahiro Yamada >
| |