Messages in this thread | | | Date | Thu, 6 Dec 2018 17:45:16 +0000 | From | Mark Rutland <> | Subject | Re: [PATCHv2] locking/atomics: build atomic headers as required |
| |
On Thu, Dec 06, 2018 at 05:13:44PM +0100, Ingo Molnar wrote: > > * Mark Rutland <mark.rutland@arm.com> wrote: > > > Andrew and Ingo report that the check-atomics.sh script is simply too > > slow to run for every kernel build, and it's impractical to make it > > faster without rewriting it in something other than shell. > > > > Rather than committing the generated headers, let's regenerate these > > as-required, if we change the data or scripts used to generate the > > atomic headers, or when building from a pristine tree. > > > > That ensures they're always up-to-date, allows them to be built in > > parallel, and avoid redundant rebuilds, which is a 2-8s saving per > > incremental build. Since the results are not committed, it's very > > obvious that they should not be modified directly. If we need to > > generate more headers in future, it's easy to extend Makefile.genheader > > to permit this. > > > > I've verified that this works in the cases we previously had issues with > > (out-of-tree builds and where scripts have no execute permissions), and > > have tested these cases for both x86_64 and arm64. > > > > The diffstat looks nice, at least... > > > > Signed-off-by: Mark Rutland <mark.rutland@arm.com> > > Cc: Andrew Morton <akpm@linux-foundation.org> > > Cc: Boqun Feng <boqun.feng@gmail.com> > > Cc: Borislav Petkov <bp@suse.de> > > Cc: Ingo Molnar <mingo@kernel.org> > > Cc: Peter Zijlstra <peterz@infradead.org> > > Cc: Will Deacon <will.deacon@arm.com> > > Cc: linux-kernel@vger.kernel.org > > --- > > Kbuild | 18 +- > > Makefile | 8 +- > > arch/arm64/include/asm/atomic.h | 2 +- > > arch/x86/include/asm/atomic.h | 2 +- > > include/asm-generic/atomic-instrumented.h | 1787 ---------------------- > > include/asm-generic/atomic-long.h | 1012 ------------- > > include/linux/atomic-fallback.h | 2294 ----------------------------- > > include/linux/atomic.h | 4 +- > > scripts/Makefile.genheader | 26 + > > scripts/atomic/check-atomics.sh | 19 - > > 10 files changed, 38 insertions(+), 5134 deletions(-) > > delete mode 100644 include/asm-generic/atomic-instrumented.h > > delete mode 100644 include/asm-generic/atomic-long.h > > delete mode 100644 include/linux/atomic-fallback.h > > create mode 100644 scripts/Makefile.genheader > > delete mode 100755 scripts/atomic/check-atomics.sh > > So these 'automatically generated' headers are actual and important code, > and I think it's bad practice to remove these from the git grep search > space and history as well.
Ok. Let's aim for that, then. :)
> Really, this whole notion of auto-generating the headers should be > implemented correctly, instead of working around deficiencies in a > short-term fashion that introduces other deficiencies.
I certainly agree.
> I never got any replies to my previous comments about this: > > <20181128083057.GA7879@gmail.com> > > Did I miss some mails of yours perhaps?
It appears my mailserver dropped that on the floor, as I can't find it in any of my mailboxes. :(
Sorry about that; I'll try to figure out what happened there.
This patch covers your suggestion of parallelizing things. On my box with -j64, that saves ~1.5s for a pristine allnoconfig build which otherwise takes ~15s.
Practically, optimizing the scripts requires moving away from shell. The cost is all down to sub-shells resulting in fork+exec, but the scripts would be illegible without this.
One option is to generate a checksum when the headers are generated and checked in, and check that at build time. I'll look into this tomorrow.
Thanks, Mark.
| |