[lkml]   [2015]   [May]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
Patch in this message
Subject[tip:x86/asm] x86: Pack loops tightly as well
Commit-ID:  24b0af706205ba49cd139913f92fea837a5724a7
Author: Ingo Molnar <>
AuthorDate: Fri, 10 Apr 2015 14:30:18 +0200
Committer: Ingo Molnar <>
CommitDate: Fri, 15 May 2015 11:17:11 +0200

x86: Pack loops tightly as well

Packing loops tightly (-falign-loops=1) is beneficial to code size:

text data bss dec filename
12566391 1617840 1089536 15273767 vmlinux.align.16-byte
12224951 1617840 1089536 14932327 vmlinux.align.1-byte
11976567 1617840 1089536 14683943 vmlinux.align.1-byte.funcs-1-byte
11903735 1617840 1089536 14611111 vmlinux.align.1-byte.funcs-1-byte.loops-1-byte

Which reduces the size of the kernel by another 0.6%, so the
the total combined size reduction of the alignment-packing
patches is ~5.5%.

The x86 decoder bandwidth and caching arguments laid out in:

be6cb02779ca ("x86: Align jump targets to 1-byte boundaries")

apply to loop alignment as well.

Furtermore, modern CPU uarchs have a loop cache/buffer that
is a L0 cache before even any uop cache, covering a few
dozen most recently executed instructions.

This loop cache generally does not have the 16-byte alignment
restrictions of the uop cache.

Now loop alignment can still be beneficial if:

- a loop is cache-hot and its surroundings are not.

- if the loop is so cache hot that the instruction
flow becomes x86 decoder bandwidth limited

But loop alignment is harmful if:

- a loop is cache-cold

- a loop's surroundings are cache-hot as well

- two cache-hot loops are close to each other

- if the loop fits into the loop cache

- if the code flow is not decoder bandwidth limited

and I'd argue that the latter five scenarios are much
more common in the kernel, as our hottest loops are

- pointer chasing: this should fit into the loop cache
in most cases and is typically data cache and address
generation limited

- generic memory ops (memset, memcpy, etc.): these generally
fit into the loop cache as well, and are likewise data
cache limited.

So this patch packs loop addresses tightly as well.

Acked-by: Denys Vlasenko <>
Cc: Andy Lutomirski <>
Cc: Aswin Chandramouleeswaran <>
Cc: Borislav Petkov <>
Cc: Brian Gerst <>
Cc: Davidlohr Bueso <>
Cc: H. Peter Anvin <>
Cc: Jason Low <>
Cc: Linus Torvalds <>
Cc: Paul E. McKenney <>
Cc: Peter Zijlstra <>
Cc: Peter Zijlstra <>
Cc: Thomas Gleixner <>
Cc: Tim Chen <>
Signed-off-by: Ingo Molnar <>
arch/x86/Makefile | 3 +++
1 file changed, 3 insertions(+)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 5c7edf9..8c7cc44 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -83,6 +83,9 @@ else
# Pack functions tightly as well:
KBUILD_CFLAGS += -falign-functions=1

+ # Pack loops tightly as well:
+ KBUILD_CFLAGS += -falign-loops=1
# Don't autogenerate traditional x87 instructions
KBUILD_CFLAGS += $(call cc-option,-mno-80387)
KBUILD_CFLAGS += $(call cc-option,-mno-fp-ret-in-387)

 \ /
  Last update: 2015-05-15 12:01    [W:0.121 / U:0.700 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site