lkml.org 
[lkml]   [2021]   [May]   [11]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Subject[PATCH v10 00/33] Memory folios
Date
Managing memory in 4KiB pages is a serious overhead.  Many benchmarks
benefit from a larger "page size". As an example, an earlier iteration
of this idea which used compound pages (and wasn't particularly tuned)
got a 7% performance boost when compiling the kernel.

Using compound pages or THPs exposes a weakness of our type system.
Functions are often unprepared for compound pages to be passed to them,
and may only act on PAGE_SIZE chunks. Even functions which are aware of
compound pages may expect a head page, and do the wrong thing if passed
a tail page.

We also waste a lot of instructions ensuring that we're not looking at
a tail page. Almost every call to PageFoo() contains one or more hidden
calls to compound_head(). This also happens for get_page(), put_page()
and many more functions. There does not appear to be a way to tell gcc
that it can cache the result of compound_head(), nor is there a way to
tell it that compound_head() is idempotent.

This patch series uses a new type, the struct folio, to manage memory.
It provides some basic infrastructure that's worthwhile in its own right,
shrinking the kernel by about 5kB of text.

Since v9:
- Rebase onto mmotm 2021-05-10-21-46
- Add folio_memcg() definition for !MEMCG (intel lkp)
- Change folio->private from an unsigned long to a void *
- Use folio_page() to implement folio_file_page()
- Add folio_try_get() and folio_try_get_rcu()
- Trim back down to just the first few patches, which are better-reviewed.
v9: https://lore.kernel.org/linux-mm/20210505150628.111735-1-willy@infradead.org/
v8: https://lore.kernel.org/linux-mm/20210430180740.2707166-1-willy@infradead.org/

Matthew Wilcox (Oracle) (33):
mm: Introduce struct folio
mm: Add folio_pgdat and folio_zone
mm/vmstat: Add functions to account folio statistics
mm/debug: Add VM_BUG_ON_FOLIO and VM_WARN_ON_ONCE_FOLIO
mm: Add folio reference count functions
mm: Add folio_put
mm: Add folio_get
mm: Add folio_try_get_rcu
mm: Add folio flag manipulation functions
mm: Add folio_young and folio_idle
mm: Handle per-folio private data
mm/filemap: Add folio_index, folio_file_page and folio_contains
mm/filemap: Add folio_next_index
mm/filemap: Add folio_offset and folio_file_offset
mm/util: Add folio_mapping and folio_file_mapping
mm: Add folio_mapcount
mm/memcg: Add folio wrappers for various functions
mm/filemap: Add folio_unlock
mm/filemap: Add folio_lock
mm/filemap: Add folio_lock_killable
mm/filemap: Add __folio_lock_async
mm/filemap: Add __folio_lock_or_retry
mm/filemap: Add folio_wait_locked
mm/swap: Add folio_rotate_reclaimable
mm/filemap: Add folio_end_writeback
mm/writeback: Add folio_wait_writeback
mm/writeback: Add folio_wait_stable
mm/filemap: Add folio_wait_bit
mm/filemap: Add folio_wake_bit
mm/filemap: Convert page wait queues to be folios
mm/filemap: Add folio private_2 functions
fs/netfs: Add folio fscache functions
mm: Add folio_mapped

Documentation/core-api/mm-api.rst | 4 +
Documentation/filesystems/netfs_library.rst | 2 +
fs/afs/write.c | 9 +-
fs/cachefiles/rdwr.c | 16 +-
fs/io_uring.c | 2 +-
include/linux/memcontrol.h | 63 ++++
include/linux/mm.h | 174 ++++++++--
include/linux/mm_types.h | 71 ++++
include/linux/mmdebug.h | 20 ++
include/linux/netfs.h | 77 +++--
include/linux/page-flags.h | 230 ++++++++++---
include/linux/page_idle.h | 99 +++---
include/linux/page_ref.h | 158 ++++++++-
include/linux/pagemap.h | 358 ++++++++++++--------
include/linux/swap.h | 7 +-
include/linux/vmstat.h | 107 ++++++
mm/Makefile | 2 +-
mm/filemap.c | 315 ++++++++---------
mm/folio-compat.c | 43 +++
mm/internal.h | 1 +
mm/memory.c | 8 +-
mm/page-writeback.c | 72 ++--
mm/page_io.c | 4 +-
mm/swap.c | 18 +-
mm/swapfile.c | 8 +-
mm/util.c | 59 ++--
26 files changed, 1374 insertions(+), 553 deletions(-)
create mode 100644 mm/folio-compat.c

--
2.30.2

\
 
 \ /
  Last update: 2021-05-11 23:48    [W:0.544 / U:0.968 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site