Messages in this thread Patch in this message | | | From | Alexander Lobakin <> | Subject | [PATCH RFC net-next v2 3/7] net: page_pool: place frag_* fields in one cacheline | Date | Fri, 14 Jul 2023 19:08:47 +0200 |
| |
On x86_64, frag_* fields of struct page_pool are scattered across two cachelines despite the summary size of 24 bytes. The last field, ::frag_users, is pushed out to the next one, sharing it with ::alloc_stats. All three fields are used in pretty much same places. There are some holes and cold members to move around. Move frag_* one block up, placing them right after &page_pool_params perfectly at the beginning of CL2. This doesn't do any meaningful to the second block, as those are some destroy-path cold structures, and doesn't do anything to ::alloc_stats, which still starts at 200 byte offset, 8 bytes after CL3 (still fitting into 1 cacheline). On my setup, this yields 1-2% of Mpps when using PP frags actively. When it comes to 32-bit architectures with 32 byte CL: &page_pool_params plus ::pad is 44 bytes, the block taken care of is 16 bytes within one CL, so there should be at least no regressions from the actual change.
Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com> --- include/net/page_pool.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 69e822021d95..68937deea4b1 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -131,16 +131,16 @@ struct page_pool { struct page_pool_params p; long pad; + long frag_users; + struct page *frag_page; + unsigned int frag_offset; + u32 pages_state_hold_cnt; + struct delayed_work release_dw; void (*disconnect)(void *); unsigned long defer_start; unsigned long defer_warn; - u32 pages_state_hold_cnt; - unsigned int frag_offset; - struct page *frag_page; - long frag_users; - #ifdef CONFIG_PAGE_POOL_STATS /* these stats are incremented while in softirq context */ struct page_pool_alloc_stats alloc_stats; -- 2.41.0
| |