lkml.org 
[lkml]   [2022]   [Nov]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: [PATCH v6 1/4] mm/slub: enable debugging memory wasting of kmalloc
    From
    On 11/1/22 11:33, John Thomson wrote:
    > On Tue, 1 Nov 2022, at 09:31, Hyeonggon Yoo wrote:
    >> On Tue, Nov 01, 2022 at 09:20:21AM +0000, John Thomson wrote:
    >>> On Tue, 1 Nov 2022, at 07:57, Feng Tang wrote:
    >>> > Hi Thomson,
    >>> >
    >>> > Thanks for testing!
    >>> >
    >>> > + mips maintainer and mail list. The original report is here
    >>> >
    >>> > https://lore.kernel.org/lkml/becf2ac3-2a90-4f3a-96d9-a70f67c66e4a@app.fastmail.com/
    >>>
    >>> I am guessing my issue comes from __kmem_cache_alloc_lru accessing s->object_size when (kmem_cache) s is NULL?
    >>> If that is the case, this change is not to blame, it only exposes the issue?
    >>>
    >>> I get the following dmesg (note very early NULL kmem_cache) with the below change atop v6.1-rc3:
    >>>
    >>> transfer started ......................................... transfer ok, time=2.02s
    >>> setting up elf image... OK
    >>> jumping to kernel code
    >>> zimage at: 80B842A0 810B4EFC
    >>>
    >>> Uncompressing Linux at load address 80001000
    >>>
    >>> Copy device tree to address 80B80EE0
    >>>
    >>> Now, booting the kernel...
    >>>
    >>> [ 0.000000] Linux version 6.1.0-rc3+ (john@john) (mipsel-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.11-4428-g6b6741b) 12.2.0, GNU ld (GNU Binutils) 2.39) #61 SMP Tue Nov 1 18:04:13 AEST 2022
    >>> [ 0.000000] slub: kmem_cache_alloc called with kmem_cache: 0x0
    >>> [ 0.000000] slub: __kmem_cache_alloc_lru called with kmem_cache: 0x0
    >>> [ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
    >>> [ 0.000000] printk: bootconsole [early0] enabled
    >>> [ 0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
    >>> [ 0.000000] MIPS: machine is MikroTik RouterBOARD 760iGS
    >>>
    >>> normal boot
    >>>
    >>>
    >>> diff --git a/mm/slub.c b/mm/slub.c
    >>> index 157527d7101b..10fcdf2520d2 100644
    >>> --- a/mm/slub.c
    >>> +++ b/mm/slub.c
    >>> @@ -3410,7 +3410,13 @@ static __always_inline
    >>> void *__kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
    >>> gfp_t gfpflags)
    >>> {
    >>> - void *ret = slab_alloc(s, lru, gfpflags, _RET_IP_, s->object_size);
    >>> + void *ret;
    >>> + if (IS_ERR_OR_NULL(s)) {
    >>> + pr_warn("slub: __kmem_cache_alloc_lru called with kmem_cache: %pSR\n", s);
    >>> + ret = slab_alloc(s, lru, gfpflags, _RET_IP_, 0);
    >>> + } else {
    >>> + ret = slab_alloc(s, lru, gfpflags, _RET_IP_, s->object_size);
    >>> + }
    >>>
    >>> trace_kmem_cache_alloc(_RET_IP_, ret, s, gfpflags, NUMA_NO_NODE);
    >>>
    >>> @@ -3419,6 +3425,8 @@ void *__kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
    >>>
    >>> void *kmem_cache_alloc(struct kmem_cache *s, gfp_t gfpflags)
    >>> {
    >>> + if (IS_ERR_OR_NULL(s))
    >>> + pr_warn("slub: kmem_cache_alloc called with kmem_cache: %pSR\n", s);
    >>> return __kmem_cache_alloc_lru(s, NULL, gfpflags);
    >>> }
    >>> EXPORT_SYMBOL(kmem_cache_alloc);
    >>> @@ -3426,6 +3434,8 @@ EXPORT_SYMBOL(kmem_cache_alloc);
    >>> void *kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
    >>> gfp_t gfpflags)
    >>> {
    >>> + if (IS_ERR_OR_NULL(s))
    >>> + pr_warn("slub: __kmem_cache_alloc_lru called with kmem_cache: %pSR\n", s);
    >>> return __kmem_cache_alloc_lru(s, lru, gfpflags);
    >>> }
    >>> EXPORT_SYMBOL(kmem_cache_alloc_lru);
    >>>
    >>>
    >>> Any hints on where kmem_cache_alloc would be being called from this early?
    >>> I will start looking from /init/main.c around pr_notice("%s", linux_banner);
    >>
    >> Great. Would you try calling dump_stack(); when we observed s == NULL?
    >> That would give more information about who passed s == NULL to these
    >> functions.
    >>
    >
    > With the dump_stack() in place:
    >
    > Now, booting the kernel...
    >
    > [ 0.000000] Linux version 6.1.0-rc3+ (john@john) (mipsel-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.11-4428-g6b6741b) 12.2.0, GNU ld (GNU Binutils) 2.39) #62 SMP Tue Nov 1 19:49:52 AEST 2022
    > [ 0.000000] slub: __kmem_cache_alloc_lru called with kmem_cache ptr: 0x0
    > [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 6.1.0-rc3+ #62
    > [ 0.000000] Stack : 810fff78 80084d98 80889d00 00000004 00000000 00000000 80889d5c 80c90000
    > [ 0.000000] 80920000 807bd380 8089d368 80923bd3 00000000 00000001 80889d08 00000000
    > [ 0.000000] 00000000 00000000 807bd380 8084bd51 00000002 00000002 00000001 6d6f4320
    > [ 0.000000] 00000000 80c97ce9 80c97d14 fffffffc 807bd380 00000000 00000003 00000dc0
    > [ 0.000000] 00000000 a0000000 80910000 8110a0b4 00000000 00000020 80010000 80010000
    > [ 0.000000] ...
    > [ 0.000000] Call Trace:
    > [ 0.000000] [<80008260>] show_stack+0x28/0xf0
    > [ 0.000000] [<8070cdc0>] dump_stack_lvl+0x60/0x80
    > [ 0.000000] [<801c1428>] kmem_cache_alloc+0x5c0/0x740
    > [ 0.000000] [<8092856c>] prom_soc_init+0x1fc/0x2b4
    > [ 0.000000] [<80928060>] prom_init+0x44/0xf0
    > [ 0.000000] [<80929214>] setup_arch+0x4c/0x6a8
    > [ 0.000000] [<809257e0>] start_kernel+0x88/0x7c0
    > [ 0.000000]
    > [ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3

    The stack means CONFIG_TRACING=n, is that right?

    That would mean
    prom_soc_init()
    soc_dev_init()
    kzalloc() -> kmalloc()
    kmalloc_trace() // after #else /* CONFIG_TRACING */
    kmem_cache_alloc(s, flags);

    Looks like this path is a small bug in the wasting detection patch, as we
    throw away size there.

    AFAICS before this patch, we "survive" "kmem_cache *s" being NULL as
    slab_pre_alloc_hook() will happen to return NULL and we bail out from
    slab_alloc_node(). But this is a side-effect, not an intended protection.
    Also the CONFIG_TRACING variant of kmalloc_trace() would have called
    trace_kmalloc dereferencing s->size anyway even before this patch.

    I don't think we should add WARNS in the slab hot paths just to prevent this
    rare error of using slab too early. At most VM_WARN... would be acceptable
    but still not necessary as crashing immediately from a NULL pointer is
    sufficient.

    So IMHO mips should fix their soc init, and we should look into the
    CONFIG_TRACING=n variant of kmalloc_trace(), to pass orig_size properly.

    \
     
     \ /
      Last update: 2022-11-02 09:22    [W:3.793 / U:0.028 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site