lkml.org 
[lkml]   [2020]   [Jul]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH] mm/hugetlb: avoid hardcoding while checking if cma is reserved
On Mon, Jul 06, 2020 at 10:30:40PM +0000, Song Bao Hua (Barry Song) wrote:
>
>
> > -----Original Message-----
> > From: Song Bao Hua (Barry Song)
> > Sent: Tuesday, July 7, 2020 10:12 AM
> > To: 'Roman Gushchin' <guro@fb.com>
> > Cc: akpm@linux-foundation.org; linux-mm@kvack.org;
> > linux-kernel@vger.kernel.org; Linuxarm <linuxarm@huawei.com>; Mike
> > Kravetz <mike.kravetz@oracle.com>; Jonathan Cameron
> > <jonathan.cameron@huawei.com>
> > Subject: RE: [PATCH] mm/hugetlb: avoid hardcoding while checking if cma is
> > reserved
> >
> >
> >
> > > -----Original Message-----
> > > From: Roman Gushchin [mailto:guro@fb.com]
> > > Sent: Tuesday, July 7, 2020 9:48 AM
> > > To: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>
> > > Cc: akpm@linux-foundation.org; linux-mm@kvack.org;
> > > linux-kernel@vger.kernel.org; Linuxarm <linuxarm@huawei.com>; Mike
> > > Kravetz <mike.kravetz@oracle.com>; Jonathan Cameron
> > > <jonathan.cameron@huawei.com>
> > > Subject: Re: [PATCH] mm/hugetlb: avoid hardcoding while checking if
> > > cma is reserved
> > >
> > > On Mon, Jul 06, 2020 at 08:44:05PM +1200, Barry Song wrote:
> > >
> > > Hello, Barry!
> > >
> > > > hugetlb_cma[0] can be NULL due to various reasons, for example,
> > > > node0 has no memory. Thus, NULL hugetlb_cma[0] doesn't necessarily
> > > > mean cma is not enabled. gigantic pages might have been reserved on
> > other nodes.
> > >
> > > Just curious, is it a real-life problem you've seen? If so, I wonder
> > > how you're using the hugetlb_cma option, and what's the outcome?
> >
> > Yes. It is kind of stupid but I once got a board on which node0 has no DDR
> > though node1 and node3 have memory.
> >
> > I actually prefer we get cma size of per node by:
> > cma size of one node = hugetlb_cma/ (nodes with memory) rather than:
> > cma size of one node = hugetlb_cma/ (all online nodes)
> >
> > but unfortunately, or the N_MEMORY infrastructures are not ready yet. I
> > mean:
> >
> > for_each_node_state(nid, N_MEMORY) {
> > int res;
> >
> > size = min(per_node, hugetlb_cma_size - reserved);
> > size = round_up(size, PAGE_SIZE << order);
> >
> > res = cma_declare_contiguous_nid(0, size, 0, PAGE_SIZE << order,
> > 0, false, "hugetlb",
> > &hugetlb_cma[nid], nid);
> > ...
> > }
> >
>
> And for a server, there are many memory slots. The best config would be
> making every node have at least one DDR. But it isn't necessarily true, it
> is totally up to the users.
>
> If we move hugetlb_cma_reserve() a bit later, we probably make hugetlb_cma size
> completely consistent by splitting it to nodes with memory rather than nodes
> which are online:
>
> void __init bootmem_init(void)
> {
> ...
>
> arm64_numa_init();
>
> /*
> * must be done after arm64_numa_init() which calls numa_init() to
> * initialize node_online_map that gets used in hugetlb_cma_reserve()
> * while allocating required CMA size across online nodes.
> */
> - #ifdef CONFIG_ARM64_4K_PAGES
> - hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT);
> - #endif
>
> ...
>
> sparse_init();
> zone_sizes_init(min, max);
>
> + #ifdef CONFIG_ARM64_4K_PAGES
> + hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT);
> + #endif
> memblock_dump_all();
> }
>
> For x86, it could be done in similar way. Do you think it is worth to try?

It sounds like a good idea to me!

Thanks.

\
 
 \ /
  Last update: 2020-07-07 01:33    [W:0.053 / U:0.292 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site