lkml.org 
[lkml]   [2022]   [Sep]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Correlation CMA size and FORCE_MAX_ZONEORDER
On 19.09.22 13:17, Michael Nazzareno Trimarchi wrote:
> Hi
>
> On Mon, Sep 19, 2022 at 1:03 PM David Hildenbrand <david@redhat.com> wrote:
>>
>> On 19.09.22 11:57, Michael Nazzareno Trimarchi wrote:
>>> Hi
>>>
>>> On Mon, Sep 19, 2022 at 11:31 AM David Hildenbrand <david@redhat.com> wrote:
>>>>
>>>> On 19.09.22 11:17, Michael Nazzareno Trimarchi wrote:
>>>>> Hi David
>>>>>
>>>>> On Mon, Sep 19, 2022 at 10:38 AM David Hildenbrand <david@redhat.com> wrote:
>>>>>>
>>>>>> On 15.09.22 23:36, Michael Nazzareno Trimarchi wrote:
>>>>>>> Hi all
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>>
>>>>>>> Working on a small device with 128MB of memory and using imx_v6_v7
>>>>>>> defconfig I found that CMA_SIZE_MBYTES, CMA_SIZE_PERCENTAGE
>>>>>>> are not respected. The calculation done does not allow the requested
>>>>>>> size. I think that this should be somehow documented and described but
>>>>>>> I did not
>>>>>>> find the documentation. Does it work this way?
>>>>>>>
>>>>>>> With CMA_SIZE of 8MB I need to have FORCE_MAX_ZONEORDER=12 if I have
>>>>>>> the default FORCE_MAX_ZONEORDER=14 the min size is 32Mb
>>>>>>
>>>>>> The underlying constraint is that CMA regions require a certain minimum
>>>>>> alignment+size. They cannot be arbitrarily in size.
>>>>>>
>>>>>> CMA_MIN_ALIGNMENT_BYTES expresses that, and corresponds in upstream
>>>>>> kernels to the size of a single pageblock.
>>>>>>
>>>>>> In previous kernels, it used to be the size of the largest buddy
>>>>>> allocation granularity (derived from MAX_ORDER, derived from
>>>>>> FORCE_MAX_ZONEORDER).
>>>>>>
>>>>>> On upstream kernels, the FORCE_MAX_ZONEORDER constraint should no longer
>>>>>> apply. On most archs, the minimum alignment+size should be 2 MiB
>>>>>> (x86-64, aarch64 with 4k base pages) -- the size of a single pageblock.
>>>>>>
>>>>>> So far the theory. Are you still running into this limitation on
>>>>>> upstream kernels?
>>>>>>
>>>>>
>>>>> I can run 6-rc2 on my board. I test again but according to it, if I
>>>>> put 4M as CMA in cma=4M in boot
>>>>> parameters, the result is 32Mb of CMA. Apart of that seems that
>>>>> process lime tiny membench can not even start
>>>>> to mblock memory
>>>>>
>>>>
>>>> The CMA alignemnt change went into v5.19. If "cma=4M" still gives you >
>>>> 4M, can you post /proc/meminfo and the early console output?
>>>>
>>>
>>> cat /proc/cmdline
>>> cma=4M mtdparts=gpmi-nand:4m(nandboot),1m(env),24m(kernel),1m(nanddtb),-(rootfs)
>>> root=ubi0:root rw ubi.mtd=ro
>>> otfs rootfstype=ubifs rootwait=1
>>> # cat /proc/meminfo
>>> MemTotal: 109560 kB
>>> MemFree: 56084 kB
>>> MemAvailable: 56820 kB
>>> Buffers: 0 kB
>>> Cached: 39680 kB
>>> SwapCached: 0 kB
>>> Active: 44 kB
>>> Inactive: 644 kB
>>> Active(anon): 44 kB
>>> Inactive(anon): 644 kB
>>> Active(file): 0 kB
>>> Inactive(file): 0 kB
>>> Unevictable: 39596 kB
>>> Mlocked: 0 kB
>>> HighTotal: 0 kB
>>> HighFree: 0 kB
>>> LowTotal: 109560 kB
>>> LowFree: 56084 kB
>>> SwapTotal: 0 kB
>>> SwapFree: 0 kB
>>> Dirty: 0 kB
>>> Writeback: 0 kB
>>> AnonPages: 628 kB
>>> Mapped: 1480 kB
>>> Shmem: 84 kB
>>> KReclaimable: 4268 kB
>>> Slab: 8456 kB
>>> SReclaimable: 4268 kB
>>> SUnreclaim: 4188 kB
>>> KernelStack: 392 kB
>>> PageTables: 88 kB
>>> NFS_Unstable: 0 kB
>>> Bounce: 0 kB
>>> WritebackTmp: 0 kB
>>> CommitLimit: 54780 kB
>>> Committed_AS: 1876 kB
>>> VmallocTotal: 901120 kB
>>> VmallocUsed: 2776 kB
>>> VmallocChunk: 0 kB
>>> Percpu: 72 kB
>>> CmaTotal: 32768 kB
>>> CmaFree: 32484 kB
>>> # uname -a
>>> Linux buildroot 6.0.0-rc5 #20 SMP Mon Sep 19 11:51:26 CEST 2022 armv7l GNU/Linux
>>> #
>>>
>>> Then here https://pastebin.com/6MUB2VBM dmesg
>>>
>>> CONFIG_ARM_MODULE_PLTS=y
>>> CONFIG_FORCE_MAX_ZONEORDER=14
>>> CONFIG_ALIGNMENT_TRAP=y
>>> ...
>>> CONFIG_CMA
>>> CONFIG_CMA_AREAS=7
>>> ...
>>>
>>> CONFIG_CMA_SIZE_MBYTES=8
>>> CONFIG_CMA_SIZE_SEL_MBYTES=y
>>>
>>
>> Thanks!
>>
>> I assume that in your setup, the pageblock size depends on MAX_ORDER
>> and, therefore, FORCE_MAX_ZONEORDER.
>>
>> This should be the case especially if CONFIG_HUGETLB_PAGE is not defined
>> (include/linux/pageblock-flags.h).
>>
>> In contrast to what I remember, the pageblock size does not seem to
>> depend on the THP size (weird) as well.
>>
>>
>> So, yes, that limitation is still in effect for some kernel configs.
>>
>> One could make the pageblock size configurable (similar to
>> CONFIG_HUGETLB_PAGE_SIZE_VARIABLE) or simply default to a smaller
>> pageblock size as default with CONFIG_CMA and !CONFIG_HUGETLB_PAGE.
>>
>> I imagine something reasonable might be to set the pageblock size to
>> 2MiB without CONFIG_HUGETLB_PAGE but with CONFIG_CMA.
>>
>
> I don't think making more configuration options makes things clear.

Yes, in an ideal case it should be automatic.

> When we enable some configuration
> we can force down the configuration. You need to explain clearly how
> you envision it. FORCE_MAX_ZONEORDER
> for me is the largest allocation that you can get from a zone (ex CMA
> one). Any request allocation that is align to the

FORCE_MAX_ZONEORDER is just a way to increase/decrease the maximum
allocation size of the buddy in general.

> CMA align and can fit inside a region should be allowed
>
> What am I missing?

I think that the issue is that the CMA alignments nowadays depend on the
pageblock size. And the pageblock size depends on *some* configurations
on the maximum allocation size of the buddy.

Documenting the interaction between FORCE_MAX_ZONEORDER and CMA size
alignment is not trivial.

For example, with CONFIG_HUGETLB_PAGE there might not be such an
interaction. With CONFIG_HUGETLB_PAGE, there clearly is one.


Let me phrase it this way: is it good enough in you setup to get 32mb vs
8mb or do you want/need to reduce it without adjusting
FORCE_MAX_ZONEORDER ?

--
Thanks,

David / dhildenb

\
 
 \ /
  Last update: 2022-09-19 13:30    [W:0.075 / U:0.192 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site