lkml.org 
[lkml]   [2015]   [May]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH 0/13] Parallel struct page initialisation v4
On 05/06/2015 01:58 PM, Waiman Long wrote:
> On 05/06/2015 06:22 AM, Mel Gorman wrote:
>> On Wed, May 06, 2015 at 08:12:46AM +0100, Mel Gorman wrote:
>>> On Tue, May 05, 2015 at 03:25:49PM -0700, Andrew Morton wrote:
>>>> On Tue, 5 May 2015 23:13:29 +0100 Mel Gorman<mgorman@suse.de> wrote:
>>>>
>>>>>> Alternatively, the page allocator can go off and synchronously
>>>>>> initialize some pageframes itself. Keep doing that until the
>>>>>> allocation attempt succeeds.
>>>>>>
>>>>> That was rejected during review of earlier attempts at this
>>>>> feature on
>>>>> the grounds that it impacted allocator fast paths.
>>>> eh? Changes are only needed on the allocation-attempt-failed path,
>>>> which is slow-path.
>>> We'd have to distinguish between falling back to other zones because
>>> the
>>> high zone is artifically exhausted and normal ALLOC_BATCH
>>> exhaustion. We'd
>>> also have to avoid falling back to remote nodes prematurely. While I
>>> have
>>> not tried an implementation, I expected they would need to be in the
>>> fast
>>> paths unless I used jump labels to get around it. I'm going to try
>>> altering
>>> when we initialise instead so that it happens earlier.
>>>
>> Which looks as follows. Waiman, a test on the 24TB machine would be
>> appreciated again. This patch should be applied instead of "mm: meminit:
>> Take into account that large system caches scale linearly with memory"
>>
>> ---8<---
>> mm: meminit: Finish initialisation of memory before basic setup
>>
>> Waiman Long reported that 24TB machines hit OOM during basic setup when
>> struct page initialisation was deferred. One approach is to
>> initialise memory
>> on demand but it interferes with page allocator paths. This patch
>> creates
>> dedicated threads to initialise memory before basic setup. It then
>> blocks
>> on a rw_semaphore until completion as a wait_queue and counter is
>> overkill.
>> This may be slower to boot but it's simplier overall and also gets
>> rid of a
>> lot of section mangling which existed so kswapd could do the
>> initialisation.
>>
>> Signed-off-by: Mel Gorman<mgorman@suse.de>
>>
>
> This patch moves the deferred meminit from kswapd to its own kernel
> threads started after smp_init(). However, the hash table allocation
> was done earlier than that. It seems like it will still run out of
> memory in the 24TB machine that I tested on.
>
> I will certainly try it out, but I doubt it will solve the problem on
> its own.

It turns out that the two new patches did work on the 24-TB DragonHawk
without the "mm: meminit: Take into account that large system caches
scale linearly with memory" patch. The bootup time was 357s which was
just a few seconds slower than the other bootup times that I sent you
yesterday.

BTW, do you want to change the following log message as kswapd will no
longer be the one doing deferred meminit?

kswapd 0 initialised 396098436 pages in 6024ms

Cheers,
Longman



\
 
 \ /
  Last update: 2015-05-07 04:41    [W:0.769 / U:1.300 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site