lkml.org 
[lkml]   [2013]   [Jan]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH] module, fix percpu reserved memory exhaustion
Hello, Rusty.

On Sat, Jan 12, 2013 at 11:36:09AM +1030, Rusty Russell wrote:
> > I've been thinking about that. The problem is that at the same time the kvm
> > problem occurs I'm attempting to load a debug module that I've written to debug
> > some cpu timer issues that allocates a large amount of percpu data (~.5K/cpu).
> > While extending PERCPU_MODULE_RESERVE to 10k might work now, it might not work
> > tomorrow if I have the need to increase the size of my log buffer.
>
> Well, it looks like PERCPU_MODULE_RESERVE is actually very generous; it
> could easily be halved. I guess this is because dynamic per-cpu data is
> now such a nice alternative (thanks to Tejun).

Heh, thanks. :)

There are two percpu constants that I basically pulled out of my ass.

* PERCPU_MODULE_RESERVE: It's used on architectures with data model
which restricts the distance between code and data. e.g. x86_64
genereates code which can't address full 64bit for static data and
as percpu addressing depends on the regular address generation to
derive percpu addresses, the percpu offsets need to be restricted
somehow. Percpu allocator currently achieves this by reserving
certain amount in the first percpu chunk which is guaranteed to have
very low (starting from 0) percpu address space offsets.

On those archs, this becomes the hard limit for all module static
percpu areas. Maybe 8k is way too high and we can go with 4k.
Worth a try I guess.

* PERCPU_DYNAMIC_RESERVE: The amount of space at the end of the first
chunk used for dynamic allocation. This isn't as critical as the
above and the only reason it exists is because the first chunk is
often embedded in the kernel linear address space instead of vmalloc
area and the former is often mapped with larger page table size than
the latter. The goal of PERCPU_DYNAMIC_RESERVE is to have just
enough space to cover the usual percpu allocations without wasting
too much space to minimize impact on TLB pressure.

The current number isn't entirely made up. I did some testing with
some random config. Don't know how it holds up now. It probably
can be increased given that we developed quite a few percpu dynamic
allocations since then.

It would be awesome to have a way to somehow dynamically adjust this
one; unfortunately, that would require persistent data across
boots. :(

Thanks.

--
tejun


\
 
 \ /
  Last update: 2013-01-14 19:01    [W:0.771 / U:0.036 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site