Messages in this thread | | | Date | Thu, 12 Jun 2014 16:02:41 +0200 | From | Stefan Bader <> | Subject | Re: fs/stat: Reduce memory requirements for stat_open |
| |
On 12.06.2014 15:41, Heiko Carstens wrote: > On Thu, Jun 12, 2014 at 03:00:17PM +0200, Stefan Bader wrote: >> When reading from /proc/stat we allocate a large buffer to maximise >> the chances of the results being from a single run and thus internally >> consistent. This currently is sized at 128 * num_possible_cpus() which, >> in the face of kernels sized to handle large configurations (256 cpus >> plus), results in the buffer being an order-4 allocation or more. >> When system memory becomes fragmented these cannot be guarenteed, leading >> to read failures due to allocation failures. >> >> There seem to be two issues in play here. Firstly the allocation is >> going to be vastly over sized in the common case, as we only consume the >> buffer based on the num_online_cpus(). Secondly, regardless of size we >> should not be requiring allocations greater than PAGE_ALLOC_COSTLY_ORDER >> as allocations above this order are significantly more likely to fail. >> >> The following patch addesses both of these issues. Does that make sense >> generally? It seemed to stop top complaining wildly for the reporter >> at least. > > Hi Stefan, > > see also https://lkml.org/lkml/2014/5/21/341
Hi Heiko,
doh, so you guys have been hit by that before. And I have missed the fact that single_open is special. Which makes the change for the upper limit do the wrong thing. While long-term it sounds like changing it to vmalloc or iterative reads sounds better, maybe the change from possible to online cpus might be something that is better acceptable as a quick fix... Not sure. At least this giving back a bit of attention to the matter and it is not only affecting zSeries. x86 starts to see hw that requires a similar high possible cpus... :)
-Stefan
> > and one possible solution: > > https://lkml.org/lkml/2014/5/30/191 > > and the other one: > > https://lkml.org/lkml/2014/6/12/92 > https://lkml.org/lkml/2014/6/12/107 > > Thanks, > Heiko >
[unhandled content-type:application/pgp-signature] | |