Messages in this thread Patch in this message | | | Date | Fri, 10 Feb 2006 15:13:29 +0100 | From | Eric Dumazet <> | Subject | Re: [PATCH] percpu data: only iterate over possible CPUs |
| |
Andrew Morton a écrit : > Eric Dumazet <dada1@cosmosbay.com> wrote: >> Andrew Morton a écrit : >>> Andi Kleen <ak@muc.de> wrote: >>>> On Thursday 09 February 2006 19:04, Andrew Morton wrote: >>>>> Ashok Raj <ashok.raj@intel.com> wrote: >>>>>> The problem was with ACPI just simply looking at the namespace doesnt >>>>>> exactly give us an idea of how many processors are possible in this platform. >>>>> We need to fix this asap - the performance penalty for HOTPLUG_CPU=y, >>>>> NR_CPUS=lots will be appreciable. >>>> What is this performance penalty exactly? >>> All those for_each_cpu() loops will hit NR_CPUS cachelines instead of >>> hweight(cpu_possible_map) cachelines. >> You mean NR_CPUS bits, mostly all included in a single cacheline, and even in >> a single long word :) for most cases (NR_CPUS <= 32 or 64) >> > > No, I mean cachelines: > > static void recalc_bh_state(void) > { > int i; > int tot = 0; > > if (__get_cpu_var(bh_accounting).ratelimit++ < 4096) > return; > __get_cpu_var(bh_accounting).ratelimit = 0; > for_each_cpu(i) > tot += per_cpu(bh_accounting, i).nr; > > That's going to hit NR_CPUS cachelines even on a 2-way. > > Or am I missing something really obvious here?
OK I see. This can be solved with this patch :
[PATCH] HOTPLUG_CPU : avoid hitting too many cachelines in recalc_bh_state()
Instead of using for_each_cpu(i), we can use for_each_online_cpu(i) : The difference matters if HOTPUG_CPU=y
When a CPU goes offline (ie removed from online map), it might have a non null bh_accounting.nr, so this patch adds a transfert of this counter to an online CPU counter.
We already have a hotcpu_notifier, (function buffer_cpu_notify()), where we can do this bh_accounting.nr transfert.
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com> --- a/fs/buffer.c 2006-02-10 15:08:21.000000000 +0100 +++ b/fs/buffer.c 2006-02-10 15:47:55.000000000 +0100 @@ -3138,7 +3138,7 @@ if (__get_cpu_var(bh_accounting).ratelimit++ < 4096) return; __get_cpu_var(bh_accounting).ratelimit = 0; - for_each_cpu(i) + for_each_online_cpu(i) tot += per_cpu(bh_accounting, i).nr; buffer_heads_over_limit = (tot > max_buffer_heads); } @@ -3187,6 +3187,9 @@ brelse(b->bhs[i]); b->bhs[i] = NULL; } + get_cpu_var(bh_accounting).nr += per_cpu(bh_accounting, cpu).nr ; + per_cpu(bh_accounting, cpu).nr = 0; + put_cpu_var(bh_accounting); } static int buffer_cpu_notify(struct notifier_block *self, | |