Messages in this thread | | | Date | Sat, 23 Aug 2008 21:55:38 -0700 | From | Jeremy Fitzhardinge <> | Subject | Re: [PATCH 2/2] smp_call_function: use rwlocks on queues rather than rcu |
| |
Andi Kleen wrote: > On Fri, Aug 22, 2008 at 11:35:46AM -0700, Jeremy Fitzhardinge wrote: > >> Andi Kleen wrote: >> >>> Right now my impression is that it is not well understood why >>> the kmalloc makes the IPI that much slower. In theory a kmalloc >>> shouldn't be all that slow, it's essentially just a >>> "disable interrupts; unlink object from cpu cache; enable interrupts" >>> with some window dressing. kfree() is similar. >>> >>> Does it bounce a cache line on freeing perhaps? >>> >> I think it's just an assumption that it would be slower. Has anyone >> measured it? >> > > It's likely slower than no kmalloc because > there will be more instructions executed, the question is just how much. > > >> (Note: The measurements I posted do not cover this path, because it was >> on a two cpu system, and it was always using the call-single path.) >> > > Ah so it was already 25% slower even without kmalloc? I thought > that was with already. That doesn't sound good. Any idea where that slowdown > comes from?
Just longer code path, I think. It calls the generic smp_call_function_mask(), which then does a popcount on the cpu mask (which it needs to do anyway), sees only one bit set, and then punts to the smp_call_function_single() path. If we maintained a cpus_online count, then we could fast-path the call to smp_call_function_single() in the two core/cpu case more efficiently (would still need to scan the mask to extract the cpu number).
Or alternatively, maybe it isn't actually worth special casing smp_call_function_single() with a multi-queue smp_call_function_mask() implementation?
J
| |