Messages in this thread | | | Subject | Re: [PATCH RFC tip/core/rcu 6/6] rcu: Reduce cache-miss initialization latencies for large systems | From | Peter Zijlstra <> | Date | Thu, 26 Apr 2012 14:51:47 +0200 |
| |
On Mon, 2012-04-23 at 09:42 -0700, Paul E. McKenney wrote: > From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > > Commit #0209f649 (rcu: limit rcu_node leaf-level fanout) set an upper > limit of 16 on the leaf-level fanout for the rcu_node tree. This was > needed to reduce lock contention that was induced by the synchronization > of scheduling-clock interrupts, which was in turn needed to improve > energy efficiency for moderate-sized lightly loaded servers. > > However, reducing the leaf-level fanout means that there are more > leaf-level rcu_node structures in the tree, which in turn means that > RCU's grace-period initialization incurs more cache misses. This is > not a problem on moderate-sized servers with only a few tens of CPUs, > but becomes a major source of real-time latency spikes on systems with > many hundreds of CPUs. In addition, the workloads running on these large > systems tend to be CPU-bound, which eliminates the energy-efficiency > advantages of synchronizing scheduling-clock interrupts. Therefore, > these systems need maximal values for the rcu_node leaf-level fanout. > > This commit addresses this problem by introducing a new kernel parameter > named RCU_FANOUT_LEAF that directly controls the leaf-level fanout. > This parameter defaults to 16 to handle the common case of a moderate > sized lightly loaded servers, but may be set higher on larger systems.
Wouldn't it be much better to match the rcu fanout tree to the physical topology of the machine?
| |