Messages in this thread | | | Date | Mon, 23 Jun 2014 20:57:35 +0200 | From | Peter Zijlstra <> | Subject | Re: [RFC PATCH 1/1] kernel/rcu/tree.c: simplify force_quiescent_state() |
| |
On Mon, Jun 23, 2014 at 10:33:08AM -0700, Paul E. McKenney wrote: > On Mon, Jun 23, 2014 at 08:57:50AM -0700, Paul E. McKenney wrote: > > On Mon, Jun 23, 2014 at 12:28:50PM +0200, Peter Zijlstra wrote: > > > On Tue, Jun 17, 2014 at 10:37:17AM -0700, Paul E. McKenney wrote: > > > > Oh, and to answer the implicit question... A properly configured 4096-CPU > > > > system will have two funnel levels, with 64 nodes at the leaf level > > > > and a single node at the root level. If the system is not properly > > > > configured, it will have three funnel levels. The maximum number of > > > > funnel levels is four, which would handle more than four million CPUs > > > > (sixteen million if properly configured), so we should be good. ;-) > > > > > > > > The larger numbers of levels are intended strictly for testing. I set > > > > CONFIG_RCU_FANOUT_LEAF=2 and CONFIG_RCU_FANOUT=2 on a 16-CPU system just > > > > to make sure that I am testing something uglier than what will be running > > > > in production. A large system should have both of these set to 64, > > > > though this requires also booting with skew_tick=1 as well. > > > > > > Right, and I think we talked about this before; the first thing one > > > should do is align the RCU fanout masks with the actual machine > > > topology. Because currently they can be all over the place. > > > > And we also talked before about how it would make a lot more sense to > > align the CPU numbering with the actual machine topology, as that would > > fix the problem in one place. But either way, in the particular case > > of the RCU fanout, does anyone have any real data showing that this is > > a real problem? Given that the rcu_node accesses are quite a ways off > > of any fastpath, I remain skeptical. > > And one way to test for this is to set CONFIG_RCU_FANOUT to the number of > cores in a socket (or to the number of hardware threads per socket for > systems that number their hardware threads consecutively), then specify > CONFIG_RCU_FANOUT_EXACT=y. This will align the rcu_node structures with > the sockets. If the number of cores/threads per socket is too large, > you can of course use a smaller number that exactly divides the number > of cores/threads per socket.
Typical Intel cpu numbering is [0..n) for SMT0 and [n..2*n) for SMT1, so that'll fall flat on its face at try 1.
| |