Messages in this thread |  | | Subject | Re: [PATCH 01/51] CPU hotplug: Provide lockless versions of callback registration functions | From | Toshi Kani <> | Date | Tue, 11 Feb 2014 09:33:56 -0700 |
| |
On Tue, 2014-02-11 at 09:27 +0000, Srivatsa S. Bhat wrote: > On 02/11/2014 06:56 AM, Toshi Kani wrote: > > On Thu, 2014-02-06 at 03:34 +0530, Srivatsa S. Bhat wrote: > > : > [...] > >> > >> Also, since cpu_maps_update_begin/done() is like a super-set of > >> get/put_online_cpus(), the former naturally protects the critical sections > >> from concurrent hotplug operations. > > > > get/put_online_cpus() is a reader-lock and concurrent executions are > > allowed among the readers. They won't be serialized until a cpu > > online/offline operation begins. By replacing this lock with > > cpu_maps_update_begin/done(), we now serialize all readers. Isn't that > > too restrictive? > > That's an excellent line of thought! It doesn't really hurt at the moment > because the for_each_online_cpu() kind of loop that the initcalls of various > subsystems run (before registering the notifier) are really tiny (typically > the loop runs for just 1 cpu, the boot-cpu). In other words, this change > represents a tiny increase in the critical section size; so its effect > shouldn't be noticeable. (Note that in the old model, register_cpu_notifier() > already takes the cpu_add_remove_lock, so they will be serialized at that > point, and this is necessary). > > However, going forward, when we start using more aggressive CPU onlining > techniques during boot (such as parallel CPU hotplug), the issue you pointed > out can become a real bottleneck, since for_each_online_cpu() can become > quite a large loop, and hence explicit (and unnecessary) mutual exclusion > will start hurting. > > > Can we fix the issue with CPU_POST_DEAD and continue > > to use get_online_cpus()? > > > > We don't want to get rid of CPU_POST_DEAD, so unfortunately we can't continue > to use get_online_cpus(). However, I am thinking of introducing a Reader-Writer > semaphore for this purpose, so that the registration routines can run in > parallel most of the time. (Basically, the rw-semaphore is like > get/put_online_cpus(), except that it protects the full hotplug critical section, > including the CPU_POST_DEAD stage.)
I agree that introducing a reader-writer semaphore allows concurrent executions. Adding yet another hotplug lock is a bit unfortunate, though.
This may be a dumb question, but can't we simply do this way?
get_online_cpus();
for_each_online_cpu(cpu) init_cpu(cpu);
put_online_cpus();
register_cpu_notifier(&foobar_cpu_notifier);
Thanks, -Toshi
|  |