Messages in this thread | | | Date | Wed, 25 Jul 2012 12:10:41 -0400 (EDT) | From | Alan Stern <> | Subject | Re: [RFC PATCH 0/6] CPU hotplug: Reverse invocation of notifiers during CPU hotplug |
| |
On Wed, 25 Jul 2012, Srivatsa S. Bhat wrote:
> On 07/25/2012 08:27 PM, Alan Stern wrote: > > On Wed, 25 Jul 2012, Srivatsa S. Bhat wrote: > > > >> Hi, > >> > >> This patchset implements the approach of invoking the CPU hotplug callbacks > >> (notifiers) in one order during CPU online and in the reverse order during CPU > >> offline. The rationale behind this is that services for a CPU are started in a > >> particular order (perhaps, with implicit dependencies between them) while > >> bringing up the CPU, and hence, it makes sense to tear down the services in > >> the opposite order, thereby honoring most of the dependencies automatically > >> (and also correctly). This is explained in more detail in Patch 6. > > > > This strongly suggests that a notifier chain may be the wrong mechanism > > to use here. Notifiers provide only limited guarantees about ordering, > > and it's hard to say much about the services a particular chain will > > provide since callbacks can be added from anywhere. > > > > True, the ability to register any random callback from anywhere is still a > problem that we are fighting... The zillions of callbacks that we have today > makes the hotplug process quite entangled.. we can't even roll-back from a > failure easily! > > > Instead of adding all this complication to the notifier mechanism, how > > about using something else for CPU hotplug? > > > > The problem is that today, many different subsystems need to know about CPUs coming > up or going down.. And CPU hotplug is not atomic, it happens in stages, and the > coordination between those subsystems is what actually drives CPU hotplug, in a way.
All this reinforces the idea that notifiers are the wrong mechanism for CPU hotplug.
> At present, I think that the best we can do is to redesign the hotplug code such that > the number of callbacks that are needed can be reduced to a minimum amount and then > have good control over what those callbacks do. For example, Thomas Gleixner posted > the park/unpark patchset[1], which not only speeds-up CPU hotplug by avoiding destruction > and creation of per-cpu kthreads on every hotplug operation, but also gets rid of quite > a few notifiers by providing a framework to manage those per-cpu kthreads...
I think the best you can do is stop using notifiers and use something else instead. For example, a simple set of function calls (assuming you know beforehand what callbacks need to be invoked).
> One of the other ideas to improve the hotplug notifier stuff that came up during some > of the discussions was to implement explicit dependency tracking between the notifiers > and perhaps get rid of the priority numbers that are currently being used to provide > some sort of ordering between the callbacks. Links to some of the related discussions > are provided below.
This seems like misplaced over-engineering.
Alan Stern
| |