[lkml]   [2015]   [Oct]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH 0/14] init: deps: dependency based (parallelized) init
Am 17.10.2015 um 19:44 schrieb Greg Kroah-Hartman:
> On Sat, Oct 17, 2015 at 07:14:13PM +0200, Alexander Holler wrote:
>> Hello,
>> here is the newest version of my patches to use a dependency based
>> initialization order. It now works without DT too.
>> Background:
>> Currently initcalls are ordered by some levels and the link order. This
>> means whenever a file is renamed, changes directory or a Makefile is
>> modified the order with which initcalls are called might change. This
>> might result in problems. Furthermore, the required dependencies are
>> often not documented, sometimes there are comments in the source or in a
>> commit message, but most often the knowledge why a specific initcall
>> belongs to a specific initcall level isn't obvious without carefully
>> examing he source. And initcalls are used by drivers and subsystems, and
>> the count of both have grown quiet a lot in the last years. So it's
>> rather difficult to maintain a proper link order.
> Files move around very rarely, is this really an issue?

No idea. You are maintaining the staging area. ;)

>> Another problem is that the link order can't be modified dynamically at
>> boot time to include dependencies dictated by the hardware. To circumvent
>> this, a brute-force trial-and-error mechanism called deferred probes has
>> been introduced, but this approach, while beeing KISS, has its own
>> problems.
> What problems does deferred probing have? Why not just fix that if
> there is issues with it, as it was supposed to solve this issue without
> needing to annotate anything.

I've not looked why deferred probes are sometimes causing such a large
delay. But giving that it's brutforce and non-deterministic, some
drivers might be probed a dozen times, or important drivers might be
probed very late, forcing all previous probed drivers to be probed again
(late). Just think at the case the link order is a, b, c but drivers
have to be called in the order c, b, a.

>> To solve these problems I've written patches to use a topological sort at
>> boot time which uses dependencies to calculate the order with which
>> initcalls are called.
>> Why? What are the benefits (assuming correct dependencies are available)?
>> - It offers a clear in-source documentation for dependencies between
>> initcalls.
>> - It is robust in regard to file or directory name changes and changes in
>> a Makefile.
>> - If enabled, the order with which drivers for interfaces are called
>> (e.g. network interfaces, hard disks), can be defined independent of
>> the link order. These might result in more stable interface names or
>> numbers.
>> - If enabled, it makes the the deferred probes obsolete, which might
>> result in faster boot times.
>> - If enabled, it is possible to call initcalls in parallel. E.g. the
>> shipped kernel for Fedora 21 (4.1.7-100.fc21.x86_64) contains around
>> 560 initcalls. These are all called in series. Also some of them use
>> asynchronous stuff by themself, most don't do.
> But that shipped kernel boots to X in less than 2 seconds, so there
> isn't really a speed issue here, right?

It's noticeable if your phone, or any other thing you want to use (like
your route planner, clock or whatever boots in one second instead of two
when you turn it on.

>> Drawbacks:
>> - It requires a small amount of time to calculate the order a boot time.
>> But this time is most often smaller than the time saved by using
>> multiple cores to call initcalls or by not needing deferred probes.
> How much time is needed?

I've measured 3ms on a slow ARM box.

>> - Dependencies are required. For everything which can be build as a
>> module, looking at modules.dep might give some pointers. Looking at
>> the help from menuconfig also might give some pointers. But in the
>> end, the most preferable way would be if maintainers or other people
>> which have a deeper knowledge about the source and functionality
>> would add the dependencies.
> How will a "normal" driver author figure out what those dependancies are
> in order to be able to write them down? That's my biggest objection

Most drivers are done c&p from an existing driver. And if someone adds
new code, he should know what these new code is for and on what it depends.

> here, I have no idea how to add these, nor how to properly review such a
> submission. What about systems that have different ordering/dependancy
> requirements for the same drivers due to different ways the hardware is
> hooked up? That is not going to work well here, unless I'm missing
> something.

Hmm, how is that solved now?

If you have dependencies dictated by a special HW, this dependencies
should come in by the HW description.

The deferred probe mechanism exists just since 1 or 2 years. And once
you had to search the source of non-displayed error in a set of several
dozens possible sources (because many errors are now non-errors but
-517) you might learn to hate deferred probes as much as I do. I'm sorry
for the harsh words in regard to deferred probes.

The way to use dependencies doesn't add new requirements, it just moves
them away from the link order. Besides that these dependencies are
making it possible to parallelize the stuff.

Alexander Holler

 \ /
  Last update: 2015-10-17 20:41    [W:0.143 / U:0.156 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site