lkml.org 
[lkml]   [2008]   [Apr]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Does process need to have a kernel-side stack all the time?
Denys Vlasenko wrote:
> Hi Ingo,
>
> You are one of the experts in processes/threads and scheduling
> in Linux kernel, I hope you can answer this question.
>
> A lot of effort went into minimizing of stack usage.
> If I understand it correctly, one of the reasons for this
> was to be efficient and not have lots of pages
> used for stacks when we have a lot of threads
> (tens of thousands).

If your application is using tens of thousands of threads on hardware that can't
spare tens of megabytes to ensure that a thread will always have a kernel stack
when it needs one, your application is horribly misdesigned.

> A random thought occurred to me: in a system with so many
> threads most of them are not executing anyway, even on
> that gigantic Altix machines. Do they all need to have
> kernel stack, all the time? I mean: the process which
> is running in user space is not using kernel stack at all.
> Process which is not running on a CPU right now
> is not using it either. But they do still consume
> at least 4k (or 8k on 64bits) of RAM.

If they're sleeping, they need a kernel stack. If they're simply scheduled out,
then your system is massively overloaded, and you need more CPUs or fewer threads.

> Process absolutely must have kernel stack only when
> it is actively running in kernel code (not sleeping),
> right?

It absolutely needs a kernel stack when it's sleeping in the kernel. It does
not really need a stack if it's simply scheduled out, but sleeping should be the
typical case, if the application is designed and configured to operate efficiently.

> Can we have per-CPU kernel stacks instead, so that process
> gets a kernel stack only every time it enters the kernel;
> and make it so that the process which is scheduled away
> from a CPU does not need to have kernel stack?

You're essentially asking us to optimize forkbombs at the expense of
well-designed applications. Unless the cost is nearly zero (and it's not) we
shouldn't do something like this.

> Currently, when process sleeps, we save some
> state in stack, and such a change may require
> some substantial surgery.

Yes, and that surgery will absolutely kill performance on the page fault and I/O
paths, while only saving a few kilobytes of RAM on well-configured systems.

> Can you tell me whether this is possible at all,
> and how difficult you estimate it to be?

It may be possible, but it's certainly not a good idea. Applications that
suffer a performance hit due to kernel stack usage while scheduled out are
poorly designed and need to be fixed. The fraction of a percent performance
boost they'd get from this change is nothing compared to the thousand percent
speedup they'd get from using threads intelligently.

-- Chris


\
 
 \ /
  Last update: 2008-04-15 01:25    [W:0.076 / U:0.476 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site