Messages in this thread | | | Date | Mon, 13 Mar 2000 09:01:14 -0500 (EST) | From | "Richard B. Johnson" <> | Subject | Re: Overcommitable memory?? Re: Some questions about linux kernel. |
| |
On Sun, 12 Mar 2000, Michael Bacarella wrote:
> > > > I take it you run your sash mlock()ed with a special > > > kernel patch to make sure procfs doesn't need to allocate > > > memory on sash's behalf :) > > > > Unfortunatelly not. > > > > But if would be posible to somehow dividie the memory into overcommitable > > and one which is not overcommitable it could help. > > Wouldn't this problem be avoided if the kernel DIDN'T overcommit memory? > > I mean, nobody tolerates their filesystem overcommitting blocks it > doesn't have (or maybe they do and my reality is a myth). Why should > it be tolerated for virtual memory? > > Are the benefits to saying "uh, sure. we only have 600 megs of VM, but uh, > feel free to let your system commit 1 gig" worth it? What makes that > behavior desirable? > > I'm not condemning, I'm just curious. > > -MB
It really isn't an option. Let's say you have a program that allocated most of the memory in your machine. Then, the program needs to fork() to do some simple command, the parent waits for the child to exit() then continues. The nature of fork() is that all address space is duplicated. The fork would not work unless the kernel over-committed. The first line of defense is called COW, where the kernel only duplicates pages if/when the child actually modifies them (Copy On Write).
You may think that this is a "special case". However, the kernel has no prior information about what is being executed during the fork() so it isn't special at all. In principle, the child could have accessed all of the parent's memory and written to its copy.
The next problem is with sparse buffers, where a program allocates a lot of data space because it might need it under certain conditions. Many/most such programs never use the entire amount of data space allocated. They just need it available for the worse-case. An example is `vi` (`vim`) and emacs.
Certainly the current `kill-the-memory-hog` situation has not found the correct solution. I think the kernel has to steal pages from the memory-hog itself, writing them to the swap device/file. This keeps a memory-hog out of everybody else's pants. But, the kernel doesn't really 'know' very much about each process at the present time. I think it will eventually have to keep track of the frequency of memory accesses (per task) so it knows what pages to steal when memory is low. And it's going to have to steal pages from the current task, not just some poor sucker that's been sleeping.
What this would do is force a memory-hog to slow down because its pages are being swapped, while better-behaved processes continue with little or no impact.
Of course, if you don't have a swap-file, there is nothing you can do. Some persons claim that a swap-file isn't necessary and should not be used. They come from the "Flat Earth Society".
Cheers, Dick Johnson
Penguin : Linux version 2.3.41 on an i686 machine (800.63 BogoMips).
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |