Messages in this thread | | | Date | Mon, 20 Mar 2000 03:23:40 +0000 | From | John Ripley <> | Subject | Re: Overcommitable memory?? |
| |
> >James Sutherland wrote: > >> On Thu, 16 Mar 2000, Paul Jakma wrote: > >> malloc() CAN be overcommitted. If you set a VM flag via /proc, then > >> malloc() will *ALWAYS* succeed, even if there isn't any memory available > >> at all. With the flag clear, malloc() does some sanity checking before > >> granting the memory. > >> > >> You CAN obtain an overcommit free malloc() by clearing the VM flag (it is > >> clear by default), then touching every page you allocate when you allocate > >> it. > > > >No, this will just result in your process SIGBUS'ing. The current > >do_mmap (in mmap_fixup) will call make_pages_present(). There's no error > >return. It'll just die if one of the pages could not be faulted. > > However it is handled, you now know that your memory allocation > succeeded and you have real memory on your hands, not just an IOU. If > there isn't any memory available, you get SIGBUSed - but then OOM > conditions usually lead to death anyway.
The key word here is "usually". If you're designing a system from the ground up, you can have absolute failsafes for OOM - but only if syscalls such as mmap and brk return ENOMEM for mappings which cannot with 100% certainty be met. I'm sure people running simulations that take weeks or embedded systems would also like such behaviour. And as for fork(), I never did like the concept of cloning the entire process only to replace it. vfork() is perhaps not elegant, but it does solve the problem. Dare I mention NT has a "nice" way of spawning a new process complete with specified file descriptor sets etc.
If you patch mmap/brk to return ENOMEM when they would overcommit and then use vfork() instead of fork() for all your apps, you can guarantee that once all memory is allocated they will never die unexpectedly. And if they can't allocate memory in the first place, they can gracefully take an appropriate course of action: inform the user, wait a few minutes, or perhaps even quit (with an error return value saying why). The exceptions here are read()/write() (and friends) which need to allocate buffers - but you can specify a minimum buffer cache size, as well as a few kernel reserved pages. And even then, you can wrap up your read/write calls to take appropriate action... Or mmap() a file which would contain the results, mlock() it into memory and write to that - no need for buffers being allocated!
This is not as silly as it sounds. If you want a guarenteed stable app, you at least want guarentees from the kernel. A way of turning on/off overcommit per-process would be absolutely ideal for some of the things I'm doing.
-- John Ripley, empeg Ltd. http://www.empeg.com
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |