Messages in this thread | | | From | Stanislav Meduna <> | Subject | Process memory vs. total vm, resource limiting? | Date | Sun, 20 Jun 1999 17:55:55 +0200 (CEST) |
| |
Hello,
I have seen some posts in the local newsgroup regarding problems with static data and I started to experiment a bit. My setup is PPro, 96 MB RAM, 135 MB of swap in one 65 MB and one 70 MB partition (total 231 MB of vm), stock 2.2.10 kernel, egcs 1.1.2, glibc 2.1.1.
The limit setting is cputime unlimited filesize unlimited datasize unlimited stacksize 8192 kbytes coredumpsize 0 kbytes memoryuse unlimited descriptors 1024 memorylocked unlimited maxproc 256 openfiles 1024
Program #1:
=== snip === #include <stdio.h> #include <malloc.h>
/* define to represent appr. half of your mem+swap */ #define N1 128
static int x[N1*1024*1024/sizeof(int)];
main() { int i;
char *p;
if (! (p=(char *)malloc(N1*1024*1024))) fprintf(stderr, "process %d: Not enough memory for malloc\n", getpid());
fprintf(stderr, "process %d setting dynamic area\n", getpid());
for (i=0; i < N1*1024*1024; i+=4096) p[i] = 1;
fprintf(stderr, "process %d set dynamic area\n", getpid());
fprintf(stderr, "process %d setting static area\n", getpid());
for (i=0; i < sizeof(x)/sizeof(int); i+=4096/sizeof(int)) x[i] = 1;
fprintf(stderr, "process %d set static area\n", getpid());
sleep(10);
fprintf(stderr, "process %d ending\n", getpid());
exit(0); } === snip ==
The program defines a static area of 128 MB, then allocates 128 MB dynamically, writes something there and proceeds to write to the static area.
The surprise #1 is that the dynamic alloc succeeded, even if the memory requested by the process was more than total amount of vm in the machine. I find the assumption that the process won't be using all of the requested memory a bit too optimistic. Is there a possibility to change this behaviour?
The process then proceeded to set the static area. When it consumed all of the available vm, it crashed with a bus error. This is _bad_ - a well written program checks for the mallocs() and such returning zero, but there is no easy way (if any) of detecting that the system cannot make a copy-on-write for a page in a static buffer. This can happen just anytime (the huge area makes it more probable) and the problem is that if I want to make a robust application, I cannot. Not good for critical applications.
Of course, there were more applications that crashed because of memory shortage - X server being one of them (so I cannot copy&paste the result of the program :-)) and IDE subsystem was having problems too:
% dmesg
hdc: timeout waiting for DMA hdc: irq timeout: status=0x58 { DriveReady SeekComplete DataRequest } hdc: DMA disabled ide1: reset: success
Out of memory for httpd.
Out of memory for klogd.
Out of memory for update.
Out of memory for actived.
Out of memory for rclock.
Out of memory for syslogd.
Out of memory for gpm.
Out of memory for init.
Well, let's check whether we can guard against such code deployed as DoS:
=== snip === #include <sys/time.h> #include <sys/resource.h> #include <unistd.h> #include <stdio.h>
static char x[128*1024*1024];
main() { int i; struct rlimit limit;
getrlimit(RLIMIT_RSS, &limit); printf("RSS limit curr, max: %d, %d\n", limit.rlim_cur, limit.rlim_max);
if (! malloc(20000000)) printf("malloc not successfull - good\n");
for (i=0; i < sizeof(x); i+=4096) x[i] = 1;
system("ps gauxww | grep a.out | grep -v grep");
exit(0); } === snip ===
% limit memoryuse 10000 % limit datasize 10000 % ./a.out RSS limit curr, max: 10240512, 2147483647 stano 1929 27.4 81.9 151660 78760 pts/3 S 16:47 0:03 ./a.out
so the process happily consumed 77 MB of real memory and allowed to allocate 20 MB dynamically instead of failing because of the limit.
Now my question is: is there any way to guard against this?
I don't know the vm subsystem and the interactions with glibc - the dynamic allocation obviously goes through mmap and is not checked against the limits. The static area is in the object header and probably gets around the check too ... Perhaps someone clarifies these issues.
Regards -- Stano
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |