lkml.org 
[lkml]   [2012]   [Dec]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: backing up ext4 fs, system unresponsive, thrashing like crazy even though swap is unused
Dear Ronald,

sorry for not replying at your first message but I didn't consider
changing kernel as a resolution to my problem.

On Sun, 2 Dec 2012, Roland Eggner wrote:
> Hello Dimitrios!
>
>
> Which part of …
>
> “0,5 … 1 G RAM is usually occupied just by kernel slab [1]; this
> memory cannot be swapped, it cannot be released by other means than rebooting,
> and there seems to be _no_ adjustment to memory pressure.”
>
> … should I explain?

Check my initial message. If you see the /proc/meminfo attached you'll see
that from all slab memory most is reclaimable, however it's not being
reclaimed despite the memory pressure:

Slab: 64880 kB
SReclaimable: 60496 kB
SUnreclaim: 4384 kB

Also I mentioned that a nice'd -20 (Realtime!) process was unresponsive
because of two nice'd + ionice'd processes. This is a bug no matter the
memory pressure, IMHO. If not I'd rather hear it from someone more
authoritative.

> When tar or gzip writes to ext4 filesystem on your external disk, the kernel
> keeps all inode data in slab memory _by design_ , not by memory leaks.

ext4_inode_cache is a *cache*. So the kernel does not keep /all/ inode
data there, but only what was needed recently. And drops it afterwards,
which didn't happen after the backup ended.

> Several 100 M slab data cannot be stored within 128 M total RAM.
> This cannot by solved by usage of /proc/sys/vm/drop_caches.

Please take a look at [2], it's what is called in that case. Via this call
all slabs that have registered a shrinker, get actually shrinked.
Other fs do, but I can't find whether ext4 actually registers a shrinker
for its slabs.

[2] http://lxr.linux.no/#linux+v3.6.8/mm/vmscan.c#L207


> This cannot by solved by oom-killer-actions.
> This cannot be solved by zram tricks.
> Unmounting the external disk would release part of slab memory, but then you
> cannot backup.

Unmounting the back'ed up fs didn't really help in my case. Of course all
my filesystems are ext[234], mounted by ext4 code, so they use the same
slab for caches. But anyway, the slab didn't shrink significantly as I was
expecting.

> Reformatting the ext4 filesystem on your external disk with 128 byte inode size,
> largest possible blocksize and with extents enabled might mitigate the memory
> pressure slightly … probably not sufficient to get a working system with 3.6
> kernel.
>
>
> One advantage of Linux compared to other OS is much more support for old
> hardware, if a _proper_ kernel version is selected. Many years ago I used
> a notebook with 40 M total RAM, with a 2.4 kernel, Blackbox window manager and
> Opera web browser … it worked flawlessly … just rather slowly due to swapping.
> Your notebook has much more RAM than 40 M, thus there is surely a Linux solution
> for you. Try a 2.6.27.62 kernel, it supports ext4 (“ext4dev”), probably it
> supports all devices of your notebook, and with a slim window manager e.g.
> WindowMaker or OpenBox your notebook will probably “fly”.

I appreciate your advice. I remember when we were running 2.4 much less
memory was needed, but still I consider the kernel /fairly/ lean
considering the time passed and the bloat in userspace applications. I
choose to run latest kernels on old hardware, hopefully this will continue
to be the case, and if things deteriorate much then let's hope we'll have
enough time to help and fix them! :-)


Thanks,
Dimitris
\
 
 \ /
  Last update: 2012-12-03 01:21    [W:0.083 / U:0.160 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site