lkml.org 
[lkml]   [2012]   [Oct]   [23]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: process hangs on do_exit when oom happens
From
On Tue, Oct 23, 2012 at 5:50 PM, Michal Hocko <mhocko@suse.cz> wrote:
> On Tue 23-10-12 15:18:48, Qiang Gao wrote:
>> This process was moved to RT-priority queue when global oom-killer
>> happened to boost the recovery of the system..
>
> Who did that? oom killer doesn't boost the priority (scheduling class)
> AFAIK.
>
>> but it wasn't get properily dealt with. I still have no idea why where
>> the problem is ..
>
> Well your configuration says that there is no runtime reserved for the
> group.
> Please refer to Documentation/scheduler/sched-rt-group.txt for more
> information.
>
>> On Tue, Oct 23, 2012 at 12:40 PM, Balbir Singh <bsingharora@gmail.com> wrote:
>> > On Tue, Oct 23, 2012 at 9:05 AM, Qiang Gao <gaoqiangscut@gmail.com> wrote:
>> >> information about the system is in the attach file "information.txt"
>> >>
>> >> I can not reproduce it in the upstream 3.6.0 kernel..
>> >>
>> >> On Sat, Oct 20, 2012 at 12:04 AM, Michal Hocko <mhocko@suse.cz> wrote:
>> >>> On Wed 17-10-12 18:23:34, gaoqiang wrote:
>> >>>> I looked up nothing useful with google,so I'm here for help..
>> >>>>
>> >>>> when this happens: I use memcg to limit the memory use of a
>> >>>> process,and when the memcg cgroup was out of memory,
>> >>>> the process was oom-killed however,it cannot really complete the
>> >>>> exiting. here is the some information
>> >>>
>> >>> How many tasks are in the group and what kind of memory do they use?
>> >>> Is it possible that you were hit by the same issue as described in
>> >>> 79dfdacc memcg: make oom_lock 0 and 1 based rather than counter.
>> >>>
>> >>>> OS version: centos6.2 2.6.32.220.7.1
>> >>>
>> >>> Your kernel is quite old and you should be probably asking your
>> >>> distribution to help you out. There were many fixes since 2.6.32.
>> >>> Are you able to reproduce the same issue with the current vanila kernel?
>> >>>
>> >>>> /proc/pid/stack
>> >>>> ---------------------------------------------------------------
>> >>>>
>> >>>> [<ffffffff810597ca>] __cond_resched+0x2a/0x40
>> >>>> [<ffffffff81121569>] unmap_vmas+0xb49/0xb70
>> >>>> [<ffffffff8112822e>] exit_mmap+0x7e/0x140
>> >>>> [<ffffffff8105b078>] mmput+0x58/0x110
>> >>>> [<ffffffff81061aad>] exit_mm+0x11d/0x160
>> >>>> [<ffffffff81061c9d>] do_exit+0x1ad/0x860
>> >>>> [<ffffffff81062391>] do_group_exit+0x41/0xb0
>> >>>> [<ffffffff81077cd8>] get_signal_to_deliver+0x1e8/0x430
>> >>>> [<ffffffff8100a4c4>] do_notify_resume+0xf4/0x8b0
>> >>>> [<ffffffff8100b281>] int_signal+0x12/0x17
>> >>>> [<ffffffffffffffff>] 0xffffffffffffffff
>> >>>
>> >>> This looks strange because this is just an exit part which shouldn't
>> >>> deadlock or anything. Is this stack stable? Have you tried to take check
>> >>> it more times?
>> >
>> > Looking at information.txt, I found something interesting
>> >
>> > rt_rq[0]:/1314
>> > .rt_nr_running : 1
>> > .rt_throttled : 1
>> > .rt_time : 0.856656
>> > .rt_runtime : 0.000000
>> >
>> >
>> > cfs_rq[0]:/1314
>> > .exec_clock : 8738.133429
>> > .MIN_vruntime : 0.000001
>> > .min_vruntime : 8739.371271
>> > .max_vruntime : 0.000001
>> > .spread : 0.000000
>> > .spread0 : -9792.255554
>> > .nr_spread_over : 1
>> > .nr_running : 0
>> > .load : 0
>> > .load_avg : 7376.722880
>> > .load_period : 7.203830
>> > .load_contrib : 1023
>> > .load_tg : 1023
>> > .se->exec_start : 282004.715064
>> > .se->vruntime : 18435.664560
>> > .se->sum_exec_runtime : 8738.133429
>> > .se->wait_start : 0.000000
>> > .se->sleep_start : 0.000000
>> > .se->block_start : 0.000000
>> > .se->sleep_max : 0.000000
>> > .se->block_max : 0.000000
>> > .se->exec_max : 77.977054
>> > .se->slice_max : 0.000000
>> > .se->wait_max : 2.664779
>> > .se->wait_sum : 29.970575
>> > .se->wait_count : 102
>> > .se->load.weight : 2
>> >
>> > So 1314 is a real time process and
>> >
>> > cpu.rt_period_us:
>> > 1000000
>> > ----------------------
>> > cpu.rt_runtime_us:
>> > 0
>> >
>> > When did tt move to being a Real Time process (hint: see nr_running
>> > and nr_throttled)?
>> >
>> > Balbir
>> --
>> To unsubscribe from this list: send the line "unsubscribe cgroups" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> Michal Hocko
> SUSE Labs
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/


maybe this is not a upstream-kernel bug. the centos/redhat kernel
would boost the process to RT prio when the process was selected
by oom-killer.

I think I should report this to redhat/centos.thanks for your attention


\
 
 \ /
  Last update: 2012-10-23 12:41    [W:0.090 / U:0.088 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site