lkml.org 
[lkml]   [2013]   [Jul]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Yet more softlockups.
On 07/12/2013 03:31 AM, Ingo Molnar wrote:
> * Dave Jones <davej@redhat.com> wrote:
>> On Wed, Jul 10, 2013 at 05:20:15PM +0200, Markus Trippelsdorf wrote:
>> > On 2013.07.10 at 11:13 -0400, Dave Jones wrote:
>> > > I get this right after booting..
>> > >
>> > > [ 114.516619] perf samples too long (4262 > 2500), lowering kernel.perf_event_max_sample_rate to 50000
>> >
>> > You can disable this warning by:
>> >
>> > echo 0 > /proc/sys/kernel/perf_cpu_time_max_percent
>>
>> Yes, but why is this even being run when I'm not running perf ?
>>
>> The only NMI source running should be the watchdog.
>
> The NMI watchdog is a perf event.
>
> I've Cc:-ed Dave Hansen, the author of those changes - is this a false
> positive or some real problem?

The warning comes from calling perf_sample_event_took(), which is only
called from one place: perf_event_nmi_handler().

So we can be pretty sure that the perf NMI is firing, or at least that
this handler code is running.

nmi_handle() says:
/*
* NMIs are edge-triggered, which means if you have enough
* of them concurrently, you can lose some because only one
* can be latched at any given time. Walk the whole list
* to handle those situations.
*/

perf_event_nmi_handler() probably gets _called_ when the watchdog NMI
goes off. But, it should hit this check:

if (!atomic_read(&active_events))
return NMI_DONE;

and return quickly. This is before it has a chance to call
perf_sample_event_took().

Dave, for your case, my suspicion would be that it got turned on
inadvertently, or that we somehow have a bug which bumped up
perf_event.c's 'active_events' and we're running some perf code that we
don't have to.

But, I'm suspicious. I was having all kinds of issues with perf and
NMIs taking hundreds of milliseconds. I never isolated it to having a
real, single, cause. I attributed it to my large NUMA system just being
slow. Your description makes me wonder what I missed, though.


\
 
 \ /
  Last update: 2013-07-12 18:21    [W:0.139 / U:0.012 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site