lkml.org 
[lkml]   [2014]   [Dec]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: frequent lockups in 3.18rc4
On Thu, Dec 4, 2014 at 10:22 AM, Dave Hansen <dave.hansen@intel.com> 
wrote:
> On 12/03/2014 09:49 PM, Linus Torvalds wrote:
>> On Wed, Dec 3, 2014 at 7:15 PM, Chris Mason <clm@fb.com> wrote:
>>>
>>> One guess is that trinity is generating a huge number of tlb
>>> invalidations over sparse and horrible ranges. Perhaps the old
>>> code was
>>> falling back to full tlb flushes before Dave Hansen's string of
>>> fixes?
>>
>> Hmm. I agree that we've had some of the backtraces look like TLB
>> flushing might be involved. Not all, though. And I'm not seeing
>> where
>> a loop over up to 33 pages should matter over doing a full TLB
>> flush.
>>
>> What *might* matter is if we somehow get that number wrong, and the
>> loops like
>>
>> addr = f->flush_start;
>> while (addr < f->flush_end) {
>> __flush_tlb_single(addr);
>> addr += PAGE_SIZE;
>> }
>>
>> ends up looping a *lot* due to some bug, and then the IPI itself
>> would
>> take so long that the watchdog could trigger.
>>
>> But I do not see how that could actually happen. As far as I can
>> tell,
>> either the number of pages is limited to less than 33, or we have
>> that
>> TLB_FLUSH_ALL case.
>>
>> Do you see something I don't?
>
> The one thing I _do_ see now is a missed TLB flush is we're flushing
> one
> page at the end of the address space. We'd overflow flush_end back so
> flush_end=0:
>
> if (!f->flush_end)
> f->flush_end = f->flush_start + PAGE_SIZE; <--
> overflow
>
> and we'll never enter the while loop where we actually do the flush:
>
> while (addr < f->flush_end) {
> __flush_tlb_single(addr);
> addr += PAGE_SIZE;
> }
>
> But we have a hole up there on x86_64, so this will never happen in
> practice there. It might theoretically apply to 32-bit, but this
> still
> doesn't help with the bug.
>
> Oh, and the tracepoint is spitting out bogus numbers because we need
> some parenthesis around the 'nr_pages' calculation.

Yeah, I didn't see any problems with your changes, but I was hoping
that even a small change like doing 33 flushes at a time was pushing
Dave's box just over the line.

-chris





\
 
 \ /
  Last update: 2014-12-04 17:01    [W:0.280 / U:0.460 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site