lkml.org 
[lkml]   [2021]   [Jul]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH 4.14 17/25] mm/thp: fix page_vma_mapped_walk() if THP mapped by ptes
    Date
    From: Hugh Dickins <hughd@google.com>

    [ Upstream commit a9a7504d9beaf395481faa91e70e2fd08f7a3dde ]

    Running certain tests with a DEBUG_VM kernel would crash within hours,
    on the total_mapcount BUG() in split_huge_page_to_list(), while trying
    to free up some memory by punching a hole in a shmem huge page: split's
    try_to_unmap() was unable to find all the mappings of the page (which,
    on a !DEBUG_VM kernel, would then keep the huge page pinned in memory).

    Crash dumps showed two tail pages of a shmem huge page remained mapped
    by pte: ptes in a non-huge-aligned vma of a gVisor process, at the end
    of a long unmapped range; and no page table had yet been allocated for
    the head of the huge page to be mapped into.

    Although designed to handle these odd misaligned huge-page-mapped-by-pte
    cases, page_vma_mapped_walk() falls short by returning false prematurely
    when !pmd_present or !pud_present or !p4d_present or !pgd_present: there
    are cases when a huge page may span the boundary, with ptes present in
    the next.

    Restructure page_vma_mapped_walk() as a loop to continue in these cases,
    while keeping its layout much as before. Add a step_forward() helper to
    advance pvmw->address across those boundaries: originally I tried to use
    mm's standard p?d_addr_end() macros, but hit the same crash 512 times
    less often: because of the way redundant levels are folded together, but
    folded differently in different configurations, it was just too
    difficult to use them correctly; and step_forward() is simpler anyway.

    Link: https://lkml.kernel.org/r/fedb8632-1798-de42-f39e-873551d5bc81@google.com
    Fixes: ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")
    Signed-off-by: Hugh Dickins <hughd@google.com>
    Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Cc: Alistair Popple <apopple@nvidia.com>
    Cc: Matthew Wilcox <willy@infradead.org>
    Cc: Peter Xu <peterx@redhat.com>
    Cc: Ralph Campbell <rcampbell@nvidia.com>
    Cc: Wang Yugui <wangyugui@e16-tech.com>
    Cc: Will Deacon <will@kernel.org>
    Cc: Yang Shi <shy828301@gmail.com>
    Cc: Zi Yan <ziy@nvidia.com>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Sasha Levin <sashal@kernel.org>
    ---
    mm/page_vma_mapped.c | 34 +++++++++++++++++++++++++---------
    1 file changed, 25 insertions(+), 9 deletions(-)

    diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
    index 96d4c4738590..16adeef76d00 100644
    --- a/mm/page_vma_mapped.c
    +++ b/mm/page_vma_mapped.c
    @@ -110,6 +110,13 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
    return true;
    }

    +static void step_forward(struct page_vma_mapped_walk *pvmw, unsigned long size)
    +{
    + pvmw->address = (pvmw->address + size) & ~(size - 1);
    + if (!pvmw->address)
    + pvmw->address = ULONG_MAX;
    +}
    +
    /**
    * page_vma_mapped_walk - check if @pvmw->page is mapped in @pvmw->vma at
    * @pvmw->address
    @@ -178,16 +185,22 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
    if (pvmw->pte)
    goto next_pte;
    restart:
    - {
    + do {
    pgd = pgd_offset(mm, pvmw->address);
    - if (!pgd_present(*pgd))
    - return false;
    + if (!pgd_present(*pgd)) {
    + step_forward(pvmw, PGDIR_SIZE);
    + continue;
    + }
    p4d = p4d_offset(pgd, pvmw->address);
    - if (!p4d_present(*p4d))
    - return false;
    + if (!p4d_present(*p4d)) {
    + step_forward(pvmw, P4D_SIZE);
    + continue;
    + }
    pud = pud_offset(p4d, pvmw->address);
    - if (!pud_present(*pud))
    - return false;
    + if (!pud_present(*pud)) {
    + step_forward(pvmw, PUD_SIZE);
    + continue;
    + }

    pvmw->pmd = pmd_offset(pud, pvmw->address);
    /*
    @@ -234,7 +247,8 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)

    spin_unlock(ptl);
    }
    - return false;
    + step_forward(pvmw, PMD_SIZE);
    + continue;
    }
    if (!map_pte(pvmw))
    goto next_pte;
    @@ -264,7 +278,9 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
    spin_lock(pvmw->ptl);
    }
    goto this_pte;
    - }
    + } while (pvmw->address < end);
    +
    + return false;
    }

    /**
    --
    2.30.2


    \
     
     \ /
      Last update: 2021-07-09 15:20    [W:3.129 / U:0.584 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site