lkml.org 
[lkml]   [2021]   [Nov]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH v6 10/23] mm/shmem: Handle uffd-wp during fork()
    Date
    Normally we skip copy page when fork() for VM_SHARED shmem, but we can't skip
    it anymore if uffd-wp is enabled on dst vma. This should only happen when the
    src uffd has UFFD_FEATURE_EVENT_FORK enabled on uffd-wp shmem vma, so that
    VM_UFFD_WP will be propagated onto dst vma too, then we should copy the
    pgtables with uffd-wp bit and pte markers, because these information will be
    lost otherwise.

    Since the condition checks will become even more complicated for deciding
    "whether a vma needs to copy the pgtable during fork()", introduce a helper
    vma_needs_copy() for it, so everything will be clearer.

    Signed-off-by: Peter Xu <peterx@redhat.com>
    ---
    mm/memory.c | 49 +++++++++++++++++++++++++++++++++++++++++--------
    1 file changed, 41 insertions(+), 8 deletions(-)

    diff --git a/mm/memory.c b/mm/memory.c
    index fef6a91c5dfb..cc625c616645 100644
    --- a/mm/memory.c
    +++ b/mm/memory.c
    @@ -859,6 +859,14 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
    if (try_restore_exclusive_pte(src_pte, src_vma, addr))
    return -EBUSY;
    return -ENOENT;
    + } else if (is_pte_marker_entry(entry)) {
    + /*
    + * We're copying the pgtable should only because dst_vma has
    + * uffd-wp enabled, do sanity check.
    + */
    + WARN_ON_ONCE(!userfaultfd_wp(dst_vma));
    + set_pte_at(dst_mm, addr, dst_pte, pte);
    + return 0;
    }
    if (!userfaultfd_wp(dst_vma))
    pte = pte_swp_clear_uffd_wp(pte);
    @@ -1227,6 +1235,38 @@ copy_p4d_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
    return 0;
    }

    +/*
    + * Return true if the vma needs to copy the pgtable during this fork(). Return
    + * false when we can speed up fork() by allowing lazy page faults later until
    + * when the child accesses the memory range.
    + */
    +bool
    +vma_needs_copy(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma)
    +{
    + /*
    + * Always copy pgtables when dst_vma has uffd-wp enabled even if it's
    + * file-backed (e.g. shmem). Because when uffd-wp is enabled, pgtable
    + * contains uffd-wp protection information, that's something we can't
    + * retrieve from page cache, and skip copying will lose those info.
    + */
    + if (userfaultfd_wp(dst_vma))
    + return true;
    +
    + if (src_vma->vm_flags & (VM_HUGETLB | VM_PFNMAP | VM_MIXEDMAP))
    + return true;
    +
    + if (src_vma->anon_vma)
    + return true;
    +
    + /*
    + * Don't copy ptes where a page fault will fill them correctly. Fork
    + * becomes much lighter when there are big shared or private readonly
    + * mappings. The tradeoff is that copy_page_range is more efficient
    + * than faulting.
    + */
    + return false;
    +}
    +
    int
    copy_page_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma)
    {
    @@ -1240,14 +1280,7 @@ copy_page_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma)
    bool is_cow;
    int ret;

    - /*
    - * Don't copy ptes where a page fault will fill them correctly.
    - * Fork becomes much lighter when there are big shared or private
    - * readonly mappings. The tradeoff is that copy_page_range is more
    - * efficient than faulting.
    - */
    - if (!(src_vma->vm_flags & (VM_HUGETLB | VM_PFNMAP | VM_MIXEDMAP)) &&
    - !src_vma->anon_vma)
    + if (!vma_needs_copy(dst_vma, src_vma))
    return 0;

    if (is_vm_hugetlb_page(src_vma))
    --
    2.32.0
    \
     
     \ /
      Last update: 2021-11-15 09:04    [W:3.574 / U:0.208 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site