lkml.org 
[lkml]   [2023]   [Feb]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    Subject[PATCH v3 25/35] mm: introduce lock_vma_under_rcu to be used from arch-specific code
    From
    Introduce lock_vma_under_rcu function to lookup and lock a VMA during
    page fault handling. When VMA is not found, can't be locked or changes
    after being locked, the function returns NULL. The lookup is performed
    under RCU protection to prevent the found VMA from being destroyed before
    the VMA lock is acquired. VMA lock statistics are updated according to
    the results.
    For now only anonymous VMAs can be searched this way. In other cases the
    function returns NULL.

    Signed-off-by: Suren Baghdasaryan <surenb@google.com>
    ---
    include/linux/mm.h | 3 +++
    mm/memory.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
    2 files changed, 49 insertions(+)

    diff --git a/include/linux/mm.h b/include/linux/mm.h
    index 3f98344f829c..36172bb38e6b 100644
    --- a/include/linux/mm.h
    +++ b/include/linux/mm.h
    @@ -701,6 +701,9 @@ static inline void vma_mark_detached(struct vm_area_struct *vma, bool detached)
    vma->detached = detached;
    }

    +struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
    + unsigned long address);
    +
    #else /* CONFIG_PER_VMA_LOCK */

    static inline void vma_init_lock(struct vm_area_struct *vma) {}
    diff --git a/mm/memory.c b/mm/memory.c
    index 8177c59ffd2d..5e1c124552a1 100644
    --- a/mm/memory.c
    +++ b/mm/memory.c
    @@ -5220,6 +5220,52 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
    }
    EXPORT_SYMBOL_GPL(handle_mm_fault);

    +#ifdef CONFIG_PER_VMA_LOCK
    +/*
    + * Lookup and lock a VMA under RCU protection. Returned VMA is guaranteed to be
    + * stable and not isolated. If the VMA is not found or is being modified the
    + * function returns NULL.
    + */
    +struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
    + unsigned long address)
    +{
    + MA_STATE(mas, &mm->mm_mt, address, address);
    + struct vm_area_struct *vma;
    +
    + rcu_read_lock();
    +retry:
    + vma = mas_walk(&mas);
    + if (!vma)
    + goto inval;
    +
    + /* Only anonymous vmas are supported for now */
    + if (!vma_is_anonymous(vma))
    + goto inval;
    +
    + if (!vma_start_read(vma))
    + goto inval;
    +
    + /* Check since vm_start/vm_end might change before we lock the VMA */
    + if (unlikely(address < vma->vm_start || address >= vma->vm_end)) {
    + vma_end_read(vma);
    + goto inval;
    + }
    +
    + /* Check if the VMA got isolated after we found it */
    + if (vma->detached) {
    + vma_end_read(vma);
    + /* The area was replaced with another one */
    + goto retry;
    + }
    +
    + rcu_read_unlock();
    + return vma;
    +inval:
    + rcu_read_unlock();
    + return NULL;
    +}
    +#endif /* CONFIG_PER_VMA_LOCK */
    +
    #ifndef __PAGETABLE_P4D_FOLDED
    /*
    * Allocate p4d page table.
    --
    2.39.1
    \
     
     \ /
      Last update: 2023-03-27 00:25    [W:4.092 / U:0.420 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site