Messages in this thread | | | Date | Wed, 13 Jul 2022 05:58:32 +0200 | Subject | Re: [PATCH v7 00/14] KVM: mm: fd-based approach for supporting KVM guest private memory | From | "Gupta, Pankaj" <> |
| |
> This is the v7 of this series which tries to implement the fd-based KVM > guest private memory. The patches are based on latest kvm/queue branch > commit: > > b9b71f43683a (kvm/queue) KVM: x86/mmu: Buffer nested MMU > split_desc_cache only by default capacity > > Introduction > ------------ > In general this patch series introduce fd-based memslot which provides > guest memory through memory file descriptor fd[offset,size] instead of > hva/size. The fd can be created from a supported memory filesystem > like tmpfs/hugetlbfs etc. which we refer as memory backing store. KVM
Thinking a bit, As host side fd on tmpfs or shmem will store memory on host page cache instead of mapping pages into userspace address space. Can we hit double (un-coordinated) page cache problem with this when guest page cache is also used?
Thanks, Pankaj
> and the the memory backing store exchange callbacks when such memslot > gets created. At runtime KVM will call into callbacks provided by the > backing store to get the pfn with the fd+offset. Memory backing store > will also call into KVM callbacks when userspace punch hole on the fd > to notify KVM to unmap secondary MMU page table entries. > > Comparing to existing hva-based memslot, this new type of memslot allows > guest memory unmapped from host userspace like QEMU and even the kernel > itself, therefore reduce attack surface and prevent bugs. > > Based on this fd-based memslot, we can build guest private memory that > is going to be used in confidential computing environments such as Intel > TDX and AMD SEV. When supported, the memory backing store can provide > more enforcement on the fd and KVM can use a single memslot to hold both > the private and shared part of the guest memory. > > mm extension > --------------------- > Introduces new MFD_INACCESSIBLE flag for memfd_create(), the file > created with these flags cannot read(), write() or mmap() etc via normal > MMU operations. The file content can only be used with the newly > introduced memfile_notifier extension. > > The memfile_notifier extension provides two sets of callbacks for KVM to > interact with the memory backing store: > - memfile_notifier_ops: callbacks for memory backing store to notify > KVM when memory gets invalidated. > - backing store callbacks: callbacks for KVM to call into memory > backing store to request memory pages for guest private memory. > > The memfile_notifier extension also provides APIs for memory backing > store to register/unregister itself and to trigger the notifier when the > bookmarked memory gets invalidated. > > The patchset also introduces a new memfd seal F_SEAL_AUTO_ALLOCATE to > prevent double allocation caused by unintentional guest when we only > have a single side of the shared/private memfds effective. > > memslot extension > ----------------- > Add the private fd and the fd offset to existing 'shared' memslot so > that both private/shared guest memory can live in one single memslot. > A page in the memslot is either private or shared. Whether a guest page > is private or shared is maintained through reusing existing SEV ioctls > KVM_MEMORY_ENCRYPT_{UN,}REG_REGION. > > Test > ---- > To test the new functionalities of this patch TDX patchset is needed. > Since TDX patchset has not been merged so I did two kinds of test: > > - Regresion test on kvm/queue (this patchset) > Most new code are not covered. Code also in below repo: > https://github.com/chao-p/linux/tree/privmem-v7 > > - New Funational test on latest TDX code > The patch is rebased to latest TDX code and tested the new > funcationalities. See below repos: > Linux: https://github.com/chao-p/linux/tree/privmem-v7-tdx > QEMU: https://github.com/chao-p/qemu/tree/privmem-v7 > > An example QEMU command line for TDX test: > -object tdx-guest,id=tdx,debug=off,sept-ve-disable=off \ > -machine confidential-guest-support=tdx \ > -object memory-backend-memfd-private,id=ram1,size=${mem} \ > -machine memory-backend=ram1 > > Changelog > ---------- > v7: > - Move the private/shared info from backing store to KVM. > - Introduce F_SEAL_AUTO_ALLOCATE to avoid double allocation. > - Rework on the sync mechanism between zap/page fault paths. > - Addressed other comments in v6. > v6: > - Re-organzied patch for both mm/KVM parts. > - Added flags for memfile_notifier so its consumers can state their > features and memory backing store can check against these flags. > - Put a backing store reference in the memfile_notifier and move pfn_ops > into backing store. > - Only support boot time backing store register. > - Overall KVM part improvement suggested by Sean and some others. > v5: > - Removed userspace visible F_SEAL_INACCESSIBLE, instead using an > in-kernel flag (SHM_F_INACCESSIBLE for shmem). Private fd can only > be created by MFD_INACCESSIBLE. > - Introduced new APIs for backing store to register itself to > memfile_notifier instead of direct function call. > - Added the accounting and restriction for MFD_INACCESSIBLE memory. > - Added KVM API doc for new memslot extensions and man page for the new > MFD_INACCESSIBLE flag. > - Removed the overlap check for mapping the same file+offset into > multiple gfns due to perf consideration, warned in document. > - Addressed other comments in v4. > v4: > - Decoupled the callbacks between KVM/mm from memfd and use new > name 'memfile_notifier'. > - Supported register multiple memslots to the same backing store. > - Added per-memslot pfn_ops instead of per-system. > - Reworked the invalidation part. > - Improved new KVM uAPIs (private memslot extension and memory > error) per Sean's suggestions. > - Addressed many other minor fixes for comments from v3. > v3: > - Added locking protection when calling > invalidate_page_range/fallocate callbacks. > - Changed memslot structure to keep use useraddr for shared memory. > - Re-organized F_SEAL_INACCESSIBLE and MEMFD_OPS. > - Added MFD_INACCESSIBLE flag to force F_SEAL_INACCESSIBLE. > - Commit message improvement. > - Many small fixes for comments from the last version. > > Links to previous discussions > ----------------------------- > [1] Original design proposal: > https://lkml.kernel.org/kvm/20210824005248.200037-1-seanjc@google.com/ > [2] Updated proposal and RFC patch v1: > https://lkml.kernel.org/linux-fsdevel/20211111141352.26311-1-chao.p.peng@linux.intel.com/ > [3] Patch v5: https://lkml.org/lkml/2022/5/19/861 > > Chao Peng (12): > mm: Add F_SEAL_AUTO_ALLOCATE seal to memfd > selftests/memfd: Add tests for F_SEAL_AUTO_ALLOCATE > mm: Introduce memfile_notifier > mm/memfd: Introduce MFD_INACCESSIBLE flag > KVM: Rename KVM_PRIVATE_MEM_SLOTS to KVM_INTERNAL_MEM_SLOTS > KVM: Use gfn instead of hva for mmu_notifier_retry > KVM: Rename mmu_notifier_* > KVM: Extend the memslot to support fd-based private memory > KVM: Add KVM_EXIT_MEMORY_FAULT exit > KVM: Register/unregister the guest private memory regions > KVM: Handle page fault for private memory > KVM: Enable and expose KVM_MEM_PRIVATE > > Kirill A. Shutemov (1): > mm/shmem: Support memfile_notifier > > Documentation/virt/kvm/api.rst | 77 +++++- > arch/arm64/kvm/mmu.c | 8 +- > arch/mips/include/asm/kvm_host.h | 2 +- > arch/mips/kvm/mmu.c | 10 +- > arch/powerpc/include/asm/kvm_book3s_64.h | 2 +- > arch/powerpc/kvm/book3s_64_mmu_host.c | 4 +- > arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +- > arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 +- > arch/powerpc/kvm/book3s_hv_nested.c | 2 +- > arch/powerpc/kvm/book3s_hv_rm_mmu.c | 8 +- > arch/powerpc/kvm/e500_mmu_host.c | 4 +- > arch/riscv/kvm/mmu.c | 4 +- > arch/x86/include/asm/kvm_host.h | 3 +- > arch/x86/kvm/Kconfig | 3 + > arch/x86/kvm/mmu.h | 2 - > arch/x86/kvm/mmu/mmu.c | 74 +++++- > arch/x86/kvm/mmu/mmu_internal.h | 18 ++ > arch/x86/kvm/mmu/mmutrace.h | 1 + > arch/x86/kvm/mmu/paging_tmpl.h | 4 +- > arch/x86/kvm/x86.c | 2 +- > include/linux/kvm_host.h | 105 +++++--- > include/linux/memfile_notifier.h | 91 +++++++ > include/linux/shmem_fs.h | 2 + > include/uapi/linux/fcntl.h | 1 + > include/uapi/linux/kvm.h | 37 +++ > include/uapi/linux/memfd.h | 1 + > mm/Kconfig | 4 + > mm/Makefile | 1 + > mm/memfd.c | 18 +- > mm/memfile_notifier.c | 123 ++++++++++ > mm/shmem.c | 125 +++++++++- > tools/testing/selftests/memfd/memfd_test.c | 166 +++++++++++++ > virt/kvm/Kconfig | 3 + > virt/kvm/kvm_main.c | 272 ++++++++++++++++++--- > virt/kvm/pfncache.c | 14 +- > 35 files changed, 1074 insertions(+), 127 deletions(-) > create mode 100644 include/linux/memfile_notifier.h > create mode 100644 mm/memfile_notifier.c >
| |