Messages in this thread | | | From | Xie Yongji <> | Subject | [PATCH v3 0/5] VDUSE: Support registering userspace memory as bounce buffer | Date | Wed, 20 Jul 2022 12:41:39 +0800 |
| |
Hi all,
This series introduces some new ioctls: VDUSE_IOTLB_GET_INFO, VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM to support registering and de-registering userspace memory for IOTLB as bounce buffer in virtio-vdpa case.
The VDUSE_IOTLB_GET_INFO ioctl can help user to query whether one IOVA region support userspace memory registration. Then user can use VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM ioctls to register and de-register userspace memory for this region.
During registering and de-registering, the DMA data in use would be copied from kernel bounce pages to userspace bounce pages and back.
With this feature, some existing application such as SPDK and DPDK can leverage the datapath of VDUSE directly and efficiently as discussed before [1][2]. They can register some preallocated hugepages to VDUSE to avoid an extra memcpy from bounce-buffer to hugepages.
The kernel and userspace codes could be found in github:
https://github.com/bytedance/linux/tree/vduse-umem https://github.com/bytedance/qemu/tree/vduse-umem
To test it with qemu-storage-daemon:
$ qemu-storage-daemon \ --chardev socket,id=charmonitor,path=/tmp/qmp.sock,server=on,wait=off \ --monitor chardev=charmonitor \ --blockdev driver=host_device,cache.direct=on,aio=native,filename=/dev/nullb0,node-name=disk0 \ --export type=vduse-blk,id=vduse-test,name=vduse-test,node-name=disk0,writable=on
[1] https://lkml.org/lkml/2021/6/27/318 [2] https://lkml.org/lkml/2022/7/4/246
Please review, thanks!
V2 to V3: - Check if the reserved fields is zero [MST] - Fix some compile errors - Rework the VDUSE_IOTLB_GET_INFO ioctl to hide some internal implementation (bounce buffer) [Jason] - Add more commit logs for patch 1 [MST]
V1 to V2: - Drop the patch that updating API version [MST] - Replace unpin_user_pages() with unpin_user_pages_dirty_lock() [MST] - Use __vmalloc(__GFP_ACCOUNT) for memory accounting [MST]
Xie Yongji (5): vduse: Remove unnecessary spin lock protection vduse: Use memcpy_{to,from}_page() in do_bounce() vduse: Support using userspace pages as bounce buffer vduse: Support registering userspace memory for IOVA regions vduse: Support querying information of IOVA regions
drivers/vdpa/vdpa_user/iova_domain.c | 102 +++++++++++++-- drivers/vdpa/vdpa_user/iova_domain.h | 8 ++ drivers/vdpa/vdpa_user/vduse_dev.c | 180 +++++++++++++++++++++++++++ include/uapi/linux/vduse.h | 48 +++++++ 4 files changed, 325 insertions(+), 13 deletions(-)
-- 2.20.1
| |