Messages in this thread | | | Subject | Re: [PATCH v2] vhost: introduce mdev based hardware backend | From | Jason Wang <> | Date | Thu, 24 Oct 2019 18:42:50 +0800 |
| |
On 2019/10/24 下午5:18, Tiwei Bie wrote: > On Thu, Oct 24, 2019 at 04:32:42PM +0800, Jason Wang wrote: >> On 2019/10/24 下午4:03, Jason Wang wrote: >>> On 2019/10/24 下午12:21, Tiwei Bie wrote: >>>> On Wed, Oct 23, 2019 at 06:29:21PM +0800, Jason Wang wrote: >>>>> On 2019/10/23 下午6:11, Tiwei Bie wrote: >>>>>> On Wed, Oct 23, 2019 at 03:25:00PM +0800, Jason Wang wrote: >>>>>>> On 2019/10/23 下午3:07, Tiwei Bie wrote: >>>>>>>> On Wed, Oct 23, 2019 at 01:46:23PM +0800, Jason Wang wrote: >>>>>>>>> On 2019/10/23 上午11:02, Tiwei Bie wrote: >>>>>>>>>> On Tue, Oct 22, 2019 at 09:30:16PM +0800, Jason Wang wrote: >>>>>>>>>>> On 2019/10/22 下午5:52, Tiwei Bie wrote: >>>>>>>>>>>> This patch introduces a mdev based hardware vhost backend. >>>>>>>>>>>> This backend is built on top of the same abstraction used >>>>>>>>>>>> in virtio-mdev and provides a generic vhost interface for >>>>>>>>>>>> userspace to accelerate the virtio devices in guest. >>>>>>>>>>>> >>>>>>>>>>>> This backend is implemented as a mdev device driver on top >>>>>>>>>>>> of the same mdev device ops used in virtio-mdev but using >>>>>>>>>>>> a different mdev class id, and it will register the device >>>>>>>>>>>> as a VFIO device for userspace to use. Userspace can setup >>>>>>>>>>>> the IOMMU with the existing VFIO container/group APIs and >>>>>>>>>>>> then get the device fd with the device name. After getting >>>>>>>>>>>> the device fd of this device, userspace can use vhost ioctls >>>>>>>>>>>> to setup the backend. >>>>>>>>>>>> >>>>>>>>>>>> Signed-off-by: Tiwei Bie <tiwei.bie@intel.com> >>>>>>>>>>>> --- >>>>>>>>>>>> This patch depends on below series: >>>>>>>>>>>> https://lkml.org/lkml/2019/10/17/286 >>>>>>>>>>>> >>>>>>>>>>>> v1 -> v2: >>>>>>>>>>>> - Replace _SET_STATE with _SET_STATUS (MST); >>>>>>>>>>>> - Check status bits at each step (MST); >>>>>>>>>>>> - Report the max ring size and max number of queues (MST); >>>>>>>>>>>> - Add missing MODULE_DEVICE_TABLE (Jason); >>>>>>>>>>>> - Only support the network backend w/o multiqueue for now; >>>>>>>>>>> Any idea on how to extend it to support >>>>>>>>>>> devices other than net? I think we >>>>>>>>>>> want a generic API or an API that could >>>>>>>>>>> be made generic in the future. >>>>>>>>>>> >>>>>>>>>>> Do we want to e.g having a generic vhost >>>>>>>>>>> mdev for all kinds of devices or >>>>>>>>>>> introducing e.g vhost-net-mdev and vhost-scsi-mdev? >>>>>>>>>> One possible way is to do what vhost-user does. I.e. Apart from >>>>>>>>>> the generic ring, features, ... related ioctls, we also introduce >>>>>>>>>> device specific ioctls when we need them. As vhost-mdev just needs >>>>>>>>>> to forward configs between parent and userspace and even won't >>>>>>>>>> cache any info when possible, >>>>>>>>> So it looks to me this is only possible if we >>>>>>>>> expose e.g set_config and >>>>>>>>> get_config to userspace. >>>>>>>> The set_config and get_config interface isn't really everything >>>>>>>> of device specific settings. We also have ctrlq in virtio-net. >>>>>>> Yes, but it could be processed by the exist API. Isn't >>>>>>> it? Just set ctrl vq >>>>>>> address and let parent to deal with that. >>>>>> I mean how to expose ctrlq related settings to userspace? >>>>> I think it works like: >>>>> >>>>> 1) userspace find ctrl_vq is supported >>>>> >>>>> 2) then it can allocate memory for ctrl vq and set its address through >>>>> vhost-mdev >>>>> >>>>> 3) userspace can populate ctrl vq itself >>>> I see. That is to say, userspace e.g. QEMU will program the >>>> ctrl vq with the existing VHOST_*_VRING_* ioctls, and parent >>>> drivers should know that the addresses used in ctrl vq are >>>> host virtual addresses in vhost-mdev's case. >>> >>> That's really good point. And that means parent needs to differ vhost >>> from virtio. It should work. >> >> HVA may only work when we have something similar to VHOST_SET_OWNER which >> can reuse MM of its owner. > We already have VHOST_SET_OWNER in vhost now, parent can handle > the commands in its .kick_vq() which is called by vq's .handle_kick > callback. Virtio-user did something similar: > > https://github.com/DPDK/dpdk/blob/0da7f445df445630c794897347ee360d6fe6348b/drivers/net/virtio/virtio_user_ethdev.c#L313-L322
This probably means a process context is required, something like kthread that is used by vhost which seems a burden for parent. Or we can extend ioctl to processing kick in the system call context.
> >> >>> But is there any chance to use DMA address? I'm asking since the API >>> then tends to be device specific. >> >> I wonder whether we can introduce MAP IOMMU notifier and get DMA mappings >> from that. > I think this will complicate things unnecessarily and may > bring pains. Because, in vhost-mdev, mdev's ctrl vq is > supposed to be managed by host.
Yes.
> And we should try to avoid > putting ctrl vq and Rx/Tx vqs in the same DMA space to prevent > guests having the chance to bypass the host (e.g. QEMU) to > setup the backend accelerator directly.
That's really good point. So when "vhost" type is created, parent should assume addr of ctrl_vq is hva.
Thanks
> >> Thanks >>
| |