lkml.org 
[lkml]   [2022]   [Feb]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Subject[RFC PATCH v2 00/19] Control VQ support in vDPA
    Date
    Hi All:

    This series tries to add the support for control virtqueue in vDPA.

    Control virtqueue is used by networking device for accepting various
    commands from the driver. It's a must to support multiqueue and other
    configurations.

    When used by vhost-vDPA bus driver for VM, the control virtqueue
    should be shadowed via userspace VMM (Qemu) instead of being assigned
    directly to Guest. This is because Qemu needs to know the device state
    in order to start and stop device correctly (e.g for Live Migration).

    This requies to isolate the memory mapping for control virtqueue
    presented by vhost-vDPA to prevent guest from accessing it directly.

    To achieve this, vDPA introduce two new abstractions:

    - address space: identified through address space id (ASID) and a set
    of memory mapping in maintained
    - virtqueue group: the minimal set of virtqueues that must share an
    address space

    Device needs to advertise the following attributes to vDPA:

    - the number of address spaces supported in the device
    - the number of virtqueue groups supported in the device
    - the mappings from a specific virtqueue to its virtqueue groups

    The mappings from virtqueue to virtqueue groups is fixed and defined
    by vDPA device driver. E.g:

    - For the device that has hardware ASID support, it can simply
    advertise a per virtqueue virtqueue group.
    - For the device that does not have hardware ASID support, it can
    simply advertise a single virtqueue group that contains all
    virtqueues. Or if it wants a software emulated control virtqueue, it
    can advertise two virtqueue groups, one is for cvq, another is for
    the rest virtqueues.

    vDPA also allow to change the association between virtqueue group and
    address space. So in the case of control virtqueue, userspace
    VMM(Qemu) may use a dedicated address space for the control virtqueue
    group to isolate the memory mapping.

    The vhost/vhost-vDPA is also extend for the userspace to:

    - query the number of virtqueue groups and address spaces supported by
    the device
    - query the virtqueue group for a specific virtqueue
    - assocaite a virtqueue group with an address space
    - send ASID based IOTLB commands

    This will help userspace VMM(Qemu) to detect whether the control vq
    could be supported and isolate memory mappings of control virtqueue
    from the others.

    To demonstrate the usage, vDPA simulator is extended to support
    setting MAC address via a emulated control virtqueue.

    Please review.

    Changes since v1:

    - Rebased the v1 patch series on vhost branch of MST vhost git repo
    git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost
    - Updates to accommodate vdpa_sim changes from monolithic module in
    kernel used v1 patch series to current modularized class (net, block)
    based approach.
    - Added new attributes (ngroups and nas) to "vdpasim_dev_attr" and
    propagated them from vdpa_sim_net to vdpa_sim
    - Widened the data-type for "asid" member of vhost_msg_v2 to __u32
    to accommodate PASID
    - Fixed the buildbot warnings
    - Resolved all checkpatch.pl errors and warnings
    - Tested both control and datapath with Xilinx Smartnic SN1000 series
    device using QEMU implementing the Shadow virtqueue and support for
    VQ groups and ASID available at:
    github.com/eugpermar/qemu/releases/tag/vdpa_sw_live_migration.d%2F
    asid_groups-v1.d%2F00

    Changes since RFC:

    - tweak vhost uAPI documentation
    - switch to use device specific IOTLB really in patch 4
    - tweak the commit log
    - fix that ASID in vhost is claimed to be 32 actually but 16bit
    actually
    - fix use after free when using ASID with IOTLB batching requests
    - switch to use Stefano's patch for having separated iov
    - remove unused "used_as" variable
    - fix the iotlb/asid checking in vhost_vdpa_unmap()

    Thanks

    Gautam Dawar (19):
    vhost: move the backend feature bits to vhost_types.h
    virtio-vdpa: don't set callback if virtio doesn't need it
    vhost-vdpa: passing iotlb to IOMMU mapping helpers
    vhost-vdpa: switch to use vhost-vdpa specific IOTLB
    vdpa: introduce virtqueue groups
    vdpa: multiple address spaces support
    vdpa: introduce config operations for associating ASID to a virtqueue
    group
    vhost_iotlb: split out IOTLB initialization
    vhost: support ASID in IOTLB API
    vhost-vdpa: introduce asid based IOTLB
    vhost-vdpa: introduce uAPI to get the number of virtqueue groups
    vhost-vdpa: introduce uAPI to get the number of address spaces
    vhost-vdpa: uAPI to get virtqueue group id
    vhost-vdpa: introduce uAPI to set group ASID
    vhost-vdpa: support ASID based IOTLB API
    vdpa_sim: advertise VIRTIO_NET_F_MTU
    vdpa_sim: factor out buffer completion logic
    vdpa_sim: filter destination mac address
    vdpasim: control virtqueue support

    drivers/vdpa/ifcvf/ifcvf_main.c | 8 +-
    drivers/vdpa/mlx5/net/mlx5_vnet.c | 11 +-
    drivers/vdpa/vdpa.c | 5 +
    drivers/vdpa/vdpa_sim/vdpa_sim.c | 100 ++++++++--
    drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 +
    drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 169 +++++++++++++----
    drivers/vhost/iotlb.c | 23 ++-
    drivers/vhost/vdpa.c | 272 +++++++++++++++++++++------
    drivers/vhost/vhost.c | 23 ++-
    drivers/vhost/vhost.h | 4 +-
    drivers/virtio/virtio_vdpa.c | 2 +-
    include/linux/vdpa.h | 46 ++++-
    include/linux/vhost_iotlb.h | 2 +
    include/uapi/linux/vhost.h | 25 ++-
    include/uapi/linux/vhost_types.h | 11 +-
    15 files changed, 566 insertions(+), 138 deletions(-)

    --
    2.25.0

    \
     
     \ /
      Last update: 2022-02-24 22:24    [W:2.426 / U:0.352 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site