lkml.org 
[lkml]   [2022]   [Nov]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [RFC 0/4] pci/sriov: support VFs dynamic addition
    On Sun, Nov 13, 2022 at 09:47:12PM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote:
    > Hi leon,
    >
    > 在 2022/11/12 0:39, Leon Romanovsky 写道:
    > > On Fri, Nov 11, 2022 at 10:27:18PM +0800, Longpeng(Mike) wrote:
    > > > From: Longpeng <longpeng2@huawei.com>
    > > >
    > > > We can enable SRIOV and add VFs by /sys/bus/pci/devices/..../sriov_numvfs, but
    > > > this operation needs to spend lots of time if there has a large amount of VFs.
    > > > For example, if the machine has 10 PFs and 250 VFs per-PF, enable all the VFs
    > > > concurrently would cost about 200-250ms. However most of them are not need to be
    > > > used at the moment, so we can enable SRIOV first but add VFs on demand.
    > >
    > > It is unclear what took 200-250ms, is it physical VF creation or bind of
    > > the driver to these VFs?
    > >
    > It is neither. In our test, we already created physical VFs before, so we
    > skipped the 100ms waiting when writing PCI_SRIOV_CTRL. And our driver only
    > probes PF, it just returns an error if the function is VF.

    It means that you didn't try sriov_drivers_autoprobe. Once it is set to
    true, It won't even try to probe VFs.

    >
    > The hotspot is the sriov_add_vfs (but no driver probe in fact) which is a
    > long procedure. Each step costs only a little, but the total cost is not
    > acceptable in some time-sensitive cases.

    This is also cryptic to me. In standard SR-IOV deployment, all VFs are
    created and configured while operator booted the machine with sriov_drivers_autoprobe
    set to false. Once this machine is ready, VFs are assigned to relevant VMs/users
    through orchestration SW (IMHO, it is supported by all orchestration SW).

    And only last part (assigning to users) is time-sensitive operation.

    >
    > What’s more, the sriov_add_vfs adds the VFs of a PF one by one. So we can
    > mostly support 10 concurrent calls if there has 10 PFs.

    I wondered, are you using real HW? or QEMU SR-IOV? What is your server
    that supports such large number of VFs?

    BTW, Your change will probably break all SR-IOV devices in the market as
    they rely on PCI subsystem to have VFs ready and configured.

    Thanks

    \
     
     \ /
      Last update: 2022-11-14 08:05    [W:2.889 / U:0.004 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site