lkml.org 
[lkml]   [2018]   [Mar]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH v3 01/11] PCI/P2PDMA: Support peer-to-peer memory
    On Tue, Mar 13, 2018 at 05:21:20PM -0600, Logan Gunthorpe wrote:
    > On 13/03/18 05:08 PM, Bjorn Helgaas wrote:
    > > On Tue, Mar 13, 2018 at 10:31:55PM +0000, Stephen Bates wrote:
    > > If it *is* necessary because Root Ports and devices below them behave
    > > differently than in conventional PCI, I think you should include a
    > > reference to the relevant section of the spec and check directly for a
    > > Root Port. I would prefer that over trying to exclude Root Ports by
    > > looking for two upstream bridges.
    >
    > Well we've established that we don't want to allow root ports.

    I assume you want to exclude Root Ports because of multi-function
    devices and the "route to self" error. I was hoping for a reference
    to that so I could learn more about it.

    While I was looking for it, I found sec 6.12.1.2 (PCIe r4.0), "ACS
    Functions in SR-IOV Capable and Multi-Function Devices", which seems
    relevant. It talks about "peer-to-peer Requests (between Functions of
    the device)". Thay says to me that multi-function devices can DMA
    between themselves.

    > But we need to, at a minimum, do two pci_upstream_bridge() calls...
    >
    > Remember, we need to check that a number of devices are behind the same
    > switch. So we need to find a _common_ upstream port for several devices.

    I agree that peers need to have a common upstream bridge. I think
    you're saying peers need to have *two* common upstream bridges. If I
    understand correctly, requiring two common bridges is a way to ensure
    that peers directly below Root Ports don't try to DMA to each other.

    So I guess the first order of business is to nail down whether peers
    below a Root Port are prohibited from DMAing to each other. My
    assumption, based on 6.12.1.2 and the fact that I haven't yet found
    a prohibition, is that they can.

    > Excluding the multifunction device case (which I don't think is
    > applicable for reasons we've discussed before), this will *never* be the
    > first upstream port for a given device.

    If you're looking for a common upstream bridge, you don't need to make
    assumptions about whether the hierarchy is conventional PCI or PCIe or
    how many levels are in the hierarchy.

    You already have upstream_bridges_match(), which takes two pci_devs.
    I think it should walk up the PCI hierarchy from the first device,
    checking whether the bridge at each level is also a parent of the
    second device.

    Bjorn

    \
     
     \ /
      Last update: 2018-03-14 03:57    [W:2.785 / U:0.032 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site