lkml.org 
[lkml]   [2006]   [Oct]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: Ordering between PCI config space writes and MMIO reads?
    Quoting r. Roland Dreier <rdreier@cisco.com>:
    > Subject: Re: Ordering between PCI config space writes and MMIO reads?
    >
    > The discussion fizzled out without really reaching a definitive
    > answer, so I'm going to apply the original patch (below), since I
    > pretty much convinced myself that only the driver doing the config
    > access has enough information to fix this reliably.
    >
    > - R.
    >
    > Author: John Partridge <johnip@sgi.com>
    > Date: Tue Oct 31 11:00:04 2006 -0800
    >
    > IB/mthca: Make sure all PCI config writes reach device before doing MMIO
    >
    > During initialization, mthca writes some PCI config space registers
    > and then does an MMIO read from one of the BARs it just enabled. This
    > MMIO read sometimes failed and caused a crash on SGI Altix machines,
    > because the PCI-X host bridge (legitimately, according to the PCI
    > spec) allowed the MMIO read to start before the config write completed.
    >
    > To fix this, add a config read after all config writes to make sure
    > they are all done before starting the MMIO read.
    >
    > Signed-off-by: John Partridge <johnip@sgi.com>
    > Signed-off-by: Roland Dreier <rolandd@cisco.com>
    >
    > diff --git a/drivers/infiniband/hw/mthca/mthca_reset.c b/drivers/infiniband/hw/mthca/mthca_reset.c
    > index 91934f2..578dc7c 100644
    > --- a/drivers/infiniband/hw/mthca/mthca_reset.c
    > +++ b/drivers/infiniband/hw/mthca/mthca_reset.c
    > @@ -281,6 +281,20 @@ good:
    > goto out;
    > }
    >
    > + /*
    > + * Perform a "flush" of the PCI config writes here by reading
    > + * the PCI_COMMAND register. This is needed to make sure that
    > + * we don't try to touch other PCI BARs before the config
    > + * writes are done -- otherwise an MMIO cycle could start
    > + * before the config writes are done and reach the HCA before
    > + * the BAR is actually enabled.
    > + */
    > + if (pci_read_config_dword(mdev->pdev, PCI_COMMAND, hca_header)) {
    > + err = -ENODEV;
    > + mthca_err(mdev, "Couldn't access HCA memory after restoring, "
    > + "aborting.\n");
    > + }
    > +
    > out:
    > if (bridge)
    > pci_dev_put(bridge);

    Here's what I don't understand: according to PCI rules, pci config read
    can bypass pci config write (both are non-posted).
    So why does doing it help flush the writes as the comment claims?

    Isn't this more the case of
    /* pci_config_write seems to complete asynchronously on Altix systems.
    * This is probably broken but its not clear what's the best
    * thing to do is - for now, do pci_read_config_dword which seems to flush
    * everything out. */

    --
    MST
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2006-10-31 20:57    [W:5.562 / U:0.308 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site