lkml.org 
[lkml]   [2022]   [Jul]   [11]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH 5.15 035/230] PCI: pciehp: Ignore Link Down/Up caused by error-induced Hot Reset
    Date
    From: Lukas Wunner <lukas@wunner.de>

    [ Upstream commit ea401499e943c307e6d44af6c2b4e068643e7884 ]

    Stuart Hayes reports that an error handled by DPC at a Root Port results
    in pciehp gratuitously bringing down a subordinate hotplug port:

    RP -- UP -- DP -- UP -- DP (hotplug) -- EP

    pciehp brings the slot down because the Link to the Endpoint goes down.
    That is caused by a Hot Reset being propagated as a result of DPC.
    Per PCIe Base Spec 5.0, section 6.6.1 "Conventional Reset":

    For a Switch, the following must cause a hot reset to be sent on all
    Downstream Ports: [...]

    * The Data Link Layer of the Upstream Port reporting DL_Down status.
    In Switches that support Link speeds greater than 5.0 GT/s, the
    Upstream Port must direct the LTSSM of each Downstream Port to the
    Hot Reset state, but not hold the LTSSMs in that state. This permits
    each Downstream Port to begin Link training immediately after its
    hot reset completes. This behavior is recommended for all Switches.

    * Receiving a hot reset on the Upstream Port.

    Once DPC recovers, pcie_do_recovery() walks down the hierarchy and
    invokes pcie_portdrv_slot_reset() to restore each port's config space.
    At that point, a hotplug interrupt is signaled per PCIe Base Spec r5.0,
    section 6.7.3.4 "Software Notification of Hot-Plug Events":

    If the Port is enabled for edge-triggered interrupt signaling using
    MSI or MSI-X, an interrupt message must be sent every time the logical
    AND of the following conditions transitions from FALSE to TRUE: [...]

    * The Hot-Plug Interrupt Enable bit in the Slot Control register is
    set to 1b.

    * At least one hot-plug event status bit in the Slot Status register
    and its associated enable bit in the Slot Control register are both
    set to 1b.

    Prevent pciehp from gratuitously bringing down the slot by clearing the
    error-induced Data Link Layer State Changed event before restoring
    config space. Afterwards, check whether the link has unexpectedly
    failed to retrain and synthesize a DLLSC event if so.

    Allow each pcie_port_service_driver (one of them being pciehp) to define
    a slot_reset callback and re-use the existing pm_iter() function to
    iterate over the callbacks.

    Thereby, the Endpoint driver remains bound throughout error recovery and
    may restore the device to working state.

    Surprise removal during error recovery is detected through a Presence
    Detect Changed event. The hotplug port is expected to not signal that
    event as a result of a Hot Reset.

    The issue isn't DPC-specific, it also occurs when an error is handled by
    AER through aer_root_reset(). So while the issue was noticed only now,
    it's been around since 2006 when AER support was first introduced.

    [bhelgaas: drop PCI_ERROR_RECOVERY Kconfig, split pm_iter() rename to
    preparatory patch]
    Link: https://lore.kernel.org/linux-pci/08c046b0-c9f2-3489-eeef-7e7aca435bb9@gmail.com/
    Fixes: 6c2b374d7485 ("PCI-Express AER implemetation: AER core and aerdriver")
    Link: https://lore.kernel.org/r/251f4edcc04c14f873ff1c967bc686169cd07d2d.1627638184.git.lukas@wunner.de
    Reported-by: Stuart Hayes <stuart.w.hayes@gmail.com>
    Tested-by: Stuart Hayes <stuart.w.hayes@gmail.com>
    Signed-off-by: Lukas Wunner <lukas@wunner.de>
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
    Cc: stable@vger.kernel.org # v2.6.19+: ba952824e6c1: PCI/portdrv: Report reset for frozen channel
    Cc: Keith Busch <kbusch@kernel.org>
    Signed-off-by: Sasha Levin <sashal@kernel.org>
    ---
    drivers/pci/hotplug/pciehp.h | 2 ++
    drivers/pci/hotplug/pciehp_core.c | 2 ++
    drivers/pci/hotplug/pciehp_hpc.c | 26 ++++++++++++++++++++++++++
    drivers/pci/pcie/portdrv.h | 2 ++
    drivers/pci/pcie/portdrv_pci.c | 3 +++
    5 files changed, 35 insertions(+)

    diff --git a/drivers/pci/hotplug/pciehp.h b/drivers/pci/hotplug/pciehp.h
    index 10d7e7e1b553..e0a614acee05 100644
    --- a/drivers/pci/hotplug/pciehp.h
    +++ b/drivers/pci/hotplug/pciehp.h
    @@ -192,6 +192,8 @@ int pciehp_get_attention_status(struct hotplug_slot *hotplug_slot, u8 *status);
    int pciehp_set_raw_indicator_status(struct hotplug_slot *h_slot, u8 status);
    int pciehp_get_raw_indicator_status(struct hotplug_slot *h_slot, u8 *status);

    +int pciehp_slot_reset(struct pcie_device *dev);
    +
    static inline const char *slot_name(struct controller *ctrl)
    {
    return hotplug_slot_name(&ctrl->hotplug_slot);
    diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c
    index e7fe4b42f039..4042d87d539d 100644
    --- a/drivers/pci/hotplug/pciehp_core.c
    +++ b/drivers/pci/hotplug/pciehp_core.c
    @@ -351,6 +351,8 @@ static struct pcie_port_service_driver hpdriver_portdrv = {
    .runtime_suspend = pciehp_runtime_suspend,
    .runtime_resume = pciehp_runtime_resume,
    #endif /* PM */
    +
    + .slot_reset = pciehp_slot_reset,
    };

    int __init pcie_hp_init(void)
    diff --git a/drivers/pci/hotplug/pciehp_hpc.c b/drivers/pci/hotplug/pciehp_hpc.c
    index 8bedbc77fe95..60098a701e83 100644
    --- a/drivers/pci/hotplug/pciehp_hpc.c
    +++ b/drivers/pci/hotplug/pciehp_hpc.c
    @@ -865,6 +865,32 @@ void pcie_disable_interrupt(struct controller *ctrl)
    pcie_write_cmd(ctrl, 0, mask);
    }

    +/**
    + * pciehp_slot_reset() - ignore link event caused by error-induced hot reset
    + * @dev: PCI Express port service device
    + *
    + * Called from pcie_portdrv_slot_reset() after AER or DPC initiated a reset
    + * further up in the hierarchy to recover from an error. The reset was
    + * propagated down to this hotplug port. Ignore the resulting link flap.
    + * If the link failed to retrain successfully, synthesize the ignored event.
    + * Surprise removal during reset is detected through Presence Detect Changed.
    + */
    +int pciehp_slot_reset(struct pcie_device *dev)
    +{
    + struct controller *ctrl = get_service_data(dev);
    +
    + if (ctrl->state != ON_STATE)
    + return 0;
    +
    + pcie_capability_write_word(dev->port, PCI_EXP_SLTSTA,
    + PCI_EXP_SLTSTA_DLLSC);
    +
    + if (!pciehp_check_link_active(ctrl))
    + pciehp_request(ctrl, PCI_EXP_SLTSTA_DLLSC);
    +
    + return 0;
    +}
    +
    /*
    * pciehp has a 1:1 bus:slot relationship so we ultimately want a secondary
    * bus reset of the bridge, but at the same time we want to ensure that it is
    diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h
    index 6126ee4676a7..41fe1ffd5907 100644
    --- a/drivers/pci/pcie/portdrv.h
    +++ b/drivers/pci/pcie/portdrv.h
    @@ -85,6 +85,8 @@ struct pcie_port_service_driver {
    int (*runtime_suspend)(struct pcie_device *dev);
    int (*runtime_resume)(struct pcie_device *dev);

    + int (*slot_reset)(struct pcie_device *dev);
    +
    /* Device driver may resume normal operations */
    void (*error_resume)(struct pci_dev *dev);

    diff --git a/drivers/pci/pcie/portdrv_pci.c b/drivers/pci/pcie/portdrv_pci.c
    index c7ff1eea225a..1af74c3d9d5d 100644
    --- a/drivers/pci/pcie/portdrv_pci.c
    +++ b/drivers/pci/pcie/portdrv_pci.c
    @@ -160,6 +160,9 @@ static pci_ers_result_t pcie_portdrv_error_detected(struct pci_dev *dev,

    static pci_ers_result_t pcie_portdrv_slot_reset(struct pci_dev *dev)
    {
    + size_t off = offsetof(struct pcie_port_service_driver, slot_reset);
    + device_for_each_child(&dev->dev, &off, pcie_port_device_iter);
    +
    pci_restore_state(dev);
    pci_save_state(dev);
    return PCI_ERS_RESULT_RECOVERED;
    --
    2.35.1


    \
     
     \ /
      Last update: 2022-07-11 11:42    [W:8.929 / U:0.144 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site