Messages in this thread | | | From | Alexandru Elisei <> | Subject | [BUG] rockpro64: PCI BAR reassignment broken by commit 9d57e61bf723 ("of/pci: Add IORESOURCE_MEM_64 to resource flags for 64-bit memory addresses") | Date | Tue, 18 May 2021 10:09:19 +0100 |
| |
After doing a git bisect I was able to trace the following error when booting my rockpro64 v2 (rk3399 SoC) with a PCIE NVME expansion card:
[..] [ 0.305183] rockchip-pcie f8000000.pcie: host bridge /pcie@f8000000 ranges: [ 0.305248] rockchip-pcie f8000000.pcie: MEM 0x00fa000000..0x00fbdfffff -> 0x00fa000000 [ 0.305285] rockchip-pcie f8000000.pcie: IO 0x00fbe00000..0x00fbefffff -> 0x00fbe00000 [ 0.306201] rockchip-pcie f8000000.pcie: supply vpcie1v8 not found, using dummy regulator [ 0.306334] rockchip-pcie f8000000.pcie: supply vpcie0v9 not found, using dummy regulator [ 0.373705] rockchip-pcie f8000000.pcie: PCI host bridge to bus 0000:00 [ 0.373730] pci_bus 0000:00: root bus resource [bus 00-1f] [ 0.373751] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff 64bit] [ 0.373777] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) [ 0.373839] pci 0000:00:00.0: [1d87:0100] type 01 class 0x060400 [ 0.373973] pci 0000:00:00.0: supports D1 [ 0.373992] pci 0000:00:00.0: PME# supported from D0 D1 D3hot [ 0.378518] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring [ 0.378765] pci 0000:01:00.0: [144d:a808] type 00 class 0x010802 [ 0.378869] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit] [ 0.379051] pci 0000:01:00.0: Max Payload Size set to 256 (was 128, max 256) [ 0.379661] pci 0000:01:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x4 link at 0000:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link) [ 0.393269] pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01 [ 0.393311] pci 0000:00:00.0: BAR 14: no space for [mem size 0x00100000] [ 0.393333] pci 0000:00:00.0: BAR 14: failed to assign [mem size 0x00100000] [ 0.393356] pci 0000:01:00.0: BAR 0: no space for [mem size 0x00004000 64bit] [ 0.393375] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x00004000 64bit] [ 0.393397] pci 0000:00:00.0: PCI bridge to [bus 01] [ 0.393839] pcieport 0000:00:00.0: PME: Signaling with IRQ 78 [ 0.394165] pcieport 0000:00:00.0: AER: enabled with IRQ 78 [..]
to the commit 9d57e61bf723 ("of/pci: Add IORESOURCE_MEM_64 to resource flags for 64-bit memory addresses"). For reference, here is the dmesg output when BAR reassignment works:
[..] [ 0.307381] rockchip-pcie f8000000.pcie: host bridge /pcie@f8000000 ranges: [ 0.307445] rockchip-pcie f8000000.pcie: MEM 0x00fa000000..0x00fbdfffff -> 0x00fa000000 [ 0.307481] rockchip-pcie f8000000.pcie: IO 0x00fbe00000..0x00fbefffff -> 0x00fbe00000 [ 0.308406] rockchip-pcie f8000000.pcie: supply vpcie1v8 not found, using dummy regulator [ 0.308534] rockchip-pcie f8000000.pcie: supply vpcie0v9 not found, using dummy regulator [ 0.374676] rockchip-pcie f8000000.pcie: PCI host bridge to bus 0000:00 [ 0.374701] pci_bus 0000:00: root bus resource [bus 00-1f] [ 0.374723] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] [ 0.374746] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) [ 0.374808] pci 0000:00:00.0: [1d87:0100] type 01 class 0x060400 [ 0.374943] pci 0000:00:00.0: supports D1 [ 0.374961] pci 0000:00:00.0: PME# supported from D0 D1 D3hot [ 0.379473] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring [ 0.379712] pci 0000:01:00.0: [144d:a808] type 00 class 0x010802 [ 0.379815] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit] [ 0.379997] pci 0000:01:00.0: Max Payload Size set to 256 (was 128, max 256) [ 0.380607] pci 0000:01:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x4 link at 0000:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link) [ 0.394239] pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01 [ 0.394285] pci 0000:00:00.0: BAR 14: assigned [mem 0xfa000000-0xfa0fffff] [ 0.394312] pci 0000:01:00.0: BAR 0: assigned [mem 0xfa000000-0xfa003fff 64bit] [ 0.394374] pci 0000:00:00.0: PCI bridge to [bus 01] [ 0.394395] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] [ 0.394569] pcieport 0000:00:00.0: enabling device (0000 -> 0002) [ 0.394845] pcieport 0000:00:00.0: PME: Signaling with IRQ 78 [ 0.395153] pcieport 0000:00:00.0: AER: enabled with IRQ 78 [..]
And here is the output of lspci when BAR reassignment works:
# lspci -v 00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, IRQ 78 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff [size=4K] Memory behind bridge: fa000000-fa0fffff [size=1M] Prefetchable memory behind bridge: 00000000-000fffff [size=1M] Capabilities: [80] Power Management version 3 Capabilities: [90] MSI: Enable+ Count=1/1 Maskable+ 64bit+ Capabilities: [b0] MSI-X: Enable- Count=1 Masked- Capabilities: [c0] Express Root Port (Slot+), MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [274] Transaction Processing Hints Kernel driver in use: pcieport lspci: Unable to load libkmod resources: error -2
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 (prog-if 02 [NVM Express]) Subsystem: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 Flags: bus master, fast devsel, latency 0, IRQ 77, NUMA node 0 Memory at fa000000 (64-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [70] Express Endpoint, MSI 00 Capabilities: [b0] MSI-X: Enable+ Count=33 Masked- Capabilities: [100] Advanced Error Reporting Capabilities: [148] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [158] Power Budgeting <?> Capabilities: [168] Secondary PCI Express Capabilities: [188] Latency Tolerance Reporting Capabilities: [190] L1 PM Substates Kernel driver in use: nvme
I can provide more information if needed (the board is sitting on my desk) and I can help with testing the fix.
Thanks,
Alex
| |