lkml.org 
[lkml]   [2022]   [Aug]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Apple Thunderbolt Display chaining
On 6/4/22 22:56, Mika Westerberg wrote:
> Hi,
>
> On Wed, Apr 06, 2022 at 10:51:41AM +0800, Brad Campbell wrote:
>> Both included in-line.
>>
>> This is cold boot with the chain plugged in. I've re-added the dbg to
>> print the link number, and I've included your path discovery debugs.
>> Boot with chain plugged in, wait for it to settle, unplug and replug.
>> First head in the chain fails with :
>>
>> [ 65.778129] [drm:radeon_dp_link_train [radeon]] *ERROR* displayport link status failed
>> [ 65.778158] [drm:radeon_dp_link_train [radeon]] *ERROR* clock recovery failed
>
> Thanks for the logs!
>
> The DP tunnels look pretty much the same except that the Apple EFI CM
> seems to assign 7 buffers for the AUX RX path first hop whereas we
> always use 1 buffer. Not sure if that really makes a difference and we
> could try to use the same number but first, I realized that the PCI
> resource allocation seems not to work properly.
>
> Can you disable PCIe tunneling (if you use Ubuntu/Fedora or similar
> there is the "Thunderbolt -> Direct Access" switch that you can turn
> off) and try again? Please also take 'sudo lspci -vv' for the resulting
> topology. I suspect this might also affect the other issues (the
> timeouts) you are seeing. Note this makes the peripherals connected to
> the monitors unusable too.
>

G'day Mika,

Unfortunately before I had a chance to progress this my poor iMac GPU finally turned
up its toes.

I have now replaced it with a Gigabyte B550 Vision-d-p which happily drives the Apple
Thunderbolt displays. This has a Titan Ridge controller and I'm running a 5.18.16 vanilla kernel.

When cold booted, this fires up, routes both PCIe and Displayport and both displays work.

With this configuration the kernel locks up when doing a warm reboot (black screen, no information). It
also fails on reset, and only a power cycle will allow the kernel to complete the boot.

If I disable the PCIe tunneling and just leave DP then it cold boots and then warm boots without issue.

If I build thunderbolt as a module and blacklist it to prevent it auto loading, I can
load it the first time which establishes the DP and PCIe tunnels. They then remain established
across a warm boot, and provided I don't try and load the thunderbolt module the system will just keep
running. I can run different kernels, memtest, whatever. The EFI doesn't break the tunnels
and the kernel works with them.

If I then try and load the thunderbolt module I get this :

[ 618.242923] ACPI: bus type thunderbolt registered
[ 618.242957] thunderbolt 0000:05:00.0: enabling device (0000 -> 0002)
[ 618.242983] thunderbolt 0000:05:00.0: total paths: 12
[ 618.243147] thunderbolt 0000:05:00.0: allocating TX ring 0 of size 10
[ 618.243163] thunderbolt 0000:05:00.0: allocating RX ring 0 of size 10
[ 618.243176] thunderbolt 0000:05:00.0: control channel created
[ 618.243180] thunderbolt 0000:05:00.0: ICM not supported on this controller
[ 618.243187] thunderbolt 0000:05:00.0: freeing RX ring 0
[ 618.243193] thunderbolt 0000:05:00.0: freeing TX ring 0
[ 618.243202] thunderbolt 0000:05:00.0: allocating TX ring 0 of size 10
[ 618.243210] thunderbolt 0000:05:00.0: allocating RX ring 0 of size 10
[ 618.243218] thunderbolt 0000:05:00.0: control channel created
[ 618.243220] thunderbolt 0000:05:00.0: using software connection manager
[ 618.243221] thunderbolt 0000:05:00.0: NHI initialized, starting thunderbolt
[ 618.243222] thunderbolt 0000:05:00.0: control channel starting...
[ 618.243223] thunderbolt 0000:05:00.0: starting TX ring 0
[ 618.243231] thunderbolt 0000:05:00.0: enabling interrupt at register 0x38200 bit 0 (0x0 -> 0x1)
[ 618.243233] thunderbolt 0000:05:00.0: starting RX ring 0
[ 618.243241] thunderbolt 0000:05:00.0: enabling interrupt at register 0x38200 bit 12 (0x1 -> 0x1001)
[ 618.243245] thunderbolt 0000:05:00.0: security level set to user
[ 618.243458] thunderbolt 0000:05:00.0: current switch config:
[ 618.243460] thunderbolt 0000:05:00.0: Thunderbolt 3 Switch: 8086:15ea (Revision: 6, TB Version: 16)
[ 618.243461] thunderbolt 0000:05:00.0: Max Port Number: 13
[ 618.243463] thunderbolt 0000:05:00.0: Config:
[ 618.243463] thunderbolt 0000:05:00.0: Upstream Port Number: 7 Depth: 0 Route String: 0x0 Enabled: 1, PlugEventsDelay: 255ms
[ 618.243465] thunderbolt 0000:05:00.0: unknown1: 0x0 unknown4: 0x0
[ 618.248577] thunderbolt 0000:05:00.0: initializing Switch at 0x0 (depth: 0, up port: 7)
[ 618.288517] thunderbolt 0000:05:00.0: 0: uid: 0xedd9a650496900
[ 618.290436] thunderbolt 0000:05:00.0: Port 1: 8086:15ea (Revision: 6, TB Version: 1, Type: Port (0x1))
[ 618.290438] thunderbolt 0000:05:00.0: Max hop id (in/out): 19/19
[ 618.290439] thunderbolt 0000:05:00.0: Max counters: 16
[ 618.290440] thunderbolt 0000:05:00.0: NFC Credits: 0x3c00000
[ 618.290441] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.292352] thunderbolt 0000:05:00.0: Port 2: 8086:15ea (Revision: 6, TB Version: 1, Type: Port (0x1))
[ 618.292354] thunderbolt 0000:05:00.0: Max hop id (in/out): 19/19
[ 618.292354] thunderbolt 0000:05:00.0: Max counters: 16
[ 618.292355] thunderbolt 0000:05:00.0: NFC Credits: 0x3c00000
[ 618.292355] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.294277] thunderbolt 0000:05:00.0: Port 3: 8086:15ea (Revision: 6, TB Version: 1, Type: Port (0x1))
[ 618.294277] thunderbolt 0000:05:00.0: Max hop id (in/out): 19/19
[ 618.294278] thunderbolt 0000:05:00.0: Max counters: 16
[ 618.294278] thunderbolt 0000:05:00.0: NFC Credits: 0x3c00000
[ 618.294279] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.296197] thunderbolt 0000:05:00.0: Port 4: 8086:15ea (Revision: 6, TB Version: 1, Type: Port (0x1))
[ 618.296198] thunderbolt 0000:05:00.0: Max hop id (in/out): 19/19
[ 618.296198] thunderbolt 0000:05:00.0: Max counters: 16
[ 618.296199] thunderbolt 0000:05:00.0: NFC Credits: 0x3c00000
[ 618.296199] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.296449] thunderbolt 0000:05:00.0: Port 5: 8086:15ea (Revision: 6, TB Version: 1, Type: DP/HDMI (0xe0101))
[ 618.296450] thunderbolt 0000:05:00.0: Max hop id (in/out): 255/255
[ 618.296450] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.296451] thunderbolt 0000:05:00.0: NFC Credits: 0x180000c
[ 618.296451] thunderbolt 0000:05:00.0: Credits (total/control): 24/0
[ 618.296764] thunderbolt 0000:05:00.0: Port 6: 8086:15ea (Revision: 6, TB Version: 1, Type: DP/HDMI (0xe0101))
[ 618.296765] thunderbolt 0000:05:00.0: Max hop id (in/out): 255/255
[ 618.296766] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.296766] thunderbolt 0000:05:00.0: NFC Credits: 0x180000c
[ 618.296767] thunderbolt 0000:05:00.0: Credits (total/control): 24/0
[ 618.297477] thunderbolt 0000:05:00.0: Port 7: 8086:15ea (Revision: 6, TB Version: 1, Type: NHI (0x2))
[ 618.297478] thunderbolt 0000:05:00.0: Max hop id (in/out): 11/11
[ 618.297478] thunderbolt 0000:05:00.0: Max counters: 16
[ 618.297479] thunderbolt 0000:05:00.0: NFC Credits: 0x1800000
[ 618.297479] thunderbolt 0000:05:00.0: Credits (total/control): 24/0
[ 618.297730] thunderbolt 0000:05:00.0: Port 8: 8086:15ea (Revision: 6, TB Version: 1, Type: PCIe (0x100101))
[ 618.297731] thunderbolt 0000:05:00.0: Max hop id (in/out): 8/8
[ 618.297731] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.297732] thunderbolt 0000:05:00.0: NFC Credits: 0x800000
[ 618.297732] thunderbolt 0000:05:00.0: Credits (total/control): 8/0
[ 618.297992] thunderbolt 0000:05:00.0: Port 9: 8086:15ea (Revision: 6, TB Version: 1, Type: PCIe (0x100101))
[ 618.297993] thunderbolt 0000:05:00.0: Max hop id (in/out): 8/8
[ 618.297993] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.297994] thunderbolt 0000:05:00.0: NFC Credits: 0x800000
[ 618.297994] thunderbolt 0000:05:00.0: Credits (total/control): 8/0
[ 618.298496] thunderbolt 0000:05:00.0: Port 10: 8086:15ea (Revision: 6, TB Version: 1, Type: DP/HDMI (0xe0102))
[ 618.298497] thunderbolt 0000:05:00.0: Max hop id (in/out): 9/9
[ 618.298498] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.298498] thunderbolt 0000:05:00.0: NFC Credits: 0x800000
[ 618.298499] thunderbolt 0000:05:00.0: Credits (total/control): 8/0
[ 618.299012] thunderbolt 0000:05:00.0: Port 11: 8086:15ea (Revision: 6, TB Version: 1, Type: DP/HDMI (0xe0102))
[ 618.299013] thunderbolt 0000:05:00.0: Max hop id (in/out): 9/9
[ 618.299014] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.299014] thunderbolt 0000:05:00.0: NFC Credits: 0x800000
[ 618.299015] thunderbolt 0000:05:00.0: Credits (total/control): 8/0
[ 618.299269] thunderbolt 0000:05:00.0: Port 12: 8086:15ea (Revision: 6, TB Version: 1, Type: Inactive (0x0))
[ 618.299270] thunderbolt 0000:05:00.0: Max hop id (in/out): 8/8
[ 618.299270] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.299271] thunderbolt 0000:05:00.0: NFC Credits: 0x800000
[ 618.299271] thunderbolt 0000:05:00.0: Credits (total/control): 8/0
[ 618.299524] thunderbolt 0000:05:00.0: Port 13: 8086:15ea (Revision: 6, TB Version: 1, Type: Inactive (0x0))
[ 618.299525] thunderbolt 0000:05:00.0: Max hop id (in/out): 8/8
[ 618.299526] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.299526] thunderbolt 0000:05:00.0: NFC Credits: 0x800000
[ 618.299527] thunderbolt 0000:05:00.0: Credits (total/control): 8/0
[ 618.299527] thunderbolt 0000:05:00.0: 0: linked ports 1 <-> 2
[ 618.299528] thunderbolt 0000:05:00.0: 0: linked ports 3 <-> 4
[ 618.304645] thunderbolt 0000:05:00.0: 0: TMU: supports uni-directional mode
[ 618.304645] thunderbolt 0000:05:00.0: 0: TMU: current mode: HiFi
[ 618.304769] thunderbolt 0000:05:00.0: 0:1: is unplugged (state: 7)
[ 618.304897] thunderbolt 0000:05:00.0: 0:3: is connected, link is up (state: 2)
[ 618.305182] thunderbolt 0000:05:00.0: current switch config:
[ 618.305183] thunderbolt 0000:05:00.0: Thunderbolt 1 Switch: 8086:1513 (Revision: 2, TB Version: 1)
[ 618.305184] thunderbolt 0000:05:00.0: Max Port Number: 13
[ 618.305184] thunderbolt 0000:05:00.0: Config:
[ 618.305184] thunderbolt 0000:05:00.0: Upstream Port Number: 1 Depth: 1 Route String: 0x3 Enabled: 1, PlugEventsDelay: 255ms
[ 618.305186] thunderbolt 0000:05:00.0: unknown1: 0x0 unknown4: 0x0
[ 618.309761] thunderbolt 0000:05:00.0: initializing Switch at 0x3 (depth: 1, up port: 1)
[ 618.327174] thunderbolt 0000:05:00.0: 3: reading drom (length: 0x97)
[ 618.820874] thunderbolt 0000:05:00.0: 3: DROM version: 1
[ 618.821902] thunderbolt 0000:05:00.0: 3: uid: 0x1000100189170
[ 618.824842] thunderbolt 0000:05:00.0: Port 1: 8086:1513 (Revision: 2, TB Version: 1, Type: Port (0x1))
[ 618.824845] thunderbolt 0000:05:00.0: Max hop id (in/out): 63/63
[ 618.824845] thunderbolt 0000:05:00.0: Max counters: 32
[ 618.824846] thunderbolt 0000:05:00.0: NFC Credits: 0x3c0000c
[ 618.824847] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.827786] thunderbolt 0000:05:00.0: Port 2: 8086:1513 (Revision: 2, TB Version: 1, Type: Port (0x1))
[ 618.827787] thunderbolt 0000:05:00.0: Max hop id (in/out): 63/63
[ 618.827788] thunderbolt 0000:05:00.0: Max counters: 32
[ 618.827788] thunderbolt 0000:05:00.0: NFC Credits: 0x3c0000c
[ 618.827789] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.830731] thunderbolt 0000:05:00.0: Port 3: 8086:1513 (Revision: 2, TB Version: 1, Type: Port (0x1))
[ 618.830732] thunderbolt 0000:05:00.0: Max hop id (in/out): 63/63
[ 618.830732] thunderbolt 0000:05:00.0: Max counters: 32
[ 618.830733] thunderbolt 0000:05:00.0: NFC Credits: 0x3c00000
[ 618.830733] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.833678] thunderbolt 0000:05:00.0: Port 4: 8086:1513 (Revision: 2, TB Version: 1, Type: Port (0x1))
[ 618.833680] thunderbolt 0000:05:00.0: Max hop id (in/out): 63/63
[ 618.833680] thunderbolt 0000:05:00.0: Max counters: 32
[ 618.833681] thunderbolt 0000:05:00.0: NFC Credits: 0x3c00000
[ 618.833681] thunderbolt 0000:05:00.0: Credits (total/control): 60/2
[ 618.833682] thunderbolt 0000:05:00.0: 3:5: disabled by eeprom
[ 618.833683] thunderbolt 0000:05:00.0: 3:6: disabled by eeprom
[ 618.834570] thunderbolt 0000:05:00.0: Port 7: 8086:1513 (Revision: 2, TB Version: 1, Type: PCIe (0x100101))
[ 618.834572] thunderbolt 0000:05:00.0: Max hop id (in/out): 8/8
[ 618.834573] thunderbolt 0000:05:00.0: Max counters: 1
[ 618.834573] thunderbolt 0000:05:00.0: NFC Credits: 0x700000
[ 618.834574] thunderbolt 0000:05:00.0: Credits (total/control): 7/0
[ 618.835468] thunderbolt 0000:05:00.0: Port 8: 8086:1513 (Revision: 2, TB Version: 1, Type: PCIe (0x100101))
[ 618.835469] thunderbolt 0000:05:00.0: Max hop id (in/out): 8/8
[ 618.835469] thunderbolt 0000:05:00.0: Max counters: 1
[ 618.835470] thunderbolt 0000:05:00.0: NFC Credits: 0x700000
[ 618.835470] thunderbolt 0000:05:00.0: Credits (total/control): 7/0
[ 618.835471] thunderbolt 0000:05:00.0: 3:9: disabled by eeprom
[ 618.836360] thunderbolt 0000:05:00.0: Port 10: 8086:1513 (Revision: 2, TB Version: 1, Type: PCIe (0x100102))
[ 618.836361] thunderbolt 0000:05:00.0: Max hop id (in/out): 8/8
[ 618.836362] thunderbolt 0000:05:00.0: Max counters: 1
[ 618.836363] thunderbolt 0000:05:00.0: NFC Credits: 0x700000
[ 618.836363] thunderbolt 0000:05:00.0: Credits (total/control): 7/0
[ 618.837518] thunderbolt 0000:05:00.0: Port 11: 8086:1513 (Revision: 2, TB Version: 1, Type: DP/HDMI (0xe0102))
[ 618.837519] thunderbolt 0000:05:00.0: Max hop id (in/out): 9/9
[ 618.837520] thunderbolt 0000:05:00.0: Max counters: 2
[ 618.837520] thunderbolt 0000:05:00.0: NFC Credits: 0x700000
[ 618.837521] thunderbolt 0000:05:00.0: Credits (total/control): 7/0
[ 618.837521] thunderbolt 0000:05:00.0: 3:c: disabled by eeprom
[ 618.837522] thunderbolt 0000:05:00.0: 3:d: disabled by eeprom
[ 618.855589] thunderbolt 0000:05:00.0: 3: TMU: current mode: bi-directional, HiFi
[ 618.855600] thunderbolt 0-3: new device found, vendor=0x1 device=0x8002
[ 618.855602] thunderbolt 0-3: Apple, Inc. Thunderbolt Display
[ 618.856101] thunderbolt 0000:05:00.0: 3:3: is connected, link is up (state: 2)
[ 618.856357] thunderbolt 0000:05:00.0: current switch config:
[ 618.856357] thunderbolt 0000:05:00.0: Thunderbolt 1 Switch: 8086:1513 (Revision: 2, TB Version: 1)
[ 618.856359] thunderbolt 0000:05:00.0: Max Port Number: 13
[ 618.856359] thunderbolt 0000:05:00.0: Config:
[ 618.856360] thunderbolt 0000:05:00.0: Upstream Port Number: 3 Depth: 2 Route String: 0x303 Enabled: 1, PlugEventsDelay: 255ms
[ 618.856361] thunderbolt 0000:05:00.0: unknown1: 0x0 unknown4: 0x0
[ 618.860965] thunderbolt 0000:05:00.0: initializing Switch at 0x303 (depth: 2, up port: 3)
[ 618.878375] thunderbolt 0000:05:00.0: 303: reading drom (length: 0x97)
[ 620.072070] pcieport 0000:2a:05.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 620.676124] pcieport 0000:2a:05.0: pciehp: pcie_do_write_cmd: no response from device
[ 621.903381] hrtimer: interrupt took 604006559 ns
[ 622.296070] pcieport 0000:2a:05.0: pciehp: pcie_do_write_cmd: no response from device
[ 622.698769] pcieport 0000:31:05.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 623.302796] pcieport 0000:31:05.0: pciehp: pcie_do_write_cmd: no response from device
[ 623.906840] pcieport 0000:31:04.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 624.511090] pcieport 0000:31:04.0: pciehp: pcie_do_write_cmd: no response from device
[ 625.115091] pcieport 0000:04:01.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 625.719304] pcieport 0000:04:01.0: pciehp: pcie_do_write_cmd: no response from device
[ 625.920747] thunderbolt 0000:05:00.0: 303: timeout writing config space 2 to 0x28
[ 625.920752] thunderbolt 0-303: reading DROM failed: -5
[ 625.920753] thunderbolt 0000:05:00.0: 303: uid: 0x0
[ 626.524964] pcieport 0000:31:05.0: pciehp: pcie_do_write_cmd: no response from device
[ 626.726306] pcieport 0000:31:04.0: pciehp: pcie_do_write_cmd: no response from device
[ 626.927661] pcieport 0000:04:01.0: pciehp: pcie_do_write_cmd: no response from device
[ 627.330332] pcieport 0000:04:02.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 627.837000] thunderbolt 0000:05:00.0: 303: timeout reading config space 1 from 0x0
[ 627.837005] thunderbolt 0-303: failed to initialize port 1
[ 628.277015] thunderbolt 0000:05:00.0: 3: timeout reading config space 1 from 0x3b
[ 628.716825] thunderbolt 0000:05:00.0: 0: timeout reading config space 1 from 0x3b
[ 629.156976] thunderbolt 0000:05:00.0: 0: timeout reading config space 1 from 0x3b
[ 629.597002] thunderbolt 0000:05:00.0: 0: timeout reading config space 1 from 0x39
[ 630.037013] thunderbolt 0000:05:00.0: 0: timeout reading config space 1 from 0x39
[ 630.477032] thunderbolt 0000:05:00.0: 0: timeout reading config space 1 from 0x39
[ 630.916994] thunderbolt 0000:05:00.0: 0: timeout reading config space 1 from 0x39
[ 631.356836] thunderbolt 0000:05:00.0: 3: timeout reading config space 1 from 0x39
[ 631.806792] thunderbolt 0000:05:00.0: 3: timeout reading config space 1 from 0x39
[ 632.246828] thunderbolt 0000:05:00.0: 0: timeout reading config space 2 from 0x10f
[ 632.686977] thunderbolt 0000:05:00.0: 0: timeout reading config space 2 from 0x10f

I'm dangerously *assuming* that when trying to configure an already configured system the thunderbolt
driver is getting stuck because booting without trying to load the driver works fine.

If I try and warm boot when its in this state, the machine doesn't reach the EFI and I have to
hard reset or power cycle.

My questions are :
A) Since I have a viable but hacky work-around, is this something you'd be interested in helping chase down?
B) If so, what debugging information can I supply?

I have made up a serial console cable, but as I can reproduce the fault by loading the module
I thought perhaps it'd be easier to debug.

Regards,
Brad

\
 
 \ /
  Last update: 2022-08-05 09:57    [W:0.149 / U:0.912 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site