Messages in this thread | | | Date | Thu, 8 Sep 2022 14:02:07 -0700 | From | Dan Williams <> | Subject | Re: [PATCH 00/15] cxl: Add support for Restricted CXL hosts (RCD mode) |
| |
Jonathan Zhang (Infra) wrote: > > > > On Sep 8, 2022, at 12:51 PM, Dan Williams <dan.j.williams@intel.com> wrote: > > > > !-------------------------------------------------------------------| > > This Message Is From an External Sender > > > > |-------------------------------------------------------------------! > > > > Jonathan Zhang (Infra) wrote: > >> > >> > >>> On Sep 7, 2022, at 10:43 PM, Dan Williams <dan.j.williams@intel.com> wrote: > >>> > >>> Apologies for the delay in getting to this I had hoped to be able to > >>> finish up some other DAX work to focus on this, but time is getting > >>> short so I will need to do both in parallel. > >>> > >>> Robert Richter wrote: > >>>> In Restricted CXL Device (RCD) mode (formerly referred to as CXL 1.1) > >>>> the PCIe enumeration hierarchy is different from CXL VH Enumeration > >>>> (formerly referred to as 2.0, for both modes see CXL spec 3.0: 9.11 > >>>> and 9.12, [1]). This series adds support for RCD mode. It implements > >>>> the detection of Restricted CXL Hosts (RCHs) and its corresponding > >>>> Restricted CXL Devices (RCDs). It does the necessary enumeration of > >>>> ports and connects the endpoints. With all the plumbing an RCH/RCD > >>>> pair is registered at the Linux CXL bus and becomes visible in sysfs > >>>> in the same way as CXL VH hosts and devices do already. RCDs are > >>>> brought up as CXL endpoints and bound to subsequent drivers such as > >>>> cxl_mem. > >>>> > >>>> For CXL VH the host driver (cxl_acpi) starts host bridge discovery > >>>> once the ACPI0017 CXL root device is detected and then searches for > >>>> ACPI0016 host bridges to enable CXL. In RCD mode an ACPI0017 device > >>>> might not necessarily exist > >>> > >>> That's a broken BIOS as far as I can see. No ACPI0017 == no OS CXL > >>> services and the CXL aspects of the device need to be 100% managed by > >>> the BIOS. You can still run the cxl_pci driver in that case for mailbox > >>> operation, but error handling must be firmware-first without ACPI0017. > >> Firmware-first or OS-first applies to CXL protocol error handling. For CXL > >> memory error handling, the device generates a DRAM error record, the OS > >> parses such record and act accordingly. According to CXL spec (section > >> 8.2.9.2.1.2 DRAM Event Record), DPA but not HPA is in such record. The OS > >> needs to translate such DPA into HPA to act on. I am taking this as an example > >> to show that OS CXL services is needed. > >> Instead of using ACPI0016 to tell whether the system is under RCH mode, > >> I suppose one way is to check “CXL version” field of CHBS structure in CEDT? > > > > Unless the OS has negotiated CXL _OSC the BIOS owns the event retrieval > > and translating it from DPA to HPA. I do want to add OS CXL services to > > Linux, but only in the case when the BIOS is actively enabling OS native > > address translation which includes populating ACPI0017, CFMWS, and > > devices with the HDM decoder capability registers instead of DVSEC range > > registers. Everything else is early-gen CXL that is 100% BIOS supported, > > similar to DDR where a driver is not expected. > > > It makes sense that the BIOS and OS need to negotiate CXL _OSC so that OS > would take care of address translation. That being said, only DVSEC range > register (but not HDM decoder capability register) is available when the device is in > RCRB mode (section 9.11.8 figure 9-7) attached to a RCH. This type of > configuration needs to be supported with OS CXL service. >
So that figure does have the HDM capabilty pictured in the RCD upstream port. However, Table 8-22 does seem to incidate that Type 3 D1 devices are not permitted to have an HDM Decoder Capabilitiy Structure.
However that then leave me confused about figure 9-8 as that shows an HDM decoder capability in the BAR and not the RCRB. Is that picture wrong with respect what Table 8-22 indicates?
| |