lkml.org 
[lkml]   [2019]   [Nov]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges
    On Thu, Nov 21, 2019 at 5:06 PM Karol Herbst <kherbst@redhat.com> wrote:
    >
    > On Thu, Nov 21, 2019 at 4:47 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
    > >
    > > On Thu, Nov 21, 2019 at 1:53 PM Karol Herbst <kherbst@redhat.com> wrote:
    > > >
    > > > On Thu, Nov 21, 2019 at 12:46 PM Mika Westerberg
    > > > <mika.westerberg@intel.com> wrote:
    > > > >
    > > > > On Thu, Nov 21, 2019 at 12:34:22PM +0100, Rafael J. Wysocki wrote:
    > > > > > On Thu, Nov 21, 2019 at 12:28 PM Mika Westerberg
    > > > > > <mika.westerberg@intel.com> wrote:
    > > > > > >
    > > > > > > On Wed, Nov 20, 2019 at 11:29:33PM +0100, Rafael J. Wysocki wrote:
    > > > > > > > > last week or so I found systems where the GPU was under the "PCI
    > > > > > > > > Express Root Port" (name from lspci) and on those systems all of that
    > > > > > > > > seems to work. So I am wondering if it's indeed just the 0x1901 one,
    > > > > > > > > which also explains Mikas case that Thunderbolt stuff works as devices
    > > > > > > > > never get populated under this particular bridge controller, but under
    > > > > > > > > those "Root Port"s
    > > > > > > >
    > > > > > > > It always is a PCIe port, but its location within the SoC may matter.
    > > > > > >
    > > > > > > Exactly. Intel hardware has PCIe ports on CPU side (these are called
    > > > > > > PEG, PCI Express Graphics, ports), and the PCH side. I think the IP is
    > > > > > > still the same.
    > > > > > >
    > > >
    > > > yeah, I meant the bridge controller with the ID 0x1901 is on the CPU
    > > > side. And if the Nvidia GPU is on a port on the PCH side it all seems
    > > > to work just fine.
    > >
    > > But that may involve different AML too, may it not?
    > >
    > > > > > > > Also some custom AML-based power management is involved and that may
    > > > > > > > be making specific assumptions on the configuration of the SoC and the
    > > > > > > > GPU at the time of its invocation which unfortunately are not known to
    > > > > > > > us.
    > > > > > > >
    > > > > > > > However, it looks like the AML invoked to power down the GPU from
    > > > > > > > acpi_pci_set_power_state() gets confused if it is not in PCI D0 at
    > > > > > > > that point, so it looks like that AML tries to access device memory on
    > > > > > > > the GPU (beyond the PCI config space) or similar which is not
    > > > > > > > accessible in PCI power states below D0.
    > > > > > >
    > > > > > > Or the PCI config space of the GPU when the parent root port is in D3hot
    > > > > > > (as it is the case here). Also then the GPU config space is not
    > > > > > > accessible.
    > > > > >
    > > > > > Why would the parent port be in D3hot at that point? Wouldn't that be
    > > > > > a suspend ordering violation?
    > > > >
    > > > > No. We put the GPU into D3hot first, then the root port and then turn
    > > > > off the power resource (which is attached to the root port) resulting
    > > > > the topology entering D3cold.
    > > > >
    > > >
    > > > If the kernel does a D0 -> D3hot -> D0 cycle this works as well, but
    > > > the power savings are way lower, so I kind of prefer skipping D3hot
    > > > instead of D3cold. Skipping D3hot doesn't seem to make any difference
    > > > in power savings in my testing.
    > >
    > > OK
    > >
    > > What exactly did you do to skip D3cold in your testing?
    > >
    >
    > For that I poked into the PCI registers directly and skipped doing the
    > ACPI calls and simply checked for the idle power consumption on my
    > laptop.

    That doesn't involve the PCIe port PM, however.

    > But I guess I should retest with calling pci_d3cold_disable
    > from nouveau instead? Or is there a different preferable way of
    > testing this?

    There is a sysfs attribute called "d3cold_allowed" which can be used
    for "blocking" D3cold, so can you please retest using that?

    \
     
     \ /
      Last update: 2019-11-21 17:40    [W:4.101 / U:0.128 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site