lkml.org 
[lkml]   [2017]   [Sep]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH 0/6] Cache coherent device memory (CDM) with HMM v5
    On Thu, Jul 20, 2017 at 08:48:20PM -0700, Dan Williams wrote:
    > On Thu, Jul 20, 2017 at 6:41 PM, Jerome Glisse <jglisse@redhat.com> wrote:
    > > On Fri, Jul 21, 2017 at 09:15:29AM +0800, Bob Liu wrote:
    > >> On 2017/7/20 23:03, Jerome Glisse wrote:
    > >> > On Wed, Jul 19, 2017 at 05:09:04PM +0800, Bob Liu wrote:
    > >> >> On 2017/7/19 10:25, Jerome Glisse wrote:
    > >> >>> On Wed, Jul 19, 2017 at 09:46:10AM +0800, Bob Liu wrote:
    > >> >>>> On 2017/7/18 23:38, Jerome Glisse wrote:
    > >> >>>>> On Tue, Jul 18, 2017 at 11:26:51AM +0800, Bob Liu wrote:
    > >> >>>>>> On 2017/7/14 5:15, Jérôme Glisse wrote:

    [...]

    > >> > Second device driver are not integrated that closely within mm and the
    > >> > scheduler kernel code to allow to efficiently plug in device access
    > >> > notification to page (ie to update struct page so that numa worker
    > >> > thread can migrate memory base on accurate informations).
    > >> >
    > >> > Third it can be hard to decide who win between CPU and device access
    > >> > when it comes to updating thing like last CPU id.
    > >> >
    > >> > Fourth there is no such thing like device id ie equivalent of CPU id.
    > >> > If we were to add something the CPU id field in flags of struct page
    > >> > would not be big enough so this can have repercusion on struct page
    > >> > size. This is not an easy sell.
    > >> >
    > >> > They are other issues i can't think of right now. I think for now it
    > >>
    > >> My opinion is most of the issues are the same no matter use CDM or HMM-CDM.
    > >> I just care about a more complete solution no matter CDM,HMM-CDM or other ways.
    > >> HMM or HMM-CDM depends on device driver, but haven't see a public/full driver to
    > >> demonstrate the whole solution works fine.
    > >
    > > I am working with NVidia close source driver team to make sure that it works
    > > well for them. I am also working on nouveau open source driver for same NVidia
    > > hardware thought it will be of less use as what is missing there is a solid
    > > open source userspace to leverage this. Nonetheless open source driver are in
    > > the work.
    >
    > Can you point to the nouveau patches? I still find these HMM patches
    > un-reviewable without an upstream consumer.

    So i pushed a branch with WIP for nouveau to use HMM:

    https://cgit.freedesktop.org/~glisse/linux/log/?h=hmm-nouveau

    Top 16 patches are HMM related (implementic logic inside the driver to use
    HMM). The next 16 patches are hardware specific patches and some nouveau
    changes needed to allow page fault.

    It is enough to have simple malloc test case working:

    https://cgit.freedesktop.org/~glisse/compote

    There is 2 program here the old one is existing way you use GPU for compute
    task while the new one is what HMM allow to achieve ie use malloc memory
    directly.


    I haven't added yet the device memory support it is in work and i will push
    update to this branch and repo for that. Probably next week if no pressing
    bug preempt my time.


    So there is a lot of ugliness in all this and i don't expect this to be what
    end up upstream. Right now there is a large rework of nouveau vm (virtual
    memory) code happening to rework completely how we do address space management
    within nouveau. This work is prerequisite for a clean implementation for HMM
    inside nouveau (it will also lift the 40bits address space limitation that
    exist today inside nouveau driver). Once that work land i will work on clean
    upstreamable implementation for nouveau to use HMM as well as userspace to
    leverage it (this is requirement for upstream GPU driver to have open source
    userspace that make use of features). All this is a lot of work and there is
    not many people working on this.


    They are other initiatives under way related to this that i can not talk about
    publicly but if they bare fruit they might help to speedup all this.

    Jérôme

    \
     
     \ /
      Last update: 2017-09-05 21:37    [W:2.902 / U:0.136 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site