lkml.org 
[lkml]   [2000]   [Mar]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: Help in DSM design
    Hi,

    On Mon, 06 Mar 2000 11:36:06 -0800, Aman Singla
    <aman@cthulhu.engr.sgi.com> said:

    > Both add that additional level of locality that a performance
    > conscious application needs to exploit; agreed that KSR was the only
    > one to have caching at the remote memory to local memory level (with a
    > page size granularity).

    Yes, and the granularity was a _big_ problem.

    > A DSM system could look very much like the current NUMA h/w with an
    > added remote-local memory caching happening at application specific
    > granularity. The main issue there is getting access control at the
    > desired granularity for the s/w DSM layer

    Not just that. On SMP, and even more on NUMA, you only get scalable
    performance by going out of your way to avoid unnecessary sharing of
    data. It's just not transparent. And when you _do_ want to pass data
    from X to Y, you only want to do it once, which requires extra
    synchronisation to make sure that the receiver of the data doesn't start
    peeking at the data structures until the data is known to arrive.

    With message passing, that synchronisation is implicit. With shared
    memory, about all you can do is spin waiting for the data to arrive,
    unless you add a separate synchronisation mechanism somewhere to do
    locking outside the shared memory channel.

    It's not just granularity. The synchronisation hit is bad, on _all_
    forms of DSM up to and including SMP. (The latest Linux development
    kernel includes "big reader" locks which allow a read-mostly data
    structure to be locked on each CPU separately with no inter-CPU memory
    traffic at all, simply because even on such tightly coupled systems the
    cost of passing even one cache line of shared memory traffic between
    CPUs is too high.)

    --Stephen

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.rutgers.edu
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-03-22 13:56    [W:3.325 / U:1.768 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site