lkml.org 
[lkml]   [2008]   [Jan]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    SubjectRe: Integration of SCST in the mainstream Linux kernel
    From
    Date
    On Thu, 2008-01-31 at 18:50 +0300, Vladislav Bolkhovitin wrote:
    > Bart Van Assche wrote:
    > > On Jan 31, 2008 2:25 PM, Nicholas A. Bellinger <nab@linux-iscsi.org> wrote:
    > >
    > >>Since this particular code is located in a non-data path critical
    > >>section, the kernel vs. user discussion is a wash. If we are talking
    > >>about data path, yes, the relevance of DD tests in kernel designs are
    > >>suspect :p. For those IB testers who are interested, perhaps having a
    > >>look with disktest from the Linux Test Project would give a better
    > >>comparision between the two implementations on a RDMA capable fabric
    > >>like IB for best case performance. I think everyone is interested in
    > >>seeing just how much data path overhead exists between userspace and
    > >>kernel space in typical and heavy workloads, if if this overhead can be
    > >>minimized to make userspace a better option for some of this very
    > >>complex code.
    > >
    > > I can run disktest on the same setups I ran dd on. This will take some
    > > time however.
    >
    > Disktest was already referenced in the beginning of the performance
    > comparison thread, but its results are not very interesting if we are
    > going to find out, which implementation is more effective, because in
    > the modes, in which usually people run this utility, it produces latency
    > insensitive workload (multiple threads working in parallel). So, such
    > multithreaded disktests results will be different between STGT and SCST
    > only if STGT's implementation will get target CPU bound. If CPU on the
    > target is powerful enough, even extra busy loops in the STGT or SCST hot
    > path code will change nothing.
    >

    I think the really interesting numbers are the difference for bulk I/O
    between kernel and userspace on both traditional iSCSI and the RDMA
    enabled flavours. I have not been able to determine anything earth
    shattering from the current run of kernel vs. userspace tests, nor which
    method of implementation for iSER, SRP, and generic Storage Engine are
    'more effective' for that case. Performance and latency to real storage
    would make alot more sense for the kernel vs. user case. Also workloads
    against software LVM and Linux MD block devices would be of interest as
    these would be some of the more typical deployments that would be in the
    field, and is what Linux-iSCSI.org uses for our production cluster
    storage today.

    Having implemented my own iSCSI and SCSI Target mode Storage Engine
    leads me to believe that putting logic in userspace is probably a good
    idea in the longterm. If this means putting the entire data IO path
    into userspace for Linux/iSCSI, then there needs to be a good reason why
    this will not not scale to multi-port 10 Gb/sec engines in traditional
    and RDMA mode if we need to take this codepath back into the kernel.
    The end goal is to have the most polished and complete storage engine
    and iSCSI stacks designs go upstream, which is something I think we can
    all agree on.

    Also, with STGT being a pretty new design which has not undergone alot
    of optimization, perhaps profiling both pieces of code against similar
    tests would give us a better idea of where userspace bottlenecks reside.
    Also, the overhead involved with traditional iSCSI for bulk IO from
    kernel / userspace would also be a key concern for a much larger set of
    users, as iSER and SRP on IB is a pretty small userbase and will
    probably remain small for the near future.

    > Additionally, multithreaded disktest over RAM disk is a good example of
    > a synthetic benchmark, which has almost no relation with real life
    > workloads. But people like it, because it produces nice looking results.
    >

    Yes, people like to claim their stacks are the fastest with RAM disk
    benchmarks. But hooking up their fast network silicon to existing
    storage hardware and OS storage subsystems and software is where the
    real game is..

    > Actually, I don't know what kind of conclusions it is possible to make
    > from disktest's results (maybe only how throughput gets bigger or slower
    > with increasing number of threads?), it's a good stress test tool,
    > but
    > not more.
    >

    Being able to have a best case baseline with disktest for kernel vs.
    user would be of interest for both transport protocol and SCSI Target
    mode Storage Engine profiling. The first run of tests looked pretty
    bandwith oriented, so disktest works well to determine maximum bandwith.
    Disktest also is nice for getting reads from cache on hardware RAID
    controllers because disktest only generates requests with LBAs from 0 ->
    disktest BLOCKSIZE.

    --nab

    > > Disktest is new to me -- any hints with regard to suitable
    > > combinations of command line parameters are welcome. The most recent
    > > version I could find on http://ltp.sourceforge.net/ is ltp-20071231.
    > >
    > > Bart Van Assche.
    > > -
    > > To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
    > > the body of a message to majordomo@vger.kernel.org
    > > More majordomo info at http://vger.kernel.org/majordomo-info.html
    > >
    >
    >



    \
     
     \ /
      Last update: 2008-01-31 18:23    [W:5.751 / U:0.208 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site