lkml.org 
[lkml]   [2012]   [Dec]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH 00/26] AIO performance improvements/cleanups, v2
On Fri, Dec 14, 2012 at 08:35:53AM +0100, Jens Axboe wrote:
> On 2012-12-14 03:26, Jack Wang wrote:
> > 2012/12/14 Jens Axboe <jaxboe@fusionio.com>:
> >> On Mon, Dec 03 2012, Kent Overstreet wrote:
> >>> Last posting: http://thread.gmane.org/gmane.linux.kernel.aio.general/3169
> >>>
> >>> Changes since the last posting should all be noted in the individual
> >>> patch descriptions.
> >>>
> >>> * Zach pointed out the aio_read_evt() patch was calling functions that
> >>> could sleep in TASK_INTERRUPTIBLE state, that patch is rewritten.
> >>> * Ben pointed out some synchronize_rcu() usage was problematic,
> >>> converted it to call_rcu()
> >>> * The flush_dcache_page() patch is new
> >>> * Changed the "use cancellation list lazily" patch so as to remove
> >>> ki_flags from struct kiocb.
> >>
> >> Kent, I ran a few tests, and the below patches still don't seem as fast
> >> as the approach I took. To keep it fair, I used your aio branch and
> >> applied by dio speedups too. As a sanity check, I ran with your branch
> >> alone as well. The quick results below - kaio is kent-aio, just your
> >> branch. kaio-dio is with the direct IO speedups too. jaio is my branch,
> >> which already has the dio changes too.
> >>
> >> Devices Branch IOPS
> >> 1 kaio ~915K
> >> 1 kaio-dio ~930K
> >> 1 jaio ~1220K
> >> 6 kaio ~3050K
> >> 6 kaio-dio ~3080K
> >> 6 jaio 3500K
> >>
> >> The box runs out of CPU driving power, which is why it doesn't scale
> >> linearly, otherwise I know that jaio at least does. It's basically
> >> completion limited for the 6 device test at the moment.
> >>
> >> I'll run some profiling tomorrow morning and get you some better
> >> results. Just thought I'd share these at least.
> >>
> >> --
> >> Jens Axboe
> >>
> >
> > A really good performance, woo.
> >
> > I think the device tested is really fast PCIe SSD builded by fusionio
> > with fusionio in house block driver?
>
> It is pci-e flash storage, but it is not fusion-io.
>
> > any compare number with current mainline?
>
> Sure, I should have included that. Here's the table again, this time
> with mainline as well.
>
> Devices Branch IOPS
> 1 mainline ~870K
> 1 kaio ~915K
> 1 kaio-dio ~930K
> 1 jaio ~1220K
> 6 kaio ~3050K
> 6 kaio-dio ~3080K
> 6 jaio ~3500K
> 6 mainline ~2850K

Cool, thanks for the numbers!

I suspect the difference is due to contention on the ringbuffer,
completion side. You didn't enable my batched completion stuff, did you?

I suspect the numbers would look quite a bit different with that,
based on my own profiling. If the driver for the device you're testing
on is open source, I'd be happy to do the conversion (it's a 5 minute
job).

Also, I don't think our approaches really conflict - it's been awhile
since I looked at your patch but you're getting rid of the aio
ringbuffer and using a linked list instead, right? My batched completion
stuff should still benefit that case.

Though - hrm, I'd have expected getting rid of the cancellation linked
list to make a bigger difference and both our patchsets do that.

What device are you testing on, and what's your fio script? I may just
have to buy some hardware so I can test this myself.


\
 
 \ /
  Last update: 2012-12-15 10:41    [W:0.092 / U:0.096 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site