Messages in this thread | | | Date | Tue, 5 Feb 2008 09:24:50 +0100 | From | Jens Axboe <> | Subject | Re: [rfc] direct IO submission and completion scalability issues |
| |
On Mon, Feb 04 2008, Arjan van de Ven wrote: > Jens Axboe wrote: > >>I was imagining the patch a little bit differently (per-cpu tasks, do a > >>wake_up from the driver instead of cpu nr testing up in blk, work > >>queues, whatever), but we know how to iron out these kinds of details ;). > > > >per-cpu tasks/wq's might be better, it's a little awkward to jump > >through hoops > > > > one caveat btw; when the multiqueue storage hw becomes available for Linux, > we need to figure out how to deal with the preference thing; since there > honoring a "non-logical" preference would be quite expensive (it means
non-local?
> you can't make the local submit queues lockless etc etc), so before we > go down the road of having widespread APIs for this stuff.. we need to > make sure we're not going to do something that's going to be really > stupid 6 to 18 months down the road.
As far as I'm concerned, so far this is just playing around with affinity (and to some extents taking it too far, on purpose). For instance, my current patch can move submissions and completions independently, with a set mask or by 'binding' a request to a CPU. Most of that doesn't make sense. 'complete on the same CPU, if possible' makes sense and would fit fine with multi-queue hw.
Moving submissions at the block layer to a defined set of CPUs is a bit silly imho, it's pretty costly and it's a lot more sane simply bind the submitters instead. So if you can set irq affinity, then just make the submitters follow that.
-- Jens Axboe
| |