lkml.org 
[lkml]   [2013]   [Feb]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: Improving AIO cancellation
    On 02/08/13 04:42, Anatol Pomozov wrote:
    > Hi,
    >
    > At Google we have several applications that heavily use asynchronous
    > IO. One thing that our userspace developers need is effective AIO
    > cancellation. You might say "sure use io_cancel syscall". Well, while
    > it cancels AIO requests it does it ineffectively. Currently (I am
    > looking at linux-next) io_cancel only marks kiocb as cancelled. The
    > bios still be issued to device even after kiocb was cancelled. Let's
    > say you have a congested device and want to cancel some AIO requests -
    > io_cancel will not make situation better. We would like to see more
    > resource effective AIO cancellation.
    >
    > I had a discussion with Ted Tso and Kent Overstreet about improving
    > this situation and would like to share the ideas with you, linux
    > community.
    >
    > Once direct async IO is submitted the request can be at several stages:
    > 1) Sitting in kernel request queue of a congested device
    > 2) Sent to device and sitting in device queue (if NCQ is enabled)
    > 3) Executing on device
    >
    > Ideally if we can cancel an IO request on any of these stages. But
    > currently we are especially interested in case #1. I do not know if
    > cancellation at stage #2 and #3 is possible and/or reasonable.
    >
    > BTW AIO cancellation makes sense only for direct IO. Buffered AIO will
    > end up in buffer soon and kiocb will be marked as completed. Later
    > (maybe much later) writeback will flush those buffers to disk, but you
    > cannot cancel it..
    >
    > And yet another thing to remember is md/RAID. Some types of raid
    > support stripes consistency. When md splits a WRITE across disks
    > either all or no of the child requests should be completed. If we do
    > partial write then the disk data will become inconsistent.
    >
    >
    > Ted and Kent suggested following solution: any time when we do forward
    > progress with request/bio we need to check its status. If user
    > cancelled the request then just skip this bio. So it covers case #1.
    >
    > The draft implementation will look like this. struct bio should have
    > some way to get current status of kiocb that generated bio. So we add
    > a pointer to bool flag.
    >
    > struct bio {
    > bool *cancelled;
    > }
    >
    > in async DIO codepath this pointer will be initialized with bool at
    > "struct kiocb"
    > bio->cancelled = &kiocb->cancelled;
    > except md. If it is RAID5 and we perform WRITE request then we do not
    > initialize this pointer.
    >
    >
    > when we do forward progress with request/bio we check its cancellation status:
    > if (bio->cancelled && *bio->cancelled)
    > goto do_not_process_bio_because_it_cancelled;
    >
    > So to cancel kiocb we do
    > kiocb->cancelled = true;
    > and all bio created from the request will not be send to device anymore.
    >
    >
    > The solution seems straightforward, but I would like to hear if there
    > are other solutions to make AIO cancellation better. Does suggested
    > implementation looks good? Are there better solutions? What about
    > cancelling requests that are already sent to device?
    >
    > If the proposal is fine then I start implementing it.

    Hello Anatol,

    Had you already noticed this message:
    http://marc.info/?l=linux-fsdevel&m=136024044202362 ?

    Bart.



    \
     
     \ /
      Last update: 2013-02-08 10:21    [W:4.830 / U:1.072 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site