Messages in this thread | | | Date | Wed, 11 Jan 2023 11:18:13 +0800 | Subject | Re: [External] Re: [PATCH v4] blk-throtl: Introduce sync and async queues for blk-throtl | From | hanjinke <> |
| |
在 2023/1/11 上午6:21, Tejun Heo 写道: > On Tue, Jan 10, 2023 at 11:59:09PM +0800, hanjinke wrote: >> >> >> 在 2023/1/10 上午4:40, Tejun Heo 写道: >>> On Sat, Jan 07, 2023 at 09:07:38PM +0800, Jinke Han wrote: >>>> + * Assumed that there were only bios queued in ASYNC queue and the SYNC >>>> + * queue was empty. The ASYNC bio was selected to dispatch and the >>>> + * disp_sync_cnt was set to 0 after each dispatching. If a ASYNC bio >>>> + * can't be dispatched because of overlimit in current slice, the process >>>> + * of dispatch should give up and the spin lock of the request queue >>>> + * may be released. A new SYNC bio may be queued in the SYNC queue then. >>>> + * When it's time to dispatch this tg, the SYNC bio was selected and pop >>>> + * to dispatch as the disp_sync_cnt is 0 and the SYNC queue is no-empty. >>>> + * If the dispatched bio is smaller than the waiting bio, the bandwidth >>>> + * may be hard to satisfied as the slice may be trimed after each dispatch. >>> >>> I still can't make a good sense of this scenario. Can you give concrete >>> example scenarios with IOs and why it would matter? >>> >>> Thanks. >>> >> >> Assumed that there are many buffer write bios queued in ASYNC queue and the >> SYNC queue is empty. The buffer write bios are all 1M in size and the bps >> limit is 1M/s. The throtl_slice is 100ms. >> >> Assumed that the start/end_slice is [jiffies1, jiffies1+100] and >> bytes_disp[w] = 0. The next ASYNC bio can't dispatch because of overlimit >> within this slice. The wait time is 900ms and the slice will be extended to >> [jiffies1, jiffies1 + 1000] in tg_may_dispatch. >> >> During the waiting of the ASYNC bio, a SYNC 4k bio be queued in SYNC queue. >> After 900ms, it's time to dispatch the ASYNC io, but the SYNC 4k bio be >> selected to be dispatched. Now the slice is [jiffies1, jiffies1+1000] and >> the byte_disp[w] = 4k. The slice may be extended to >> [jiffies1, jiffies1+1100]. In tg_dispatch_one_bio, the slice will be trimed >> to [jiffies1+1000, jiffies1+1100], the byte_disp[w] will set 0. >> >> After the 4k SYNC bio be dispatched, the WAITING ASYNC bio still cann't >> be dispatched because of overlimit within this slice. >> >> The same thing may happen DISPACH_SYNC_FACTOR times if alway there is a SYNC >> bio be queued in the SYNC queue when the ASYNC bio is waiting. >> >> This means that in nearly 5s, we have dispathed 4 4k SYNC bios and a 1m >> ASYNC bio. >> >> In our test, with 100M/s bps limit setted, the bps only reach to ~80m/s >> when a fio generate buffer write ios and fsync continuous generated by >> dbench in same cgroup. > > That makes sense to me. Thanks for the explanation. It'd probably a good > idea to expand the explanation in the comment tho. > > Thanks. >
I will be glad to do so and send the v5.
Thanks. Jinke
| |