Messages in this thread | | | Date | Mon, 26 Sep 2022 21:08:27 +0100 | Subject | Re: [PATCH net-next v4 00/27] io_uring zerocopy send | From | Pavel Begunkov <> |
| |
On 7/24/22 19:28, David Ahern wrote: > On 7/17/22 8:19 PM, David Ahern wrote: >>> >>> Haven't seen it back then. In general io_uring doesn't stop submitting >>> requests if one request fails, at least because we're trying to execute >>> requests asynchronously. And in general, requests can get executed >>> out of order, so most probably submitting a bunch of requests to a single >>> TCP sock without any ordering on io_uring side is likely a bug. >> >> TCP socket buffer fills resulting in a partial send (i.e, for a given >> sqe submission only part of the write/send succeeded). io_uring was not >> handling that case. >> >> I'll try to find some time to resurrect the iperf3 patch and try top of >> tree kernel. > > With your zc_v5 branch (plus the init fix on using msg->sg_from_iter), > iperf3 with io_uring support (non-ZC case) no longer shows completions > with incomplete sends. So that is good improvement over the last time I > tried it. > > However, adding in the ZC support and that problem resurfaces - a lot of > completions are for an incomplete size. > > liburing comes from your tree, zc_v4 branch. Upstream does not have > support for notifications yet, so I can not move to it. > > Changes to iperf3 are here: > https://github.com/dsahern/iperf mods-3.10-io_uring
Tried it out, the branch below fixes a small problem, adds a couple of extra optimisations and now it actually uses registered buffers.
https://github.com/isilence/iperf iou-sendzc
Still, the submission loop looked a bit weird, i.e. it submits I/O to io_uring only when it exhausts sqes instead of sending right away with some notion of QD and/or sending in batches. The approach is good for batching (SQ size =16 here), but not so for latency.
I also see some CPU cycles being burnt in select(2). io_uring wait would be more natural and perhaps more performant, but I didn't spend enough time with iperf to say for sure.
-- Pavel Begunkov
| |