Messages in this thread | | | Date | Tue, 10 May 2022 16:03:47 -0700 | From | Peilin Ye <> | Subject | Re: [PATCH RFC v1 net-next 0/4] net: Qdisc backpressure infrastructure |
| |
Hi Eric,
On Mon, May 09, 2022 at 08:26:27PM -0700, Eric Dumazet wrote: > On 5/6/22 12:43, Peilin Ye wrote: > > From: Peilin Ye <peilin.ye@bytedance.com> > > > > Hi all, > > > > Currently sockets (especially UDP ones) can drop a lot of skbs at TC > > egress when rate limited by shaper Qdiscs like HTB. This experimental > > patchset tries to improve this by introducing a backpressure mechanism, so > > that sockets are temporarily throttled when they "send too much". > > > > For now it takes care of TBF, HTB and CBQ, for UDP and TCP sockets. Any > > comments, suggestions would be much appreciated. Thanks! > > This very much looks like trying to solve an old problem to me. > > If you were using EDT model, a simple eBPF program could get rid of the > HTB/TBF qdisc > > and you could use MQ+FQ as the packet schedulers, with the true multiqueue > sharding. > > FQ provides fairness, so a flow can not anymore consume all the qdisc limit.
This RFC tries to solve the "when UDP starts to drop (whether because of per-flow or per-Qdisc limit), it drops a lot" issue described in [I] of the cover letter; its main goal is not to improve fairness.
> (If your UDP sockets send packets all over the place (not connected > sockets), > > then the eBPF can also be used to rate limit them)
I was able to reproduce the same issue using EDT: default sch_fq flow_limit (100 packets), with a 1 Gbit/sec rate limit. Now if I run this:
$ iperf -u -c 1.2.3.4 -p 5678 -l 3K -i 0.5 -t 30 -b 3g
[ ID] Interval Transfer Bandwidth [ 3] 0.0- 0.5 sec 137 MBytes 2.29 Gbits/sec [ 3] 0.5- 1.0 sec 142 MBytes 2.38 Gbits/sec [ 3] 1.0- 1.5 sec 117 MBytes 1.96 Gbits/sec [ 3] 1.5- 2.0 sec 105 MBytes 1.77 Gbits/sec [ 3] 2.0- 2.5 sec 132 MBytes 2.22 Gbits/sec <...> ^^^^^^^^^^^^^^
On average it tries to send 2.31 Gbits per second, dropping 56.71% of the traffic:
$ tc -s qdisc show dev eth0 <...> qdisc fq 5: parent 1:4 limit 10000p flow_limit 100p buckets 1024 orphan_mask 1023 quantum 18030 initial_quantum 90150 low_rate_threshold 550Kbit refill_delay 40.0ms Sent 16356556 bytes 14159 pkt (dropped 2814461, overlimits 0 requeues 0) ^^^^^^^^^^^^^^^
This RFC does not cover EDT though, since it does not use Qdisc watchdog or friends.
Thanks, Peilin Ye
| |