Messages in this thread | | | From | Jonathan Maxwell <> | Date | Thu, 22 Dec 2022 16:39:21 +1100 | Subject | Re: [net-next] ipv6: fix routing cache overflow for raw sockets |
| |
On Wed, Dec 21, 2022 at 3:31 PM Jonathan Maxwell <jmaxwell37@gmail.com> wrote: > > On Wed, Dec 21, 2022 at 8:55 AM Jonathan Maxwell <jmaxwell37@gmail.com> wrote: > > > > On Wed, Dec 21, 2022 at 2:10 AM David Ahern <dsahern@kernel.org> wrote: > > > > > > On 12/20/22 5:35 AM, Paolo Abeni wrote: > > > > On Mon, 2022-12-19 at 10:48 +1100, Jon Maxwell wrote: > > > >> Sending Ipv6 packets in a loop via a raw socket triggers an issue where a > > > >> route is cloned by ip6_rt_cache_alloc() for each packet sent. This quickly > > > >> consumes the Ipv6 max_size threshold which defaults to 4096 resulting in > > > >> these warnings: > > > >> > > > >> [1] 99.187805] dst_alloc: 7728 callbacks suppressed > > > >> [2] Route cache is full: consider increasing sysctl net.ipv6.route.max_size. > > > >> . > > > >> . > > > >> [300] Route cache is full: consider increasing sysctl net.ipv6.route.max_size. > > > > > > > > If I read correctly, the maximum number of dst that the raw socket can > > > > use this way is limited by the number of packets it allows via the > > > > sndbuf limit, right? > > > > > > > > Are other FLOWI_FLAG_KNOWN_NH users affected, too? e.g. nf_dup_ipv6, > > > > ipvs, seg6? > > > > > > > > @DavidA: why do we need to create RTF_CACHE clones for KNOWN_NH flows? > > > > > > > > Thanks, > > > > > > > > Paolo > > > > > > > > > > If I recall the details correctly: that sysctl limit was added back when > > > ipv6 routes were managed as dst_entries and there was a desire to allow > > > an admin to limit the memory consumed. At this point in time, IPv6 is > > > more inline with IPv4 - a separate struct for fib entries from dst > > > entries. That "Route cache is full" message is now out of date since > > > this is dst_entries which have a gc mechanism. > > > > > > IPv4 does not limit the number of dst_entries that can be allocated > > > (ip_rt_max_size is the sysctl variable behind the ipv4 version of > > > max_size and it is a no-op). IPv6 can probably do the same here? > > > > > > > diff --git a/net/ipv6/route.c b/net/ipv6/route.c > > index dbc224023977..701aba7feaf5 100644 > > --- a/net/ipv6/route.c > > +++ b/net/ipv6/route.c > > @@ -6470,7 +6470,7 @@ static int __net_init ip6_route_net_init(struct net *net) > > #endif > > > > net->ipv6.sysctl.flush_delay = 0; > > - net->ipv6.sysctl.ip6_rt_max_size = 4096; > > + net->ipv6.sysctl.ip6_rt_max_size = INT_MAX; > > net->ipv6.sysctl.ip6_rt_gc_min_interval = HZ / 2; > > net->ipv6.sysctl.ip6_rt_gc_timeout = 60*HZ; > > net->ipv6.sysctl.ip6_rt_gc_interval = 30*HZ; > > > > The above patch resolved it for the Ipv6 reproducer. > > > > Would that be sufficient? > > > > Otherwise if you prefer to make Ipv6 behaviour similar to IPv4. > Rather than upping max_size. > > Here is prototype patch that removes the max_size check for Ipv6: >
There are some mistakes in this prototype patch. Let me come up with a better one. I'll submit a new patch in the new year for review. Thanks for the suggestion DavidA.
Regards
Jon
> diff --git a/include/net/dst_ops.h b/include/net/dst_ops.h > index 88ff7bb2bb9b..632086b2f644 100644 > --- a/include/net/dst_ops.h > +++ b/include/net/dst_ops.h > @@ -16,7 +16,7 @@ struct dst_ops { > unsigned short family; > unsigned int gc_thresh; > > - int (*gc)(struct dst_ops *ops); > + void (*gc)(struct dst_ops *ops); > struct dst_entry * (*check)(struct dst_entry *, __u32 cookie); > unsigned int (*default_advmss)(const struct dst_entry *); > unsigned int (*mtu)(const struct dst_entry *); > diff --git a/net/core/dst.c b/net/core/dst.c > index 497ef9b3fc6a..dcb85267bc4c 100644 > --- a/net/core/dst.c > +++ b/net/core/dst.c > @@ -82,12 +82,8 @@ void *dst_alloc(struct dst_ops *ops, struct net_device *dev, > > if (ops->gc && > !(flags & DST_NOCOUNT) && > - dst_entries_get_fast(ops) > ops->gc_thresh) { > - if (ops->gc(ops)) { > - pr_notice_ratelimited("Route cache is full: > consider increasing sysctl net.ipv6.route.max_size.\n"); > - return NULL; > - } > - } > + dst_entries_get_fast(ops) > ops->gc_thresh) > + ops->gc(ops); > > dst = kmem_cache_alloc(ops->kmem_cachep, GFP_ATOMIC); > if (!dst) > diff --git a/net/ipv6/route.c b/net/ipv6/route.c > index dbc224023977..8db7c5436da4 100644 > --- a/net/ipv6/route.c > +++ b/net/ipv6/route.c > @@ -91,7 +91,7 @@ static struct dst_entry *ip6_negative_advice(struct > dst_entry *); > static void ip6_dst_destroy(struct dst_entry *); > static void ip6_dst_ifdown(struct dst_entry *, > struct net_device *dev, int how); > -static int ip6_dst_gc(struct dst_ops *ops); > +static void ip6_dst_gc(struct dst_ops *ops); > > static int ip6_pkt_discard(struct sk_buff *skb); > static int ip6_pkt_discard_out(struct net *net, struct > sock *sk, struct sk_buff *skb); > @@ -3295,32 +3295,21 @@ struct dst_entry *icmp6_dst_alloc(struct > net_device *dev, > return dst; > } > > -static int ip6_dst_gc(struct dst_ops *ops) > +static void ip6_dst_gc(struct dst_ops *ops) > { > struct net *net = container_of(ops, struct net, ipv6.ip6_dst_ops); > - int rt_min_interval = net->ipv6.sysctl.ip6_rt_gc_min_interval; > - int rt_max_size = net->ipv6.sysctl.ip6_rt_max_size; > int rt_elasticity = net->ipv6.sysctl.ip6_rt_gc_elasticity; > int rt_gc_timeout = net->ipv6.sysctl.ip6_rt_gc_timeout; > - unsigned long rt_last_gc = net->ipv6.ip6_rt_last_gc; > int entries; > > entries = dst_entries_get_fast(ops); > - if (entries > rt_max_size) > - entries = dst_entries_get_slow(ops); > - > - if (time_after(rt_last_gc + rt_min_interval, jiffies) && > - entries <= rt_max_size) > - goto out; > > net->ipv6.ip6_rt_gc_expire++; > fib6_run_gc(net->ipv6.ip6_rt_gc_expire, net, true); > entries = dst_entries_get_slow(ops); > if (entries < ops->gc_thresh) > net->ipv6.ip6_rt_gc_expire = rt_gc_timeout>>1; > -out: > net->ipv6.ip6_rt_gc_expire -= net->ipv6.ip6_rt_gc_expire>>rt_elasticity; > - return entries > rt_max_size; > } > > static int ip6_nh_lookup_table(struct net *net, struct fib6_config *cfg, > > > > I do not believe the suggested flag is the right change. > > > > Regards > > > > Jon
| |