Messages in this thread | | | Date | Wed, 30 Aug 2023 17:02:28 +0200 | Subject | Re: [Networking] ERSPAN decapsulation drops DHCP unicast packets | From | Volodymyr Litovka <> |
| |
Hi colleagues,
sorry bothering you, but can anyone shed light on this issue? This stops me and I will be glad to hear, where I'm wrong and/or where try to look into the problem.
Thank you very much.
On 8/27/23 10:34, Volodymyr Litovka wrote: > Hi Bagas, > > this tested on: > > - 5.19.0-42 on Intel 82599ES 10-Gigabit SFI/SFP+ Network Connection > -- this is host hardware > - 6.2.0-32 on Virtio network device (under KVM 6.2 on host hardware > above) > - 6.5.0-060500rc7 on Virtio network device (under KVM 6.2 on host > hardware above) > > Result is the same for all cases. > > Thank you. > > On 8/27/23 04:07, Bagas Sanjaya wrote: >> On Sat, Aug 26, 2023 at 09:55:30PM +0200, Volodymyr Litovka wrote: >>> Hi colleagues, >>> >>> I'm trying to catch and process (in 3rd party analytics app) DHCP >>> packets >>> from ERSPAN session, but cannot do this due to absence of DHCP unicast >>> packets after decapsulation. >>> >>> The model is pretty simple: there is PHY interface (enp2s0) which >>> receive >>> ERSPAN traffic and erspan-type interface to get decapsulated packets >>> (inspan, created using command "ip link add inspan type erspan seq >>> key 10 >>> local 10.171.165.65 erspan_ver 1", where 10.171.165.65 is ERSPAN >>> target). >>> Then I'm going to rewrite headers in the proper ways (nftable's netdev >>> family) and forward packets to the pool of workers. >>> >>> Having this, I'm expecting everything, which is encapsulated inside >>> ERSPAN, >>> on 'inspan' interface. And there is _almost_ everything except DHCP >>> unicast >>> packets - tcpdump shows about 1kps on this interface of decapsulated >>> packets, but no DHCP unicast (see below traces). >>> >>> To avoid any interactions, I removed and disabled everything that >>> can catch >>> DHCP in userspace - systemd-networkd, netplan, dhcp-client. There is >>> no DHCP >>> server and ifupdown - for test purposes, I'm bringing networking >>> manually. >>> Apparmor disabled as well. Kernel (Linux 5.19.0-42-generic >>> #43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC) compiled without CONFIG_IP_PNP >>> (according to /boot/config-5.19.0-42-generic). Nothing in userspace >>> listens >>> on UDP/68 and UDP/67: >> Can you reproduce this on latest mainline? >> >>> # netstat -tunlpa >>> Active Internet connections (servers and established) >>> Proto Recv-Q Send-Q Local Address Foreign Address >>> State PID/Program name >>> tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 544/sshd: >>> /usr/sbin >>> tcp6 0 0 :::22 :::* LISTEN 544/sshd: >>> /usr/sbin >>> >>> I have no ideas, why this is happening. Decapsulation itself works, but >>> particular kind of packets get lost. >>> >>> I will appreciate if anyone can help me understand where is the bug >>> - in my >>> configuration or somewhere inside the kernel? >>> >>> Evidence of traffic presence/absence is below. >>> >>> Thank you. >>> >>> Encapsulated ERSPAN session (udp and port 67/68) contains lot of >>> different >>> kinds of DHCP packets: >>> >>> # tcpdump -s0 -w- -i enp2s0 'proto gre and ether[73:1]=17 and >>> (ether[84:2]=67 or ether[84:2]=68)' | tshark -r- -l >>> [ ... ] >>> 7 0.001942 0.0.0.0 → 255.255.255.255 DHCP 392 DHCP Discover - >>> Transaction ID 0x25c096fc >>> 8 0.003432 z.z.z.z → a.a.a.a DHCP 418 DHCP ACK - >>> Transaction ID 0x5515126a >>> 9 0.005170 m.m.m.m → z.z.z.z DHCP 435 DHCP Discover - >>> Transaction ID 0xa7b7 >>> 10 0.005171 m.m.m.m → z.z.z.z DHCP 435 DHCP Discover - >>> Transaction ID 0xa7b7 >>> 11 0.015399 n.n.n.n → z.z.z.z DHCP 690 DHCP Request - >>> Transaction ID 0x54955233 >>> 12 0.025537 z.z.z.z → n.n.n.n DHCP 420 DHCP ACK - >>> Transaction ID 0x54955233 >>> 13 0.030313 z.z.z.z → m.m.m.m DHCP 413 DHCP Offer - >>> Transaction ID 0xa7b7 >>> >>> but decapsulated traffic (which I'm seeing on inspan interface) >>> contains >>> just the following: >>> >>> # tcpdump -i inspan 'port 67 or port 68' >>> listening on inspan, link-type EN10MB (Ethernet), snapshot length >>> 262144 >>> bytes >>> 17:23:36.540721 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, >>> Request from 00:1a:64:33:8d:fa (oui Unknown), length 300 >>> 17:23:39.760036 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, >>> Request from 00:1a:64:33:8d:fa (oui Unknown), length 300 >>> 17:23:44.135711 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, >>> Request from 00:1a:64:33:8d:fa (oui Unknown), length 300 >>> 17:23:52.008504 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, >>> Request from 00:1a:64:33:8d:fa (oui Unknown), length 300 >>> >> What hardware? >> -- Volodymyr Litovka "Vision without Execution is Hallucination." -- Thomas Edison
| |