Messages in this thread | | | Date | Tue, 06 Sep 2022 10:10:50 +0200 | From | Michael Walle <> | Subject | Re: [PATCH devicetree] arm64: dts: ls1028a-rdb: add more ethernet aliases |
| |
Am 2022-09-06 01:54, schrieb Vladimir Oltean: > On Tue, Sep 06, 2022 at 12:17:29AM +0200, Michael Walle wrote: >> First, let me say, I'm fine with this patch. But I'm not sure, >> how many MAC addresses are actually reserved on your >> RDB/QDS boards? > > AFAIK, the Reference Design Boards are sold with an unprogrammed I2C > EEPROM, but with a sticker containing 5 MAC addresses on the bottom of > the board. It doesn't have a clear correspondence between MAC addresses > and their intended use, although I suspect that one MAC address is > intended for each RJ45 port (although that isn't how I use them). > > For the QIXIS Development Boards, I have no clue, it's probably even > nonsensical to talk about MAC address reservations since there is just > one onboard Ethernet port (RGMII) and the rest is routed via SERDES to > PCIe slots, to pluggable riser cards, from which Linux/U-Boot don't > bother > too much to read back any info, even though I can't exclude something > like an EEPROM may be available on those cards too. In any case, I > think > QDS boards don't leave the lab, so it doesn't matter too much. > > The way I use the MAC addresses from the sticker of my RDBs, on a day > to > day basis, is: > > ethaddr (eno0) - #1 > eth1addr (eno2) - #2 > eth2addr (swp0) - #2 > eth3addr (swp1) - #2 > eth4addr (swp2) - #2 > eth5addr (swp3) - #2
Ah, I never thought of handing out the same MAC address.
> And now I'm adding these new env variables: > > eth6addr (swp4) - #2 > eth7addr (swp5) - #2 > eth8addr (eno3) - #3 > > So I still have 2 more unique MAC addresses to burn through. > >> I guess, they being evaluation boards you don't care? ;) > > I do care a bit, but not that much. > >> On the Kontron sl28 boards we reserve just 8 and that is >> already a lot for a board with max 6 out facing ports. 4 of >> these ports used to be a switch, so in theory it should work > > /used/ to be a switch? What happened to them? Details? Or you mean > "4 ports are used as a switch"?
I shouldn't probably write mails right before going to sleep. Yes it should read "the 4 ports (swp0..swp3) are usually configured to as a switch."
>> with 3 MAC addresses, right? > > Which 3 MAC addresses would those be? Not sure I'm following. enetc #0, > enetc #1, enetc #2? That could work, multiple DSA user ports can share > the same MAC address (inherited from the DSA master or not) and things > would work just fine unless you connect them to each other.
enetc #0, #1 and swp0. As you mentioned, swp1..3 should inherit the address from swp0 then if swp0 is added as the first device, right?
So why would enetc#2 (or #3) need a non-random mac address? I must be missing something here.
>> Or even just 2 if there is no need to terminate any traffic on the >> switch interfaces. > > And here, which 2? enetc #0 and enetc #1?
Yes. The switch would just be a dumb ethernet switch.
>> Anyway, do we really need so many addresses? > > idk, who's "we" and what does "need" mean? (serious questions)
We as in the users of the ls1028a SoC. And as I said, I thought of *unique* MAC addresses.
> I'm not sure I can give you any answer to this question. As an engineer > working with the kernel, I need to roll the LS1028A Ethernet around on > all its sides. The Linux RDB/QDS support will inevitably reflect what > we > need to test. Everybody else will have a fixed configuration, and the > user reviews will vary from 'internet works! 5 stars!' to 'internet > doesn't work! 1 star!'. > > To offer that quality of service for all front-facing ports, you don't > need much. I know of a 12 port industrial switch that entered > production > with 1 MAC address, the "termination" address. It's fine, when it's > marketed as a switch, people come to expect that and don't wonder too > much. > >> What are the configurations here? For what is the address of the >> internal ports used? > > By internal ports you mean swp4/swp5, or eno2/eno3?
eno2/eno3.
> If eno2/eno3, then a > configuration where having MAC addresses on these interfaces is useful > to me is running some of the kselftests on the LS1028A-RDB, which does > not have enough external enetc ports for 2 loopback pairs, so I do > this, thereby having 1 external loopback through a cable, and 1 > internal > loopback in the SoC: > > ./psfp.sh eno0 swp0 swp4 eno2 > https://github.com/torvalds/linux/blob/master/tools/testing/selftests/drivers/net/ocelot/psfp.sh > > Speaking of kselftests, it actually doesn't matter that much what the > MAC addresses *are*, since we don't enter any network, just loop back > traffic. In fact we have an environment variable STABLE_MAC_ADDRS, > which > when set, configures the ports to use some predetermined MAC addresses. > > There are other configurations where it is useful for eno2 to see DSA > untagged traffic. These are downstream 802.1CB (where this hardware can > offload redundant streams in the forwarding plane, but not in the > termination plane, so we use eno2 as forwarding plane, for > termination),
I'm not that familiar with 802.1CB. Is this MAC address visible outside of the switch or can it be a random one?
> DPDK on eno2 (which mainline Linux doesn't care about), and vfio-pci + > QEMU, where DSA switch control still belongs to the Linux host, but the > guest has 'internet'.
For me, all of that is kind of a trade off. If you want to use virtual interfaces, you might need to borrow a MAC address from one of the switch ports (where you have 3 unique addresses left if you combine all 4 ports to one bridge).
>> Let's say we are in the "port extender mode" and use the >> second internal port as an actual switch port, that would >> then be: >> 2x external enetc >> 1x internal enetc >> 4x external switch ports in port extender mode >> >> Which makes 7 addresses. The internal enetc port doesn't >> really make sense in a port extender mode, because there >> is no switching going on. > > It can make sense. You can run ptp4l -i eno2, and ptp4l -i swp4, as > separate processes, and you can get high quality synchronization > between > /dev/ptp0 (enetc) and /dev/ptp1 (felix) over internal Ethernet (there > isn't any other mechanism in the SoC to keep them in sync if that is > needed for some use case like a boundary_clock_jbod between eno0 + eno1 > + swp0-swp3).
Ok, could make sense.
>> So uhm, 6 addresses are the maximum? > > No, the maximum is given by the number of ports, PFs and VFs. But > that's > a high number. It's the theoretical maximum. Then there's the practical > maximum, which is given by what kind of embedded system is built with > it. > I think that the more general-purpose the system is, the more garden > variety the networking use cases will be. I also think it would be very > absurd for everybody to reserve a number of MAC addresses equal to the > number of possibilities in which the LS1028A can expose IP termination > points, if they're likely to never need them.
I think we are on the same track here. I was ignoring any VFs for now. So I guess, what I'm still missing here is why enet#2 and enet#3 (or even swp4 and swp5) would need a non-random MAC address. Except from your example above. Considering the usecase where swp0..3 is one bridge with eno2 and eno3 being the CPU ports. Then I'd only need a unique MAC address for eno0, eno1 and swp0, correct?
-michael
>> This is the MAC address distribution for now on the >> sl28 boards: >> https://lore.kernel.org/linux-devicetree/20220901221857.2600340-19-michael@walle.cc/ >> >> Please tell me if I'm missing something here. > > My 2 cents, if you don't need anything special like in-SoC PTP, > 802.1CB, > virtualization, and don't habitually connect ports of the same ports to > each other or do some other sorts of redundant networking without > VLANs, > then there isn't too much wrong with one MAC address per RJ45 port, but > best discuss with those who are actually marketing the devices.
| |