testing NICs on the same machine
I wanted to try load testing (specifically: very approximately see the
power draw under load) of a new machine. Just setting up the NICs and
then trying to do iperf3 across them did nothing - it wasn't even
shortcutting in the NIC, the kernel didn't even seem to bother to
sending the data over the PCIe bus.
After some digging, I found this, which showed how to put each NIC in a unique network namespace, which forces the kernel to actually send data to the NIC and the NIC to actually send the data out of the lasers.
Minor consequence of this - all your monitoring will be wrong, since the devices are in namespaces and so are their stats, so you need to query them in the namespace as well:
# ip netns exec ns_1b ip -s -s link show dev eno8