State of fast path networking in Linux – Damir Samardzić [OpenWRT Summit 2018]

To implement a fast path in Linux, there are some userspace fastpath mechansisms but also in-kernel. The Linux network stack is complex and therefore packet processing takes a lot of CPU time - 10Gb and faster networks become difficult. Offload (e.g. checksum or segmentation) is not enough. So other approaches: filter and drop as early as possible; bypass network stack entirely.

Completely bypassing the stack usually requires modifying the network driver, and of course it means that everything has to be done by userspace. DPDK (Data Plane Development Kit) is the best known project in this area. It supports a lot of different network cards, with drivers running in userspace through UIO. On top of that you have e.g. Open vSwitch, which does a software defined network for virtual machines.

netmap is a different approach to kernel bypass. It’s a set of modules that gives userspace direct access to NIC buffer rings.

A few other technologies were mentioned. All of them require some kernel modules.

Another approach is to stay in the kernel, but to process as much data as possible as early as possible - before SKB is allocated, can be done on NIC itself. For packet inspection and filtering, mostly. eBPF can now also be hooked into XDP (eXpress Data Path), which allows eBPF filtering before anything else is done with the packet. On some NICs, the eBPF program can be offloaded onto it. The filter returns PASS, DROP, TX (retransmit on same interface, possibly after modification), REDIRECT (output on a different interface), ABORT (internal error in the program). AF_XDP sockets allow to pass buffers in zerocopy to userspace - similar to DPDK.

None of the bypass solutions except netmap are supported in OpenWRT. XDP and eBPF are in mainline, but driver support is limited. E.g. only Intel ixgbe supports REDIRECT as of 4.14.

Measurements: drop functionality can be sped up by factor of 1.5 to 5.5 compared with iptables.