Porting any Service to any Linux-based OS using OpenWrt Build System – Marko Ratkaj [OpenWRT Summit 2018]

With OpenWRT, it is possible to put a service on top of an existing OS (so not reflashing the device with OpenWRT, just adding an additional service).

When you create a service, replacing the original firmware is usually not an option. So the service should be OS-independent, and deployable on a wide range of devices. Typically, you get a device, but without SDK, hardware specificationk information of what is running on it, documentation, … Often you don’t have OpenWRT support, or OpenWRT is not good enough for the customer. Rootfs is readonly. You do get access to the shell though.

So, you have to keep the running kernel (which may be old and lack some features). You can’t reuse the libraries already on the filesystem, because you don’t know which version it is or with which toolchain it was built.

The first step is to understand the system. At least, know the arch and ABI. Check if there is enough free storage and RAM. Check the kernel configuration, work around missing features. Check if the tools needed for any scripts you run are available. You can derive the libc by looking at /lib/ld*.so*. You can know the ABI by looking at any executable with readelf.

To build with OpenWRT, make a new, simplified target that doesn’t generate an image and has no target-specific patches.

Compiling statically simplifies deployment. If you do compile dynamically, you probably have to add non-standard locations for shared libraries. patchelf is needed to set the correct runtime paths. Alternatively, export LD_LIBRARY_PATH.

The service has to be integrated with the init system. That requires some hacking. Usually there is some writable partition, and with some luck, there is something on that partition that is executed or sourced. That way you can inject the start of your service. In addition to starting the service, extra manipulations may be needed, e.g. updating firewall rules. This is tricky, with a risk of breaking something from the original OS.