PREEMPT_RT isn't Just for Lasers: The Perfect Match for Hearing Aid Research! - Christopher Obbard & Daniel James, 64 Studio Ltd [Open Source Summit EU 2018]

Digital hearing aid do processing of audio to filter out instead of amplifying unwanted noise. This needs to be calibrated per user.

Hearing aids are important but underfunded.

They consist of a microphone, processing and an output. Analog hearing aids also had processing, but can’t be changed. Nowadays other electronics also have to be integrated with the hearing aid, so the TV or phone don’t need to go through an analog path. It’s not possible for small companies to work on hearing aids because of the high initial cost. Processing adds latency, if that becomes more than 50ms it becomes annoying.

OpenMHA (Master Hearing Aid) is a platform for research. C++ software that runs on Linux, Windows and MacOS. Realtime for latency. Includes filtering, beamforming (multiple microphones). The hardware is PC based - doesn’t make sense to optimise for DSP until the algorithm is proven. Audio interface is external. Laptop and other hardware put in a backpack with cables coming out to the hearing aid. Now they’re looking at SBCs, e.g. BBB. Lots less power so longer battery life, lots of hardware peripherals, and they have Linux support. There may be a problem with availability though - they need that so different labs can do long-term research and don’t have to swap out their systems midway. For the SBC, USB audio is less interesting because it adds overhead/latency. Instead use I2S directly. RPi has only 2 channels, but you need more for multiple microphones. Fortunately, home theatre surround sound introduced more channels, BBB has more channels. BBB is reasonably priced and you can produce your own device if needed. Single core 1GHz processing may be limiting though.

They designed a cape for the BBB so it is portable. The cape hase 3 stereo mic inputs and 2 stereo hearing aid outputs. It also has BT and Wifi connectivity. The PDK is Debian based, customized, with upstream kernel and U-Boot and debootstrapped. This gives researchers the ability to install extra packages when they want.

They wrote a small glue driver for the I2S devices (<500 lines). Tested with ALSA and jack. However, even without any processing you get 17ms latency and 70% CPU usage. PREEMPT_RT cuts that down to 4ms latency and 21% CPU. TI kernel has patches waiting to go upstream for the McASP that improve performace quite a lot.

To get realtime performance, first use cyclictest to verify there are no latency spikes. Then look for the interrupts used by your application and increase their priority. Also make sure the DMA IRQ and threads are covered. Finally increase the priority of your application above 50 so it goes over other interrupts.

The device launches a Wifi hotspot and a BLE peripheral.

In the future they’ll look at other CPUs (Allwinner?) with more processing power.