Accelerating RNN-Based Speech Enhancement on a Multi-core MCU with Mixed FP16-INT8 Post-training Quantization

This paper presents an optimized methodology to design and deploy Speech Enhancement (SE) algorithms based on Recurrent Neural Networks (RNNs) a state-of-the-art MicroController Unit (MCU), with 1+8 general-purpose RISC-V cores. To achieve low-latency execution, we propose software pipeline interleaving parallel computation of LSTM or GRU recurrent blocks, featuring vectorized 8-bit integer (INT8) 16-bit floating-point (FP16) compute units, manually-managed memory transfers model parameters. ensure minimal accuracy degradation respect the full-precision models, novel FP16-INT8 Mixed-Precision Post-Training Quantization (PTQ) scheme that compresses layers while bit precision remaining is kept FP16. Experiments are conducted multiple SE models trained Valentini dataset, up 1.24M Thanks proposed approaches, speed-up by 4 $$\times $$ lossless FP16 baselines. Differently from uniform quantization degrades PESQ score 0.3 average, PTQ leads low-degradation only 0.06, achieving 1.4–1.7 saving. this compression, cut power cost external fitting large limited on-chip non-volatile gain MCU saving 2.5 reducing supply voltage 0.8 V 0.65 still matching real-time constraints. Our results >10 more energy efficient than solutions deployed single-core MCUs make use smaller quantization-aware training.