CNN Sensor Analytics With Hybrid-Float6 Quantization on Low-Power Embedded FPGAs

The use of artificial intelligence (AI) in sensor analytics is entering a new era based on the ubiquitous embedded connected devices. This transformation requires adoption design techniques that reconcile accurate results with sustainable system architectures. As such, improving efficiency AI hardware engines as well backward compatibility must be considered. In this paper, we present Hybrid-Float6 (HF6) quantization and its dedicated design. We propose an optimized multiply-accumulate (MAC) by reducing mantissa multiplication to multiplexor-adder operation. exploit intrinsic error tolerance neural networks further reduce approximation. To preserve model accuracy, quantization-aware training (QAT) method, which some cases improves accuracy. demonstrate concept 2D convolution layers. lightweight tensor processor (TP) implementing pipelined vector dot-product. For portability, 6-bit floating-point (FP) wrapped standard FP format, automatically extracted proposed hardware. hardware/software architecture compatible TensorFlow (TF) Lite. evaluate applicability our approach CNN-regression for anomaly localization structural health monitoring (SHM) application acoustic emission (AE). framework demonstrated XC7Z007S smallest Zynq-7000 SoC. implementation achieves peak power run-time acceleration 5.7 GFLOPS/s/W $48.3\times $ , respectively.