DeeperThings: Fully Distributed CNN Inference on Resource-Constrained Edge Devices

Abstract Performing inference of Convolutional Neural Networks (CNNs) on Internet Things (IoT) edge devices ensures both privacy input data and possible run time reductions when compared to a cloud solution. As most are memory- compute-constrained, they cannot store execute complex CNNs. Partitioning distributing layer information across multiple reduce the amount computation each device presents solution this problem. In article, we propose DeeperThings, an approach that supports full distribution CNN tasks by partitioning fully-connected as well feature- weight-intensive convolutional layers. Additionally, jointly optimize memory, communication demands. This is achieved using techniques combine feature weight with communication-aware fusion method, enabling holistic optimization For given number devices, schemes applied Integer Linear Programming (ILP) formulations minimize exchanged between times find entire model’s minimal memory footprint. Experimental results from real-world hardware setup running four different models confirm scheme able evenly balance footprint devices. six 100 Mbit/s connections integration additionally leads reduction demands up 28.8%. in speed-up task 1.52x without fusing.