Lazy Aggregation for Heterogeneous Federated Learning

Federated learning (FL) is a distributed neural network training paradigm with privacy protection. With the premise of ensuring that local data isn’t leaked, multi-device cooperation trains model and improves its normalization. Unlike centralized training, FL susceptible to heterogeneous data, biased gradient estimations hinder convergence global model, traditional sampling techniques cannot apply due constraints. Therefore, this paper proposes novel framework, federated lazy aggregation (FedLA), which reduces frequency obtain high-quality gradients improve robustness in non-IID. To judge aggregating timings, change rate models’ weight divergence (WDR) introduced FL. Furthermore, collected also facilitate walking out saddle point without extra communications. The cross-device momentum (CDM) mechanism could significantly upper limit performance We evaluate several popular algorithms, including FedLA (FedLAM). results show FedLAM achieves best most scenarios can be improved IID scenarios.