Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning

Federated learning (FL), as an emerging edge artificial intelligence paradigm, enables many devices to collaboratively train a global model without sharing their private data. To enhance the training efficiency of FL, various algorithms have been proposed, ranging from first-order second-order methods. However, these cannot be applied in scenarios where gradient information is not available, e.g., federated black-box attack and hyperparameter tuning. address this issue, paper we propose derivative-free zeroth-order optimization (FedZO) algorithm featured by performing multiple local updates based on stochastic estimators each communication round enabling partial device participation. Under non-convex settings, derive convergence performance FedZO non-independent identically distributed data characterize impact numbers iterates participating convergence. enable communication-efficient over wireless networks, further over-the-air computation (AirComp) assisted algorithm. With appropriate transceiver design, show that AirComp-assisted can still preserved under certain signal-to-noise ratio conditions. Simulation results demonstrate effectiveness validate theoretical observations.