Communication-Efficient Federated Learning: A Second Order Newton-Type Method With Analog Over-the-Air Aggregation

Owing to their fast convergence, second-order Newton-type learning methods have recently received attention in the federated (FL) setting. However, current solutions are based on communicating Hessian matrices from devices parameter server, at every iteration, incurring a large number of communication rounds; calling for novel communication-efficient methods. In this article, we propose method that, similarly its first-order counterpart, requires device share only model-sized vector each iteration while hiding gradient and information. doing so, proposed approach is significantly more privacy-preserving. Furthermore, by leveraging over-the-air aggregation principle, our inherits privacy guarantees obtains much higher efficiency gains. particular, formulate problem inverse Hessian-gradient product as quadratic that solved distributed way. The framework alternates between updating using few alternating direction multipliers (ADMM) steps, global model Newton’s method. Numerical results show scalable under noisy channels different scenarios across multiple datasets.