Decentralized Asynchronous Non-convex Stochastic Optimization on Directed Graphs

We consider a decentralized stochastic optimization problem over network of agents, modeled as directed graph: Agents aim to asynchronously minimize the average their individual losses (possibly non-convex), each one having access only noisy estimate gradient its own function. propose an asynchronous distributed algorithm for such class problems. The combines gradients with tracking in push-sum framework and obtains sublinear convergence rate, matching rate centralized descent applied nonconvex minimization. Our experiments on non-convex image classification task using convolutional neural validate our proposed across different number nodes graph connectivity percentages.