Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning

This article deals with distributed policy optimization in reinforcement learning, which involves a central controller and group of learners. In particular, two typical settings encountered several applications are considered: multiagent learning (RL) xmlns:xlink="http://www.w3.org/1999/xlink">parallel RL , where frequent information exchanges between the learners required. For many practical systems, however, overhead caused by these communication is considerable, becomes bottleneck overall performance. To address this challenge, novel gradient approach developed for solving RL. The adaptively skips during iterations, can reduce without degrading learning It established analytically that: i) algorithm has convergence rate identical to that plain-vanilla gradient; while ii) if heterogeneous terms their reward functions, number rounds needed achieve desirable accuracy markedly reduced. Numerical experiments corroborate reduction attained compared alternatives.