Multi-Agent Deep Reinforcement Learning

This work introduces a novel approach for solving reinforcement learning problems in multi-agent settings. We propose a state reformulation of multi-agent problems in R that allows the system state to be represented in an image-like fashion. We then apply deep reinforcement learning techniques with a convolution neural network as the Q-value function approximator to learn distributed multi-agent policies. Our approach extends the traditional deep reinforcement learning algorithm by making use of stochastic policies during execution time and stationary policies for homogenous agents during training. We also use a residual neural network as the Q-value function approximator. The approach is shown to generalize multi-agent policies to new environments, and across varying numbers of agents. We also demonstrate how transfer learning can be applied to learning policies for large groups of agents in order to decrease convergence time. We conclude with other potential applications and extensions for future work.