Hierarchical Reinforcement Learning on the Virtual Battlefield

This paper investigates the potential of flat and hierarchical reinforcement learning (HRL) for solving problems within strategy games. A HRL method, Max-Q, is applied to a unit transportation task modelled within a simplified, discrete real-time strategy game engine, and its performance compared to that of flat Q-learning. It is shown that reinforcement learning approaches, and especially hierarchical reinforcement learning approaches, to strategy game AI are effective for learning such tasks. They are also efficient, in particular when such tasks decompose naturally into subtasks.