Policy Learning in Imperfect-information Infinite Dynamic Games

Dynamic games (DGs) play an important role in distributed decision making and control in complex environments. Finding optimal/approximate solutions for these games in the imperfect-information setting is currently a challenge for mathematicians and computer scientists, especially when state and action spaces are infinite. This paper presents an approach to this problem by using multi-agent reinforcement learning techniques. We hence propose a method of policy-space search based on variable resolution state/action abstraction. Stated precisely, a non-uniform partitioning of the state-action space of a perfect-information game version is used to parameterize stochastic policies to learn. We study in detail the application of this method to two-player zero-sum pursuit-evasion games (PEGs), including an antimissile game. Experimental evaluations on this realistic PEG demonstrate good performance of our learning method and show that it gives better solutions than those given by traditional analytical methods.