A Q-learning Based Evolutionary Algorithm for Sequential Decision Making Problems

Both Evolutionary Dynamic Optimization (EDO) methods and Reinforcement Learning (RL) methods tackle forms of Sequential Decision Making Problems (SDMPs), yet with different key assumptions. In this paper, we combine the strength of both EDO methods and RL methods to develop a new algorithm for SDMPs. Assuming that the environmental state is observable and that a computational model of the reward function is available, the key idea in our algorithm is to employ an evolutionary algorithm to search on the reward function at each time step, the outcome of which is exploited to speed up convergence to optimal policies in RL methods. Some preliminary experimental studies demonstrate that our algorithm is a promising approach for SDMPs.