Evolutionary Planning in Latent Space

Planning is a powerful approach to reinforcement learning with several desirable properties such as sampling efficiency. However, it requires world model, which not readily available in many real-life problems. In this paper, we propose learn model that enables Evolutionary Latent Space (EPLS). We use Variational Auto Encoder (VAE) compressed latent representation of individual observations and extend Mixture Density Recurrent Neural Network (MDRNN) stochastic, multi-modal forward the used for planning. Random Mutation Hill Climbing (RMHC) algorithm find sequence actions maximize expected reward learned world. demonstrate how build by bootstrapping rollouts from random policy iteratively refining an increasingly accurate planning using model. After few iterations, our agents exceed standard model-free approaches, demonstrates viability approach. Code reproduce experiments at https://github.com/two2tee/WorldModelPlanning videos https://youtu.be/3M39QgeF27U.