Symbolic Regression Methods for Reinforcement Learning

Reinforcement learning algorithms can be used to optimally solve dynamic decision-making and control problems. With continuous-valued state input variables, reinforcement must rely on function approximators represent the value policy mappings. Commonly numerical approximators, such as neural networks or basis expansions, have two main drawbacks: they are black-box models offering no insight in mappings learned, require significant trial error tuning of their meta-parameters. In this paper, we propose a new approach constructing smooth functions form analytic expressions by means symbolic regression. We introduce three off-line methods for finding based transition model: iteration, direct solution Bellman equation. The illustrated four nonlinear problems: velocity under friction, one-link two-link pendulum swing-up, magnetic manipulation. results show that not only yield well-performing policies, but also compact, mathematically tractable easy plug into other algorithms. This makes them potentially suitable further analysis closed-loop system. A comparison with an alternative using shows our method outperforms network-based one.