Reinforcement Learning and Function Approximation
نویسندگان
چکیده
Relational reinforcement learning combines traditional reinforcement learning with a strong emphasis on a relational (rather than attribute-value) representation. Earlier work used relational reinforcement learning on a learning version of the classic Blocks World planning problem (a version where the learner does not know what the result of taking an action will be). “Structural” learning results have been obtained, such as learning in a mixed 3–5 block environment and being able to perform in a 3 or 10 block environment. Here we instead take a function approximation approach to reinforcement learning for this same problem. We obtain similar learning accuracies, with much better running times, allowing us to consider much larger problem sizes. For instance, we can train on a mix of 3–7 blocks and then perform well on worlds with 100–800 blocks—using less running time than the relational method required for 3–10 blocks.
منابع مشابه
Function Approximation in Hierarchical Relational Reinforcement Learning
Recently there have been a number of dif ferent approaches developed for hierarchi cal reinforcement learning in propositional setting We propose a hierarchical version of relational reinforcement learning HRRL We describe a value function approximation method inspired by logic programming which is suitable for HRRL
متن کاملA-Learning for Approximate Planning
Abstract We consider a new algorithm for reinforcement learning called A-learning. A-learning learns the advantages from a single training set. We compare A-learning with function approximation to Q-learning with function approximation and find that because A-learning approximates only the advantages it is less likely to exhibit bias due to the function approximation as compared to Q-learning.W...
متن کاملReinforcement Learning in a Noisy Environment: Light-seeking Robot
Despite many promising results from the use of reinforcement learning in simulated robot worlds, its use in real robot worlds is relatively rare. This paper addresses challenges related to real robot worlds and shows how reinforcement learning combined with linear function approximation can solve many of them. Experiments are performed using a light-seeking robot built with the Lego Mindstorms ...
متن کاملA Formal Framework for Reinforcement Learning with Function Approximation in Learning Classifier Systems
To fully understand the properties of Accuracy-based Learning Classifier Systems, we need a formal framework that captures all components of classifier systems, that is, function approximation, reinforcement learning, and classifier replacement, and permits the modelling of them separately and in their interaction. In this paper we extend our previous work on function approximation [22] to rein...
متن کاملA Function Approximation Method for Model-based High-Dimensional Inverse Reinforcement Learning
This works handles the inverse reinforcement learning problem in high-dimensional state spaces, which relies on an efficient solution of model-based high-dimensional reinforcement learning problems. To solve the computationally expensive reinforcement learning problems, we propose a function approximation method to ensure that the Bellman Optimality Equation always holds, and then estimate a fu...
متن کاملIssues in Using Function Approximation for Reinforcement Learning
Reinforcement learning techniques address the problem of learning to select actions in unknown, dynamic environments. It is widely acknowledged that to be of use in complex domains, reinforcement learning techniques must be combined with generalizing function approximation methods such as artificial neural networks. Little, however, is understood about the theoretical properties of such combina...
متن کامل