Comparison of CMACs and Radial Basis Functions for Local Function Approximators in Reinforcement Learning
نویسندگان
چکیده
CMACs and Radial Basis Functions are often used in reinforcement learning to learn value function approximations having local generalization properties. We examine the similarities and differences between CMACs, RBFs and normalized RBFs and compare the performance of Q-learning with each representation applied to the mountain car problem. We discuss ongoing research efforts to exploit the flexibility of adaptive units to better represent the local characteristics of the state space.
منابع مشابه
Novel Radial Basis Function Neural Networks based on Probabilistic Evolutionary and Gaussian Mixture Model for Satellites Optimum Selection
In this study, two novel learning algorithms have been applied on Radial Basis Function Neural Network (RBFNN) to approximate the functions with high non-linear order. The Probabilistic Evolutionary (PE) and Gaussian Mixture Model (GMM) techniques are proposed to significantly minimize the error functions. The main idea is concerning the various strategies to optimize the procedure of Gradient ...
متن کاملConvergence and Divergence in Standard and Averaging Reinforcement Learning
Although tabular reinforcement learning (RL) methods have been proved to converge to an optimal policy, the combination of particular conventional reinforcement learning techniques with function approximators can lead to divergence. In this paper we show why off-policy RL methods combined with linear function approximators can lead to divergence. Furthermore, we analyze two different types of u...
متن کاملThe use of radial basis functions by variable shape parameter for solving partial differential equations
In this paper, some meshless methods based on the local Newton basis functions are used to solve some time dependent partial differential equations. For stability reasons, used variably scaled radial kernels for constructing Newton basis functions. In continuation, with considering presented basis functions as trial functions, approximated solution functions in the event of spatial variable wit...
متن کاملUsing Temporal Neighborhoods to Adapt Function Approximators in Reinforcement Learning
To avoid the curse of dimensionality, function approximators are used in reinforcement learning to learn value functions for individual states. In order to make better use of computational resources (basis functions) many researchers are investigating ways to adapt the basis functions during the learning process so that they better t the value-function landscape. Here we introduce temporal neig...
متن کاملTHE COMPARISON OF EFFICIENT RADIAL BASIS FUNCTIONS COLLOCATION METHODS FOR NUMERICAL SOLUTION OF THE PARABOLIC PDE’S
In this paper, we apply the compare the collocation methods of meshfree RBF over differential equation containing partial derivation of one dimension time dependent with a compound boundary nonlocal condition.
متن کامل