Neurocontrol is a crucial area of fundamental research within the neural network eld. Adaptive Heuristic Critic learning is a key algorithm for real time adaptation in neurocontrollers. In this paper we present how an unsupervised neural network model with adaptable structure can be used to speed-up Adaptive Heuristic Critic learning, its FPGA design , and how it adapts the neurocontroller to t...

We present a novel class of actor-critic algorithms for actors consisting of sets of interacting modules. We present, analyze theoretically, and empirically evaluate an update rule for each module, which requires only local information: the module’s input, output, and the TD error broadcast by a critic. Such updates are necessary when computation of compatible features becomes prohibitively dif...

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic reinforcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods...

Actor-Critic architectures of reinforcement learning were found to show a strong resemblance with known anatomy and function of a part of the vertebrate's brain: the basal ganglia. Based on this analogy, a large number of Actor-Critic models were simulated to reproduce behaviours of rats performing laboratory tasks. However, most of these models were tested in different tasks and it is often di...

In this article, we propose an architecture of a bio-inspired controller that addresses the problem of learning different locomotion gaits for different robot morphologies. The modeling objective is split into two: baseline motion modeling and dynamics adaptation. Baseline motion modeling aims to achieve fundamental functions of a certain type of locomotion and dynamics adaptation provides a "r...

Adaptive control proves to be an important eld of investigation as several control problems change its behaviour in time or there is no analytical model available at all. We present a new approach in this context: A fuzzy controller is adapted according to an evaluation function by a critic that is trained using reinforcement learning. In this paper we address the modeling and adaptation of the...

This paper proposes a control algorithm based on adaptive dynamic programming to solve the infinite-horizon optimal control problem for known deterministic nonlinear systems with saturating actuators and non-quadratic cost functionals. The algorithm is based on an actor/critic framework where a critic neural network is used to learn the optimal cost and an actor neural network is used to learn ...

Critic-proofing is a modified heuristic evaluation technique, specifically designed to provide a fine-grained, prioritized list of heuristic violations. The critic-proofing technique weights the severity of a problem based on the frequency that similar problems are found in similar games. The severity ratings are calculated using data collected from game reviews, and the severity assigned to a ...

This article argues that Russell McCutcheon’s notion of the religion scholar as a critic is crucial for envisioning a distinct relevance to the academic study of religion in multidisciplinary conversations concerning questions of religion and conflict. However, McCutcheon’s critical approach is insufficient for thinking about transforming conflicts and underlying structures of injustice. To act...

Adaptive critic (AC) based controllers are typically discrete and/or yield a uniformly ultimately bounded stability result because of the presence of disturbances and unknown approximation errors. A continuous-time AC controller is developed that yields asymptotic tracking of a class of uncertain nonlinear systems with bounded disturbances. The proposed AC-based controller consists of two neura...