Action-Planning in Anticipatory Classifier Systems

Learning consists in the acquisition of knowledge. In Reinforcement Learning this is knowledge about how to reach a maximum of environmental reward. We are interested in the acquisition of knowledge that consists in having expectations of behavioral consequences. Behavioral consequences depend on the current situation, so it is necessary to learn in which situation S which behavior/reaction R leads to which behavioral consequences C. In other words, SRC units are learned. It was the psychologist Edward Tolman (1932) who firstly stated that animals learn SRC units. Seward (1949) proved that rats are able to learn in the absence of reward and confirmed Tolman’s assumption. Learning in the absence of reinforcement is called ‘latent learning’ and cannot be explained by usual reinforcement learning techniques. In the field of Learning Classifier Systems (LCS) latent learning is realized in Riolo’s CFSC2 (Riolo, 1991) and Stolzmann’s ACS (Stolzmann, 1997, 1998). Both authors prove the performance of their learning algorithms with a simulation of Seward’s experiment. This experiment consists in a learning phase without any reward followed by a test phase where the rats have to use the knowledge they acquired during the learning phase to do action-planning. Action-planning and latent learning occur at different times. This paper focuses on the integration of actionplanning and latent learning in ACS. Using an example about learning of the hand-eye coordination of a robot arm in conjunction with a camera it will be shown, that a combination of action-planning and latent learning in ACS induces a substantial reduction of the number of trials which are required to learn a complete model of a prototypically environment.