Intrinsic Motivation For Reinforcement Learning Systems

Motivation is a key factor in human learning. We learn best when we are highly motivated to learn. Psychologists distinguish between extrinsically-motivated behavior, which is behavior undertaken to achieve some externally supplied reward, such as a prize, a high grade, or a high-paying job, and intrinsically-motivated behavior, which is behavior done for its own sake. Is there an analogous distinction for machine learning systems? Can we say of a machine learning system that it is motivated to learn, and if so, can it be meaningful to distinguish between extrinsic and intrinsic motivation? In this paper, we argue that the answer to both questions is “yes,” and we describe some computational experiments that explore these ideas within the framework of computational reinforcement learning. In particular, we describe an approach by which artificial agents can learn hierarchies of reusable skills through a computational analog of intrinsic motivation.