Blending Autonomous Exploration and Apprenticeship Learning
نویسندگان
چکیده
We present theoretical and empirical results for a framework that combines the benefits of apprenticeship and autonomous reinforcement learning. Our approach modifies an existing apprenticeship learning framework that relies on teacher demonstrations and does not necessarily explore the environment. The first change is replacing previously used Mistake Bound model learners with a recently proposed framework that melds the KWIK and Mistake Bound supervised learning protocols. The second change is introducing a communication of expected utility from the student to the teacher. The resulting system only uses teacher traces when the agent needs to learn concepts it cannot efficiently learn on its own.
منابع مشابه
Notes in Artificial Intelligence 7523
Robots are typically far less capable in autonomous mode than in tele-operated mode. The few exceptions tend to stem from long days (and more oftenweeks, or even years) of expert engineering for a specific robot and its operatingenvironment. Current control methodology is quite slow and labor intensive. I be-lieve advances in machine learning have the potential to revolutionize ...
متن کاملEfficient Apprenticeship Learning with Smart Humans
This report describes a generalized apprenticeship learning protocol for reinforcement-learning agents with access to a teacher. The teacher interacts with the agent by providing policy traces (transition and reward observations). We characterize sufficient conditions of the underlying models for efficient apprenticeship learning and link this criteria to two established learnability classes (K...
متن کاملAutonomous Helicopter Aerobatics through Apprenticeship Learning
Autonomous helicopter flight is widely regarded to be a highly challenging control problem. Despite this fact, human experts can reliably fly helicopters through a wide range of maneuvers, including aerobatic maneuvers at the edge of the helicopter’s capabilities. We present apprenticeship learning algorithms, which leverage expert demonstrations to efficiently learn good controllers for tasks ...
متن کاملBuilding Adaptive Autonomous Agents for Adversarial Domains
This paper presents a methodology, called CAPTAIN, to build adaptive agents in an integrated framework that facilitates both building agents through knowledge elicitation and interactive apprenticeship learning from subject matter experts, and making these agents adapt and improve during their normal use through autonomous learning. Such an automated adaptive agent consists of an adversarial pl...
متن کاملGoal-Directed Online Learning of Predictive Models
We present an algorithmic approach for integrated learning and planning in predictive representations. The approach extends earlier work on predictive state representations to the case of online exploration, by allowing exploration of the domain to proceed in a goal-directed fashion and thus be more efficient. Our algorithm interleaves online learning of the models, with estimation of the value...
متن کامل