Transferring State Abstractions Between MDPs
نویسندگان
چکیده
Decision makers that employ state abstraction (or state aggregation) usually find solutions faster by treating groups of states as indistinguishable by ignoring irrelevant state information. Identifying irrelevant information is essential for the field of knowledge transfer where learning takes place in a general setting for multiple domains. We provide a general treatment and algorithm for transferring state abstractions in MDPs.
منابع مشابه
Counterexample-guided Planning
Planning in adversarial and uncertain environments can be modeled as the problem of devising strategies in stochastic perfect information games. These games are generalizations of Markov decision processes (MDPs): there are two (adversarial) players, and a source of randomness. The main practical obstacle to computing winning strategies in such games is the size of the state space. In practice ...
متن کاملDiscovering hidden structure in factored MDPs
Markov Decision Processes (MDPs) describe a wide variety of planning scenarios ranging from military operations planning to controlling a Mars rover. However, today’s solution techniques scale poorly, limiting MDPs’ practical applicability. In this work, we propose algorithms that automatically discover and exploit the hidden structure of factored MDPs. Doing so helps solve MDPs faster and with...
متن کاملPOMDP Homomorphisms
The problem of finding hidden state in a POMDP and the problem of finding abstractions for MDPs are closely related. In this paper, we analyze the connection between existing Predictive State Representation methods [3] and homomorphic reductions of Markov Processes [5, 6]. We formally define a POMDP homomorphism, then extend PSR reduction methods to find POMDP homomorphisms when the original PO...
متن کاملAbstraction and Approximate Decision-Theoretic Planning
ion and Approximate Decision Theoretic Planning Richard Dearden and Craig Boutiliery Department of Computer Science University of British Columbia Vancouver, British Columbia CANADA, V6T 1Z4 email: dearden,[email protected] Abstract Markov decision processes (MDPs) have recently been proposed as useful conceptual models for understanding decision-theoretic planning. However, the utility of the as...
متن کاملPolicy-contingent state abstraction for hierarchical MDPs
Hierarchically structured planning problems often provide great opportunities for state abstraction: high-level planners can ignore fine details, while low-level planners can focus only on specific tasks. Most previous hierarchical MDP algorithms rely on hand-designed, knowledgeintensive state abstractions. We propose instead an automatic, lazy algorithm which plans from the bottom of the hiera...
متن کامل