Solving Markov Decision Processes by Graphical Modelling: the DT-Planner Program
نویسندگان
چکیده
In this paper we will describe a software package , called DT-Planner, able to represent and solve nite-state Markov Decision Processes, by exploiting a novel graphical formalism, called Innuence View. An Innuence View is a directed acyclic graph that depicts the probabilistic relationships between the problems state variables in a generic time transition ; additional variables, called event variables , may be added, in order to describe the conditional independencies between state variables. By using the speciied conditional independence structure, an Innuence View may allow a parsimonious speciication of a Markov Decision Process. DT-Planner lets the user specify and manage models through a user-friendly graphical interface, and implements eecient for policy determination algorithms .
منابع مشابه
DT-Planner: an environment for managing dynamic decision problems.
The problem of formulating plans under uncertainty and coping with dynamic decision problems is a major task of both artificial intelligence and control theory applications in medicine. In this paper we will describe a software package, called DT-Planner, designed to represent and solve dynamic decision problems that can be modelled as Markov decision processes, by exploiting a novel graphical ...
متن کاملInference strategies for solving semi-Markov decision processes
Semi-Markov decision processes are used to formulate many control problems and also play a key role in hierarchical reinforcement learning. In this chapter we show how to translate the decision making problem into a form that can instead be solved by inference and learning techniques. In particular, we will establish a formal connection between planning in semiMarkov decision processes and infe...
متن کاملPii: S1386-5056(00)00099-x
The aim of this paper is to point out the difference between static and dynamic approaches to choosing the optimal time for intervention. The paper demonstrates that classical approaches, such as decision trees and influence diagrams, hardly cope with dynamic problems: they cannot simulate all the real-world strategies and consequently can only calculate suboptimal solutions. A dynamic formalis...
متن کاملSymbolic Heuristic Search for Probabilistic Planning
We describe a planner that participates in the Probabilistic Planning Track of the 2004 International Planning Competition. Our planner integrates two approaches to solving Markov decision processes with large state spaces. State abstraction is used to avoid evaluating states individually. Forward search from a start state, guided by an admissible heuristic, is used to avoid evaluating all states.
متن کاملAbstraction and Refinement for Solving Continuous Markov Decision Processes
ion and Refinement for Solving Continuous Markov Decision Processes Alberto Reyesand Pablo Ibargüengoytia Inst. de Inv. Eléctricas Av. Reforma 113, Palmira, Cuernavaca, Mor., México {areyes,pibar}@iie.org,mx L. Enrique Sucar and Eduardo Morales INAOE Luis Enrique Erro 1, Sta. Ma. Tonantzintla, Pue., México {esucar,emorales}@inaoep.mx
متن کامل