Solving Stochastic Shortest-Path Problems with RTDP
نویسندگان
چکیده
We present a modification of the Real-Time Dynamic Programming (rtdp) algorithm that makes it a genuine off-line algorithm for solving Stochastic Shortest-Path problems. Also, a new domainindependent and admissible heuristic is presented for Stochastic Shortest-Path problems. The new algorithm and heuristic are compared with Value Iteration over benchmark problems with large state spaces. The results show that the modified rtdp algorithm can beat standard Value Iteration by several orders of magnitude in problems with large state space.
منابع مشابه
Robust Planning with (L)RTDP
Stochastic Shortest Path problems (SSPs), a subclass of Markov Decision Problems (MDPs), can be efficiently dealt with using Real-Time Dynamic Programming (RTDP). Yet, MDP models are often uncertain (obtained through statistics or guessing). The usual approach is robust planning: searching for the best policy under the worst model. This paper shows how RTDP can be made robust in the common case...
متن کاملPlanning with Robust (L)RTDP
Stochastic Shortest Path problems (SSPs), a subclass of Markov Decision Problems (MDPs), can be efficiently dealt with using Real-Time Dynamic Programming (RTDP). Yet, MDP models are often uncertain (obtained through statistics or guessing). The usual approach is robust planning: searching for the best policy under the worst model. This paper shows how RTDP can be made robust in the common case...
متن کاملRobust LRTDP: Reachability Analysis
Stochastic Shortest Path problems (SSPs) can be efficiently dealt with by the Real-Time Dynamic Programming algorithm (RTDP). Yet, RTDP requires that a goal state is always reachable. This paper presents an algorithm checking for goal reachability, especially in the complex case of an uncertain SSP where only a possible interval is known for each transition probability. This gives an analysis m...
متن کاملFast Reachability Analysis for Uncertain SSPs
Stochastic Shortest Path problems (SSPs) can be efficiently dealt with by the Real-Time Dynamic Programming algorithm (RTDP). Yet, RTDP requires that a goal state is always reachable, what can be checked easily for a certain SSP, and with a more complex algorithm for an uncertain SSP, i.e. where only a possible interval is known for each transition probability. This paper makes a simplified des...
متن کاملPlanification robuste avec (L)RTDP
Résumé : Les problèmes de chemin le plus court stochastique (SSP : Stochastic Shortest Path problem), un sous-ensemble des problèmes de décision markoviens (MDPs), peuvent être efficacement traı̂tés en utilisant l’algorithme Real-Time Dynamic Programming (RTDP). Toutefois, les modèles des MDPs sont souvent incertains (obtenus à l’aide de statistiques ou par intuition). Une approche usuelle est a...
متن کامل