We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exist...

Reinforcement learning often involves assuming Markov characteristics. However, the agent cannot always observe the environment completely, and in such cases, different states are observed as the same state. In this research, the authors develop a Delayed Reward-based Genetic Algorithm for POMDP (DRGA) as a means to solve a partially observable Markov decision problem (POMDP) which has such per...

We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exist...

For a given problem, the optimal Markov policy over a finite horizon is a conditional plan containing a potentially large number of branches. However, there are applications where it is desirable to strictly limit the number of decision points and branches in a plan. This raises the question of how one goes about finding optimal plans containing only a limited number of branches. In this paper,...

We present a logic inspired by partially observable Markov decision process (POMDP) theory for specifying agent domains where the agent’s actuators and sensors are noisy (causing uncertainty). The language features modalities for actions and predicates for observations. It includes a notion of probability to represent the uncertainties, and the expression of rewards and costs are also catered f...

—Prediction of spectrum sensing and access is one of the keys in cognitive radio (CR). It is necessary to know the channel state transition probabilities to predict the spectrum. By the use of the model of partially observable Markov decision process (POMDP), this paper addressed the spectrum sensing and access in cognitive radio and proposed an estimation algorithm of channel state transition...

The planning of clinical treatment actions for children with congenital heart disease requires a subtle trade-o between their immediate and long-term consequences, where most of these consequences cannot be predicted with certainty. It is described how this problem can be cast as a nite-horizon, partially observable Markov decision process. The complexity of the resulting model is reduced by us...

For over a decade, the Hidden Markov Model (HMM) has been the primary tool used for acoustic modeling in the field of speech recognition. In this paper we examine a more general approach using a Partially Observable Markov Decision Process (POMDP) to mo del the base phonetic unit. We introduce the concept of multiple phonetic context classes, one for each of the infinite possible contexts a pho...

RÉSUMÉ. Cet article se focalise sur des approches formelles pour la construction de systèmes multi-agents. Ce travail a cherché à proposer des apprentissages décentralisés pour construire les comportements d’agents sociaux. Cet article propose un formalisme original, l’interacDEC-POMDP inspiré des modèles markoviens au sein duquel les agents peuvent interagir directement et localement entre eux...

Markov models have been a keystone in Artificial Intelligence for many decades. However, they remain unsatisfactory when the environment modelled is partially observable. There are pathological examples where no history of fixed length is sufficient for accurate prediction or decision making. On the other hand, working with a hidden state (like in Hidden Markov Models or Partially Observable Ma...