We discuss the problem of comparing the behavioural equivalence of partially observable systems with observations. We examine different types of equivalence relations on states, and show that branching equivalence relations are stronger than linear ones. Finally, we discuss how this hierarchy can be used in duality theory.

We formulate an automatic strategy acquisition problem for the multi-agent card game “Hearts” as a reinforcement learning (RL) problem. Since there are often a lot of unobservable cards in this game, RL is approximately dealt with in the framework of a partially observable Markov decision process (POMDP). This article presents a POMDP-RL method based on estimation of unobservable state variable...

We present a major improvement to the incremental pruning algorithm for solving partially observable Markov decision processes. Our technique targets the cross-sum step of the dynamic programming (DP) update, a key source of complexity in POMDP algorithms. Instead of reasoning about the whole belief space when pruning the cross-sums, our algorithm divides the belief space into smaller regions a...

Despite the intractability of generic optimal partially observable Markov decision process planning, there exist important problems that have highly structured models. Previous researchers have used this insight to construct more efficient algorithms for factored domains, and for domains with topological structure in the flat state dynamics model. In our work, motivated by findings from the edu...

A novel algorithm to speed up online planning in partially observable Markov decision processes (POMDPs) is introduced. I propose a method for compressing nodes in beliefdecision-trees while planning occurs. Whereas belief-decision-trees branch on actions and observations, with my method, they branch only on actions. This is achieved by unifying the branches required due to the nondeterminism o...

The focus of this paper is the framework of partially observable Markov decision processes (POMDPs) and its role in modeling and solving complex dynamic decision problems in stochastic and partially observable medical domains. The paper summarizes some of the basic features of the POMDP framework and explores its potential in solving the problem of the management of the patient with chronic isc...

Fiber type specificity of pyruvate dehydrogenase (PDH) phosphatase (PDP) was determined in fed (CON) and 48-h food-deprived (FD) rats. PDP activity and isoform protein content were determined in soleus (slow-twitch oxidative), red gastrocnemius (RG; fast-twitch oxidative glycolytic), and white gastrocnemius (WG; fast-twitch glycolytic) muscles. When normalized for mitochondrial volume, there wa...

BACKGROUND/AIMS With the increased efficacy of current therapy for wet age-related macular degeneration (AMD), better ways to detect wet AMD are needed. This study was designed to test the ability of three-dimensional contrast threshold Amsler grid (3D-CTAG) testing to distinguish wet AMD from dry AMD. METHODS Conventional paper Amsler grid and 3D-CTAG tests were performed in 90 eyes: 63 with...

A promising application area for proactive assistant agents is automated tutoring and training. Intelligent tutoring systems (ITSs) assist tutors and tutees by automating diagnosis and adaptive tutoring. These tasks are well modeled by a partially observable Markov decision process (POMDP) since it accounts for the uncertainty inherent in diagnosis. However, an important aspect of making POMDP ...

Although a partially observable Markov decision process (POMDP) provides an appealing model for problems of planning under uncertainty, exact algorithms for POMDPs are intractable. This motivates work on approximation algorithms, and grid-based approximation is a widely-used approach. We describe a novel approach to grid-based approximation that uses a variable-resolution regular grid, and show...