Partially Observable Markov Decision Processes (POMDPs) are rich environments often used in machine learning. But the issue of information and causal structures in POMDPs has been relatively little studied. This paper presents the concepts of equivalent and counterfactually equivalent POMDPs, where agents cannot distinguish which environment they are in though any observations and actions. It s...

This paper introduces the even odd POMDP an approxi mation to POMDPs Partially Observable Markov Decision Problems in which the world is assumed to be fully observable every other time step This approximation works well for problems with a delayed need to observe The even odd POMDP can be converted into an equivalent MDP the MDP whose value function V MDP can be combined online with a step loo...

Dialogue Management (DM) is a key issue in Spoken Dialogue System (SDS). Most of the existing studies on DM use Dialogue Act (DA) to represent semantic information of sentence, which might not represent the nuanced meaning sometimes. In this paper, we model DM based on sentence clusters which have more powerful semantic representation ability than DAs. Firstly, sentences are clustered not only ...

Partially observable Markov decision processes (POMDPs) are widely used in probabilistic planning problems in which an agent interacts with an environment using noisy and imprecise sensors. We study a setting in which the sensors are only partially defined and the goal is to synthesize"weakest"additional sensors, such that in the resulting POMDP, there is a small-memory policy for the agent tha...

Session search is an information retrieval task that involves a sequence of queries for a complex information need. It is characterized by rich user-system interactions and temporal dependency between queries and between consecutive user behaviors. Recent efforts have been made in modeling session search using the Partially Observable Markov Decision Process (POMDP). To best utilize the POMDP m...

The goal of this paper is to present a first step toward integrating Incremental Speech Recognition (ISR) and Partially-Observable Markov Decision Process (POMDP) based dialogue systems. The former provides support for advanced turn-taking behavior while the other increases the semantic accuracy of speech recognition results. We present an Incremental Interaction Manager that supports the use o...

This paper presents the first demonstration of a statistical spoken dialogue system that uses automatic belief compression to reason over complex user goal sets. Reasoning over the power set of possible user goals allows complex sets of user goals to be represented, which leads to more natural dialogues. The use of the power set results in a massive expansion in the number of belief states main...

In this work we propose a decision-theoretic approach to Intelligent Tutoring Systems (ITSs) that seeks to alleviate the need for extensive development and hand-tuning in the design of such systems. Given a set of available learning materials, our approach enables the ITS to track the students’ difficulties and provide the right material at the right time. We model the learning process as a Par...

In multi-agent systems, anticipating the behavior of other agents constitutes a difficult problem. In this paper we present the case where a cognitive agent is inserted into an unknown environment composed of different kinds of other objects and agents; our cognitive agent needs to incrementally learn a model of the environment dynamics, doing it only from its interaction experience; the learne...

Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a set of imperfect or noisy observations. The modeling advantage of POMDPs, however, comes at a price — exact methods for solving them are computationally very...