This paper investigates how to automatically create a dialogue control component of a listening agent to reduce the current high cost of manually creating such components. We collected a large number of listening-oriented dialogues with their user satisfaction ratings and used them to create a dialogue control component using partially observable Markov decision processes (POMDPs), which can le...

Unlike traditional reinforcement learning (RL), market-based RL is in principle applicable to worlds described by partially observable Markov Decision Processes (POMDPs), where an agent needs to learn short-term memories of relevant previous events in order to execute optimal actions. Most previous work, however, has focused on reactive settings (MDPs) instead of POMDPs. Here we reimplement a r...

Interactive POMDPs (I-POMDPs) are a useful framework for describing POMDPs that interact with other POMDPs. I-POMDPs are solved recursively in levels: a level1 I-POMDP assumes the opponent acts randomly, and a levelk I-POMDP assumes the opponent is a level-(k-1) I-POMDP. In this paper, we introduce fully-nested I-POMDPs, which are uncertain about the physical state of the game, the level of the...

The partially observable Markov decision process (POMDP) framework has been applied in dialogue systems as a formal framework to represent uncertainty explicitly while being robust to noise. In this context, estimating the dialogue POMDP model components is a significant challenge as they have a direct impact on the optimized dialogue POMDP policy. To achieve such an estimation, we propose meth...

Given the problem of planning actions for situations with uncertainty about the action outcomes, Markov models can eeectively model this uncertainty and ooer optimal actions. When the information about the world state is itself uncertain, partially observable Markov models are an appropriate extension to the basic Markov model. However , nding optimal actions for partially observable Markov mod...

We consider a distributionally robust partially observable Markov decision process (DR-POMDP), where the distribution of transition-observation probabilities is unknown at beginning each period, but their realizations can be inferred using side information end period after an action being taken. build ambiguity set joint bounded moments via conic constraints and seek optimal policy to maximize ...

Partially observable Markov decision processes (POMDP) are well-suited for realizing sequential decision making capabilities that respect uncertainty in Companion systems that are to naturally interact with and assist human users. Unfortunately, their complexity prohibits modeling the entire Companion system as a POMDP. We therefore propose an approach that makes use of abstraction to enable em...

This paper examines the problem of finding an optimal policy for a Partially Observable Markov Decision Process (POMDP) when the model is not known or is only poorly specified. We propose two formulations of the problem. The first formulation relies on a model of the uncertainty that is added directly into the POMDP planning problem. This has some interesting theoretical properties, but is impr...