Optimising Turn-Taking Strategies With Reinforcement Learning
نویسندگان
چکیده
In this paper, reinforcement learning (RL) is used to learn an efficient turn-taking management model in a simulated slotfilling task with the objective of minimising the dialogue duration and maximising the completion task ratio. Turn-taking decisions are handled in a separate new module, the Scheduler. Unlike most dialogue systems, a dialogue turn is split into microturns and the Scheduler makes a decision for each one of them. A Fitted Value Iteration algorithm, Fitted-Q, with a linear state representation is used for learning the state to action policy. Comparison between a non-incremental and an incremental handcrafted strategies, taken as baselines, and an incremental RL-based strategy, shows the latter to be significantly more efficient, especially in noisy environments.
منابع مشابه
Reinforcement Learning for Turn-Taking Management in Incremental Spoken Dialogue Systems
In this article, reinforcement learning is used to learn an optimal turn-taking strategy for vocal human-machine dialogue. The Orange Labs’ Majordomo dialogue system, which allows the users to have conversations within a smart home, has been upgraded to an incremental version. First, a user simulator is built in order to generate a dialogue corpus which thereafter is used to optimise the turn-t...
متن کاملEvaluating State Representations for Reinforcement Learning of Turn-Taking Policies in Tutorial Dialogue
Learning and improving natural turn-taking behaviors for dialogue systems is a topic of growing importance. In task-oriented dialogue where the user can engage in task actions in parallel with dialogue, unrestricted turn taking may be particularly important for dialogue success. This paper presents a novel Markov Decision Process (MDP) representation of dialogue with unrestricted turn taking an...
متن کاملA Bidding Approach to Turn-Taking
We propose a new turn-taking framework for spoken dialogue systems in which conversants bid for the turn. This differs from most current turn-taking approaches, where the turn only changes after the holder has released it. Our new framework uses Reinforcement Learning to choose appropriate turn bids, which indirectly estimates the utterance importance. We evaluate this approach in an artificial...
متن کاملInverse reinforcement learning for micro-turn management
Existing spoken dialogue systems are typically not designed to provide natural interaction since they impose a strict turn-taking regime in which a dialogue consists of interleaved system and user turns. To allow more responsive and natural interaction, this paper describes a system in which turn-taking decisions are taken at a more fine-grained micro-turn level. A decision-theoretic approach i...
متن کامل