In this paper, a proposal of a multi-modal dialogue system oriented to multilingual questionanswering is presented. This system includes the following ways of access: voice, text, avatar, gestures and signs language. The proposal is oriented to the question-answering task as a user interaction mechanism. The proposal here presented is in the first stages of its development phase and the archite...

This paper describes a probabilistic mechanism for the interpretation of sentence sequences developed for a spoken dialogue system mounted on a robotic agent. The mechanism receives as input a sequence of sentences, and produces an interpretation which integrates the interpretations of individual sentences. For our evaluation, we collected a corpus of hypothetical requests to a robot. Our mecha...

We describe several architectural features of the Athos agentbased framework for the construction of spoken dialogue systems, and additional features in two application demonstrators, AthosMail and AthosNews, that contribute to the robustness of their operation. These include adaptation of recognition resources to personal and application data, use of a broad-coverage robust parser as an interm...

The central module of any natural language dialogue system is the dialogue manager, which plays the role of an intermediate agent between the user and the information source. Its cooperativity and portability highly determines the efficiency of the dialogue system. Therefore, as the basis for cooperativity of information-providing dialogue systems we propose a knowledge representation of the in...

Much work has been done in dialogue mod-eling for spoken and multi-modal human-computer interaction. Problems can arise in situations that do not correspond to the dialogue model. For this reason, we propose information-centered dialogue processing in which the actions to be taken by the dialogue system are determined as a function of the information available in the discourse, the database and...

Dialogue state tracking is a key sub-task of dialogue management. The fourth Dialog State Tracking Challenge (DSTC-4) focuses on dialogue state tracking for real world human-tohuman dialogues. The task is more challenging than previous challenges because of more complex domain and coreferences, more synonyms and abbreviations, sub-dialogue level labelled utterances, and no spoken language under...

Instead of clustering video shots into scenes using low level image features, in this paper, we propose a rule-based model to extract simple dialog or action scenes. Through analyzing video editing rules and observing temporal appearance patterns of shots in dialog scenes of movies, we deduce a set of rules to recognize dialog or action scenes. Based on these rules, a finite state machine is de...

The overall objective of the DARPA COMMUNICATOR project is to support rapid, cost-effective development of multi-modal speechenabled dialogue systems with advanced conversational capabilities, such as plan optimization, explanation and negotiation. In order to make this a reality, we need to find methods for evaluating the contribution of various techniques to the users’ willingness and ability...

User modeling is an iml>ortant COlnponents of dialog systems. Most previous approaches are rule-based methods, hi this paper, we proimse to represent user models th rough Bayesian networks. Some advantages of the Bayesian approach over the rule-based approach are as follows. First, rules for upda t ing user models are not necessary because up<lating is directly performed by the ewduation of the...

The syntactic component of a speech understanding system is described. The interface between acoustic recognition and syntax is sketched, and the set-up of the lexicon is outlined. The main aspects of the paper concern the ru/e format of the APSG (augmented phrase structure grammar), the syntactic concept for handling the sentence topology of German, and the treatment of discontinuous structure...