The popularity of image sharing on social media and the engagement it creates between users reflect the important role that visual context plays in everyday conversations. We present a novel task, ImageGrounded Conversations (IGC), in which natural-sounding conversations are generated about a shared image. To benchmark progress, we introduce a new multiplereference dataset of crowd-sourced, eve...

Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. When recognizing visual feedback, people use more than their visual perception. Knowledge about the current topic and expectations from previous utterances help guide our visual perception in recognizing nonverbal cues. In this chapter, we investigate how dial...

Parallel with the orthographic streams of words in conversation are multiple layered epiphenomena, short in duration and with a communicative purpose. These paralinguistic events regulate the interaction flow via gaze, gestures and intonation. This thesis focus on how to compute, model, discover and analyze prosody and it’s applications for spoken dialog systems. Specifically it addresses autom...

Natural (spoken) language, combined with gestures and other human modalities, provides a promising alternative for interacting with computers, but such benefit has not been explored for interactions with geographical information systems. This paper presents a conceptual framework for enabling conversational humanGIS interactions. Conversations with a GIS are modeled as human-computer collaborat...

Cooperativity is usually seen as a central concept in the pragmatics of dialogue. There are a number of accounts of dialogue performance and interpretation that require some notion of cooperation or collaboration as part of the explanatory mechanism of communication. For instance, Grice’s cooperativity principle and associated maxims are used to explain conversational implicature (Grice, 1975)....

Videoconference has become the dominant technology for remote meetings. Embodied Virtual Reality is a potential alternative that employs motion tracking in order to place people shared virtual environment as avatars. This paper describes 210 participant study focused on behavioral measures compares multiparty interaction videoconference and embodied VR across range of task types: factual intell...

The notion of “common ground” (Clark & Brennan 1991; Clark 1996) presupposes communication or conversation as “the basic setting for language use” 1996: 11). serialisation Japanese sentence-final particles is highly sensitive to the likelihood relevant utterance being part common ground. This paper reconsiders conception ground and grounding processes, investigating monologic well conversat...

People with age-related or congenital cognitive impairments require assistance in daily tasks to enable them to maintain a self-determined lifestyle in their own home. We developed and evaluated a prototype of an autonomous spoken dialogue assistant to support these user groups in the domain of week planning. Based on insights from previous work with a WOz study, we designed a dialogue system w...

Outside of Western contexts, natural-conversation-based research on intergenerational communication is relatively rare. To help redress this imbalance, this paper explores the conversational styles of first-encounter talks between five pairs of college students and older adults in Taiwan, and infers the interactional norms that underlie them. It is found that younger Taiwanese adults tend to ex...