Using listener gaze to augment speech generation in a virtual 3D environment
نویسندگان
چکیده
Listeners tend to gaze at objects to which they resolve referring expressions. We show that this remains true even when these objects are presented in a virtual 3D environment in which listeners can move freely. We further show that an automated speech generation system that uses eyetracking information to monitor listener’s understanding of referring expressions outperforms comparable systems that do not draw on listener gaze.
منابع مشابه
Grounding spoken interaction with real-time gaze in dynamic virtual environments
Gaze is an important cue in visually situated dialog, grounding referring expressions to objects in the environment. We present a new technique which demonstrates that monitoring real-time listener gaze – and giving appropriate feedback – enhances reference resolution by the listener: In a 3D virtual environment, users followed directional instructions, including pressing a number of buttons th...
متن کاملThe Impact of Listener Gaze on Predicting Reference Resolution
We investigate the impact of listener’s gaze on predicting reference resolution in situated interactions. We extend an existing model that predicts to which entity in the environment listeners will resolve a referring expression (RE). Our model makes use of features that capture which objects were looked at and for how long, reflecting listeners’ visual behavior. We improve a probabilistic mode...
متن کاملA New Trust Model for B2C E-Commerce Based on 3D User Interfaces
Lack of trust is one of the key bottle necks in e-commerce development. Nowadays many advanced technologies are trying to address the trust issues in e-commerce. One among them suggests using suitable user interfaces. This paper investigates the functionality and capabilities of 3D graphical user interfaces in regard to trust building in the customers of next generation of B2C e-commerce websit...
متن کاملPedagogical mediation using the virtual learning environment and the new generation: A search for improved performance in medical education
Introduction: Medical schools face increasing challenges toaccommodate new health care needs. However, little has changedregarding the students’ education. In addition, educators face thechallenge of improving their teaching approach, making it suitableand effective for the new generation. The purpose of this work wasto describe and contextualize the application of active pedagogicalmethodologi...
متن کاملThe influence of speaker gaze on listener comprehension: contrasting visual versus intentional accounts.
Previous research has shown that listeners follow speaker gaze to mentioned objects in a shared environment to ground referring expressions, both for human and robot speakers. What is less clear is whether the benefit of speaker gaze is due to the inference of referential intentions (Staudte and Crocker, 2011) or simply the (reflexive) shifts in visual attention. That is, is gaze special in how...
متن کامل