Speech, gaze and gesturing: multimodal conversational interaction with Nao robot
نویسندگان
چکیده
The paper presents a multimodal conversational interaction system for the Nao humanoid robot. The system was developed at the 8th International Summer Workshop on Multimodal Interfaces, Metz, 2012. We implemented WikiTalk, an existing spoken dialogue system for open-domain conversations, on Nao. This greatly extended the robot’s interaction capabilities by enabling Nao to talk about an unlimited range of topics. In addition to speech interaction, we developed a wide range of multimodal interactive behaviours by the robot, including facetracking, nodding, communicative gesturing, proximity detection and tactile interrupts. We made video recordings of user interactions and used questionnaires to evaluate the system. We further extended the robot’s capabilities by linking Nao with Kinect.
منابع مشابه
Multimodal Open-Domain Conversations with the Nao Robot
In this paper we discuss the design of human-robot interaction focussing especially on social robot communication and multimodal information presentation. As a starting point we use the WikiTalk application, an open-domain conversational system which has been previously developed using a robotics simulator. We describe how it can be implemented on the Nao robot platform, enabling Nao to make in...
متن کاملOpen-Domain Conversation with a NAO Robot
for demonstration Abstract—In this demo, we present a multimodal conversation system, implemented using a Nao robot and Wikipedia. The system was developed at the 8th International Workshop on Multimodal Interfaces in Metz, France, 2012. The system is based on an interactive, open-domain spoken dialogue system called WikiTalk, which guides the user through conversations based on the link struct...
متن کاملEvaluation of WikiTalk - User Studies of Human-Robot Interaction
The paper concerns the evaluation of Nao WikiTalk, an application that enables a Nao robot to serve as a spoken open-domain knowledge access system. With Nao WikiTalk the robot can talk about any topic the user is interested in, using Wikipedia as its knowledge source. The robot suggests some topics to start with, and the user shifts to related topics by speaking their names after the robot men...
متن کاملOlga - a conversational agent with gestures
The Olga project has developed an animated agent interface for information services. The interface combines a graphical interface, spoken dialogue and an animated 3D ‘human-like’ character for multimodal input and output. The interaction is intelligently managed using techniques derived from spoken dialogue but extended for the graphical modality. The Olga agent is innovative in combining a spo...
متن کاملCombining dynamic head pose-gaze mapping with the robot conversational state for attention recognition in human-robot interactions
The ability to recognize the Visual Focus of Attention (VFOA, i.e. what or whom a person is looking at) of people is important for robots or conversational agents interacting with multiple people, since it plays a key role in turn-taking, engagement or intention monitoring. As eye gaze estimation is often impossible to achieve, most systems currently rely on head pose as an approximation, creat...
متن کامل