Many of the current GPS (Global Positioning Systems) navigation aids use an audio method to deliver navigation information to the user. For the visually impaired person this method can be problematic. The visually impaired pedestrian relies heavily on information contained within the ambient sound environment; for location and orientation information, navigation information, and importantly, sa...

This study examined the manners of audiovisual speech perception, using the “McGurk effect”, when the speakers were foreigners. The McGurk effect demonstrates that visual (lip movement) information is used during speech perception even when it is discrepant with auditory information. Subjects, 17 Chinese and 23 Japanese reported what they heard while looking at and listening to the speakers’ fa...

This paper presents the work on a prototype spatial auditory display. Using high-definition audio rendering a sample application was presented to a mixed group of users with visual disabilities and normal sighted users. The evaluation of the prototype provided insights into how effective spatial presentation of sound can be in terms of human-computer interaction (HCI). It showed that typical ap...

Visual speech information is known to improve accuracy and noise robustness of automatic speech recognizers. However, todate, all audio-visual ASR work has concentrated on “visually clean” data with limited variation in the speaker’s frontal pose, lighting, and background. In this paper, we investigate audiovisual ASR in two practical environments that present significant challenges to robust v...

This paper describes the development and evaluation of a computer game for blind and visually impaired people using a new haptic audio virtual environment (HAVE). The GRAB HAVE consists of a new two-finger haptic interface and Haptic Geometric Modeller which enables people to locate and interact with 3D computer-generated objects using their sense of touch and audio feedback. A simple 3D search...

A blind method for suppressing late reverberation from speech and audio signals is presented. The proposed technique operates both on the spectral and on the sub-band domains employing a single input channel. At first, a preliminary rough clean signal estimation is required and for this, any standard technique may be applied; however here the estimate is obtained through spectral subtraction. T...

In this paper we describe a final-year undergraduate honours course that requires multimedia computing students to create narrative auditory and visual displays. The aim was to use discontinuity as a catalyst for creativity by getting students to tell a story in non-speech sound and in vision in which each medium conveyed a different mood.

Visual speech information from the speaker’s mouth region has been successfully shown to improve noise robustness of automatic speech recognizers, thus promising to extend their usability in the human computer interface. In this paper, we review the main components of audio-visual automatic speech recognition and present novel contributions in two main areas: First, the visual front end design,...

Information and communication technologies have played a fundamental role in teaching and learning for many years. Technologies, such as radio and TV, were used during the 50s and 60s for delivering instructional material in audio and/or video format. More recently, the spread of computer-based educational systems has transformed the processes of teaching and learning (Squires, Conole, & Jacobs...