This paper describes the incorporation of a visual lip tracking and lip-reading algorithm that utilizes the affine-invariant Fourier descriptors from parametric lip contours to improve the audio-visual speech recognition systems. The audio-visual speech recognition system presented here uses parallel hidden Markov models (HMMs), where a joint decision, using an optimal decision rule, is made af...

In this paper, we present our approach of face and lip detection, lip modeling, and tracking. A new lip model based on Bézier Curves is used to capture the dynamics of the lips efficiently. The model is defined only through few points which are modeled using the Active Shape Model (ASM). Accurate detection of lip details is implemented using multiple independent feature templates. The method de...

Systems that attempt to recover the spoken word from image sequences usually require complicated models of the mouth and its motions. Here we describe a new approach based on a fast mathematical morphology transform called the sieve. We form statistics of scale measurements in one and two dimensions and these are used as a feature vector for standard Hidden Markov Models (HMMs).

Lip segmentation is an essential stage in many multimedia systems such as video-conferencing, lip reading, or low bit rate coding communication systems. It is also useful in various image / video acquisition devices encouraged the development of many computer vision applications, such as vision -based surveillance, vision based man machine interfaces, vision-based biometrics, and so on. Hence e...

A critical assumption of all current visual speech recognition systems is that there are visual speech units called visemes which can be mapped to units of acoustic speech, the phonemes. Despite there being a number of published maps it is infrequent to see the effectiveness of these tested, particularly on visual-only lip-reading (many works use audio-visual speech). Here we examine 120 mappin...

A lipase gene (lip) and its activator gene (act) on a 2.9 kb BglII-EcoRI fragment from Pseudomonas sp. KWI-56 were cloned in Escherichia coli using pUC19 as a vector plasmid. From the sequencing results, the open reading frames of the lip and the act were found to contain 1092 and 1032 nucleotides, respectively. The act existed downstream of the lip with the same orientation. When the lip was e...

Automatic speech recognition systems that utilise the visual modality of speech often are investigated within a speakerdependent or a multi-speaker paradigm. That is, during training the recogniser will have had prior exposure to example speech from each of the possible test speakers. In a previous paper we highlighted the danger of not using different speakers in the training and test sets, an...

Lip-reading has been shown to improve the intelligibility of speech in multitalker situations, where auditory stream segregation naturally takes place. This study investigated whether the benefit of lip-reading is a result of a primary audiovisual interaction that enhances the obligatory streaming mechanism. Two behavioral experiments were conducted involving sequences of French vowels that alt...

It may be difficult for hearing-impaired people to communicate with people who hear. In the health care area, there is often little awareness of the communication barriers faced by the deaf and, in dentistry, the attitude adopted towards the deaf is not always correct. A review is given of the basic rules and advice given for communicating with the hearing-impaired. The latter are classified in...