Speech and manual gesture coordination in a pointing task
نویسندگان
چکیده
This study explores the coordination between manual pointing gestures and gestures of the vocal tract. Using a novel methodology that allows for concurrent collection of audio, kinematic body and speech articulator trajectories, we ask 1) which particular gesture (vowel gesture, consonant gesture, or tone gesture) the pointing gesture is coordinated with, and 2) with which landmarks the two gestures are coordinated (for example, whether the pointing gesture is coordinated to the speech gesture by the onset or maximum displacement). Preliminary results indicate coordination of the intonation gesture and the pointing gesture.
منابع مشابه
Gesture and speech coordination: the influence of the relationship between manual gesture and speech
Communication is multimodal. In particular, speech is often accompanied by manual gestures. Moreover, their coordination has often been related to prosody. The aim of this study was to further explore the coordination between prosodic focus and different manual gestures (pointing, beat and control gestures) on ten speakers using motion capture. As compared to previous studies, results show that...
متن کاملCo-Production of Speech and Pointing Gestures in Clear and Perturbed Interactive Tasks: Multimodal Designation Strategies
Designation consists in attracting an interlocutor’s attention on a specific object and/or location. It is most often achieved using both speech (e.g., demonstratives) and gestures (e.g., manual pointing). This study aims at analyzing how speech and pointing gestures are co-produced in a semi-directed interactive task involving designation. 20 native speakers of French were involved in a cooper...
متن کاملCo-Production of Contrastive Prosodic Focus and Manual Gestures: Temporal Coordination and Effects on the Acoustic and Articulatory Correlates of Focus
Speech, and prosody in particular, is tightly linked to manual gestures. This study investigates the coordination of prosodic contrastive focus and different manual gestures (pointing, beat and control gestures). We used motion capture on ten speakers to explore this issue. The results show that prosodic focus "attracts" the manual gesture whichever its type, the temporal alignment being strict...
متن کاملThe speech focus position effect on jaw-finger coordination in a pointing task.
PURPOSE This article investigates jaw-finger coordination in a task involving pointing to a target while naming it with a CVCV (e.g., /papa/) versus CVCV (e.g., /papa/) word. According to the authors' working hypothesis, the pointing apex (gesture extremum) would be synchronized with the apex of the jaw-opening gesture corresponding to the stressed syllable. METHOD Jaw and finger motions were...
متن کاملFinger-jaw coordination during a deictic gesture with CVCV utterances: the effect of stress position
The present study investigates the hypothesis of a synchronization between the extremum (apex) of the pointing gesture and the apex of the jaw opening gesture for the stress vowel in 'CVCV vs CV'CV utterances in a deictic task (consisting in showing a target while naming it). The results show that for 'CVCV sequences (e.g. /pápa/) the pointing apex tends to be phased with the jaw apex for the f...
متن کامل