Spanish and Swedish interpretations of Spanish and Swedish emotions – the influence of facial expressions

This study presents an experiment in crosslinguistic multimodal interpretation of emotional expressions. Earlier studies on multimodal communication have shown an interaction between the visual and auditive modalities. Other, cross-cultural, studies indicate that facial expression of emotions is more universal than prosody is. Cross-linguistic interpretation of emotions could then be more successful multimodally than only vocally. The specific questions asked in the present study are to what degree Spanish and Swedish listeners can interpret Spanish emotional prosody, and whether simultaneously presented faces, expressing the same emotions, improve the interpretation. Audio recordings of Spanish emotional expressions were presented to Spanish and Swedish listeners, in two experimental settings. In the first setting the listeners only attended to prosody, in the second one they also saw a face, expressing different emotions. The results indicate that intra-linguistic as well as cross-linguistic interpretation of emotional prosody is improved by visual stimuli. Cross-linguistic interpretation of prosody is more poorly accomplished than inter-linguistic, but seems to be greatly augmented by multimodal stimuli.