Facial activity of attitudinal speech in German

The current study is a continuation of previous work on how attitudes are expressed in speech. Generally, we found that the availability of facial expression (together with speech) leads to a higher plausibility of an expressed attitude. Therefore, a further analysis of facial cues in the expression of attitudinal states appears useful. For this purpose, we selected a subset of sixteen attitudes portrayed by five performers, who have shown to be most convincing in their portrayal of attitudinal states. The Facial Action Coding System (FACS) is then applied on the output of OpenFace, which automatically detects the presence and the intensity of Action Units using a computer vision algorithm. Based on detected Action Units (Aus), a cluster analysis on one representative frame for each video is carried out. Furthermore, a linear discriminant analysis of the clusters is carried out including the whole dataset. The analysis shows that based on two linear discriminants, no clear separation of the cluster groups is possible. Some of the results for the facial expressions can be explained by their corresponding attitudes' distribution in emotional space. Not surprisingly, we also found that the facial expressions are strongly influenced by their phonetic content, thus, the attitudinal expression related to lip movements may be difficult to detect in speech.