Cross-modal description of sentiment information embedded in speech

Looking for new possibilities to describe the information embedded in speech, we have carried out sentiment correlation analysis between speech features and color attributes. Using single vowel utterances with different prosody and sound pressure level, we have asked subjects to select colors based on their perceptual impressions after listening them. By analyzing selected color attributes using Value, Saturation and Hue, we found high correlations between mean F0 and Value, sound pressure level and Saturation, and Formants and Hue. These correlations coincided with previous observations using speech and color categories, which suggests a possibility for visualization of sentiment information embedded in speech based on cross-modal sentiment correlations.