On Search for Emotion in Hindusthani Vocal Music

Indian music is primarily melodic. Appreciation of it is supposed to be mainly based on the melodic content and rhythm, and not so much on the timbral content. In north Indian classical vocal music (also known as Hindusthani vocal music) raga forms the base bone. The most common style of performance is khayal. The first part in khayal is known as alap, wherein one tries to establish the image and emotional distinctiveness in slow tempo. The present study concentrates on the alap part of professional khayal performances. For each raga there are some distinctive sequences known as chalans and pakads. A raga is said to elicit some specific emotions from 8 categories referred to in Indian treatises as Rasas. The database for the experiments consists of short segments extracted from khayal performances from eleven most emotive ragas selected from the archives of ITCSRA. The first part of the study consists of identifying significant emotions evoked by the aforesaid segments in western listeners as well as native Indian listeners and the cross-cultural differences, if any. This also includes an examination of to what extent the raga–rasa relationship indicated in treatises matches with the observed data. The second part consists of the extraction of the distinctive note-sequences in these segments. The signal processing part performed here includes extraction of pitch contour, and segmentation of it into sequences of notes. The result of the examination of the relation of these sequences with the perceived emotion is also presented. Thirty seconds long music segments were used in our experiments, to check what specific emotion(s) they elicit. The listening tests showed that different segments from the same raga do not generally correspond to the emotions prescribed in Indian treaties. The responses from the two cultural groups are found to be similar. Melodic sequences vaguely relate with emotional response. INTRODUCTION Emotions give meaning to our lives. No aspect of our mental life is more important to the quality and meaning of our existence than emotions. They make life worth living, or sometimes ending. The English word 'emotion' is derived from the French word émouvoir which means ‘move’. Great classical philosophers—Plato, Aristotle, Spinoza, Descartes conceived emotion as responses to certain sorts of events triggering bodily changes and typically motivating characteristic behavior. It is difficult to find a consensus on the definition of emotion [4]. Most researchers would probably agree that emotions are relatively brief and intense reactions to goal-relevant changes in the environment that consist of many subcomponents: cognitive appraisal, subjective feeling, physiological arousal, expression, action tendency, and regulation. It therefore suggests that some part of the brain would be selectively activated [10]. Origin of emotion may be traced back to 200,000 years ago to semi-nomadic hunter–gatherer [11]. It is argued that their way of living, which involved cooperating in such activities as hunting, avoiding predators, finding food, rearing children, and also competing for resources, could be related to the origin of emotion. Most emotions are presumably adapted to living this way. Several of the activities are associated with basic survival problems that most organisms have in common. These problems, in turn, require specific types of adaptive reactions. A number of authors have suggested that such adaptive reactions are the prototypes of emotions as seen in humans [13], [14]. If various emotions are cognitively differentiable, as is again likely to be publicly agreed upon, there should be differences in sites of brain being excited for different emotions. Bower further suggested [1] that every emotion is associated with autonomic reactions and expressive behaviors. These expressive behaviors or responses to the same stimuli can vary depending on many factors external to the stimuli, like the mood of a person [4], [8], memory association of the person to the applied stimuli [4], [5], etc. Objects or a sequence of objects elicit feeling through a sequence of psychophysical processes. The sensory organs convert the signals from the objects to neural pulses. These pulse trains are processed in sub-cortical neural structures, which are primarily inherited. These processed signals produce perception in brain. Through the process of learning and experience we cognize the objects or sequence of objects from these perceived signals. Again through the process of learning we learn to associate these with some emotive environment in the past. This ultimately evokes emotion or feeling. The evaluation of emotional appraisals of stimuli may be done by having the person report the emotions they perceive as reaction to the stimuli. This can be done in several different ways such as verbal descriptions, choosing emotional terms from a list, or rating how well several different emotional terms describe the appraisal [4], [7]. The emotional terms used should be limited in number and as unambiguous as possible. It is also possible to represent these terms in vector forms. The splitting of emotion into dimensions is consistent with Bower’s network theory of emotion [3], [6]. However, the number of components and the type of components vary between studies [2], [6], [7], [9]. While sound stimuli may cause general physiological changes ("arousal"), these changes must be interpreted cognitively in order for a specific emotion to emerge. The listener does not come to the listening experience as a blank slate. He or she already has existing musically pertinent knowledge. Even for a musically untrained listener the general exposure to listening to music since childhood is also learning, though not formal. Thus any emotional behavior, even habitual and seemingly automatic and natural, is actually learned. In case of music this behavior serves as a means of communication, since often emotional behavior is differentiable and intelligible. One major problem that arises in the study of the emotional power of music is that the emotional content of music is very subjective. A piece of music may be undeniably emotionally powerful, and at the same time be experienced in very different ways by each person who hears it. The emotion created by a piece of music may be affected by memories associated with the piece, by the environment it is being played in, by the mood of the person listening and their personality, by the culture they were brought up in; by any number of factors both impossible to control and impossible to quantify. Under such circumstances, it is extremely difficult to deduce what intrinsic quality of the music, if any, created a specific emotional response in the listener. It seems that the listeners experience gross emotion through the unfolding of successive events. If the successive events are always predictable, the emotion is boredom, unless an association of past evokes a specific emotion. Again, extensive uncertainties are likely to lead to apprehension and anxiety. These two emotional experiences may therefore be the robust ones. As soon as the unexpected is experienced, the listener attempts to fit it into the general system of beliefs relevant to the theme. This may happen in one of the following three ways: (1) The mind may suspend judgment, expecting that the subsequent event will bring in clarity. (2) If no clarification takes place, irritation will set in. (3) The expected consequent may be seen as a purposeful blunder. When we expect music to convey emotion, we accept it to be a language, as emotion is built upon the meaning. In order to extract the right emotion, it is essential that the listener is acquainted with the grammar, and thus the extra-musical world of concepts, actions, emotional states, and character. Even if he does not have the appropriate grammar in his mind, he can still extract some emotional meaning out of it, with some of his own stock grammar. In this sense emotion in music may be called referential. A piece of music can be universally pleasant or universally irritating. On the other hand, a musical stimulus or series of stimuli can be considered to indicate and point to other musical events, which are about to happen, rather than extra-musical concepts and objects. That is, one musical event (be it a tone, a phrase, or a whole section) has meaning because it points to and makes us expect another musical event. Even then the affective experience is still dependent on cognition (involving a process of intellection, conscious or unconscious) that cannot be restricted to the musical concepts alone. The musical expectations and experience grow out of the innate processes of grouping, closure, and good continuation in Gestalt psychology. Hevner [12] studied as early as in 1936 grouping of emotions described by listeners using adjectives through listening experiments. The experiments substantiated a hypothesis that music inherently carries emotional meaning. Hevner discovered the existence of clusters of descriptive adjectives and laid them out in a circle. These are: a) cheerful, gay, happy, b) fanciful, light, c) delicate, graceful, d) dreamy, leisurely, e) longing, pathetic, f) dark, depressing, g) sacred, spiritual, h) dramatic, emphatic, i) agitated, exciting, j) frustrated, k) mysterious, spooky, l) passionate, m) bluesy. Actually, emotion detection in musical information is better considered as a ‘Multi-label Classification problem’, where the music sounds are classified into multiple classes simultaneously. That means that a single music sound may be characterized by more than one label, e.g. both “dreamy” and “cheerful.” Figure 1: Possible descriptions of emotion using valence and arousal [22]. Emotions recognized in music can be represented in a two-dimensional space (Figure 1), with valence (positive vs. negative feelings) and arousal (high–low) as principal axes [17]. These are the dimensions suggested by Russell to describe emotion [15]. Valence refers to the happiness or sadness of the emotion and arousal is the activeness or passiveness of the emotion [16]. A positive valence corresponds with positive emotions such as joy, happiness, relaxing and a negative valence corresponds with negative emotions such as fear, anger and sadness. Emotions are denotative signs. When a listener reports an emotion, particularly in case of ragas in Indian Music, he may actually be describing only what he believes the passage is supposed to indicate, not anything he has experienced by himself. Even when a genuine emotional experience is reported, it is liable to become garbled and perverted in the process of verbalization. Some emotional states are much more subtle and varied than are the few crude and standardized words which we use to denote them. In such cases reports may contain a large amount of what psychiatrists call "distortion". In India, music (geet) has been a subject of aesthetic and intellectual discourse since the times of Vedas (samaveda). Rasa was examined critically as an essential part of the theory of art by Bharata in Natya Sastra, (200 century BC). The rasa is considered as a state of enhanced emotional perception produced by the presence of musical energy. It is perceived as a sentiment, which could be described as an aesthetic experience. Although unique, one can distinguish several flavors according to the emotion that colors it. Several emotional flavors are listed, namely erotic love (sringara), pathetic (karuna), devotional (bhakti), comic (hasya), horrific (bhayanaka), repugnant (bibhatsa), heroic (vira), fantastic, furious (roudra), peaceful (shanta). Italics represent the corresponding emotion given in the Indian treatises. The individual feels immersed in that mood to the exclusion of anything else including himself. It may be noted that during the musical experience, the mind experiences conscious joy even in the representation of painful events because of the integration of perceptual, emotional, and cognitive faculties in a more expanded and enhanced auditory perception, completed by the subtle aesthetic of sensing, feeling, understanding and hearing all at the same time. The Eastern approach to emotional aesthetics and intelligence treats rasa as a multi-dimensional principle that explains thoroughly the relation between a sentiment, a mood, the creative process and its transpersonal qualities. This transpersonal domain includes the super-conscious or spiritual state and therefore acts as an interface between individual and collective unconscious states. This transpersonal quality is a germinating power hidden behind aspects of great musical creation that can reveal it, and is able to induce the complete chromatic range of each emotion. Rasa conveys the idea of an aesthetic beauty knowable only through the feeling. This aesthetic experience is a transformation of not merely feeling, but equally of cognition, a comprehensive understanding in the mode of ecstasy of the intellect, itself inscrutable and illuminating. In the Vedas the experience of rasa is described as a flash of inner consciousness, which appears to whom the knowledge of ideal beauty is innate and intuitive. Rasa is not the unique property of the art itself. It unites the art with the creator and the observer in the same state of consciousness, and requires the power of imagination and representation and therefore a kind of intellectual sensibility. Indian musicological treaties since Bharata hold that even notes bear the potential of producing emotional effects. Tembe listed eight of them (Table 1). However no rational or scientific scrutiny was provided. It seems that the list was drawn from the proposals presented in Natyashastra. Table 1 Emotional attributes of notes according to G.S. Tembe [21] Notes Emotional attribute Shadja like a yogi beyond any attachment Rishabha (komala) rather sluggish Rishabha (shuddha) reminding of indolence of a person waking up from sleep Gandhara (komala) bewildered, helpless and pitiable Gandhara (shuddha) fresh and pleasant Madhyama (shuddha) grave, noble and powerful Madhyama (tivra) sensitive, luxurious Panchama brilliant, self composing Dhaivata (komala) grief, pathos Dhaivata (shuddha) robust, lustful Nishada (komala) gentle, happy, affectionate Nishada (shuddha) piercing appeal However, he agrees that only four rasas, namely Karuna, Shanta, Shringara and Vira may actually be experienced from a single note. He further proposes that when Shuddha madhyama dominates a melody, it creates a serene and sublime atmosphere, while a dominant Panchama creates an invigorating and erotic feeling. Pandit V.N. Bhatkhande [24] in his work suggested the inadequacy of vadi svara (i.e. the main melodic tone of the raga) in determining the rasa of ragas. However, he mentions that Ragas employing Shuddha (Rishabha, Dhaivata and Gandhara) emote Shringara rasa, and those employing Komala (Dhaivata and Nishad) emote Vira rasa. This view is contradicted by Ratanjankar [21]. According to him, individual notes cannot produce emotion, and they may do so only in a specific context. This implies that expression is born by the melodic content. Konishi et.al [27] reported that listeners can correctly decode emotions like anger, fear, happiness, and sadness from single notes from vibrato effect in Western music. Having noted all these, it appears that the notion of a single note conveying emotion in general may be somewhat contrived particularly in Indian music. In Indian music, a note does not have a specific frequency. It is related to the scale where the base note Sa can be assigned any arbitrary frequency. Kamani [22] noticed inconsistency between the rasa of a raga traditionally prescribed and experienced. He holds that since a raga represents a complex set of feelings, a simple relationship between a raga and rasa is unlikely. Some empirical studies on the relationship of raga and rasa in Hindustani music are available. Deva [25] and Virmani reported consistent judgment of Indian listeners on the mood, color, season, and time of day for excerpts from Hindustani ragas. Gregory and Varney [28] used both Western and Indian listeners to assess the emotional content of Hindustani ragas, Western classical music, and Western new-age music. He used a list of mood terms taken from Hevner for the assessment of the emotional contents. Both the Western and Indian listeners were reported to be sensitive to intended emotions in Western music, but not in the Hindustani ragas. Also the textbook descriptions of ragas did not always reflect the mood intended in a given performance. On the other hand, Balkwill and Thompson [26] reported high ratings in respected categories of correct emotion detection by Western listeners for joy, sadness and anger in Hindustani ragas. They believe that listeners can appreciate affective qualities of unfamiliar music by attending to acoustic cues. Consciously or intuitively, composers and performers draw upon acoustic cues too. When cultural-specific cues are absent, listeners may still attend to acoustic cues such as tempo and loudness. These cues provide listeners with a general understanding of the intended emotion. In Hindustani music, ragas are said to be associated with different rasas (emotions). However, one particular raga is not necessarily associated with one emotion. Moreover, opinion varies; a comprehensive summary is available in Semiosis in Hindusthani Music [18]. For the present study we have selected 11 ragas (Table 2) to represent different rasas/emotions residing therein. Of the eight emotions listed in the opinion score sheets, only six represent rasas. These are Heroic (Vira), Anger (Raudra), Serenity (Santa), Devotion (Bhakti), Sorrow (Karuna), Romantic (Sringara). Other two emotions namely Joy and Anxiety have been considered additionally. Table 2. Selected ragas and corresponding rasas Name of the Raga Rasas Adana Vira Bhairav Raudra, Santa, Bhakti, Karuna Chayanat Sringara Darbari Kannada Santa Hindol Vira, Raudra Jayjayvanti Sringara Jogiya Karuna, Sringara, Bhakti Kedar Santa Mian-ki-Malhar Karuna Mian-ki-Todi Bhakti, Srigara, Karuna Shree Santa The objective of the present study was to find whether: 1) An oral music segment of short length extracted randomly from a raga elicit any emotion, 2) The elicited emotion from an oral music segment can be specified into prescribed categories, 3) The elicited emotion from different segments from the same raga has some specificity, 4) To what extent the emotional responses from the segments of a raga correspond to those given in table 2, 5) Whether the elicited response have any cross-cultural similarity, 6) To what extent the melodic sequence (sequence of musical notes) relate with emotional response. For the purpose of this research, we selected the ragas from ITC Sangeet Research Academy archive, and after signal processing we continued our work in order to find the sequences for listening experiments. These sequences would essentially be the fragments of ragas. In order to find which sequences evoke particular emotions, many possible sequences of various lengths can be taken into account (considering the grammar of the raga). We decided to extract about 30-seconds long sequences from the sound signal, which might evoke emotions, and use them for perceptual tests. The tests were performed by both western and Indian listeners, in order to observe and compare emotions evoked by each sequence. Since each raga has a specific set of notes and sequences used, we could assign short sequences of notes to particular emotions evoked in both Indian and western listeners. EXPERIMENTAL DETAILS There are different styles for executing a raga in vocal music. The most common is khayal. The performance of khayal has two distinctive parts. The first one is known as alap wherein one tries to establish the image and emotional distinctiveness in slow tempo. The second part is known as bistar where the tempo is faster and faster and the performer tries to expose his skills through use of various embellishments (alankaras) while keeping the mood and emotion of the raga intact. The present study concentrates on the alap part of professional khayal performances as this really establishes the characteristics of raga. The investigated excerpts represent vocal music (with some accompaniment). Songs by eminent singers in the ragas mentioned in Table 2 were selected from the archives of ITC Sangeet Research Academy. From the alap portions of the ragas, segments of about 30 seconds were taken out for the audition test. From each song, 4 segments were collected. The collection of these segments was then randomized. The listeners’ opinions were collected in the score sheet, presented in Table 3. There were two groups of informants: western listeners (24), and native Indians (12), as we wanted to investigate whether the perceived emotions and the required minimal length of audio segments to listen to were coincident, and also conforming to the theory presented in treatises on ragas. For western listeners, this music was very different than what they were used to listen, as Indian and western music and melodies are based on different scales. Each raga has specific set of notes used, sequences, prolonged notes, etc. Some western listeners reported difficulties in perceiving emotions when listening to this music. Initially, the test set consisted of 124 segments, but this test was too long and difficult for all listeners, even educated in music. Therefore, we decided to limit the test to 44 excerpts. The listeners were asked to assign each excerpt to only one emotional category if possible. Two choices were also allowed, but the listener had to order them and mark the first and the second choice. Statistical elaboration of results in this research was performed taking into account only the first choices. Table 3. Opinion Score Sheet Name of the informant: Age: Sex: M/F Knowledge of Music: y/n No. Anger Joy SorrowHeroicRomantic Serenity Devotion Anxiety Any other,mention Nil Emotion 1 2 3 ... ... ... 42 43 44 The correlation matrix was obtained using a Pearson product-moment correlation coefficient between different types or category of emotion. The Pearson product-moment correlation coefficient is a common measure of the correlation (linear dependence) between two variables. In our experiment, each category of emotion was compared to every other category, yielding a value in a range [1, -1] and the obtained correlation between the two was plotted in a matrix. The statistics is defined as the sum of the products of the standard scores of the two measures divided by the number of degrees of freedom. If the data comes from a sample, then