Evolving Expression of Emotions in Virtual Humans Using Lights and Pixels

An intelligent virtual human should not be limited to gesture, face and voice for the expression of emotion. The arts have shown us that complex affective states can be expressed resorting to lights, shadows, sounds, music, shapes, colors, motion, among many others [1]. Thus, we previously proposed that lighting and the pixels in the screen could be used to express emotions in virtual humans [2]. Lighting expression inspires in the principles of lighting, which are regularly explored in theatre or film production [3]. Screen expression acknowledges that, at the meta level, virtual humans are no more than pixels in the screen which can be manipulated to convey emotions, in a way akin to the visual arts [4]. In particular, we explore the filtering technique where the scene is rendered to a temporary texture, modified using shaders and, then, presented to the user. Now, having defined the expression channels, how should we express emotions through them? We are presently exploring an evolutionary approach which relies on genetic algorithms (GAs) to learn mappings between emotions and lighting and screen expression. The GAs’ clear separation between generation and evaluation of alternatives is convenient for this problem. Alternatives can be generated using biologically inspired operators – selection, mutation, crossover, etc. Evaluation, in turn, can rely on artificial critics, which define fitness functions from art theory, or on human critics. Humans can be used to fill in the gaps in the literature as well as accommodate the individual, social and cultural values with respect to the expression of emotion in art [1].