Abstract
Artists express emotions through art. To accomplish this they rely on lines, shapes, textures, color, light, sounds, music, words, and the body. The field of virtual humans has been neglecting the kind of expression we see in the arts. In fact, researchers have tended to focus on gesture, face, and voice for the expression ofemotions. But why limit ourselves to the body? In this context, drawing on accumulated knowledge from the arts, this chapter describes an evolutionary model for the expression of emotions in virtual humans using lights, shadows, filters, and composition. Lighting expression uses lighting techniques from the visual arts to convey emotions through the lights in the environment. Screen expression uses filters and composition to manipulate the virtual human's pixels themselves in a way akin to painting. Emotions are synthesized using the OCC model. To learn how to map affective states into lighting and screen expression, an evolutionary model that relies on genetic algorithms is used. The crossover and mutation operators generate alternatives for the expression of some affective state and a critic ensemble, composed of artificial and human critics, selects among the alternatives.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alton, J. (1949). Painting with light. New York: Macmillan.
Amaya, K., Bruderlin, A., & Calvert, T. (1996). Emotion from motion. In Proceedings of Graphics Interface'96 (pp. 222–229).
Angeline, P. (1996). Evolving fractal movies. In Proceedings of the First Annual Conference in Genetic Programming (pp. 503–511).
Arijon, D. (1976). Grammar of film language. New York: Hastings House.
Averill, J., Nunley, E., & Tassinary, L. (1995). Voyages of the heart: Living an emotionally creative life. Contemporary Psychology, 40(6), 530.
Batteux, C. (1969). Les Beaux Arts Réduits à un meme Principe. Genève: Slatkine Reprints.
Birn, J. (2006). [digital] Lighting and rendering (2nd ed.). London: New Riders.
Block, B. (2001). The visual story: Seeing the structure of film, TV, and new media. Boston: Focal Press.
Blumberg, B., & Galyean, T. (1995). Multi-level direction of autonomous creatures for real-time virtual environments. Proceedings of SIGGRAPH'95, 30(3), 173–182.
Cassell, J. (2000). Nudge, nudge, wink, wink: Elements of face-to-face conversation for embodied conversational agents. In J. Cassell, J. Sullivan, and E. Churchill (Eds.), Embodied conversational agents (pp. 1–27). Cambridge, MA: MIT Press.
Chi, D., Costa, M., Zhao, L., et al. (2000). The EMOTE model for effort and shape. In Proceedings of SIGGRAPH 2000 (pp. 173–182).
Collingwood, R. (1938). The principles of art. Oxford, UK: Clarendon Press.
de Melo, C., & Paiva, A. (2005). Environment expression: Expressing emotions through cameras, lights and music. In Proceedings of Affective Computing Intelligent Interaction (ACII'05) (pp. 715–722).
de Melo, C., & Paiva, A. (2006a). Multimodal expression in virtual humans. Computer Animation and Virtual Worlds Journal, 17(3),215–220.
de Melo, C., & Paiva, A. (2006b) A story about gesticulation expression. In Proceedings of Intelligent Virtual Agents (IVA'06) (pp. 270–281).
de Melo, C., & Paiva, A. (2007). The expression of emotions in virtual humans using lights, shadows, composition and filters. In Proceedings of Affective Computing Intelligent Interaction (ACII'07) (pp. 546–557).
Dissanayake, E. (1988). What is art for? Seattle: University of Washington Press.
Ekman, P. (1999). Facial expressions. In: T. Dalgleish and M. Power (Eds.), Handbook of cognition and emotion. New York: John Wiley & Sons.
Elliot, R. (1966). Aesthetic theory and the experience of art paper. In Proceedings of the Aristotelian Society, NS 67, III–26.
Fraser, T., & Banks, A. (2004). Designer's color manual: The complete guide to color theory and application. San Francisco: Chronicle Books.
Geertz, C. (1976). Art as a cultural system. Modern Language Notes, 91(6), 1473–1499.
Gombrich, E. (1960). Art and illusion; A study in the psychology of pictorial representation. New York: Pantheon Books.
Gratch, J., Rickel, J., Andre, E., et al (2002). Creating interactive virtual humans: Some assembly required. IEEE Intelligent Systems, 17(4), 54–63.
Gross, L., & Ward, L. (2007). Digital moviemaking (6th ed.). Belmont, CA: Thomson/Wadsworth.
Hartmann, B., Mancini, A., & Pelachaud, C. (2005). Implementing expressive gesture synthesis for embodied conversational agents. In Proceedings of Gesture Workshop (pp. 173–182).
Hunt, R. (2004). The reproduction of colour. West Sussex, UK: John Wiley & Sons,
Juslin, P., & Sloboda, J. (2001). Music and emotion: Theory and research. New York: Oxford University Press.
Kant, I., & Bernard, J. (1951). Critique of judgment. New York: Hafner.
Katz, S. (1991). Film directing shot by shot: Visualizing from concept to screen. Studio City, CA: Focal Press.
Keltner, D., Ekman, P., Gonzaga, G., et al. (2003). Facial expression of emotion. In: R. Davidson, K. Scherer, & J. Goldsmith (Eds.), Handbook of affective sciences (pp. 415–433). New York: Oxford University Press.
Lewis, M. (2001). Aesthetic evolutionary design with data flow networks. PhD thesis, Ohio State University.
Machado, F. (2006). Inteligencia artificial e arte. PhD thesis, Universidade de Coimbra.
Malkiewicz, K., & Gryboski, B. (1986). Film lighting: Talks with Hollywood's cinematographers and gaffers. New York: Prentice Hall Press.
Malkiewicz, K., & Mullen, D. (2005). Cinematography: A guide for filmmakers and film teachers. New York: Simon & Schuster.
Mesquita, B. (2003). Emotions as dynamic cultural phenomena. In: R. Davidson, K. Scherer, & J. Goldsmith (Eds.), Handbook of affective sciences (pp. 871–890). New York: Oxford University Press.
Millerson, G. (1999). Lighting for television and film (3rd ed.). Oxford: Focal Press.
Mitchell, M. (1998). An introduction to genetic algorithms. Cambridge, MA: MIT Press.
Moller, T., & Haines, E. (2002). Real-time rendering (2nd ed.). Wellesley, MA: AK Peters.
Noh, J., & Neumann, U. (1998). A survey of facial modelling and animation techniques. Technical report, USC Technical Report: 99–705.
Oatley, K. (2003). Creative expression and communication of emotions in the visual and narrative arts. In: R. Davidson, K. Scherer, & J. Goldsmith (Eds.), Handbook of affective sciences (pp. 481–502). New York: Oxford University Press.
Ortony, A., Clore, G., & Collins, A. (1988). The cognitive structure of emotions. Cambridge, MA: Cambridge University Press.
Perlin, K., & Goldberg, A. (1996). Improv: A system for scripting interactive actors. Virtual worlds. In Proceedings of SIGGRAPH'96 (pp. 205–216).
Picard, R.W. (1997). Affective computing. Cambridge, MA: MIT Press.
Sayre, H. (2007). A world of art (5th ed.). Upper Saddle River, NJ: Prentice Hall.
Schroder, M. (2004). Speech and emotion research: An overview of research frameworks and a dimensional approach to emotional speech synthesis. PhD thesis, Institute of Phonetics, Saarland University.
Sims, K. (1991). Artificial evolution for computer graphics. ACM Computer Graphics, 25, 319– 328.
St-Laurent, S. (2004). Shaders for game programmers and artists. Boston, MA: Thomson/Course Technology.
Todd, S., & Latham, W. (1992) Evolutionary art and computers. San Diego, CA: Academic Press.
Ventrella, J. (1995). Disney meets Darwin—The evolution of funny animated figures. In IEEE Proceedings of Computer Animation (pp. 19–21).
World, L. (1996). Aesthetic selection: The evolutionary art of Steven Rooke. IEEE Computer Graphics and Applications, 16(1), 4–5.
Zettl, H. (2008). Sight, sound, motion: Applied media aesthetics (5th ed.). Belmont, CA: Thomson/ Wadsworth.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag London Limited
About this chapter
Cite this chapter
Melo, C.M.d., Paiva, A. (2009). Evolutionary Expression of Emotions in Virtual Humans Using Lights and Pixels. In: Tao, J., Tan, T. (eds) Affective Information Processing. Springer, London. https://doi.org/10.1007/978-1-84800-306-4_17
Download citation
DOI: https://doi.org/10.1007/978-1-84800-306-4_17
Publisher Name: Springer, London
Print ISBN: 978-1-84800-305-7
Online ISBN: 978-1-84800-306-4
eBook Packages: Computer ScienceComputer Science (R0)