Abstract
In a seminal paper, Ekman (1979) remarks that brows can play an accentuation role (e.g., to signal focus). However, the literature about eyebrows is inconclusive about their exact role and as a consequence there is no agreement among developers of embodied conversational agents about their precise timing and placement. In addition, it is unclear whether eyebrow movements perform the same role in different languages. In this chapter, an analysis-by-synthesis technique is used to find out what the role of eyebrow movements is for the perception of focus and to see whether this role is the same across different languages. Three experiments are performed, both for Dutch and Italian, investigating where subjects prefer eyebrow movements, whether brows influence the perceived prominence of words and whether they are used in a functional way when subjects interpret utterances. The results for Dutch and Italian are indeed different, but it is argued that these differences can be reduced to prosodic differences between the two languages. The advantages and potential limitations of studies via analysis-by-synthesis are discussed, and an approach to compensate for the limitations is offered.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bolinger, D. (1985). Intonation and its parts. Edward Arnold, London.
Birdwhistell, R. (1970). Kinesics and context. University of Pennsylvania Press.
Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors (2000). Embodied Conversational Agents. The MIT Press, Cambridge, MA.
Cassell, J., Vihjálmsson, H., and Bickmore, T. (2001). BEAT: The Behavior Expression Animation Toolkit. In Proceedings of SIG-GRAPH’01, pp. 477–486, Los Angeles.
Cavé, C., Guaïtella, I., Bertrand, R., Santi, S., Harlay, F., and Espesser, R. (1996). About the relationship between eyebrow movements and F 0 variations. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), pp. 2175–2179, Philadelphia.
Chafe, W. (1974). Language and consciousness. Language 50: 111–133.
Condon, W. (1976). An analysis of behavioral organization. Sign Language Studies, 13: 285–318.
Cutler, A. (1984). Stress and accent in language production and understanding. In Gibbon, D. and Richter, H., editors, Intonation, accent and rhythm. Studies in Discourse Phonology. pp. 77–90, de Gruyter, Berlin.
Cruttenden, A. (1997). Intonation, 2nd edition. Cambridge University Press, Cambridge.
Darwin, Ch. (1872). The Expression of the emotions in man and animals. Philosophical Library, New York.
Doherty-Sneddon, G., Bonner, L., and Bruce, V. (2001). Cognitive demands of face monitoring: Evidence for visuospatial overload. Memory and Cognition, 29(7): 909–919.
Efron, D. (1941). Gesture and environment. King’s Crown Press, New York.
Eibl-Eibesfelt, I. (1972). Similarities and differences between cultures in expressive movements. In Hinde, R., editor, Non-verbal communication. Cambridge University Press, Cambridge.
Ekman, P. (1979). About brows: Emotional and conversational signals. In von Cranach, M., Foppa, K., Lepenies, W. and Ploog, D., editors, Human ethology: Claims and limits of a new discipline. pp. 169–202, Cambridge University Press, Cambridge.
Ekman, P. and Friesen, W. (1978). Facial Action Coding System. Consulting Psychologists Press, Inc, Palo Alto.
Erickson, D., Fujimura, O., and Pardo, B. (1998). Articulatory correlates of prosodic control: Emotion and emphasis. Language and Speech, 41(3–4): 399–417.
Granström, B., House, D., and Lundeberg, M. (1999). Prosodic cues to multimodal speech perception. In Proceedings 14th International Conference of the Phonetic Sciences (ICPhS), pp. 655658, San Francisco.
Granström, B., House, D., and Swerts, M. (2002). Multimodal feedback cues in human-machine interactions. In Proceedings of Speech Prosody 2002. pp. 347–350, Aix en Provence, France.
Gratch, J., Rickel, J., André, E., Badler, N., Cassell, J., and Petajan, E. (2002). Creating interactive virtual humans: Some assembly required. IEEE Intelligent Systems, 17(4): 54–63.
Hirschberg, J. (1993). Pitch accents in context: predicting intonational prominence from text. Artificial Intelligence, 63: 305–340.
Keating, P., Baroni, M., Mattys, S., Scarborough, R., Alwan, A., Auer, E., and Berstein, L. (2003). Optical phonetics and visual perception of lexical and phrasal stress in English. In Proceedings 16th International Conference of the Phonetic Sciences (ICPhS), pp. 2071–2074, Barcelona, Spain.
Krahmer, E. and Swerts, M. (2001). On the alleged existence of contrastive accents, Speech Communication, 34: 391–405.
Krahmer, E., Ruttkay, Zs., Swerts, M., and Wesselink, W. (2002a). Pitch, eyebrows and the perception of focus. In Proceedings of Speech Prosody 2002, pp. 443–446, Aix en Provence, France.
Krahmer, E., Ruttkay, Zs., Swerts, M., and Wesselink, W. (2002b). Perceptual evaluation of audio-visual cues to prominence. In Proceedings of International Conference on Spoken Language Processing (IC-SLP’02), pp. 1933–1936, Denver, CO.
Ladd, D. (1996). Intonational phonology. Cambridge University Press, Cambridge.
van de Laar, L. (2003). Influence of eyes on the interpretation of utterances of embodied conversational agents: An experimental inquiry. MA thesis, Tilburg University.
Morgan, B. (1953). Question melodies in American English. American Speech, 2: 181–191.
Nass, C., Isbister, K., and Lee, E. (2000). Truth is beauty: Researching embodied conversational agents. In Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors, Embodied Conversational Agents. The MIT Press, Cambridge, MA.
O’Sullivan, M. and Eyman, J. (1978). The signal value of eyebrow movements in conversation. J. Western Psychological Association Convention. San Francisco.
Pelachaud, C., Badler, N., and Steedman, M. (1996). Generating facial expressions for speech. Cognitive Science, 20: 1–46.
Perlin, K. (1995). Real time responsive animation with personality. IEEE Transactions on Visualization and Computer Graphics, 1(1): 5–15.
Rimé, B. and Schiaratura, L. (1991). Gesture and speech. In Feldman, R. and Rimé, B., editors, Fundamentals of nonverbal behavior. pp. 239–281, Cambridge University Press, Cambridge.
Rubin, P. and Vatikiotis-Bateson, E. (1998). Talking heads. In Burnham, D., Robert-Ribes, J. and Vatikiotis-Bateson, E., editors, International Conference on Auditory-Visual Speech Processing (AVSP’98), pp. 233238, Sydney, Australia
Ruttkay, Zs. (2001). Constraint-based facial animation. Journal of Constraints, 6: 85–113
Ruttkay, Zs. and Noot, H. (2000). Animated CharToon Faces. In Proceedings of NPAR 2000 — First International Symposium on Non Photorealistic Animation and Rendering, pp. 91–100, Annecy, France.
Sanderman, A. and Collier, R. (1997). Prosodic phrasing and comprehension. Language and Speech, 40(4): 391–409.
Swerts, M., Krahmer, E., and Avesani, C. (2002). Prosodic marking of information status in Dutch and Italian: A comparative analysis. Journal of Phonetics, 30(4): 629–654.
Terken, J. (1984). The distribution of pitch accents in instructions as a function of discourse structure. Language and Speech, 27: 269–289.
Terken, J. and Nooteboom, S. (1987). Opposite effects of accentuation and deaccentuation on verification latencies for Given and New information. Language and Cognitive Processes, 2(3–4): 145–163.
Zappa, F. (1989). The Real Frank Zappa Book, Poseidon Press, New York.
Editor information
Rights and permissions
Copyright information
© 2004 Kluwer Academic Publishers
About this chapter
Cite this chapter
Krahmer, E., Swerts, M. (2004). More About Brows. In: Ruttkay, Z., Pelachaud, C. (eds) From Brows to Trust. Human-Computer Interaction Series, vol 7. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2730-3_7
Download citation
DOI: https://doi.org/10.1007/1-4020-2730-3_7
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-2729-1
Online ISBN: 978-1-4020-2730-7
eBook Packages: Computer ScienceComputer Science (R0)