Skip to main content

More About Brows

A Cross-Linguistic Study via Analysis-by-Synthesis

  • Chapter
From Brows to Trust

Part of the book series: Human-Computer Interaction Series ((HCIS,volume 7))

  • 290 Accesses

Abstract

In a seminal paper, Ekman (1979) remarks that brows can play an accentuation role (e.g., to signal focus). However, the literature about eyebrows is inconclusive about their exact role and as a consequence there is no agreement among developers of embodied conversational agents about their precise timing and placement. In addition, it is unclear whether eyebrow movements perform the same role in different languages. In this chapter, an analysis-by-synthesis technique is used to find out what the role of eyebrow movements is for the perception of focus and to see whether this role is the same across different languages. Three experiments are performed, both for Dutch and Italian, investigating where subjects prefer eyebrow movements, whether brows influence the perceived prominence of words and whether they are used in a functional way when subjects interpret utterances. The results for Dutch and Italian are indeed different, but it is argued that these differences can be reduced to prosodic differences between the two languages. The advantages and potential limitations of studies via analysis-by-synthesis are discussed, and an approach to compensate for the limitations is offered.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  • Bolinger, D. (1985). Intonation and its parts. Edward Arnold, London.

    Google Scholar 

  • Birdwhistell, R. (1970). Kinesics and context. University of Pennsylvania Press.

    Google Scholar 

  • Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors (2000). Embodied Conversational Agents. The MIT Press, Cambridge, MA.

    Google Scholar 

  • Cassell, J., Vihjálmsson, H., and Bickmore, T. (2001). BEAT: The Behavior Expression Animation Toolkit. In Proceedings of SIG-GRAPH’01, pp. 477–486, Los Angeles.

    Google Scholar 

  • Cavé, C., Guaïtella, I., Bertrand, R., Santi, S., Harlay, F., and Espesser, R. (1996). About the relationship between eyebrow movements and F 0 variations. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), pp. 2175–2179, Philadelphia.

    Google Scholar 

  • Chafe, W. (1974). Language and consciousness. Language 50: 111–133.

    Article  Google Scholar 

  • Condon, W. (1976). An analysis of behavioral organization. Sign Language Studies, 13: 285–318.

    Article  Google Scholar 

  • Cutler, A. (1984). Stress and accent in language production and understanding. In Gibbon, D. and Richter, H., editors, Intonation, accent and rhythm. Studies in Discourse Phonology. pp. 77–90, de Gruyter, Berlin.

    Google Scholar 

  • Cruttenden, A. (1997). Intonation, 2nd edition. Cambridge University Press, Cambridge.

    Book  Google Scholar 

  • Darwin, Ch. (1872). The Expression of the emotions in man and animals. Philosophical Library, New York.

    Book  Google Scholar 

  • Doherty-Sneddon, G., Bonner, L., and Bruce, V. (2001). Cognitive demands of face monitoring: Evidence for visuospatial overload. Memory and Cognition, 29(7): 909–919.

    Article  Google Scholar 

  • Efron, D. (1941). Gesture and environment. King’s Crown Press, New York.

    Google Scholar 

  • Eibl-Eibesfelt, I. (1972). Similarities and differences between cultures in expressive movements. In Hinde, R., editor, Non-verbal communication. Cambridge University Press, Cambridge.

    Google Scholar 

  • Ekman, P. (1979). About brows: Emotional and conversational signals. In von Cranach, M., Foppa, K., Lepenies, W. and Ploog, D., editors, Human ethology: Claims and limits of a new discipline. pp. 169–202, Cambridge University Press, Cambridge.

    Google Scholar 

  • Ekman, P. and Friesen, W. (1978). Facial Action Coding System. Consulting Psychologists Press, Inc, Palo Alto.

    Google Scholar 

  • Erickson, D., Fujimura, O., and Pardo, B. (1998). Articulatory correlates of prosodic control: Emotion and emphasis. Language and Speech, 41(3–4): 399–417.

    Google Scholar 

  • Granström, B., House, D., and Lundeberg, M. (1999). Prosodic cues to multimodal speech perception. In Proceedings 14th International Conference of the Phonetic Sciences (ICPhS), pp. 655658, San Francisco.

    Google Scholar 

  • Granström, B., House, D., and Swerts, M. (2002). Multimodal feedback cues in human-machine interactions. In Proceedings of Speech Prosody 2002. pp. 347–350, Aix en Provence, France.

    Google Scholar 

  • Gratch, J., Rickel, J., André, E., Badler, N., Cassell, J., and Petajan, E. (2002). Creating interactive virtual humans: Some assembly required. IEEE Intelligent Systems, 17(4): 54–63.

    Article  Google Scholar 

  • Hirschberg, J. (1993). Pitch accents in context: predicting intonational prominence from text. Artificial Intelligence, 63: 305–340.

    Article  Google Scholar 

  • Keating, P., Baroni, M., Mattys, S., Scarborough, R., Alwan, A., Auer, E., and Berstein, L. (2003). Optical phonetics and visual perception of lexical and phrasal stress in English. In Proceedings 16th International Conference of the Phonetic Sciences (ICPhS), pp. 2071–2074, Barcelona, Spain.

    Google Scholar 

  • Krahmer, E. and Swerts, M. (2001). On the alleged existence of contrastive accents, Speech Communication, 34: 391–405.

    Article  MATH  Google Scholar 

  • Krahmer, E., Ruttkay, Zs., Swerts, M., and Wesselink, W. (2002a). Pitch, eyebrows and the perception of focus. In Proceedings of Speech Prosody 2002, pp. 443–446, Aix en Provence, France.

    Google Scholar 

  • Krahmer, E., Ruttkay, Zs., Swerts, M., and Wesselink, W. (2002b). Perceptual evaluation of audio-visual cues to prominence. In Proceedings of International Conference on Spoken Language Processing (IC-SLP’02), pp. 1933–1936, Denver, CO.

    Google Scholar 

  • Ladd, D. (1996). Intonational phonology. Cambridge University Press, Cambridge.

    Google Scholar 

  • van de Laar, L. (2003). Influence of eyes on the interpretation of utterances of embodied conversational agents: An experimental inquiry. MA thesis, Tilburg University.

    Google Scholar 

  • Morgan, B. (1953). Question melodies in American English. American Speech, 2: 181–191.

    Article  Google Scholar 

  • Nass, C., Isbister, K., and Lee, E. (2000). Truth is beauty: Researching embodied conversational agents. In Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors, Embodied Conversational Agents. The MIT Press, Cambridge, MA.

    Google Scholar 

  • O’Sullivan, M. and Eyman, J. (1978). The signal value of eyebrow movements in conversation. J. Western Psychological Association Convention. San Francisco.

    Google Scholar 

  • Pelachaud, C., Badler, N., and Steedman, M. (1996). Generating facial expressions for speech. Cognitive Science, 20: 1–46.

    Article  Google Scholar 

  • Perlin, K. (1995). Real time responsive animation with personality. IEEE Transactions on Visualization and Computer Graphics, 1(1): 5–15.

    Article  Google Scholar 

  • Rimé, B. and Schiaratura, L. (1991). Gesture and speech. In Feldman, R. and Rimé, B., editors, Fundamentals of nonverbal behavior. pp. 239–281, Cambridge University Press, Cambridge.

    Google Scholar 

  • Rubin, P. and Vatikiotis-Bateson, E. (1998). Talking heads. In Burnham, D., Robert-Ribes, J. and Vatikiotis-Bateson, E., editors, International Conference on Auditory-Visual Speech Processing (AVSP’98), pp. 233238, Sydney, Australia

    Google Scholar 

  • Ruttkay, Zs. (2001). Constraint-based facial animation. Journal of Constraints, 6: 85–113

    Article  MATH  Google Scholar 

  • Ruttkay, Zs. and Noot, H. (2000). Animated CharToon Faces. In Proceedings of NPAR 2000 — First International Symposium on Non Photorealistic Animation and Rendering, pp. 91–100, Annecy, France.

    Google Scholar 

  • Sanderman, A. and Collier, R. (1997). Prosodic phrasing and comprehension. Language and Speech, 40(4): 391–409.

    Google Scholar 

  • Swerts, M., Krahmer, E., and Avesani, C. (2002). Prosodic marking of information status in Dutch and Italian: A comparative analysis. Journal of Phonetics, 30(4): 629–654.

    Article  Google Scholar 

  • Terken, J. (1984). The distribution of pitch accents in instructions as a function of discourse structure. Language and Speech, 27: 269–289.

    Google Scholar 

  • Terken, J. and Nooteboom, S. (1987). Opposite effects of accentuation and deaccentuation on verification latencies for Given and New information. Language and Cognitive Processes, 2(3–4): 145–163.

    Article  Google Scholar 

  • Zappa, F. (1989). The Real Frank Zappa Book, Poseidon Press, New York.

    Google Scholar 

Download references

Authors

Editor information

Zsófia Ruttkay Catherine Pelachaud

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Kluwer Academic Publishers

About this chapter

Cite this chapter

Krahmer, E., Swerts, M. (2004). More About Brows. In: Ruttkay, Z., Pelachaud, C. (eds) From Brows to Trust. Human-Computer Interaction Series, vol 7. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2730-3_7

Download citation

  • DOI: https://doi.org/10.1007/1-4020-2730-3_7

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-1-4020-2729-1

  • Online ISBN: 978-1-4020-2730-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics