skip to main content
10.1145/3204493.3204533acmconferencesArticle/Chapter ViewAbstractPublication PagesetraConference Proceedingsconference-collections
research-article

Dynamics of emotional facial expression recognition in individuals with social anxiety

Published:14 June 2018Publication History

ABSTRACT

This paper demonstrates the utility of ambient-focal attention and pupil dilation dynamics to describe visual processing of emotional facial expressions. Pupil dilation and focal eye movements reflect deeper cognitive processing and thus shed more light on the dynamics of emotional expression recognition. Socially anxious individuals (N = 24) and non-anxious controls (N = 24) were asked to recognize emotional facial expressions that gradually morphed from a neutral expression to one of happiness, sadness, or anger in 10-sec animations. Anxious cohorts exhibited more ambient face scanning than their non-anxious counterparts. We observed a positive relationship between focal fixations and pupil dilation, indicating deeper processing of viewed faces, but only by non-anxious participants, and only during the last phase of emotion recognition. Group differences in the dynamics of ambient-focal attention support the hypothesis of vigilance to emotional expression processing by socially anxious individuals. We discuss the results by referring to current literature on cognitive psychopathology.

References

  1. Ahern, S. and Beatty, J. (1979). Pupillary Responses During Information Processing Vary With Scholastic Aptitude Test Scores. Science, 205(4412):1289--1292.Google ScholarGoogle ScholarCross RefCross Ref
  2. Allison, T., Puce, A., and McCarthy, G. (2000). Social Perception from Visual Cues: Role of the STS Region. Trends in Cognitive Sciences, 4(7):267--278.Google ScholarGoogle ScholarCross RefCross Ref
  3. Anderson, A. K., Christoff, K., Panitz, D., De Rosa, E., and Gabrieli, J. D. (2003). Neural Correlates of the Automatic Processing of Threat Facial Signals. Journal of Neuroscience, 23(13):5627--5633.Google ScholarGoogle ScholarCross RefCross Ref
  4. Armstrong, T. and Olatunji, B. (2012). Eye Tracking of Attention in the Affective Disorders: a Meta-Analytic Review and Synthesis. Clinical Psychology Review, 32:704--723.Google ScholarGoogle ScholarCross RefCross Ref
  5. Bombari, D., Schmid, P. C., Mast, M. S., Birri, S., Mast, F. W., and Lobmaier, J. S. (2013). Emotion recognition: The role of featural and configural face information. Quarterly Journal of Experimental Psychology, 66(12):2426--2442. PMID: 23679155.Google ScholarGoogle ScholarCross RefCross Ref
  6. Bradley, M. M., Miccoli, L., Escrig, M. A., and Lang, P. J. (2008). The Pupil as a Measure of Emotional Arousal and Autonomic Activation. Psychophysiology, 45(4):602--607.Google ScholarGoogle ScholarCross RefCross Ref
  7. Chen, N. T., Clarke, P. J., MacLeod, C., and Guastella, A.J. (2012). Biased Attentional Processing of Positive Stimuli in Social Anxiety Disorder: an eye Movement Study. Cognitive Behaviour Therapy, 41(2):96--107.Google ScholarGoogle ScholarCross RefCross Ref
  8. Clark, D. M. and Wells, A. (1995). A Cognitive Model of Social Phobia. Social Phobia: Diagnosis, Assessment, and Treatment, 41(68):00022--3.Google ScholarGoogle Scholar
  9. Craig, B. M., Becker, S. I., and Lipp, O. V. (2014). Different Faces in the Crowd: a Happiness Superiority Effect for Schematic Faces in Heterogeneous Backgrounds. Emotion, 14(4):794--803.Google ScholarGoogle ScholarCross RefCross Ref
  10. Cunningham, D. W. and Wallraven, C. (2009). Dynamic Information for the Recognition of Conversational Expressions. Journal of Vision, 9(13):1--17.Google ScholarGoogle ScholarCross RefCross Ref
  11. de Gee, J. W., Knapen, T., and Donner, T. H. (2014). Decision-related Pupil Dilation Reflects Upcoming Choice and Individual Bias. Proceedings of the National Academy of Sciences, 111(5):E618--E625.Google ScholarGoogle ScholarCross RefCross Ref
  12. Delis, I., Chen, C., Jack, R. E., Garrod, O. G., Panzeri, S., and Schyns, P. G. (2016). Space-by-Time Manifold Representation of Dynamic Facial Expressions for Emotion Categorization. Journal of Vision, 16(8):14--14.Google ScholarGoogle ScholarCross RefCross Ref
  13. Duchowski, A. T., Krejtz, K., Krejtz, I., Biele, C., Niedzielska, A., Kiefer, P., Raubal, M., and Giannopoulos, I. (2018). The Index of Pupillary Activity. In Proceedings of ACM CHI 2018 Conference on Human Factors in Computing Systems, Montréal, QC, Canada. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ekman, P. (1978). Facial Expression. Nonverbal Behavior and Communication, pages 97--116.Google ScholarGoogle Scholar
  15. Ekman, P. (1993). Facial expression and emotion. American psychologist, 48(4):384.Google ScholarGoogle Scholar
  16. Garner, M., Mogg, K., and Bradley, B. P. (2006). Orienting and Maintenance of Gaze to Facial Expressions in Social Anxiety. Journal of Abnormal Psychology, 115(4):760.Google ScholarGoogle ScholarCross RefCross Ref
  17. Haxby, J. F., Hoffman, E. A., and Gobbini, I. M. (2000). The Distributed Human Neural System for Face Perception. Trends in Cognitive Sciences, 4(6):223--233.Google ScholarGoogle ScholarCross RefCross Ref
  18. Heimberg, R. G., Brozovich, F. A., and Rapee, R. M. (2010). A Cognitive-behavioral Model of Social Anxiety Disorder: Update and Extension. Social Anxiety: Clinical, Developmental, and Social Perspectives, 2:395--422.Google ScholarGoogle Scholar
  19. Hess, E. H. and Polt, J. M. (1960). Pupil Size as Related to Interest Value of Visual Stimuli. Science, 132(3423):349--350.Google ScholarGoogle ScholarCross RefCross Ref
  20. Holas, P., Krejtz, I., Cypryańska, M., and Nezlek, J. B. (2014). Orienting and Maintenance of Attention to Threatening Facial Expressions in Anxiety - An Eye Movement Study. Psychiatry Research, 220:362--369.Google ScholarGoogle ScholarCross RefCross Ref
  21. Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., and van de Weijer, J. (2011). Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford University Press, Oxford, UK.Google ScholarGoogle Scholar
  22. Horley, K., Williams, L. M., Gonsalvez, C., and Gordon, E. (2003). Social Phobics do not see eye to eye: a Visual Scanpath Study of Emotional Expression Processing. Journal of Anxiety Disorders, 17(1):33--44.Google ScholarGoogle ScholarCross RefCross Ref
  23. Horley, K., Williams, L. M., Gonsalvez, C., and Gordon, E. (2004). Face to Face: Visual Scanpath Evidence for Abnormal Processing of Facial Expressions inSocial Phobia. Psychiatry Research, 127(1):43--53.Google ScholarGoogle ScholarCross RefCross Ref
  24. Horstmann, G. (2003). What do Facial Expressions Convey: Feeling States, Behavioral Intentions, or Actions Requests? Emotion, 3(2):150.Google ScholarGoogle ScholarCross RefCross Ref
  25. Hsiao, J. H. and Cottrell, G. (2008). Two Fixations Suffice in Face Recognition. Psychological Science, 19(10):998--1006.Google ScholarGoogle ScholarCross RefCross Ref
  26. Iacoboni, M., Molnar-Szakacs, I., Gallese, V., Buccino, G., Mazziotta, J. C., and Rizzolatti, G. (2005). Grasping the Intentions of Others with One's Own Mirror Neuron System. PLoS Biology, 3(3):e79.Google ScholarGoogle ScholarCross RefCross Ref
  27. Irwin, D. E. and Zelinsky, G.J. (2002). Eye Movements and Scene Perception: Memory for Things Observed. Attention, Perception, & Psychophysics, 64(6):882--895.Google ScholarGoogle ScholarCross RefCross Ref
  28. Jack, R. E., Garrod, O. G., and Schyns, P. G. (2014). Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time. Current Biology, 24(2):187--192.Google ScholarGoogle ScholarCross RefCross Ref
  29. Krejtz, I., Szarkowska, A., Krejtz, K., Walczak, A., and Duchowski, A. T. (2012). Audio Description as an Aural Guide of Children's Visual Attention: Evidence from an Eye-tracking Study. In ACM Proceedings of the Symposium on Eye TrackingResearch and Applications, ETRA '12, pages 99--106. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Krejtz, K., Çöltekin, A., Duchowski, A., and Niedzielska, A. (2017). Using Coefficient K to Distinguish Ambient/Focal Visual Attention During Map Viewing. Journal of Eye Movement Research, 10(2).Google ScholarGoogle ScholarCross RefCross Ref
  31. Krejtz, K., Duchowski, A., Krejtz, I., Szarkowska, A., and Kopacz, A. (2016). Discerning Ambient/Focal Attention with Coefficient K. ACM Transactions on Applied Perception, 13(3):11:1--11:20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kret, M. E., Stekelenburg, J. J., Roelofs, K., and De Gelder, B. (2013). Perception of Face and Body Expressions Using Electromyography, Pupillometry and Gaze Measures. Frontiers in Psychology, 4.Google ScholarGoogle Scholar
  33. Leyman, L., De Raedt, R., Vaeyens, R., and Philippaerts, R. M. (2011). Attention for Emotional Facial Expressions in Dysphoria: an Eye-Movement Registration Study. Cognition and Emotion, 25(1):111--120.Google ScholarGoogle ScholarCross RefCross Ref
  34. Liebowitz, M. R. (1987). Social Phobia. In D.F., K., editor, Anxiety, volume 22, pages 141--173. Karger Publishers.Google ScholarGoogle Scholar
  35. Oathes, D. J., Siegle, G. J., and Ray, W. J. (2011). Chronic worry and the temporal dynamics of emotional processing. Emotion, 11(1):101.Google ScholarGoogle ScholarCross RefCross Ref
  36. Olszanowski, M., Pochwatko, G., Kukliński, K., Ścibor-Rylski, M., Lewinski, P., and Ohme, R.K.(2015). Warsaw Set of Emotional Facial Expression Pictures: a Validation Study of Facial Display Photographs. Frontiers in Psychology, 5:1516.Google ScholarGoogle ScholarCross RefCross Ref
  37. Pannasch, S., Helmert, J. R., Roth, K., Herbold, A.-K., and Walter, H. (2008). Visual Fixation Durations and Saccade Amplitudes: Shifting Relationship in a Variety of Conditions. Journal of Eye Movement Research, 2(2).Google ScholarGoogle ScholarCross RefCross Ref
  38. Pineles, S. L. and Mineka, S. (2005). Attentional Biases to Internal and External Sources of Potential Threat in Social Anxiety. Journal of Abnormal Psychology, 114(2):314.Google ScholarGoogle ScholarCross RefCross Ref
  39. Price, R. B., Siegle, G. J., Silk, J. S., Ladouceur, C., McFarland, A., Dahl, R. E., and Ryan, N. D. (2013). Sustained neural alterations in anxious youth performing an attentional bias task: a pupilometry study. Depression and anxiety, 30(1):22--30.Google ScholarGoogle Scholar
  40. R Core Team (2017). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.Google ScholarGoogle Scholar
  41. Radloff, L. S. (1977). The CES-D Scale: A Self-Report Depression Scale for Research in the General Population. Applied Psychological Measurement, 1(3):385--401.Google ScholarGoogle ScholarCross RefCross Ref
  42. Rapee, R. M. and Heimberg, R. G. (1997). A Cognitive-behavioral Model of Anxiety in Social Phobia. Behaviour Research and Therapy, 35(8):741--756.Google ScholarGoogle ScholarCross RefCross Ref
  43. Recio, G., Sommer, W., and Schacht, A. (2011). Electrophysiological Correlates of Perceiving and Evaluating Static and Dynamic Facial Emotional Expressions. Brain Research, 1376:66--75.Google ScholarGoogle ScholarCross RefCross Ref
  44. Schofield, C. A., Inhoff, A., and Coles, M. (2013). Social Anxiety and Difficulty Disengaging Threat: Evidence from Eye-Tracking. Journal of Anxiety Disorders, 27:661--669.Google ScholarGoogle ScholarCross RefCross Ref
  45. Schofield, C. A., Johnson, A. L., Inhoff, A. W., and Coles, M. E. (2012). Social Anxiety and Difficulty Disengaging Threat: Evidence from Eye-Tracking. Cognition & Emotion, 26(2):300--311.Google ScholarGoogle ScholarCross RefCross Ref
  46. Schurgin, M. W., Nelson, J., Iida, S., Ohira, H., Chiao, J., and Franconeri, S. (2014). Eye Movements During Emotion Recognition in Faces. Journal of Vision, 14(13):1--16.Google ScholarGoogle ScholarCross RefCross Ref
  47. Schyns, P. G., Petro, L., and Smith, M. (2008). Transmission of Facial Expressions of Emotion Coevolved with Their Efficient Decoding in the Brain: Behavioral and Brain Evidence. PLoS One, 4(5):e5625.Google ScholarGoogle ScholarCross RefCross Ref
  48. Siegle, G. J., Steinhauer, S. R., Carter, C. S., Ramel, W., and Thase, M. E. (2003). Do the Seconds Turn Into Hours? Relationships Between Sustained Pupil Dilation in Response to Emotional Information and Self-reported Rumination. Cognitive Therapy and Research, 27(3):365--382.Google ScholarGoogle ScholarCross RefCross Ref
  49. Silvia, P. J., Allan, W. D., Beauchamp, D. L., Maschauer, E. L., and Workman, J. O. (2006). Biased Recognition of Happy Facial Expressions in Social Anxiety. Journal of Social and Clinical Psychology, 25(6):585--602.Google ScholarGoogle ScholarCross RefCross Ref
  50. Smith, M. L., Cottrell, G. W., Gosselin, F., and Schyns, P. G. (2005). Transmitting and Decoding Facial Expressions. Psychological Science, 16(3):184--189.Google ScholarGoogle ScholarCross RefCross Ref
  51. Staugaard, S. R. (2010). Threatening Faces and Social Anxiety: a Literature Review. Clinical Psychology Review, 30(6):669--690.Google ScholarGoogle ScholarCross RefCross Ref
  52. Torro-Alves, N., Bezerra, I. A. d. O., Rodrigues, M. R., Machado-de Sousa, J. P., Osório, F. d. L., Crippa, J. A., et al. (2016). Facial Emotion Recognition in Social Anxiety: the Influence of Dynamic Information. Psychology & Neuroscience, 9(1):1.Google ScholarGoogle ScholarCross RefCross Ref
  53. Unema, P. J. A., Pannasch, S., Joos, M., and Velichkovsky, B. M. (2005). Time Course of Information Processing During Scene Perception: The Relationship Between Saccade Amplitude and Fixation Duration. Visual Cognition, 12(3):473--494.Google ScholarGoogle ScholarCross RefCross Ref
  54. Urai, A. E., Braun, A., and Donner, T. H. (2017). Pupil-linked Arousal is Driven by Decision Uncertainty and Alters Serial Choice Bias. Nature Communications, 8.Google ScholarGoogle Scholar
  55. Van Kleef, G. A., Van Doorn, E. A., Heerdink, M. W., and Koning, L. F. (2011). Emotion is for Influence. European Review of Social Psychology, 22(1):114--163.Google ScholarGoogle ScholarCross RefCross Ref
  56. Vassallo, S., Cooper, S. L., and Douglas, J. M. (2009). Visual scanning in the recognition of facial affect: Is there an observer sex difference? Journal of Vision, 9(3):11.Google ScholarGoogle ScholarCross RefCross Ref
  57. Velichkovsky, B. M., Joos, M., Helmert, J. R., and Pannasch, S. (2005). Two Visual Systems and Their eye Movements: Evidence from Static and Dynamic Scene Perception. In Proceedings of the XXVII Conference of the Cognitive Science Society, pages 2283--2288. Mahwah, NJ.Google ScholarGoogle Scholar
  58. Waters, K. (1987). A Muscle Model for Animation Three-dimensional Facial Expression. SIGGRAPH Comput. Graph., 21(4):17--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Weierich, M. R., Treat, T. A., and Hollingworth, A. (2008). Theories and Measurement of Visual Attentional Processing in Anxiety. Cognition and Emotion, 22(6):985--1018.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Dynamics of emotional facial expression recognition in individuals with social anxiety

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      ETRA '18: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications
      June 2018
      595 pages
      ISBN:9781450357067
      DOI:10.1145/3204493

      Copyright © 2018 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 14 June 2018

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate69of137submissions,50%

      Upcoming Conference

      ETRA '24
      The 2024 Symposium on Eye Tracking Research and Applications
      June 4 - 7, 2024
      Glasgow , United Kingdom

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader