ABSTRACT
This paper demonstrates the utility of ambient-focal attention and pupil dilation dynamics to describe visual processing of emotional facial expressions. Pupil dilation and focal eye movements reflect deeper cognitive processing and thus shed more light on the dynamics of emotional expression recognition. Socially anxious individuals (N = 24) and non-anxious controls (N = 24) were asked to recognize emotional facial expressions that gradually morphed from a neutral expression to one of happiness, sadness, or anger in 10-sec animations. Anxious cohorts exhibited more ambient face scanning than their non-anxious counterparts. We observed a positive relationship between focal fixations and pupil dilation, indicating deeper processing of viewed faces, but only by non-anxious participants, and only during the last phase of emotion recognition. Group differences in the dynamics of ambient-focal attention support the hypothesis of vigilance to emotional expression processing by socially anxious individuals. We discuss the results by referring to current literature on cognitive psychopathology.
- Ahern, S. and Beatty, J. (1979). Pupillary Responses During Information Processing Vary With Scholastic Aptitude Test Scores. Science, 205(4412):1289--1292.Google ScholarCross Ref
- Allison, T., Puce, A., and McCarthy, G. (2000). Social Perception from Visual Cues: Role of the STS Region. Trends in Cognitive Sciences, 4(7):267--278.Google ScholarCross Ref
- Anderson, A. K., Christoff, K., Panitz, D., De Rosa, E., and Gabrieli, J. D. (2003). Neural Correlates of the Automatic Processing of Threat Facial Signals. Journal of Neuroscience, 23(13):5627--5633.Google ScholarCross Ref
- Armstrong, T. and Olatunji, B. (2012). Eye Tracking of Attention in the Affective Disorders: a Meta-Analytic Review and Synthesis. Clinical Psychology Review, 32:704--723.Google ScholarCross Ref
- Bombari, D., Schmid, P. C., Mast, M. S., Birri, S., Mast, F. W., and Lobmaier, J. S. (2013). Emotion recognition: The role of featural and configural face information. Quarterly Journal of Experimental Psychology, 66(12):2426--2442. PMID: 23679155.Google ScholarCross Ref
- Bradley, M. M., Miccoli, L., Escrig, M. A., and Lang, P. J. (2008). The Pupil as a Measure of Emotional Arousal and Autonomic Activation. Psychophysiology, 45(4):602--607.Google ScholarCross Ref
- Chen, N. T., Clarke, P. J., MacLeod, C., and Guastella, A.J. (2012). Biased Attentional Processing of Positive Stimuli in Social Anxiety Disorder: an eye Movement Study. Cognitive Behaviour Therapy, 41(2):96--107.Google ScholarCross Ref
- Clark, D. M. and Wells, A. (1995). A Cognitive Model of Social Phobia. Social Phobia: Diagnosis, Assessment, and Treatment, 41(68):00022--3.Google Scholar
- Craig, B. M., Becker, S. I., and Lipp, O. V. (2014). Different Faces in the Crowd: a Happiness Superiority Effect for Schematic Faces in Heterogeneous Backgrounds. Emotion, 14(4):794--803.Google ScholarCross Ref
- Cunningham, D. W. and Wallraven, C. (2009). Dynamic Information for the Recognition of Conversational Expressions. Journal of Vision, 9(13):1--17.Google ScholarCross Ref
- de Gee, J. W., Knapen, T., and Donner, T. H. (2014). Decision-related Pupil Dilation Reflects Upcoming Choice and Individual Bias. Proceedings of the National Academy of Sciences, 111(5):E618--E625.Google ScholarCross Ref
- Delis, I., Chen, C., Jack, R. E., Garrod, O. G., Panzeri, S., and Schyns, P. G. (2016). Space-by-Time Manifold Representation of Dynamic Facial Expressions for Emotion Categorization. Journal of Vision, 16(8):14--14.Google ScholarCross Ref
- Duchowski, A. T., Krejtz, K., Krejtz, I., Biele, C., Niedzielska, A., Kiefer, P., Raubal, M., and Giannopoulos, I. (2018). The Index of Pupillary Activity. In Proceedings of ACM CHI 2018 Conference on Human Factors in Computing Systems, Montréal, QC, Canada. ACM Press. Google ScholarDigital Library
- Ekman, P. (1978). Facial Expression. Nonverbal Behavior and Communication, pages 97--116.Google Scholar
- Ekman, P. (1993). Facial expression and emotion. American psychologist, 48(4):384.Google Scholar
- Garner, M., Mogg, K., and Bradley, B. P. (2006). Orienting and Maintenance of Gaze to Facial Expressions in Social Anxiety. Journal of Abnormal Psychology, 115(4):760.Google ScholarCross Ref
- Haxby, J. F., Hoffman, E. A., and Gobbini, I. M. (2000). The Distributed Human Neural System for Face Perception. Trends in Cognitive Sciences, 4(6):223--233.Google ScholarCross Ref
- Heimberg, R. G., Brozovich, F. A., and Rapee, R. M. (2010). A Cognitive-behavioral Model of Social Anxiety Disorder: Update and Extension. Social Anxiety: Clinical, Developmental, and Social Perspectives, 2:395--422.Google Scholar
- Hess, E. H. and Polt, J. M. (1960). Pupil Size as Related to Interest Value of Visual Stimuli. Science, 132(3423):349--350.Google ScholarCross Ref
- Holas, P., Krejtz, I., Cypryańska, M., and Nezlek, J. B. (2014). Orienting and Maintenance of Attention to Threatening Facial Expressions in Anxiety - An Eye Movement Study. Psychiatry Research, 220:362--369.Google ScholarCross Ref
- Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., and van de Weijer, J. (2011). Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford University Press, Oxford, UK.Google Scholar
- Horley, K., Williams, L. M., Gonsalvez, C., and Gordon, E. (2003). Social Phobics do not see eye to eye: a Visual Scanpath Study of Emotional Expression Processing. Journal of Anxiety Disorders, 17(1):33--44.Google ScholarCross Ref
- Horley, K., Williams, L. M., Gonsalvez, C., and Gordon, E. (2004). Face to Face: Visual Scanpath Evidence for Abnormal Processing of Facial Expressions inSocial Phobia. Psychiatry Research, 127(1):43--53.Google ScholarCross Ref
- Horstmann, G. (2003). What do Facial Expressions Convey: Feeling States, Behavioral Intentions, or Actions Requests? Emotion, 3(2):150.Google ScholarCross Ref
- Hsiao, J. H. and Cottrell, G. (2008). Two Fixations Suffice in Face Recognition. Psychological Science, 19(10):998--1006.Google ScholarCross Ref
- Iacoboni, M., Molnar-Szakacs, I., Gallese, V., Buccino, G., Mazziotta, J. C., and Rizzolatti, G. (2005). Grasping the Intentions of Others with One's Own Mirror Neuron System. PLoS Biology, 3(3):e79.Google ScholarCross Ref
- Irwin, D. E. and Zelinsky, G.J. (2002). Eye Movements and Scene Perception: Memory for Things Observed. Attention, Perception, & Psychophysics, 64(6):882--895.Google ScholarCross Ref
- Jack, R. E., Garrod, O. G., and Schyns, P. G. (2014). Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time. Current Biology, 24(2):187--192.Google ScholarCross Ref
- Krejtz, I., Szarkowska, A., Krejtz, K., Walczak, A., and Duchowski, A. T. (2012). Audio Description as an Aural Guide of Children's Visual Attention: Evidence from an Eye-tracking Study. In ACM Proceedings of the Symposium on Eye TrackingResearch and Applications, ETRA '12, pages 99--106. Google ScholarDigital Library
- Krejtz, K., Çöltekin, A., Duchowski, A., and Niedzielska, A. (2017). Using Coefficient K to Distinguish Ambient/Focal Visual Attention During Map Viewing. Journal of Eye Movement Research, 10(2).Google ScholarCross Ref
- Krejtz, K., Duchowski, A., Krejtz, I., Szarkowska, A., and Kopacz, A. (2016). Discerning Ambient/Focal Attention with Coefficient K. ACM Transactions on Applied Perception, 13(3):11:1--11:20. Google ScholarDigital Library
- Kret, M. E., Stekelenburg, J. J., Roelofs, K., and De Gelder, B. (2013). Perception of Face and Body Expressions Using Electromyography, Pupillometry and Gaze Measures. Frontiers in Psychology, 4.Google Scholar
- Leyman, L., De Raedt, R., Vaeyens, R., and Philippaerts, R. M. (2011). Attention for Emotional Facial Expressions in Dysphoria: an Eye-Movement Registration Study. Cognition and Emotion, 25(1):111--120.Google ScholarCross Ref
- Liebowitz, M. R. (1987). Social Phobia. In D.F., K., editor, Anxiety, volume 22, pages 141--173. Karger Publishers.Google Scholar
- Oathes, D. J., Siegle, G. J., and Ray, W. J. (2011). Chronic worry and the temporal dynamics of emotional processing. Emotion, 11(1):101.Google ScholarCross Ref
- Olszanowski, M., Pochwatko, G., Kukliński, K., Ścibor-Rylski, M., Lewinski, P., and Ohme, R.K.(2015). Warsaw Set of Emotional Facial Expression Pictures: a Validation Study of Facial Display Photographs. Frontiers in Psychology, 5:1516.Google ScholarCross Ref
- Pannasch, S., Helmert, J. R., Roth, K., Herbold, A.-K., and Walter, H. (2008). Visual Fixation Durations and Saccade Amplitudes: Shifting Relationship in a Variety of Conditions. Journal of Eye Movement Research, 2(2).Google ScholarCross Ref
- Pineles, S. L. and Mineka, S. (2005). Attentional Biases to Internal and External Sources of Potential Threat in Social Anxiety. Journal of Abnormal Psychology, 114(2):314.Google ScholarCross Ref
- Price, R. B., Siegle, G. J., Silk, J. S., Ladouceur, C., McFarland, A., Dahl, R. E., and Ryan, N. D. (2013). Sustained neural alterations in anxious youth performing an attentional bias task: a pupilometry study. Depression and anxiety, 30(1):22--30.Google Scholar
- R Core Team (2017). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.Google Scholar
- Radloff, L. S. (1977). The CES-D Scale: A Self-Report Depression Scale for Research in the General Population. Applied Psychological Measurement, 1(3):385--401.Google ScholarCross Ref
- Rapee, R. M. and Heimberg, R. G. (1997). A Cognitive-behavioral Model of Anxiety in Social Phobia. Behaviour Research and Therapy, 35(8):741--756.Google ScholarCross Ref
- Recio, G., Sommer, W., and Schacht, A. (2011). Electrophysiological Correlates of Perceiving and Evaluating Static and Dynamic Facial Emotional Expressions. Brain Research, 1376:66--75.Google ScholarCross Ref
- Schofield, C. A., Inhoff, A., and Coles, M. (2013). Social Anxiety and Difficulty Disengaging Threat: Evidence from Eye-Tracking. Journal of Anxiety Disorders, 27:661--669.Google ScholarCross Ref
- Schofield, C. A., Johnson, A. L., Inhoff, A. W., and Coles, M. E. (2012). Social Anxiety and Difficulty Disengaging Threat: Evidence from Eye-Tracking. Cognition & Emotion, 26(2):300--311.Google ScholarCross Ref
- Schurgin, M. W., Nelson, J., Iida, S., Ohira, H., Chiao, J., and Franconeri, S. (2014). Eye Movements During Emotion Recognition in Faces. Journal of Vision, 14(13):1--16.Google ScholarCross Ref
- Schyns, P. G., Petro, L., and Smith, M. (2008). Transmission of Facial Expressions of Emotion Coevolved with Their Efficient Decoding in the Brain: Behavioral and Brain Evidence. PLoS One, 4(5):e5625.Google ScholarCross Ref
- Siegle, G. J., Steinhauer, S. R., Carter, C. S., Ramel, W., and Thase, M. E. (2003). Do the Seconds Turn Into Hours? Relationships Between Sustained Pupil Dilation in Response to Emotional Information and Self-reported Rumination. Cognitive Therapy and Research, 27(3):365--382.Google ScholarCross Ref
- Silvia, P. J., Allan, W. D., Beauchamp, D. L., Maschauer, E. L., and Workman, J. O. (2006). Biased Recognition of Happy Facial Expressions in Social Anxiety. Journal of Social and Clinical Psychology, 25(6):585--602.Google ScholarCross Ref
- Smith, M. L., Cottrell, G. W., Gosselin, F., and Schyns, P. G. (2005). Transmitting and Decoding Facial Expressions. Psychological Science, 16(3):184--189.Google ScholarCross Ref
- Staugaard, S. R. (2010). Threatening Faces and Social Anxiety: a Literature Review. Clinical Psychology Review, 30(6):669--690.Google ScholarCross Ref
- Torro-Alves, N., Bezerra, I. A. d. O., Rodrigues, M. R., Machado-de Sousa, J. P., Osório, F. d. L., Crippa, J. A., et al. (2016). Facial Emotion Recognition in Social Anxiety: the Influence of Dynamic Information. Psychology & Neuroscience, 9(1):1.Google ScholarCross Ref
- Unema, P. J. A., Pannasch, S., Joos, M., and Velichkovsky, B. M. (2005). Time Course of Information Processing During Scene Perception: The Relationship Between Saccade Amplitude and Fixation Duration. Visual Cognition, 12(3):473--494.Google ScholarCross Ref
- Urai, A. E., Braun, A., and Donner, T. H. (2017). Pupil-linked Arousal is Driven by Decision Uncertainty and Alters Serial Choice Bias. Nature Communications, 8.Google Scholar
- Van Kleef, G. A., Van Doorn, E. A., Heerdink, M. W., and Koning, L. F. (2011). Emotion is for Influence. European Review of Social Psychology, 22(1):114--163.Google ScholarCross Ref
- Vassallo, S., Cooper, S. L., and Douglas, J. M. (2009). Visual scanning in the recognition of facial affect: Is there an observer sex difference? Journal of Vision, 9(3):11.Google ScholarCross Ref
- Velichkovsky, B. M., Joos, M., Helmert, J. R., and Pannasch, S. (2005). Two Visual Systems and Their eye Movements: Evidence from Static and Dynamic Scene Perception. In Proceedings of the XXVII Conference of the Cognitive Science Society, pages 2283--2288. Mahwah, NJ.Google Scholar
- Waters, K. (1987). A Muscle Model for Animation Three-dimensional Facial Expression. SIGGRAPH Comput. Graph., 21(4):17--24. Google ScholarDigital Library
- Weierich, M. R., Treat, T. A., and Hollingworth, A. (2008). Theories and Measurement of Visual Attentional Processing in Anxiety. Cognition and Emotion, 22(6):985--1018.Google ScholarCross Ref
Index Terms
Dynamics of emotional facial expression recognition in individuals with social anxiety
Recommendations
Emotional facial expression interface: effects of displayed facial designs
Ergo'IA '10: Proceedings of the Ergonomie et Informatique Avancee ConferenceIn supporting the visual-mediated emotional recognition, little research has centred on analysing the effect of presenting different combinatorial facial designs. Moreover, in the theoretical field of emotions, research on the recognition of emotional ...
Emotional facial expression model building
Human face plays a crucial role in interpersonal communication. If we synthesize vivid expressional face in cyberspace, we could make the interaction between computer and human more natural and friendly. In this paper, we present a simple methodology ...
Expression-invariant face recognition by facial expression transformations
In this paper, we present a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. When a new face image with an arbitrary expression is ...
Comments