skip to main content
10.1145/3317958.3319809acmconferencesArticle/Chapter ViewAbstractPublication PagesetraConference Proceedingsconference-collections
short-paper

Art facing science: Artistic heuristics for face detection: tracking gaze when looking at faces

Published: 25 June 2019 Publication History

Abstract

Automatic Area Of Interest (AOI) demarcation of facial regions is not yet commonplace in applied eye-tracking research, partially because automatic AOI labeling is prone to error. Most previous eye-tracking studies relied on manual frame-by-frame labeling of facial AOIs. We present a fully automatic approach for facial AOI labeling (i.e., eyes, nose, mouth) and gaze registration within those AOIs, based on modern computer vision techniques combined with heuristics drawn from art. We discuss details in computing gaze analytics, provide proof-of-concept, and a short validation against what we consider ground truth. Relative dwell time over expected AOIs exceeded 98% showing efficacy of the approach.

References

[1]
Auyeung, B., Lombardo, M. V., Heinrichs, M., Chakrabarti, B., Sule, A., Deakin, J. B., Bethlehem, R. A. I., Dickens, L., Mooney, N., Sipple, J. A. N., Thiemann, P., and Baron-Cohen, S. (2015). Oxytocin increases eye contact during a real-time, naturalistic social interaction in males with and without autism. Translational Psychiatry, 5(e507).
[2]
Brooker, S. (2010). Portrait Painting Atelier: OldMaster Techniques and Contemporary Applications. Watson-Guptill Publications, New York, NY. Castrillón-Santana, M., Déniz-Suárez, O., Antón-Canalis, L., and Lorenzo-Navarro, J. (2008). Face and Facial Feature Detection Evaluation: Performance Evaluation of Public Domain Haar Detectors for Face and Facial Feature Detection. In Ranchordas, A. and Araujo, H., editors, IIIInternational Conference on Computer Vision Theory and Applications, volume 2 of VISAPP, pages 167--172.
[3]
Chrysos, G. G., Antonakos, E., Snape, P., Asthana, A., and Zafeiriou, S. (2018). A Comprehensive Performance Evaluation of Deformable Face Tracking "In-the-Wild". International Journal of Computer Vision, 126(2):198--232.
[4]
Coutrot, A., Binetti, N., Harrison, C., Mareschal, I., and Johnston, A. (2016). Face exploration dynamics differentiate men and women. Journal of Vision, 16(14):1--19.
[5]
deMartin, J. (2016). Drawing Atelier: The Figure. North Light Books, Blue Ash, OH.
[6]
Duchowski, A. T. (2017). Eye Tracking Methodology: Theory & Practice. Springer-Verlag, Inc., London, UK, 3rd edition.
[7]
Freeth, M. and Bugembe, P. (2018). Social partner gaze direction and conversational phase; factors affecting social attention during face-to-face conversations in autistic adults. Autism, 23(2):503--513.
[8]
Freeth, M., Foulsham, T., and Kingstone, A. (2013). What Affects Social Attention? Social Presence, Eye Contact and Autistic Traits. PLOS ONE, 8(1): 1--10.
[9]
Gehrer, N. A., Schönenberg, M., Duchowski, A. T., and Krejtz, K. (2018). Implementing Innovative Gaze Analytic Methods in Clinical Psychology: A Study on Eye Movements in Antisocial Violent Offenders. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, ETRA '18, pages 41:1-41:9, New York, NY. ACM.
[10]
Gorry, P. A. (1990). General least-squares smoothing and differentiation by the convolution (Savitzky-Golay) method. Analytical Chemistry, 62(6):570--573.
[11]
Hanley, M., Riby, D. M., Carty, C., McAteer, A. M., Kennedy, A., and McPhillips, M. (2015). The use of eye-tracking to explore social difficulties in cognitively able students with autism spectrum disorder: A pilot investigation. Autism, 19(7):868--873.
[12]
Hanley, M., Riby, D. M., McCormack, T., Carty, C., Coyle, L., Crozier, N., Robinson, J., and McPhillips, M. (2014). Attention during social interaction in children with autism: Comparison to specific language impairment, typical development, and links to social cognition. Research in Autism Spectrum Disorders, 8(7):908--924.
[13]
Hessels, R. S., Benjamins, J. S., Cornelissen, T. H. W., and Hooge, I. T. C. (2018a). A Validation of Automatically-Generated Areas-of-Interest in Videos of a Face for Eye-Tracking Research. Frontiers in Psychology, 9(1367):1--8.
[14]
Hessels, R. S., Holleman, G. A., Cornelissen, T. H. W., Hooge, I. T. C., and Kemner, C. (2018b). Eye contact takes two - autistic and social anxiety traits predict gaze behavior in dyadic interaction. Journal of Experimental Psychopathology, 9(2):1--17.
[15]
Hessels, R. S., Holleman, G. A., Kingstone, A., Hooge, I. T. C., and Kemner, C. (2019). Gaze allocation in face-to-face communication is affected primarily by task structure and social context, not stimulus-driven factors. Cognition, 184:28--43.
[16]
Hessels, R. S., Kemner, C., van den Boomen, C., and Hooge, I. T. C. (2016). The area-of-interest problem in eyetracking research: A noise-robust solution for face and sparse stimuli. Behavior Research Methods, 48(4):1694--1712.
[17]
Kaehler, A. and Bardski, G. (2017). Learning OpenCV 3. O'Reilly Media Inc., Sebastopol, CA.
[18]
Kassner, M., Patera, W., and Bulling, A. (2014). Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, UbiComp '14 Adjunct, pages 1151--1160, New York, NY. ACM.
[19]
Mogilevtsev, V. A. (2016). Fundamentals of Drawing: Textbook. 4art Ltd, St. Petersburg, Russia.
[20]
Peterson, M. F. and Eckstein, M. P. (2012). Looking just below the eyes is optimal across face recognition tasks. Proceedings of the National Academy of Sciences, pages E3314--E3323.
[21]
Savitzky, A. and Golay, M. J. E. (1964). Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry, 36(8):1627--1639.
[22]
Schwaller, A. C. (2014). Combining eye- and head-tracking signals for improved event detection. Master's thesis, Lund University, Lund, Sweden.
[23]
Spezio, M. L., Huang, P.-Y. S., Castelli, F., and Adolphs, R. (2007). Amygdala Damage Impairs Eye Contact During Conversations with Real People. Journal of Neuroscience, 27(15):3994--3997.
[24]
Vabalas, A. and Freeth, M. (2016). Brief Report: Patterns of Eye Movements in Face to Face Conversation are Associated with Autistic Traits: Evidence from a Student Sample. Journal of Autism and Developmental Disorders, 46(1):305--314.
[25]
Viola, P. and Jones, M. (2001). Rapid Object Detection using a Boosted Cascade of Simple Features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 1 of CVPR, pages I--I.
[26]
Zollhöfer, M., Thies, J., Garrido, P., Bradley, D., Beeler, T., Pérez, P., Stamminger, M., Nießner, M., and Theobalt, C. (2018). State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications. Computer Graphics Forum, 37(2):523--550.

Cited By

View all
  • (2024)Implementing mobile eye tracking in psychological research: A practical guideBehavior Research Methods10.3758/s13428-024-02473-656:8(8269-8288)Online publication date: 15-Aug-2024
  • (2024)Assessment of body-related attention processes via mobile eye tracking: A pilot study to validate an automated analysis pipelineProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653354(1-7)Online publication date: 4-Jun-2024
  • (2024)Initial Investigations into Information Retention and Perception on Virtual Human Race2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00335(1078-1079)Online publication date: 16-Mar-2024
  • Show More Cited By

Index Terms

  1. Art facing science: Artistic heuristics for face detection: tracking gaze when looking at faces

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications
    June 2019
    623 pages
    ISBN:9781450367097
    DOI:10.1145/3314111
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 June 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. eye tracking
    2. face tracking
    3. gaze analytics

    Qualifiers

    • Short-paper

    Conference

    ETRA '19

    Acceptance Rates

    Overall Acceptance Rate 69 of 137 submissions, 50%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)36
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Implementing mobile eye tracking in psychological research: A practical guideBehavior Research Methods10.3758/s13428-024-02473-656:8(8269-8288)Online publication date: 15-Aug-2024
    • (2024)Assessment of body-related attention processes via mobile eye tracking: A pilot study to validate an automated analysis pipelineProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653354(1-7)Online publication date: 4-Jun-2024
    • (2024)Initial Investigations into Information Retention and Perception on Virtual Human Race2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00335(1078-1079)Online publication date: 16-Mar-2024
    • (2024)“The way I see it makes me believe you intentionally did it”: Intentionality ascription and gaze transition entropy in violent offendersBiological Psychology10.1016/j.biopsycho.2024.108962(108962)Online publication date: Dec-2024
    • (2023)Attributional and attentional patterns in the perception of ambiguous harmful encounters involving peer and authority figuresCurrent Issues in Personality Psychology10.5114/cipp/166751Online publication date: 21-Sep-2023
    • (2022)Automatic Generation of Customized Areas of Interest and Evaluation of Observers' Gaze in Portrait VideosProceedings of the ACM on Human-Computer Interaction10.1145/35308856:ETRA(1-14)Online publication date: 13-May-2022
    • (2022)Reduced attention toward faces, intentionality and blame ascription in violent offenders and community‐based adults: Evidence from an eye‐tracking studyAggressive Behavior10.1002/ab.2201848:2(264-274)Online publication date: 17-Jan-2022
    • (2021)Eye-tracking glasses in face-to-face interactions: Manual versus automated assessment of areas-of-interestBehavior Research Methods10.3758/s13428-021-01544-253:5(2037-2048)Online publication date: 19-Mar-2021

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media