skip to main content
10.1145/3314111.3319819acmconferencesArticle/Chapter ViewAbstractPublication PagesetraConference Proceedingsconference-collections
research-article

A fast approach to refraction-aware eye-model fitting and gaze prediction

Published: 25 June 2019 Publication History

Abstract

By temporally integrating information about pupil contours extracted from eye images, model-based methods for glint-free gaze estimation can mitigate pupil detection noise. However, current approaches require time-consuming iterative solving of a nonlinear minimization problem to estimate key parameters, such as eyeball position. Based on the method presented by [Swirski and Dodgson 2013], we propose a novel approach to glint-free 3D eye-model fitting and gaze prediction using a single near-eye camera. By recasting model optimization as a least-squares intersection of lines, we make it amenable to a fast non-iterative solution. We further present a method for estimating deterministic refraction-correction functions from synthetic eye images and validate them on both synthetic and real eye images. We demonstrate the robustness of our method in the presence of pupil detection noise and show the benefit of temporal integration of pupil contour information on eyeball position and gaze estimation accuracy.

References

[1]
G. K. Aguirre. 2018. A model of the entrance pupil of the human eye. bioRxiv (2018).
[2]
A. D. Barsingerhorn, F. N. Boonstra, and H. H. L. M. Goossens. 2017. Optics of the Human Cornea Influence the Accuracy of Stereo Eye-Tracking Methods: a Simulation Study. Biomedical Optics Express 8, 2 (2017), 712--725.
[3]
I. Bekerman, P. Gottlieb, and M. Vaiman. 2014. Variations in Eyeball Diameters of the Healthy Adults. Journal of Ophthalmology 2014, 3 (2014).
[4]
J. Chen and Q. Ji. 2008. 3D Gaze Estimation With a Single Camera Without IR Illumination. In ICPR 2008. 19th International Conference on Pattern Recognition. IEEE, 1--4.
[5]
J. Chen, Y. Tong, W. Gray, and Q. Ji. 2008. A Robust 3D Eye Gaze Tracking System Using Noise Reduction. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (ETRA '08). ACM, New York, NY, USA, 189--196.
[6]
K. Dierkes, M. Kassner, and A. Bulling. 2018. A novel approach to single camera, glint-free 3D eye model fitting including corneal refraction. Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications - ETRA '18 9 (2018), 1--9.
[7]
C. Elmadjian, A. D. Tula, and C.H. Morimoto. 2018. 3D gaze estimation in the scene volume with a head-mounted eye tracker. COGAIN'18: Workshop on Communication by Gaze Interaction (2018).
[8]
C. Fedtke, F. Mann, and A. Ho. 2010. The Entrance Pupil of the Human Eye: a Three-Dimensional Model as a Function of Viewing Angle. Optics Express 18 (2010), 1--13.
[9]
W. Fuhl, T. Santini, G. Kasneci, W. Rosenstiel, and E. Kasneci. 2017. PupilNet v2.0: Convolutional Neural Networks for CPU Based Real Time Robust Pupil Detection. CoRR (2017). https://arxiv.org/abs/1711.00112
[10]
W. Fuhl, M. Tonsen, A. Bulling, and E. Kasneci. 2016. Pupil detection for head-mounted eye tracking in the wild: An evaluation of the state of the art. Springer Machine Vision and Applications 27, 8 (2016), 1275--1288.
[11]
J. Gehrmann and D. A. Atchison. 2018. Pupil shape as viewed along the horizontal visual field Ankit Mathur. 13, 2013 (2018), 1--8.
[12]
H. Gross (Ed.). 2008. Handbook of Optical Systems: Vol. 4 Survey of Optical Instruments. Wiley-VCH Verlag GmbH+Co. KGaA, Weinheim.
[13]
E. D. Guestrin and M. Eizenman. 2006. General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal Reflections. IEEE Transactions on Biomedical Engineering 53, 6 (2006), 1124--1133.
[14]
D. W. Hansen and Q. Ji. 2010. In the Eye of the Beholder: A Survey of Models for Eyes and Gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 3 (2010), 478--500.
[15]
C. Hennessey, B. Noureddin, and P. Lawrence. 2006. A Single Camera Eye-Gaze Tracking System With Free Head Motion. In Proceedings of the 2006 Symposium on Eye Tracking Research & Applications (ETRA '06). ACM, 87--94.
[16]
A.-H. Javadi, Z. Hakimi, M. Barati, V. Walsh, and L. Tcheang. 2015. SET: a Pupil Detection Method Using Sinusoidal Approximation. Frontiers in Neuroengineering 8 (2015).
[17]
M. Kassner, W. Patera, and A. Bulling. 2014. Pupil: an Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-Based Interaction. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. ACM, 1151--1160.
[18]
C.-C. Lai, S.-W. Shih, and Y.-P. Hung. 2015. Hybrid Method for 3-D Gaze Tracking Using Glint and Contour Features. IEEE Transactions on Circuits and Systems for Video Technology 25, 1 (2015), 24--37.
[19]
Y. Le Grand. 1957. Light, Color and Vision. Wiley, New York, NY, USA.
[20]
J. Li and S. Li. 2016. Two phase approach - Calibration and iris contour estimation - For gaze tracking of head-mounted eye camera. 2016 IEEE International Conference on Image Processing (ICIP) (2016), 3136--3140.
[21]
J. Li, S. Li, T. Chen, and Y. Liu. 2018. A Geometry-Appearance-Based Pupil Detection Method for Near-Infrared Head-Mounted Cameras. IEEE Access 6 (2018), 23242--23252.
[22]
M. Mansouryar, J. Steil, Y. Sugano, and A. Bulling. 2016. 3D Gaze Estimation from 2D Pupil Positions on Monocular Head-Mounted Eye Trackers. In Proc. of the 9th ACM International Symposium on Eye Tracking Research & Applications (ETRA). 197--200.
[23]
A. Mayberry, P. Hu, B. Marlin, C. Salthouse, and D. Ganesan. 2014. iShadow: Design of a Wearable, Real-time Mobile Gaze Tracker. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services. 82--94.
[24]
T. Ohno, N. Mukawa, and A. Yoshikawa. 2002. FreeGaze: A Gaze Tracking System for Everyday Gaze Interaction. Proceedings of the 2002 Symposium on Eye Tracking Research & Applications (ETRA '02) (2002), 1--8.
[25]
R. Safaee-Rad, I. Tchoukanov, K. C. Smith, and B. Banhabib. 1992. Three-Dimensional Location Estimation of Circular Features for Machine Vision. IEEE Transactions on Robotics and Automation 8, 5 (1992), 624--640.
[26]
T. Santini, W. Fuhl, and E. Kasneci. 2018. PuRe: Robust Pupil Detection for Real-Time Pervasive Eye Tracking. Computer Vision and Image Understanding (2018).
[27]
S. W. Shih and J. Liu. 2004. A Novel Approach to 3-D Gaze Tracking Using Stereo Cameras. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) 34, 1 (2004), 234--245.
[28]
Y. Sugano and A. Bulling. 2015. Self-calibrating Head-mounted Eye Trackers Using Egocentric Visual Saliency. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST). 363--372.
[29]
L. Świrski, A. Bulling, and N. Dodgson. 2012. Robust Real-Time Pupil Tracking in Highly Off-Axis Images. In Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA '12). 173--176.
[30]
L. Świrski and N. A. Dodgson. 2013. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013. http://www.cl.cam.ac.uk/research/rainbow/projects/eyemodelfit/
[31]
M. Tonsen, J. Steil, Y. Sugano, and A. Bulling. 2017. InvisibleEye: Mobile Eye Tracking Using Multiple Low-Resolution Cameras and Learning-Based Gaze Estimation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (2017).
[32]
M. Tonsen, X. Zhang, Y. Sugano, and A. Bulling. 2016. Labelled pupils in the wild: A dataset for studying pupil detection in unconstrained environments. In Proc. of the 9th ACM International Symposium on Eye Tracking Research & Applications (ETRA). 139--142.
[33]
A. Tsukada and T. Kanade. 2012. Automatic Acquisition of a 3D Eye Model for a Wearable First-Person Vision Device. In Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA '12).
[34]
A. Tsukada, M. Shino, M. Devyver, and T. Kanade. 2011. Illumination-Free Gaze Estimation Method for First-Person Vision Wearable Device. IEEE International Conference on Computer Vision Workshops (Nov. 2011), 2085--2091.
[35]
A. Villanueva and R. Cabeza. 2008. Evaluation of Corneal Refraction in a Model of a Gaze Tracking System. IEEE Transactions on Biomedical Engineering 55, 12 (2008), 2812--2822.
[36]
J.-G. Wang, E. Sung, and R. Venkateswarlu. 2003. Eye Gaze Estimation from a Single Image of One Eye. Proceedings of the Ninth IEEE Internatinal Conference on Computer Vision (ICCV) (2003), 1--8.

Cited By

View all
  • (2024)The Impact of Pupil Constriction on the Relationship Between Melanopic EDI and Melatonin Suppression in Young Adult MalesJournal of Biological Rhythms10.1177/0748730424122646639:3(282-294)Online publication date: 13-Feb-2024
  • (2024)EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event SlicingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997458:4(1-32)Online publication date: 21-Nov-2024
  • (2024)Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual RealityProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36547057:2(1-16)Online publication date: 17-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications
June 2019
623 pages
ISBN:9781450367097
DOI:10.1145/3314111
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. 3D eye model
  2. contour-based
  3. eye tracking
  4. glint-free
  5. pupil detection
  6. refraction

Qualifiers

  • Research-article

Conference

ETRA '19

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)88
  • Downloads (Last 6 weeks)17
Reflects downloads up to 12 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)The Impact of Pupil Constriction on the Relationship Between Melanopic EDI and Melatonin Suppression in Young Adult MalesJournal of Biological Rhythms10.1177/0748730424122646639:3(282-294)Online publication date: 13-Feb-2024
  • (2024)EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event SlicingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997458:4(1-32)Online publication date: 21-Nov-2024
  • (2024)Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual RealityProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36547057:2(1-16)Online publication date: 17-May-2024
  • (2024)Low-Latency Ocular Parallax Rendering and Investigation of Its Effect on Depth Perception in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337207830:5(2228-2238)Online publication date: 5-Mar-2024
  • (2024)E-Gaze: Gaze Estimation With Event CameraIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.335960646:7(4796-4811)Online publication date: Jul-2024
  • (2024)Wearable Eye-Tracking System for Synchronized Multimodal Data AcquisitionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.333281434:6(5146-5159)Online publication date: Jun-2024
  • (2024)Pupillary responses to directional uncertainty while intercepting a moving targetRoyal Society Open Science10.1098/rsos.24060611:10Online publication date: 2-Oct-2024
  • (2023)Noise estimation for head-mounted 3D binocular eye tracking using Pupil Core eye-tracking gogglesBehavior Research Methods10.3758/s13428-023-02150-056:1(53-79)Online publication date: 27-Jun-2023
  • (2023)A novel end-to-end dual-camera system for eye gaze synchrony assessment in face-to-face interactionAttention, Perception, & Psychophysics10.3758/s13414-023-02679-486:7(2221-2230)Online publication date: 26-Apr-2023
  • (2023)An Automatic Calibration Method for Kappa Angle Based on a Binocular Gaze ConstraintSensors10.3390/s2308392923:8(3929)Online publication date: 12-Apr-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media