short-paper

Neural networks for optical vector and eye ball parameter estimation

Authors:

Enkelejda KasneciAuthors Info & Claims

ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and Applications

Article No.: 4, Pages 1 - 5

https://doi.org/10.1145/3379156.3391346

Published: 02 June 2020 Publication History

Abstract

In this work we evaluate neural networks, support vector machines and decision trees for the regression of the center of the eyeball and the optical vector based on the pupil ellipse. In the evaluation we analyze single ellipses as well as window-based approaches as input. Comparisons are made regarding accuracy and runtime. The evaluation gives an overview of the general expected accuracy with different models and amounts of input ellipses. A simulator was implemented for the generation of the training and evaluation data. For a visual evaluation and to push the state of the art in optical vector estimation, the best model was applied to real data. This real data came from public data sets in which the ellipse is already annotated by an algorithm. The optical vectors on real data and the generator are made publicly available. Link to the generator and models.

References

[1]

AD Barsingerhorn, FN Boonstra, and HHLM Goossens. 2017. Optics of the human cornea influence the accuracy of stereo eye-tracking methods: a simulation study. Biomedical optics express 8, 2 (2017), 712–725.

[2]

Andreas Bulling and Thorsten O Zander. 2014. Cognition-aware computing. IEEE Pervasive Computing 13, 3 (2014), 80–83.

[3]

Nora Castner, Enkelejda Kasneci, Thomas Kübler, Katharina Scheiter, Juliane Richter, Thérése Eder, Fabian Hüttig, and Constanze Keutel. 2018. Scanpath comparison in medical image reading skills of dental students: distinguishing stages of expertise development. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. 1–9.

Digital Library

[4]

Jixu Chen and Qiang Ji. 2008. 3d gaze estimation with a single camera without ir illumination. In 2008 19th International Conference on Pattern Recognition. IEEE, 1–4.

[5]

Jixu Chen, Yan Tong, Wayne Gray, and Qiang Ji. 2008. A robust 3D eye gaze tracking system using noise reduction. In Proceedings of the 2008 symposium on Eye tracking research & applications. ACM, 189–196.

Digital Library

[6]

Kai Dierkes, Moritz Kassner, and Andreas Bulling. 2018. A novel approach to single camera, glint-free 3D eye model fitting including corneal refraction. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. ACM, 9.

Digital Library

[7]

Kai Dierkes, Moritz Kassner, and Andreas Bulling. 2019. A fast approach to refraction-aware 3D eye-model fitting and gaze prediction. In Proc. International Symposium on Eye Tracking Research and Applications (ETRA). https://doi.org/10.1145/3314111.3319819

Digital Library

[8]

Shahram Eivazi, Roman Bednarik, Ville Leinonen, Mikael von und zu Fraunberg, and Juha E Jääskeläinen. 2015. Embedding an eye tracker into a surgical microscope: Requirements, design, and implementation. IEEE Sensors Journal 16, 7 (2015), 2070–2078.

[9]

Tobias Fischer, Hyung Jin Chang, and Yiannis Demiris. 2018. Rt-gene: Real-time eye gaze estimation in natural environments. In Proceedings of the European Conference on Computer Vision (ECCV). 334–352.

Digital Library

[10]

Clinton Fookes, Anthony Maeder, Sridha Sridharan, and George Mamic. 2010. Gaze based personal identification. In Behavioral Biometrics for Human Identification: Intelligent Applications. IGI Global, 237–263.

[11]

F Dan Foresee and Martin T Hagan. 1997. Gauss-Newton approximation to Bayesian learning. In Proceedings of International Conference on Neural Networks (ICNN’97), Vol. 3. IEEE, 1930–1935.

[12]

Wolfgang Fuhl, David Geisler, Wolfgang Rosenstiel, and Enkelejda Kasneci. 2019a. The applicability of Cycle GANs for pupil and eyelid segmentation, data generation and image refinement. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 0–0.

[13]

Wolfgang Fuhl, Thomas Kübler, Katrin Sippel, Wolfgang Rosenstiel, and Enkelejda Kasneci. 2015. Excuse: Robust pupil detection in real-world scenarios. In International Conference on Computer Analysis of Images and Patterns. Springer, 39–51.

[14]

Wolfgang Fuhl, Wolfgang Rosenstiel, and Enkelejda Kasneci. 2019b. 500,000 images closer to eyelid and pupil segmentation. In International Conference on Computer Analysis of Images and Patterns. Springer, 336–347.

[15]

Wolfgang Fuhl, Thiago Santini, and Enkelejda Kasneci. 2017a. Fast and robust eyelid outline and aperture detection in real-world scenarios. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1089–1097.

[16]

Wolfgang Fuhl, Thiago Santini, and Enkelejda Kasneci. 2017b. Fast camera focus estimation for gaze-based focus control. arXiv preprint arXiv:1711.03306(2017).

[17]

Wolfgang Fuhl, Thiago Santini, Gjergji Kasneci, Wolfgang Rosenstiel, and Enkelejda Kasneci. 2017c. PupilNet v2. 0: Convolutional Neural Networks for CPU based real time Robust Pupil Detection. arXiv preprint arXiv:1711.00112(2017).

[18]

Wolfgang Fuhl, Thiago Santini, Carsten Reichert, Daniel Claus, Alois Herkommer, Hamed Bahmani, Katharina Rifai, Siegfried Wahl, and Enkelejda Kasneci. 2016b. Non-intrusive practitioner pupil detection for unmodified microscope oculars. Computers in biology and medicine 79 (2016), 36–44.

[19]

Wolfgang Fuhl, Thiago C Santini, Thomas Kübler, and Enkelejda Kasneci. 2016a. Else: Ellipse selection for robust pupil detection in real-world environments. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. ACM, 123–130.

Digital Library

[20]

Elias Daniel Guestrin and Moshe Eizenman. 2006. General theory of remote gaze estimation using the pupil center and corneal reflections. IEEE Transactions on biomedical engineering 53, 6 (2006), 1124–1133.

[21]

Martin T Hagan and Mohammad B Menhaj. 1994. Training feedforward networks with the Marquardt algorithm. IEEE transactions on Neural Networks 5, 6 (1994), 989–993.

[22]

Dan Witzner Hansen and Qiang Ji. 2009. In the eye of the beholder: A survey of models for eyes and gaze. IEEE transactions on pattern analysis and machine intelligence 32, 3(2009), 478–500.

[23]

Craig Hennessey, Borna Noureddin, and Peter Lawrence. 2006. A single camera eye-gaze tracking system with free head motion. In Proceedings of the 2006 symposium on Eye tracking research & applications. ACM, 87–94.

Digital Library

[24]

Chi Jian-nan, Zhang Chuang, Yan Yan-tao, Liu Yang, and Zhang Han. 2009. Eye gaze calculation based on nonlinear polynomial and generalized regression neural network. In 2009 Fifth International Conference on Natural Computation, Vol. 3. IEEE, 617–623.

[25]

Enkelejda Kasneci, Katrin Sippel, Kathrin Aehling, Martin Heister, Wolfgang Rosenstiel, Ulrich Schiefer, and Elena Papageorgiou. 2014. Driving with binocular visual field loss? A study on a supervised on-road parcours with simultaneous eye and head tracking. PloS one 9, 2 (2014), e87470.

[26]

Petr Kellnhofer, Adria Recasens, Simon Stent, Wojciech Matusik, and Antonio Torralba. 2019. Gaze360: Physically unconstrained gaze estimation in the wild. In Proceedings of the IEEE International Conference on Computer Vision. 6912–6921.

[27]

Jeff Klingner, Rakshit Kumar, and Pat Hanrahan. 2008. Measuring the task-evoked pupillary response with a remote eye tracker. In Proceedings of the 2008 symposium on Eye tracking research & applications. ACM, 69–72.

Digital Library

[28]

Stefan Kohlbecher, Stanislavs Bardinst, Klaus Bartl, Erich Schneider, Tony Poitschke, and Markus Ablassmeier. 2008. Calibration-free eye tracking by reconstruction of the pupil ellipse in 3D space. In Proceedings of the 2008 symposium on Eye tracking research & applications. ACM, 135–138.

Digital Library

[29]

Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, and Antonio Torralba. 2016. Eye tracking for everyone. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2176–2184.

[30]

Sofia Ira Ktena, William Abbott, and A Aldo Faisal. 2015. A virtual reality platform for safe evaluation and training of natural gaze-based wheelchair driving. In 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, 236–239.

[31]

Chih-Chuan Lai, Sheng-Wen Shih, and Yi-Ping Hung. 2014. Hybrid method for 3-D gaze tracking using glint and contour features. IEEE Transactions on Circuits and Systems for Video Technology 25, 1(2014), 24–37.

[32]

Jianfeng Li and Shigang Li. 2016. Two-phase approach—Calibration and iris contour estimation—For gaze tracking of head-mounted eye camera. In 2016 IEEE International Conference on Image Processing (ICIP). IEEE, 3136–3140.

[33]

Jianfeng Li, Shigang Li, Tong Chen, and Yiguang Liu. 2018. A Geometry-Appearance-Based Pupil Detection Method for Near-Infrared Head-Mounted Cameras. IEEE Access 6(2018), 23242–23252.

[34]

Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics (TOG) 35, 6 (2016), 179.

Digital Library

[35]

Alessandro Roncone, Ugo Pattacini, Giorgio Metta, and Lorenzo Natale. 2016. A Cartesian 6-DoF Gaze Controller for Humanoid Robots. In Robotics: science and systems, Vol. 2016.

[36]

Laura Sesma-Sanchez, Yanxia Zhang, Andreas Bulling, and Hans Gellersen. 2016. Gaussian processes as an alternative to polynomial gaze estimation functions. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. ACM, 229–232.

Digital Library

[37]

Sheng-Wen Shih and Jin Liu. 2004. A novel approach to 3-D gaze tracking using stereo cameras. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34, 1(2004), 234–245.

Digital Library

[38]

Yusuke Sugano and Andreas Bulling. 2015. Self-calibrating head-mounted eye trackers using egocentric visual saliency. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, 363–372.

Digital Library

[39]

Lech Swirski and Neil Dodgson. 2013. A fully-automatic, temporal approach to single camera, glint-free 3d eye model fitting. Proc. PETMEI (2013).

[40]

Marc Tonsen, Xucong Zhang, Yusuke Sugano, and Andreas Bulling. 2016. Labelled pupils in the wild: a dataset for studying pupil detection in unconstrained environments. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. ACM, 139–142.

Digital Library

[41]

Arantxa Villanueva, Rafael Cabeza, 2008. Evaluation of corneal refraction in a model of a gaze tracking system. IEEE Transactions on Biomedical Engineering 55, 12 (2008), 2812–2822.

[42]

Arantxa Villanueva, Gintautas Daunys, Dan Witzner Hansen, Martin Böhme, Rafael Cabeza, André Meyer, and Erhardt Barth. 2009. A geometric approach to remote eye tracking. Universal Access in the Information Society 8, 4 (2009), 241.

Digital Library

[43]

Jian-Gang Wang, Eric Sung, and Ronda Venkateswarlu. 2005. Estimating the eye gaze from one eye. Computer Vision and Image Understanding 98, 1 (2005), 83–103.

Digital Library

[44]

Yafei Wang, Tianyi Shen, Guoliang Yuan, Jiming Bian, and Xianping Fu. 2016. Appearance-based gaze estimation using deep features and random forest regression. Knowledge-Based Systems 110 (2016), 293–301.

Digital Library

[45]

Sunu Wibirama, Hanung A Nugroho, and Kazuhiko Hamamoto. 2017. Evaluating 3D gaze tracking in virtual space: A computer graphics approach. Entertainment computing 21 (2017), 11–17.

[46]

Erroll Wood, Tadas Baltrušaitis, Louis-Philippe Morency, Peter Robinson, and Andreas Bulling. 2016. Learning an appearance-based gaze estimator from one million synthesised images. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research and Applications. ACM, 131–138.

Digital Library

[47]

Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. 2015. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755(2015).

[48]

Xucong Zhang, Yusuke Sugano, and Andreas Bulling. 2019. Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 416.

Digital Library

[49]

Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2015. Appearance-based gaze estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4511–4520.

[50]

Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2017. It’s written all over your face: Full-face appearance-based gaze estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 51–60.

[51]

Yanxia Zhang, Jörg Müller, Ming Ki Chong, Andreas Bulling, and Hans Gellersen. 2014. GazeHorizon: enabling passers-by to interact with public displays by gaze. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 559–563.

Digital Library

[52]

Zhiwei Zhu, Qiang Ji, and Kristin P Bennett. 2006. Nonlinear eye gaze mapping function estimation via support vector regression. In 18th International Conference on Pattern Recognition (ICPR’06), Vol. 1. IEEE, 1132–1135.

Cited By

Fuhl W(2024)Zero Shot Learning in Pupil DetectionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655641(1-3)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655641
Fuhl WWeber DEivazi S(2024)Pistol: Pupil Invisible Supportive Tool in the WildSN Computer Science10.1007/s42979-024-02606-w5:3Online publication date: 21-Feb-2024
https://doi.org/10.1007/s42979-024-02606-w
Dai LLiu JJu ZGao Y(2022)Attention-Mechanism-Based Real-Time Gaze Tracking in Natural Scenes With Residual BlocksIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2021.306428014:2(696-707)Online publication date: Jun-2022
https://doi.org/10.1109/TCDS.2021.3064280
Show More Cited By

Neural networks for optical vector and eye ball parameter estimation
1. Computing methodologies

Recommendations

A Multimodal Eye Movement Dataset and a Multimodal Eye Movement Segmentation Analysis
ETRA '21 Short Papers: ACM Symposium on Eye Tracking Research and Applications

We present a new dataset with annotated eye movements. The dataset consists of over 800,000 gaze points recorded during a car ride in the real world and in the simulator. In total, the eye movements of 19 subjects were annotated. In this dataset, there ...
Eye&Head: Synergetic Eye and Head Movement for Gaze Pointing and Selection
UIST '19: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology

Eye gaze involves the coordination of eye and head movement to acquire gaze targets, but existing approaches to gaze pointing are based on eye-tracking in abstraction from head motion. We propose to leverage the synergetic movement of eye and head, and ...
Rule-based learning for eye movement type detection
MCPMD '18: Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data

Eye movements hold information about human perception, intention, and cognitive state. Various algorithms have been proposed to identify and distinguish eye movements, particularly fixations, saccades, and smooth pursuits. A major drawback of existing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and Applications

June 2020

305 pages

ISBN:9781450371346

DOI:10.1145/3379156

Editors:
Andreas Bulling
University of Stuttgart, Germany
,
Anke Huckauf
Ulm University, Germany
,
Eakta Jain
University of Florida, USA
,
Ralph Radach
University of Wuppertal, Germany
,
Daniel Weiskopf
University of Stuttgart, Germany

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

ETRA '20

Sponsor:

ETRA '20: 2020 Symposium on Eye Tracking Research and Applications

June 2 - 5, 2020

Stuttgart, Germany

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

ETRA '25

Sponsor:
sigchi
sigchi

2025 Symposium on Eye Tracking Research and Applications

May 26 - 29, 2025

Tokyo , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
438
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)3

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fuhl W(2024)Zero Shot Learning in Pupil DetectionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655641(1-3)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655641
Fuhl WWeber DEivazi S(2024)Pistol: Pupil Invisible Supportive Tool in the WildSN Computer Science10.1007/s42979-024-02606-w5:3Online publication date: 21-Feb-2024
https://doi.org/10.1007/s42979-024-02606-w
Dai LLiu JJu ZGao Y(2022)Attention-Mechanism-Based Real-Time Gaze Tracking in Natural Scenes With Residual BlocksIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2021.306428014:2(696-707)Online publication date: Jun-2022
https://doi.org/10.1109/TCDS.2021.3064280
Lu CChakravarthula PLiu KLiu XLi SFuchs H(2022)Neural 3D Gaze: 3D Pupil Localization and Gaze Tracking based on Anatomical Eye Model and Neural Refraction Correction2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00053(375-383)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00053
Fuhl WJasneci E(2022)Maximum and Leaky Maximum Propagation2022 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN55064.2022.9892955(1-8)Online publication date: 18-Jul-2022
https://doi.org/10.1109/IJCNN55064.2022.9892955
Palmero CSharma ABehrendt KKrishnakumar KKomogortsev OTalathi S(2021)OpenEDS2020 Challenge on Gaze Tracking for VR: Dataset and ResultsSensors10.3390/s2114476921:14(4769)Online publication date: 13-Jul-2021
https://doi.org/10.3390/s21144769
Martinez-Marquez DPingali SPanuwatwanich KStewart RMohamed S(2021)Application of Eye Tracking Technology in Aviation, Maritime, and Construction Industries: A Systematic ReviewSensors10.3390/s2113428921:13(4289)Online publication date: 23-Jun-2021
https://doi.org/10.3390/s21134289
Fuhl WKasneci E(2021)A Multimodal Eye Movement Dataset and a Multimodal Eye Movement Segmentation AnalysisACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3458004(1-7)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3458004
Kübler TFuhl WWagner EKasneci E(2021)55 Rides: attention annotated head and gaze data during naturalistic drivingACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3457993(1-8)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3457993
Fuhl WKasneci GKasneci E(2021)TEyeD: Over 20 Million Real-World Eye Images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR52148.2021.00053(367-375)Online publication date: Oct-2021
https://doi.org/10.1109/ISMAR52148.2021.00053
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten