skip to main content
10.1145/3379156.3391354acmconferencesArticle/Chapter ViewAbstractPublication PagesetraConference Proceedingsconference-collections
short-paper

EyeLinks: Methods to compute reliable stereo mappings used for eye gaze tracking

Published: 02 June 2020 Publication History

Abstract

We present methods for extracting corneal images and estimating pupil centers continuously and reliably using head worn glasses that consists of two eye cameras. An existing CNN was modified for detecting pupils in IR and RGB images, and stereo vision together with 2D and 3D models are used. We confirm the feasibility of the proposed methods through user study results, which show that the methods can be used in future real gaze estimation systems.

References

[1]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. (2015). https://www.tensorflow.org/ Software available from tensorflow.org.
[2]
Muhammad Arsalan, Hyung Gil Hong, Rizwan Ali Naqvi, Min Beom Lee, Min Cheol Kim, Dong Seop Kim, Chan Sik Kim, and Kang Ryoung Park. 2017. Deep learning-based iris segmentation for iris recognition in visible light environment. Symmetry 9, 11 (2017), 263.
[3]
Inessa Bekerman, Paul Gottlieb, and Michael Vaiman. 2014. Variations in eyeball diameters of the healthy adults. Journal of ophthalmology 2014, Article 503645(2014), 5 pages.
[4]
Warapon Chinsatit and Takeshi Saitoh. 2017. CNN-based pupil center detection for wearable gaze estimation system. Applied Computational Intelligence and Soft Computing 2017, Article 718956(2017), 10 pages.
[5]
HSM Coxeter. 1969. Introduction to Geometry John Wiley & Sons. Inc., New York 1(1969), 969.
[6]
Wolfgang Fuhl, Thomas Kübler, Katrin Sippel, Wolfgang Rosenstiel, and Enkelejda Kasneci. 2015. Excuse: Robust pupil detection in real-world scenarios. In International Conference on Computer Analysis of Images and Patterns. 39–51.
[7]
Wolfgang Fuhl, Thiago C. Santini, Thomas Kübler, and Enkelejda Kasneci. 2016. ElSe: Ellipse Selection for Robust Pupil Detection in Real-world Environments. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications(ETRA ’16). ACM, New York, NY, USA, 123–130. https://doi.org/10.1145/2857491.2857505
[8]
Frank R Hampel, Elvezio M Ronchetti, Peter J Rousseeuw, and Werner A Stahel. 2011. Robust statistics: the approach based on influence functions. Vol. 196. John Wiley & Sons.
[9]
Dan Witzner Hansen and Qiang Ji. 2010. In the Eye of the Beholder: A Survey of Models for Eyes and Gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32, 3 (March 2010), 478–500. https://doi.org/10.1109/TPAMI.2009.30
[10]
Robert J. K. Jacob. 1990. What You Look at is What You Get: Eye Movement-Based Interaction Techniques. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’90). ACM, New York, NY, USA, 11–18. https://doi.org/10.1145/97243.97246
[11]
Moritz Kassner, William Patera, and Andreas Bulling. 2014. Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction. In Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing: Adjunct publication. 1151–1160.
[12]
Pupil Labs. 2020 (accessed February 3, 2020). Pupil Core. https://pupil-labs.com/products/core/
[13]
Christian Lander, Markus löchtefeld, and Antonio Krüger. 2018. hEYEbrid: A Hybrid Approach for Mobile Calibration-free Gaze Estimation. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 149 (Jan. 2018), 29 pages. https://doi.org/10.1145/3161166
[14]
Libor Masek 2003. Recognition of human iris patterns for biometric identification. Master’s thesis. University of Western Australia.
[15]
Moayad Mokatren, Tsvi Kuflik, and Ilan Shimshoni. 2018. Exploring the potential of a mobile eye tracker as an intuitive indoor pointing device: A case study in cultural heritage. Future Generation Computer Systems 81 (2018), 528–541.
[16]
Atsushi Nakazawa, Christian Nitschke, and Toyoaki Nishida. 2015. Non-Calibrated and real-time Human View Estimation Using a Mobile Corneal Imaging Camera. In Multimedia & Expo Workshops (ICMEW), 2015 IEEE International Conference on. IEEE, 1–6. https://doi.org/10.1109/ICMEW.2015.7169846
[17]
Rizwan Naqvi, Muhammad Arsalan, Ganbayar Batchuluun, Hyo Yoon, and Kang Park. 2018. Deep learning-based gaze detection system for automobile drivers using a NIR camera sensor. Sensors 18, 2 (2018), 456.
[18]
Ko Nishino and Shree K Nayar. 2004. The World in an Eye. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1. IEEE, I–444. https://doi.org/10.1109/CVPR.2004.248
[19]
Ko Nishino and Shree K. Nayar. 2006. Corneal Imaging System: Environment from Eyes. Int. J. Comput. Vision 70, 1 (Oct. 2006), 23–40. https://doi.org/10.1007/s11263-006-6274-9
[20]
Christian Nitschke, Atsushi Nakazawa, and Toyoaki Nishida. 2013. I See What You See: Point of Gaze Estimation from Corneal Images. In Proceedings of the 2013 IAPR Asian Conference on Pattern Recognition(ACPR ’13). IEEE Computer Society, Washington, DC, USA, 298–304. https://doi.org/10.1109/ACPR.2013.84
[21]
Franco P Preparata and Michael I Shamos. 2012. Computational geometry: an introduction. Springer Science & Business Media.
[22]
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems. 91–99.
[23]
Thiago Santini, Diederick C Niehorster, and Enkelejda Kasneci. 2019. Get a grip: slippage-robust and glint-free gaze estimation for real-time pervasive head-mounted eye tracking. In Proceedings of the 11th ACM symposium on eye tracking research & applications. 1–10.
[24]
J. Steil and A. Bulling. 2015. Discovery of Everyday Human Activities From Long-term Visual Behaviour Using Topic Models. In Proc. UbiComp 2015. 75–85. https://doi.org/10.1145/2750858.2807520
[25]
Charles V Stewart. 1999. Robust parameter estimation in computer vision. SIAM review 41, 3 (1999), 513–537.
[26]
Mingxing Tan, Ruoming Pang, and Quoc V Le. 2019. Efficientdet: Scalable and efficient object detection. arXiv preprint arXiv:1911.09070(2019).
[27]
Tobii. 2020 (accessed February 3, 2020). Tobii Pro Glasses 2. https://www.tobiipro.com/product-listing/tobii-pro-glasses-2/
[28]
Etienne Vincent and Robert Laganiére. 2001. Detecting planar homographies in an image pair. In ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat. IEEE, 182–187.
[29]
Zijing Wan and Xiangjun Wang. 2017. A method of gaze estimation for wearable multi-camera devices using stereo vision. In AOPC 2017: Optical Sensing and Imaging Technology and Applications, Vol. 10462. International Society for Optics and Photonics, 104620G.
[30]
Zhengyou Zhang. 2000. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence 22, 11(2000), 1330–1334.
[31]
Zijing Zhao and Kumar Ajay. 2015. An accurate iris segmentation framework under relaxed imaging constraints using total variation model. In Proceedings of the IEEE International Conference on Computer Vision. 3828–3836.

Cited By

View all
  • (2022)3D Gaze Estimation Using RGB-IR CamerasSensors10.3390/s2301038123:1(381)Online publication date: 29-Dec-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and Applications
June 2020
305 pages
ISBN:9781450371346
DOI:10.1145/3379156
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Corneal Imaging
  2. Gaze Estimation
  3. Mapping Transformation

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Funding Sources

Conference

ETRA '20

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)3D Gaze Estimation Using RGB-IR CamerasSensors10.3390/s2301038123:1(381)Online publication date: 29-Dec-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media