Skip to main content
SpringerLink
Log in

A real-time high precision eye center localizer

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Precise eye center localization remains a very promising but challenging task, while its real-time performance constitutes a critical constraint in many human interaction applications. In this paper a new hybrid framework that combines the shape-based Modified Fast Radial Symmetry Transform (MFRST) and a Convolutional Neural Network (CNN), is introduced. The motivation of this work is to exploit the circularity of the iris to reduce the search space and consequently, the computational complexity of the fed CNN. Thus, the proposed hybrid scheme not only achieves real-time performance, but also increases substantially the localization accuracy by reducing the false detections of the MFRST. The experimental results that stemmed from the most challenging face databases demonstrated high accuracy, outperforming state of the art techniques even those that are based on end-to-end deep neural networks. To deal with unreliable data and provide valid evaluation, we manually annotated the FERET database, making the annotations publicly available. Moreover, the reduced computational time of the proposed scheme reveals that it can be incorporated in low-cost eye trackers, where the real-time performance is a basic prerequisite.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://xanthippi.ceid.upatras.gr/people/psarakis/publications/feret-ground-truth.zip.

References

  1. Kar, A., Corcoran, P.: A review and analysis of eye-gaze estimation systems, algorithms and performance evaluation methods in consumer platforms. IEEE Access 5, 16495–16519 (2017)

    Article  Google Scholar 

  2. Krafka, K., Khosla, A., Kellnhofer, P., Kannan, H., Bhandarkar, S., Matusik, W., Torralba, A.: Eye tracking for everyone. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2176–2184 (2016)

  3. Poulopoulos, N., Psarakis, E. Z.: Real time eye localization and tracking. In: Proceedings of the 27th International Conference on Robotics in Alpe-Adria Danube Region, RAAD, Patras (2018)

  4. Ian, G., Yoshua, B., Aaron, C.: Deep learning. MIT Press, London (2016)

    MATH  Google Scholar 

  5. Ghosh, S., Srinivasa, S. K. K., Amon, P., Hutter, A., Kaup, A.: Deep network pruning for object detection. In: IEEE International Conference on Image Processing (ICIP), pp. 3915–3919 (2019)

  6. Poulopoulos, N., Psarakis, E. Z.: A new high precision eye center localization technique. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 2806–2810 (2017)

  7. Dobes, M., Martinek, J., Skoupil, D., Dobesova, Z., Pospisil, J.: Human eye localization using the modified Hough transform. Opt. Int. J. Light Electron Opt. 117(10), 468–473 (2006)

    Article  Google Scholar 

  8. Valenti, R., Gevers, T.: Accurate eye center location through invariant isocentric patterns. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1785–1798 (2012)

    Article  Google Scholar 

  9. Xia, Y., et al.: Hybrid regression and isophote curvature for accurate eye center localization. Multimed. Tools Appl. 79, 805–824 (2020)

    Article  Google Scholar 

  10. Loy, G., Zelinsky, A.: Fast radial symmetry for detecting points of interest. IEEE Trans. Pattern Anal. Mach. Intell. 25, 959–973 (2003)

    Article  Google Scholar 

  11. Bai, L., Shen, L., Wang, Y.: A novel eye location algorithm based on radial symmetry transform. IEEE Int. Conf. Pattern Recognit. 3, 511–514 (2006)

    Google Scholar 

  12. Yang, P., Du, B., Shan, S., Gao, W.: A novel pupil localization method based on GaborEye model and radial symmetry operator. IEEE Int. Conf. Image Process. 1, 67–70 (2004)

    Google Scholar 

  13. Skodras, E., Fakotakis, N.: Precise localization of eye centers in low resolution color images. J. Image Vis. Comput. 31, 51–60 (2015)

    Article  Google Scholar 

  14. Niu, Z., Shan, S., Yan, S., Chen, X., Gao, W.: 2D cascaded adaboost for eye localization. In: IEEE International Conference on Pattern Recognition (ICPR’06) (2006)

  15. Campadelli, P., Lanzarotti, R., Lipori, G.: Precise eye localization through a general-to- specific model definition. BMVC 1, 187–196 (2006)

    MATH  Google Scholar 

  16. Everingham, M., Zisserman, A.: Regression and classification approaches to eye localization in face images. In: IEEE 7th International Conference on Automatic Face and Gesture Recognition, pp. 441–446 (2006)

  17. Shaoqing, R., Kaiming, H., Ross, G., Jian, S.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp. 91–99. Springer, Berlin (2015)

    Google Scholar 

  18. Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement, arXiv:1804.02767 (2018)

  19. Hong, D., Gao, L., Yokoya, N., Yao, J., Chanussot, J., Du, Q., Zhang, B.: More diverse means better: Multimodal deep learning meets remote-sensing imagery classification. IEEE Trans. Geosci. Remote Sens. 59, 4340–4354 (2021)

    Article  Google Scholar 

  20. Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., Chanussot, J.: Graph convolutional networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 59, 5966–5978 (2021)

    Article  Google Scholar 

  21. Chinsatit, W., Saitoh, T.: CNN-based pupil center detection for wearable gaze estimation system, applied computational intelligence and soft computing, pp. 1–10. Springer, Berlin (2017)

    Google Scholar 

  22. Fuhl, W., Santini, T., Kasneci, G., Kasneci, E.: PupilNet: convolutional neural networks for robust pupil detection, arXiv:1601.04902 (2016)

  23. Li, B., Fu, H.: Real time eye detector with cascaded convolutional neural networks. Appl. Comput. Intell. Soft Comput. (2018). https://doi.org/10.1155/2018/1439312

    Article  Google Scholar 

  24. Xia, Y., Yu, H., Wang, F.: Accurate and robust eye center localization via fully convolutional networks. IEEE/CAA J. Autom. Sin. 6, 1127–1138 (2019)

    Article  Google Scholar 

  25. Choi, J.H., Lee, K.I., Song, B.C.: Eye pupil localization algorithm using convolutional neural networks. J. Multimed. Tools Appl. 79, 32563–32574 (2020)

    Article  Google Scholar 

  26. Lee, K.I., Jeon, J.H., Song B.C.: Deep learning based pupil center detection for fast and accurate eye tracking system. In: Springer European Conference on Computer Vision (ECCV), pp. 1127–1138 (2020)

  27. Gou, C., Zhang, K., Wang, K., Wang, F., Ji, Q.: Cascade learning from adversarial synthetic images for accurate pupil detection. J. Pattern Recognit. 88, 584–594 (2019)

    Article  Google Scholar 

  28. Larumbe-Bergera, A., Garde, G., Porta, S., Cabeza, R., Villanueva, A.: Accurate pupil center detection in off-the-shelf eye tracking systems using convolutional neural networks. Sensors 21, 6847 (2021)

    Article  Google Scholar 

  29. Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004)

    Article  Google Scholar 

  30. Wang, H., Li, S.Z., Wang, Y., Zhang, J.: Self quotient image for face recognition. ICIP 2, 1397–1400 (2004)

    Google Scholar 

  31. Jesorsky, O., Kirchbergand, K. J., Frischholz, R.: Robust face detection using the hausdorff distance. In: Audio and Video Biom. Pers. Authentication, pp. 90–95 (2001)

  32. Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: IEEE International Conference on Learning Representations (2015)

  33. Milborrow, S., Morkel, J., Nicolls, F.: The MUCT landmarked face database. Pattern Recognit. Assoc. S. Afr. 201, 20 (2010)

    Google Scholar 

  34. Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 22(10), 1090–1104 (2000)

    Article  Google Scholar 

  35. Krizhevsky, A., Sutskever, I., Hinton, G.: imagenet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)

  36. Timm, F.E.: Barth, accurate eye center localization by means of gradients, VISAPP, pp. 125–130 (2011)

  37. Ahmed, N.Y.: Real-time accurate eye center localization for low-resolution grayscale images. J. Real-Time Image Process. 18(1), 193–220 (2021)

    Article  Google Scholar 

  38. Ahmed, M., Laskar, R.H.: Eye center localization in a facial image based on geometric shapes of iris and eyelid under natural variability. Elsevier J. Image Vis. Comput. 88, 52–66 (2019)

    Article  Google Scholar 

  39. Xiao, F., Huang, K., Qiu, Y.H.: Shen, Accurate iris center localization method using facial landmark, snakuscule, circle fitting and binary connected component. J. Multimed. Tools Appl. 77, 25333–25353 (2018)

    Article  Google Scholar 

  40. Wang, Z., Cai, H., Liu, H.: Based, robust eye center localization on an improved SVR method. In: Neural information processing. ICONIP, lecture notes in computer science, 11307th edn. Springer, Berlin (2018)

    Google Scholar 

  41. Araujo, G., Ribeiro, F., Junior, W., da Silva, E., Goldenstein, S.: Weak classifier for density estimation in eye localization and tracking. IEEE Trans. Image Process. 26, 3410–3424 (2017)

    Article  MathSciNet  Google Scholar 

  42. George, A., Routray, A.: Fast and accurate algorithm for eye localization for gaze tracking in low-resolution images. IET Comput. Vis. 10(7), 660–669 (2016)

    Article  Google Scholar 

  43. Cai, H.-Bi, et al.: Convolution-based means of gradient for fast eye center localization. IEEE International Conference on Machine Learning and Cybernetics (ICMLC) (2015)

  44. Markus, N., Frljak, M., Pandzic, I.S., Ahlberg, J., Forchheimer, R.: Eye pupil localization with an ensemble of randomized trees. Pattern Recognit. 47(2), 578–587 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Informatics, University of Patras, Patras, Greece

    Nikolaos Poulopoulos & Emmanouil Z. Psarakis

Authors
  1. Nikolaos Poulopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emmanouil Z. Psarakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emmanouil Z. Psarakis.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Poulopoulos, N., Psarakis, E.Z. A real-time high precision eye center localizer. J Real-Time Image Proc 19, 475–486 (2022). https://doi.org/10.1007/s11554-022-01200-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-022-01200-8

Keywords

Search

Navigation