Skip to main content

Advertisement

SpringerLink
Log in

Real-time accurate eye center localization for low-resolution grayscale images

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Eye center localization is considered a crucial step for many human–computer interaction (HCI) real-time applications. Detecting the center of eye (COE), accurately and in real time, is very challenging due to the wide variation of poses, eye appearance and specular reflection, especially in low-resolution images. In this paper, an accurate real-time detection algorithm of the COE is proposed. The proposed approach depends on the image gradient to detect the COE. The computational complexity is minimized and the accuracy is improved by down sampling the face resolution and applying a rough-to-fine algorithms, to reduce the search area, in accordance with the Eye Region Of Interest (EROI) and the number of COE candidates, tested by the proposed algorithm. Also, the detection algorithm is applied on a limited number of pixels that represent the iris boundary of the COE candidates. The Look Up Tables (LUTs) are implemented to, initially, store the invariant elements of the proposed image gradient-based algorithm, to reduce the detection time. Before applying the proposed COE detection approach, a modified specular reflection method is used to improve the detection accuracy. The performance of the proposed algorithm has been evaluated by applying it to three benchmark databases: the BIOID, GI4E and Talking Face video datasets, at different face resolutions. Experimental results revealed that the accuracy of the proposed algorithm is up to 91.68% and 96.7% for BIOID and GI4E datasets, respectively, while the minimum achieved average detection time is 2.7 ms. The promising results highlight the potential of the proposed algorithm to be used in some eye gaze-based real-time applications. Comparing the proposed method with the most state-of-the-art approaches showed that the system outperforms most of them and has a comparable performance with the others, in terms of the COE localization accuracy and detection speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31

Similar content being viewed by others

References

  1. Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. 32(3), 478–500 (2010)

    Article  Google Scholar 

  2. Sambrekar, U., Ramdasi, D.: Human computer interaction for disabled using eye motion tracking. In: IEEE International Conference on Information Processing, pp. 745–750, Pune, India (2015)

  3. Sun, L., Liu, Z., Sun, M.T.: Real time gaze estimation with a consumer depth camera. Elsevier J. Inf. Sci. 320, 346–360 (2015)

    Article  MathSciNet  Google Scholar 

  4. Fu, X., Guan, X., Peli, E., Liu, H., Luo, G.: Automatic calibration method for driver’s head orientation in natural driving environment. IEEE Trans Intell. Transp. Syst. 14(1), 303–312 (2013)

    Article  Google Scholar 

  5. Zhang, Y. F., Gao, X. Y., Zhu, J. Y.: A novel approach to driving fatigue detection using forehead eog. In: 7th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 707–710 (2015)

  6. Paperno, E., Semyonov, D.: A new method for eye location tracking. IEEE Trans. Biomed. Eng. 50(10), 1174–1179 (2003)

    Article  Google Scholar 

  7. Hyoki, K., Shigeta, M., Tsuno, N., Kawamuro, Y., Kinoshita, T.: Quantitative electro-oculography and electroencephalography as indexes of alertness. Electroencephalogr. Clin. Neurophysiol. 106, 213–219 (1998)

    Article  Google Scholar 

  8. Manabe, H., Fukumoto, M., Yagi, T.: Direct gaze estimation based on nonlinearity of EOG. IEEE Trans. Biomed. Eng. 62(6), 1553–1562 (2015)

    Article  Google Scholar 

  9. Lupu, R. G., Ungureanu, F., Siriteanu, V.:Eye tracking mouse for human computer interaction. In: Proceedings on 4th IEEE International Conference on E-Health Bioengineering, pp. 1-4, Romania (2013)

  10. Bates, R., Istance, H., Oosthuizen L., Majaranta, P.: Survey of de-facto standards in eye tracking. In: Conference on Communication by Gaze Inter (COGAIN) (2005)

  11. Hansen, D. W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32, 478–500 (2010)

    Article  Google Scholar 

  12. Zhu, Z., Ji, Q.: Novel eye gaze tracking techniques under natural head movement. IEEE Trans. Biomed. Eng. 54(12), 2246–2260 (2007)

    Article  Google Scholar 

  13. Shin, Y.G., Choi, K.A., Kim, S.T., Ko, S.J.: A novel single IR light based gaze estimation method using virtual glints. IEEE Trans. Consumer Electron. 61(2), 254–260 (2015)

    Article  Google Scholar 

  14. Choi, K.A., Baek, S.J. Ma, C., Park, S., Ko, S.J.: Improved pupil center localization method for eye-gaze tracking-based human-device interaction. In: IEEE International Conference on Consumer Electron. pp. 524-525, Las Vegas, USA (2014)

  15. Huang, J. B., Cai, Q. Z., Ahuja, Liu, N., Zhang, Z.: Towards accurate and robust cross-ratio based gaze trackers through learning from simulation. In: Proceedings of ACM Symposium on Eye Tracking Research and Applications, pp. 75–82 (2014)

  16. Guestrin, E., Eizenman, M.: General theory of remote gaze estimation using the pupil center and corneal reflections. IEEE Trans. Biomed. Eng. 53(6), 1124–1133 (2006)

    Article  Google Scholar 

  17. Shin, Y.-G., Choi, K.A., Kim, S.T., Ko, S.-J.: A novel single IR light based gaze estimation method using virtual glints. IEEE Trans. on Consumer Electron. 61, 254–260 (2015)

    Article  Google Scholar 

  18. Sesma-Sanchez, L., Villanueva, A., Cabeza, R.: Gaze estimation interpolation methods based on binocular data. IEEE Trans. Biomed. Eng. 59(8), 2235–2243 (2012)

    Article  Google Scholar 

  19. Coetzer, R., Hancke, G.: Eye detection for a real-time vehicle drive fatigue monitoring system. In: IEEE Intelligent Vehicles Symposium, pp. 66-71, Germany (2011)

  20. Sambrekar, U., Ramdasi, D.: Human Computer Interaction For Disabled Using Eye Motion Tracking. In: IEEE International Conference on Information Processing, pp. 745-750, Pune, India (2015)

  21. Chinsatit, W., Saitoh, T.: CNN-based pupil center detection for wearable gaze estimation system. Appl. Comput. Intell. Soft Comput. 20(17), 1–10 (2017)

    Google Scholar 

  22. Dan, W. H., Ji, Q.: In the eye of the beholder: A survey of models for eyes and gaze. IEEE Trans. on Pattern Analysis and actions on Machine Intelligence. 32(3), 478–500 (2010)

  23. Wu, Y., Ji, Q.: Facial landmark detection: a literature survey. Int. J. Comput. Vis. 127(2), 1–28 (2018)

  24. Hassaballah, M., Bekhet, S., Amal A.M.R., Gang, Z.: Facial features detection and localization. In: Recent Advances in Computer Vision—Theories and Applications. Studies in Computational Intelligence Series, Springer, 2019

  25. Yiu, Y. H., Aboulatta, M., Raiser, T., Ophey, L., Flanagin, V. L., Zu Eulenburg, P., Ahmadi, S.A.: DeepVOG: open-source pupil segmentation and gaze estimation in neuroscience using deep learning. J. Neurosci. Methods. 324, 108307 (2019)

  26. Asteriadis, S., Nikolaidis, N., Hajdu, A., Pitas, I.: An eye detection algorithm using pixel to edge information. ISCCPSP (2006)

  27. Vassilios, C., Fotios P., Vassilios M.: Face to Iris area ratio as a feature for children detection in digital forensics applications. IEEE Digital Media Industry & Academic Forum (DMIAF), pp. 121–124, Greece (2016)

  28. Hamouz, M., Kittler, J., Kamarainen, J.K., Paala-nen, P., Kalviainen H., Matas, J.: Feature-based affine-invariant localization of faces. IEEE Trans. Pattern Anal. Mach. Intell. 27(9),1490–1495 (2005)

  29. Kim, B. s., Lee, H., Kim, W.Y.: Rapid eye detection method for non-glasses type 3d display on portable devices. IEEE Trans. Consumer Electron. 56(4), 2498–2505 (2010)

  30. Skodras, E., Fakotakis, N.: Precise localization of eye centers in low resolution color images. Elsevier Image Vis. Comput. 36, 51–60 (2015)

    Article  Google Scholar 

  31. Cai, H., Liu, B., Ju, Z., Thill, S., Belpaeme, T., Vanderborght, B., Liu, H.: Accurate eye center localization via hierarchical adaptive convolution. British Machine Vision Conference, 2018

  32. Hansen, A.D., Pece, W.E.: Eye tracking in the wild. Comput. Vis. Image Understand. 98(1), 155–181 (2005)

    Article  Google Scholar 

  33. Roberto, V., Theo G.: Accurate eye center location through invariant isocentric patterns. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1785–1798 (2012)

  34. Soelistio, Y. E., Postma, E., Maes, A.: Circle-based eye center localization (CECL) Machine Vision Applications (MVA). In: 14th IEEE Internatonal Conference on IAPR. pp. 349–352, Japan (2015)

  35. Illingworth, J., Kittler, J.: A survey of the Hough transform. Elsevier Comp. Vis. Graph. Image Process. 44(1), 87–116 (1988)

    Article  Google Scholar 

  36. Świrski, L., Bulling, A., Dodgson, N.: Robust real-time pupil tracking in highly off-axis images. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 173–176, USA (2012)

  37. Wang, J., Sung, E., Venkateswarlu, R.: Eye gaze estimation from a single image of one eye. In: 9th IEEE International Conference on Computer Vision, pp. 136–143, France (2003)

  38. Baek, S.J., Choi, K.A, Ma, C.: Eyeball model-based iris center localization for visible image-based eye-gaze tracking systems. IEEE Trans. Consum. Electron. 59(2), 415–421 (2013)

  39. Chao G. Yue, W., Kang, W. Kunfeng, W., Fei-Yue, W., Qiang, J.: A joint cascaded framework simultaneous eye detection and eye state estimation. Elsevier Trans. Pattern Recognit. 67, 23–31 (2017)

  40. Li, X., Dou, Y., Niu, X., Xu, J., Xiao, R.: An efficient robust eye localization by learning the convolution distribution using eye template. Comput. Intell. Neurosci. 2015, 709–721 (2015)

  41. Gabriel, M.A., Felipe, M.L.R., Waldir, S.S.J., Eduardo, A.B.: Weak classifier for density estimation in eye localization and tracking. IEEE Trans. Image Process. 26(7), 3410–3424 (2017)

    Article  MathSciNet  Google Scholar 

  42. Dong, T., Guanghui, H., Jiaxiang, W., Hongtao, C., Yong, J.: An Accurate Eye Pupil Localization Approach Based on Adaptive Gradient Boosting Decision Tree. In: IEEE Conference on Visual Communication and Image Processing. pp. 1–4, China (2016)

  43. Hassaballah, M., Kanazawa, T., Ido, S.: Efficient eye detection method based on grey intensity variance and independent components analysis. Comput. Vis. IET 4(4), 261–271 (2010)

    Article  Google Scholar 

  44. Hyvarien, A., Oja, E.: Independent component analysis: algorithms and applications. Neural Netw. 13, 411–430 (2000)

    Article  Google Scholar 

  45. Roberto, V., Theo, G.: Accurate eye center location through invariant isocentric patterns. IEEE Trans Pattern Anal. Mach. Intell. 34(9), 1785–1798 (2012)

    Article  Google Scholar 

  46. Bin, L.: Eye centre localisation with hybrid regression framework. IET Electron. Lett. 54(13), 819–821 (2018)

    Article  Google Scholar 

  47. Xia, Y., Lou, J., Dong, J., Li, G., Yu, H.: SDM-based means of gradient for eye center localization. In: IEEE 16th International Conference on Dependable, Autonomic and Secure Computing, pp. 862–867, Greece (2018)

  48. Anjith, G., Aurobinda, R.: Fast and accurate algorithm for eye localization for gaze tracking in low resolution images. IET Comput. Vis. Pattern Recognit. 10(7), 660–669 (2016)

    Google Scholar 

  49. Roberto, V., Theo, G.: Accurate Eye Center Location and Tracking Using Isophote Curvature. In: IEEE Confderence on Computer Vision and Pattern Recognition, pp. 1–8, USA (2018)

  50. Valenti, R., Gevers, T.: Accurate eye center location through invariant isocentric patterns. IEEE Trans. Pattern Anal. Mach. Intel. 34(9), 1785–1798 (2012)

    Article  Google Scholar 

  51. Chen, S., Liu, C.: Eye detection using discriminatory haar features and a new efficient SVM. Elsevier, Image Vis. Comput. 33, 68–77 (2015)

    Article  Google Scholar 

  52. Kawulok, M., Szymanek, J.: Precise multi-level face detector for advanced analysis of facial images. Image Process. IET 6(2), 95–103 (2012)

    Article  MathSciNet  Google Scholar 

  53. Kim, H., Kim, J., Park, R.: Efficient and fast iris localization using binary radial gradient features for human–computer interaction. Int. J. Pattern Recognit. Artif. Intell. 31(11), 1756015 (2017)

    Article  Google Scholar 

  54. Li, B., Fu, H.: Real time eye detector with cascaded convolutional neural networks. Appl. Comput. Intell. Soft Comput. 2018, 1–8 (2018)

    Google Scholar 

  55. Jacek, N., Michał, W., Stanisław, S., Dariusz, K., Michał, T.: Webcam-based system for video-oculography. IET Comput. Vis. 11(2), 173–180 (2017)

    Article  Google Scholar 

  56. Zhou, Z., Yao, P., Zhuang, Z..: A robust algorithm for iris localization based on radial symmetry and circular integro differential operator. In: IEEE International Conference on Industrial Electronics and Applications, 1742–1745, China (2011)

  57. Villanueva, A., Ponz, V., Sesma-Sanchez, L., Ariz, M., Porta, S., Cabeza, R.: Hybrid method based on topography for robust detection of iris center and eye corners. ACM Trans. Multimed. Comput. Commun. Appl. 9(4), 1–20 (2013)

    Article  Google Scholar 

  58. Guoqing X., Yangsheng W., Jituo L., Xiaoxu Z.: Real Time Detection of Eye Corners and Iris Center from Images Acquired by Usual Camera. In: IEEE 2nd International Conference on Intelligent Networks and Intelligent Systems, China (2009)

  59. J. Ahlberg. A system for face localization and facial feature extraction. Proc. Siggraph 98, ACM Press, July, 1998, pp:124–139

  60. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE International Conference on Computer Vision and Pattern Recognition, USA (2001)

  61. André, C.M.P.: The effect of corneal diameter on soft lens fitting, part 1. Contact Lens Spect. 17(4), 56 (2002)

    Google Scholar 

  62. Robert H. Spector. Clinical Methods: The History, Physical, and Laboratory Examinations. 3rd edition, Chapter 58

  63. Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face recognition algorithms. PAMI 22, 1090–1104 (2000)

    Article  Google Scholar 

  64. Fei, Y., Junzhou, H., Peng, Y., Dimitris, M.: Eye localization through multiscale sparse dictionaries. In: IEEE International Conference on. Automatic Face & Gesture Recognition, pp. 414–518, USA (2011)

  65. Timm, F., Barth, E.: Accurate eye centre localisation by means of gradients. In: International Conference on Computer Vision Theory and Application, pp. 125–130, Portugal (2011)

  66. Krupa, N.J., Aakash, N., Upena, D.D.: A real time robust eye center localization using geometric eye model and edge gradients in unconstrained visual environment. Int. J. Comput. Appl. 128(1), 22–27 (2015)

    Google Scholar 

  67. Prendergast, P.: Facial Proportions Advanced Surgical Facial Rejuvenation: Art and Clinical Practice, pp. 15–22. Springer, Berlin (2012)

    Book  Google Scholar 

  68. Tieniu, T., Zhaofeng, H., Zhenan, S.: Efficient and robust segmentation of noisy iris images for non-cooperative iris recognition. Elsevier, Image Vis. Comput. 28(2), 223–230 (2010)

    Article  Google Scholar 

  69. BioID-Database. https://www.bioid.com/About/BioID-Face-Database. (2014). Accessed Nov 2018

  70. Alessandro, A., Francesco, B., Dmitry, C.: A survey of specularity removal methods. J. Comput. Graph. Forum 30, 2208–2230 (2011)

    Article  Google Scholar 

  71. Zhaofeng, H., Tieniu, T., Zhenan, S., Xianchao, Q.: Towards accurate and fast iris segmentation for iris biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1670–1684 (2009)

    Article  Google Scholar 

  72. Nianfeng, L., Haiqing, L., Man, Z., Jing, L., Zhenan, S., Tieniu, T.: Accurate iris segmentation in non-cooperative environments using fully convolutional networks. In: IEEE International Conference on Biometrics, 1–8, Sweden (2016)

  73. Nadia,O., Bernadette, d., Sonia, G.: OSIRIS: An open source iris recognition software. Elsevier Pattern Recognition Letters. 82(2), 124-131 (2016)

  74. Smereka, M., Dulęba, I.: Circular object detection using a modified Hough transform. Int. J. Appl. Math. Comput. Sci. 18(1), 85–91 (2008)

    Article  MathSciNet  Google Scholar 

  75. Ponz, V., Villanueva, A., Cabeza, R.: Dataset for the evaluation of eye detector for gaze estimation. ACM Conf. on Ubiquitous Computing, 681–684 (2012)

  76. F. IST-2000-26434. Talking face video. http://wwwprima.inrialpes.fr/FGnet/data/01-TalkingFace/talking_face.html, 2004. Accessed 12 Aug 2015

  77. Jesorsky, O., Kirchberg, K.J., Frischholz, R.W.: Robust face detection using the Hausdorff distance. Springer Int. Conf. on Audio- and Video-Based Biometric Person Authentication, 90–95 (2001)

  78. Bradski, G.: The OpenCV library. Doct. Dobbs J. 25(11), 120–126 (2000)

    Google Scholar 

  79. Wu, Y., Ji, Q.: Robust facial landmark detection under significant head poses and occlusion. In: The IEEE International Conference on Computer Vision (ICCV), Dec 2015

Download references

Author information

Authors and Affiliations

  1. Department of radiation engineering, National Center for Radiation Research and Technology (NCRRT), Atomic Energy Authority in Egypt, Cairo, Egypt

    Noha Younis Ahmed

Authors
  1. Noha Younis Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noha Younis Ahmed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, N.Y. Real-time accurate eye center localization for low-resolution grayscale images. J Real-Time Image Proc 18, 193–220 (2021). https://doi.org/10.1007/s11554-020-00955-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00955-2

Keywords

Search

Navigation