Skip to main content

Advertisement

SpringerLink
Log in

EyeLSD a Robust Approach for Eye Localization and State Detection

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Improving the safety of public roads and industrial factories requires more reliable and robust computer vision-based approaches for monitoring the eye state (open or closed) of human operators. Getting this information in real time when humans are driving cars or using hazardous machinery will help to prevent accidents and deaths. This paper proposes a new framework called EyeLSD to localize the eyes and detect their states without face detection step. For EyeLSD aims, two novel descriptors are proposed: enhanced Pyramidal Local Binary Pattern Histogram (ePLBPH) and Multi-Three-Patch LBP histogram (Multi-TPLBP). The performance of EyeLSD with ePLBPH and Multi-TPLBP is evaluated and compared against other approaches. For this evaluation three independent and public datasets were used: BioID, CAS-PEAL-R1 and ZJU datasets. The set EyeLSD, ePLBPH and Multi-TPLBP have a greater performance when compared against the state-of-the-art algorithms. The proposed approach is very stable under large range of eye appearances caused by expression, rotation, lighting, head pose, and occlusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13

Similar content being viewed by others

References

  1. (1998). PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance. In Indianapolis, Federal Highway Administration, Office of Motor Carriers. Tech. Rep. MCRT-98-006.

  2. Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face description with local binary patterns: application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2037–2041.

    Article  MATH  Google Scholar 

  3. Benrachou, D. E., Boulebtateche, B., & Bensaoula, S. (2013). Gabor/pca/svm-based face detection for drivers monitoring. Journal of Automation and Control Engineering, 1, 115–118.

    Article  Google Scholar 

  4. Benrachou, D. E., dos Santos, F. N., Boulebtateche, B., & Bensaoula, S. (2015). Automatic eye localization multi-block lbp vs. pyramidal lbp three-levels image decomposition for eye visual appearance description. In Pattern recognition and image analysis (pp. 718–726). Springer.

  5. Benrachou, D. E., dos Santos, F. N., Boulebtateche, B., & Bensaoula, S. (2015). Online vision-based eye detection: Lbp/svm vs lbp/lstm-rnm. In CONTROLO’2014–proceedings of the 11th Portuguese conference on automatic control (pp. 659–668). Springer.

  6. Chang, C. C., & Lin, C. J. (2011). Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.

    Google Scholar 

  7. Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models-their training and application. Computer Vision and Image Understanding, 61(1), 38–59.

    Article  Google Scholar 

  8. Feng, G. C., & Yuen, P. C. (1998). Variance projection function and its application to eye detection for human face recognition. Pattern Recognition Letters, 19(9), 899–906.

    Article  Google Scholar 

  9. Flores, M. J., Armingol, J. M., & de la Escalera, A. (2010). Real-time warning system for driver drowsiness detection using visual information. Journal of Intelligent & Robotic Systems, 59(2), 103–125.

  10. Gao, W., Cao, B., Shan, S., Chen, X., Zhou, D., Zhang, X., & Zhao, D. (2008). The cas-peal large-scale chinese face database and baseline evaluations. IEEE Transactions on Systems, Man and Cybernetics Part a Systems and Humans, 38(1), 149.

    Article  Google Scholar 

  11. Ge, S., Yang, R., Wen, H., Chen, S., & Sun, L. (2014). Eye localization based on correlation filter bank. In 2014 IEEE international conference on multimedia and expo (ICME) (pp. 15). IEEE.

  12. Gonzalez, R. C., & Woods, R. E. (2002). Digital image processing, 2nd edn. (pp. 523–532). Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  13. González-Ortega, D., Díaz-Pernas, F., Antón-Rodríguez, M., Martínez-Zarzuela, M., & Díez-Higuera, J. (2013). Real-time vision-based eye state detection for driver alertness monitoring. Pattern Analysis and Applications, 16(3), 285–306.

    Article  MathSciNet  Google Scholar 

  14. Harwood, D., Ojala, T., Pietikäinen, M., Kelman, S., & Davis, L. (1995). Texture classification by center-symmetric auto-correlation, using kullback discrimination of distributions. Pattern Recognition Letters, 16(1), 1–10.

    Article  Google Scholar 

  15. Hassaballah, M., Kanazawa, T., & Ido, S. (2010). Efficient eye detection method based on grey intensity variance and independent components analysis. Computer Vision, IET, 4(4), 261–271.

    Article  Google Scholar 

  16. Hassner, T., Harel, S., Paz, E., & Enbar, R. (2015). Effective face frontalization in unconstrained images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4295–4304).

  17. Hollingsworth, K., Clark, S., Thompson, J., Flynn, P. J., & Bowyer, K. W. (2013). Eyebrow segmentation using active shape models, SPIE Defense, Security, and Sensing (pp. 871,208–871,208). International Society for Optics and Photonics.

  18. Kalbkhani, H., Shayesteh, M. G., & Mohsen Mousavi, S. (2013). Efficient algorithms for detection of face, eye and eye state. Computer Vision, IET, 7(3), 184–200.

    Article  Google Scholar 

  19. Lazebnik, S., Schmid, C., & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 2169–2178). IEEE.

  20. Lehmann, A., Leibe, B., & Van Gool, L. (2011). Fast prism: Branch and bound hough transform for object class detection. International Journal of Computer Vision, 94(2), 175–197.

  21. Leibe, B., Leonardis, A., & Schiele, B. (2008). Robust object detection with interleaved categorization and segmentation. International Journal of Computer Vision, 77(1-3), 259–289.

    Article  Google Scholar 

  22. Li, W., Wang, Y., & Wang, Y. (2011). Eye location via a novel integral projection function and radial symmetry transform. International Journal of Digital Content Technology and its Applications, 5(8), 70–80.

    Article  Google Scholar 

  23. Mäenpää, T. (2003). The Local binary pattern approach to texture analysis: extensions and applications. Oulun Yliopisto.

  24. Mäenpää, T., & Pietikäinen, M. (2003). Multi-scale binary patterns for texture analysis. In Image analysis (pp. 885–892). Springer.

  25. Mahalingam, G., & Ricanek, Jr K. (2013). Lbp-based periocular recognition on challenging face EURASIP. Journal on Image and Video Processing, 2013(1), 1–13.

  26. Ojala, T., Pietikäinen, M., & Harwood, D. (1996). A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1), 51–59.

    Article  Google Scholar 

  27. Ojala, T., Pietikäinen, M., & Mäenpää, T. (2000). Gray scale and rotation invariant texture classification with local binary patterns. In Computer vision-ECCV 2000 (pp. 404–420). Springer.

  28. Ojala, T., Pietikainen, M., & Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.

    Article  MATH  Google Scholar 

  29. Oyini Mbouna, R., Kong, S. G., & Chun, M. G. (2013). Visual analysis of eye state and head pose for driver alertness monitoring. IEEE Transactions on Intelligent Transportation Systems, 14(3), 1462–1469.

    Article  Google Scholar 

  30. Pang, Y., Yuan, Y., & Li, X. (2008). Gabor-based region covariance matrices for face recognition. IEEE Transactions on Circuits and Systems for Video Technology, 18(7), 989–993.

    Article  Google Scholar 

  31. Qian, X., Hua, X. S., Chen, P., & Ke, L. (2011). Plbp: An effective local binary patterns texture descriptor with pyramid representation. Pattern Recognition, 44(10), 2502–2515.

    Article  Google Scholar 

  32. Pan, G., Sun, L., Wu, Z., & Lao, S. (2007). Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In IEEE 11th international conference on computer vision, 2007. ICCV 2007 (pp. 1–8). IEEE.

  33. Ren, Y., Wang, S., Hou, B., & Ma, J. (2014). A novel eye localization method with rotation invariance. IEEE Transactions on Image Processing, 23(1), 226–239.

    Article  MathSciNet  MATH  Google Scholar 

  34. Shi, Y., Yan, Z., Ge, H., & Mei, L. (2014). Visual objects location based on hand eye coordination. In Future information technology (pp. 403–408). Springer.

  35. Sirohey, S., Rosenfeld, A., & Duric, Z. (2002). A method of detecting and tracking irises and eyelids in video. Pattern Recognition, 35(6), 1389–1401.

    Article  MATH  Google Scholar 

  36. Smith, B., Brandt, J., Lin, Z., & Zhang, L. (2014). Nonparametric context modeling of local appearance for pose-and expression-robust facial landmark localization. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1741–1748).

  37. Song, F., Tan, X., Chen, S., & Zhou, Z. H. (2013). A literature survey on robust and efficient eye localization in real-life scenarios. Pattern Recognition, 46(12), 3157–3173.

    Article  Google Scholar 

  38. Song, F., Tan, X., Liu, X., & Chen, S. (2014). Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recognition, 47(9), 2825– 2838.

    Article  Google Scholar 

  39. Tan, X., Song, F., Zhou, Z. H., & Chen, S. (2009). Enhanced pictorial structures for precise eye localization under incontrolled conditions. In IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009 (pp. 1621–1628). IEEE.

  40. Tan, X., & Triggs, B. (2010). Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Transactions on Image Processing, 19(6), 1635–1650.

    Article  MathSciNet  MATH  Google Scholar 

  41. Turtinen, M., Pietikäinen, M., Turtinen, M., & Pietikäinen, M. (2006). Contextual analysis of textured scene images. In Proceedings of the 17th British machine vision conference (BMVC 2006) (Vol. 2, pp. 849–858). Edinburgh, UK.

  42. Vapnik, V. N., & Vapnik, V. (1998). Statistical learning theory (Vol. 1). New York: Wiley.

  43. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001 (Vol. 1, pp. I–511). IEEE.

  44. Wang, Q., & Yang, J. (2006). Eye detection in facial images with unconstrained background. Journal of Pattern Recognition Research, 1(1), 55–62.

    Article  Google Scholar 

  45. Wolf, L., Hassner, T., & Taigman, Y. (2008). Descriptor based methods in the wild. In Workshop on faces in ‘real-Life’ images: detection, alignment, and recognition.

  46. Wu, J., & Mei, L. (2013). A face recognition algorithm based on asm and gabor features of key points. In 2012 international conference on graphic and image processing (pp. 87,686L–87,686L). International Society for Optics and Photonics.

  47. Wu, Y., & Ji, Q. (2014). Learning the deep features for eye detection in uncontrolled conditions. In 22nd international conference on pattern recognition (ICPR) 2014 (pp. 455–459). IEEE.

  48. Wu, Y. S., Lee, T. W., Wu, Q. Z., & Liu, H. S. (2010). An eye state recognition method for drowsiness detection. In Vehicular technology conference (VTC 2010-Spring), 2010 IEEE 71st (pp. 1–5). IEEE.

  49. Xiang, C., Ding, S. Q., & Lee, T. H. (2005). Geometrical interpretation and architecture selection of mlp. IEEE Transactions on Neural Networks/a Publication of the IEEE Neural Networks Council, 16(1), 84–96.

    Article  Google Scholar 

  50. Xu, C., Zheng, Y., & Wang, Z. (2008). Eye states detection by boosting local binary pattern histogram features. In 15th IEEE international conference on image processing, 2008. ICIP 2008 (pp. 1480–1483). IEEE.

  51. Yi, D., Lei, Z., & Li, S. Z. (2011). A robust eye localization method for low quality face images. In International joint conference on biometrics (IJCB), 2011 (pp. 1–6). IEEE.

  52. Zhang, B., Shan, S., Chen, X., & Gao, W. (2007). Histogram of gabor phase patterns (hgpp): a novel object representation approach for face recognition. IEEE Transactions on Image Processing, 16(1), 57–68.

    Article  MathSciNet  Google Scholar 

  53. Zhao, G., Ahonen, T., Matas, J., & Pietikainen, M. (2012). Rotation-invariant image and video description with local binary pattern features. IEEE Transactions on Image Processing, 21(4), 1465–1477.

    Article  MathSciNet  MATH  Google Scholar 

  54. Zhou, M., Wang, X., Wang, H., Heo, J., & Nam, D. (2015). Precise eye localization with improved sdm. In IEEE international conference on image processing (ICIP), 2015 (pp. 4466–4470). IEEE.

  55. Zhou, Z.H., & Geng, X. (2004). Projection functions for eye detection. Pattern Recognition, 37(5), 1049–1056.

    Article  MATH  Google Scholar 

Download references

Acknowledgments

Project NORTE-01-0145-FEDER-000020” is financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF).

Author information

Authors and Affiliations

  1. University Badji Mokhtar Annaba, B.P.12, Annaba, Algeria

    Benrachou Djamel Eddine, Brahim Boulebtateche & Salah Bensaoula

  2. INESC TEC FEUP campus, Rua Dr. Roberto Frias, 4200 - 465, Porto, Portugal

    Filipe Neves dos Santos

Authors
  1. Benrachou Djamel Eddine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Filipe Neves dos Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brahim Boulebtateche
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salah Bensaoula
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Filipe Neves dos Santos.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eddine, B.D., dos Santos, F.N., Boulebtateche, B. et al. EyeLSD a Robust Approach for Eye Localization and State Detection. J Sign Process Syst 90, 99–125 (2018). https://doi.org/10.1007/s11265-016-1219-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-016-1219-1

Keywords

Search

Navigation