Skip to main content
Springer Nature Link
Log in

Camera-based eye blinks pattern detection for intelligent mouse

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Human–computer interface systems provide an alternative input modality to allow people with severe disabilities to access computer systems. One of the inexpensive and unobtrusive methods for this purpose is image-based eye blinks detection. Currently, available human–computer interface systems are often intrusive, limit in head rotation, require special hardware, and have special lighting or manual initialization. This paper presented a new robust method for real-time eye blinks detection. This method enables interaction using “blink patterns,” which are sequences of long and short blinks interpreted as semiotic messages. The precise location of the eye is determined automatically through multi-cues, accompanied by integration of eye variance feature and Gaussian Mixture Model classifier. The detected eye window is converted into a binary image. The eyelid’s distance is extracted by applying a variance projection derivative function. By following the eyelid’s distance in a finite-state machine, the blink patterns can be detected. The performance of the presented algorithm is evaluated using several frame streams. The experimental results show a robust eye blink pattern detection system in real environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Song, F., Tan, X., Chen, S., Zhou, Z.H.: A literature survey on robust and efficient eye localization in real-life scenarios, Pattern Recognit. (2013). doi:10.1016/j.patcog.2013.05.009

  2. Lajevardi, S.M., Hussain, Z.M.: Automatic facial expression recognition: feature extraction and selection. Signal Image Video Process. 6(1), 159–169 (2012)

    Article  Google Scholar 

  3. Moallem, P., Mousavi, B.S., Monadjemi, S.A.: A novel fuzzy rule base system for pose independent faces detection. Appl. Soft Comput. 11, 1801–1810 (2011)

    Article  Google Scholar 

  4. Baltzakis, H., Pateraki, M., Trahanias, P.: Visual tracking of hands, faces and facial features of multiple persons. Mach. Vis. Appl. (2012). doi:10.1007/s00138-012-0409-5

  5. Fathi, A.H., Manzuri, M.T.: Eye detection and tracking in video stream. In: International Symposium on Communication and Information Technologies 2004 (ISCIT 2004) Sapporo, Japan, pp. 1258–1261, 26–29 October 2004

  6. Magee, J.J., Scott, M.R., Waber, B.N., Betke, M.: EyeKeys: a real-time vision interface based on gaze detection from a Low-grade video camera. Comput. Vis. Hum. Comput. Interact. 3766, 90–99 (2005)

    Article  Google Scholar 

  7. Benoit, A., Caplier, A.: Fusing bio-inspired vision data for simplified high level scene interpretation: application to face motion analysis. Comput. Vis. Image Underst. 114(7), 774–789 (2010)

    Article  Google Scholar 

  8. Sirohey, S., Rosenfeld, A., Duric, Z.: A method of detecting and tracking irises and eyelids in video. Pattern recognit. 35, 1389–1401 (2002)

    Article  MATH  Google Scholar 

  9. Meynet, J., Popovici, V., Thiran, J.-P.: Mixtures of boosted classifiers for frontal face detection. SIViP 1(1), 29–38 (2007)

    Article  Google Scholar 

  10. Wang, Q., Yang, W., Wang, H., Yang, J., Zheng, Y.: Face detection using binary template matching and SVM. In: Book Series Lecture Notes in Computer Science vol. 4099, pp. 1237–1241 (2006)

  11. Kaminski, J.Y., Knaan, D., Shavit, A.: Single image face orientation and gaze detection. Mach. Vis. Appl. 21, 85–98 (2009)

    Article  Google Scholar 

  12. Nabati, M., Behrad, A.: 3D Head pose estimation and camera mouse implementation using a monocular video camera. Signal Image Video Process. (2012, december). doi:10.1007/s11760-012-0421-2

  13. Zhang, Z., Potamianos, G., Senior, A.W., Huang, T.S.: Joint face and head tracking inside multi-camera smart rooms. SIViP 1(2), 163–178 (2007)

    Article  MATH  Google Scholar 

  14. Yan, C., Wang, Y., Zhang, Z.: Robust real-time multi-user pupil detection and tracking under various illumination and large-scale head motion. Comput. Vis. Image Underst. 115, 1223–1238 (2011)

    Article  Google Scholar 

  15. Zhu, Z., Ji, Q.: Eye and gaze tracking for interactive graphic display. Mach. Vis. Appl. 15, 139–148 (2004)

    Article  Google Scholar 

  16. Lee, W.O., Lee, E.C., Park, K.R.: Blink detection robust to various facial poses. J. Neurosci. Methods 193, 356–372 (2010)

    Article  Google Scholar 

  17. Li, Y., Wang, S., Ding, X.: Eye/eyes tracking based on a unified deformable template and particle filtering. Pattern Recognit. Lett. 31, 1377–1387 (2010)

    Article  Google Scholar 

  18. Park, C.W., Lee, T.: A robust facial feature detection on mobile robot platform. Mach. Vis. Appl. 21, 981–988 (2010)

    Article  Google Scholar 

  19. Torricelli, D., Goffredo, M., Conforto, S., Schmid, M.: An adaptive blink detector to initialize and update a view-based remote eye gaze tracking system in a natural scenario. Pattern Recognit. Lett. 30, 1144–1150 (2009)

    Article  Google Scholar 

  20. Santis, A.D., Iacoviello, D.: Robust real time eye tracking for computer interface for disabled people. Comput. Methods Programs Biomed. 96, 1–11 (2009)

    Article  Google Scholar 

  21. Krolak, A., Strumillo, P.: Eye-blink detection system for human–computer interaction. Univers. Access Inf. Soc. 11, 409–419 (2012)

    Article  Google Scholar 

  22. Song, F., Tan, X., Liu, X., Chen, S.: Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recognit. (2014). doi:10.1016/j.patcog.2014.03.024i

  23. Feraund, R., Bernier, O.J., Viallet, J., Collobert, M.: A fast and accurate face detector based on neural network. IEEE Trans. Pattern Anal. Mach. Intell. 23, 42–53 (2001)

    Article  Google Scholar 

  24. Wang, P., Ji, Q.: Multi-view face and eye detection using discriminant features. Comput. Vis. Image Underst. 105(2), 99–111 (2007)

    Article  Google Scholar 

  25. Li, Y., Qi, X., Wang, Y.: Eye detection using fuzzy template matching and feature-parameter-based judgement. Pattern Recognit. Lett. 22(10), 1111–1124 (2001)

    Article  MATH  Google Scholar 

  26. Jiang, X., Tien, G., Huang, D., Zheng, B., Atkins, M.S.: Capturing and evaluating blinks from video-based eye trackers. Behav. Res. (2012). doi:10.3758/s13428-012-0294-x

    Google Scholar 

  27. Vidal, M., Turner, J., Bulling, A., Gellersen, H.: Wearable eye tracking for mental health monitoring. Comput. Commun. 35, 1306–1311 (2012)

  28. Nguyen, T., Nguyen, T.H., Truong, K.Q.D., Toi Van Vo.: A mean threshold algorithm for human eye blinking detection using EEG. In: IFMBE Proceedings 4th International Conference on Biomedical Engineering in Vietnam, vol. 40, pp. 275–279. (2013)

  29. Picot, A., Charbonnier, S., Caplier, A., Vu, N.S.: Using retina modelling to characterize blinking: comparison between EOG and video analysis. Mach. Vis. Appl. 23, 1195–1208 (2012)

    Article  Google Scholar 

  30. Yang, S.W., Lin, C.S., Lin, S.K., Lee, C.H.: Design of virtual keyboard using blink control method for the severely disabled. Comput. Methods Programs Biomed. (2013). doi:10.1016/j.cmpb.2013.04.012

  31. Erdem, A., Erdem, E., Yardimci, Y., Atalay, V., Çetin, A.E.: Computer vision based mouse. In: Proceeding of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’02), vol. 4, pp. 4178–4181. Orlando, FL. 13–17 May 2002

  32. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, New York (2001)

    MATH  Google Scholar 

  33. Fathi, A.H., Abdali-Mohammadi, F., Manzuri, M.T.: The eyelids distance detection in gray scale images. In: International Symposium on Communication and Information Technologies 2006 (ISCIT 2006) Bangkok, Thiland, pp. 937–940 (2006)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Razi University, Kermanshah, Iran

    Abdolhossein Fathi & Fardin Abdali-Mohammadi

Authors
  1. Abdolhossein Fathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fardin Abdali-Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdolhossein Fathi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fathi, A., Abdali-Mohammadi, F. Camera-based eye blinks pattern detection for intelligent mouse. SIViP 9, 1907–1916 (2015). https://doi.org/10.1007/s11760-014-0680-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-014-0680-1

Keywords

Search

Navigation