Skip to main content
Springer Nature Link
Log in

On designing an unconstrained tri-band pupil detection system for human identification

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The majority of facial recognition systems depend on the accurate location of both the left and right eye centers in an effort to geometrically normalize the face images available in a database under study. In this paper, we propose a novel pupil detection algorithmic approach that automatically and efficiently locates the eye centers of face images captured using visible and infrared sensors when operating under challenging conditions. Our proposed approach includes the usage of long-range and night-time face images captured in the infrared (IR) band under active illumination. It also efficiently deals with partial face obstruction (subjects wearing eye glasses) as well as face pose and illumination variation. Our scenario-adaptable methodological approach involves a number of algorithmic steps, including (i) situation classification (to automatically determine the acquisition scenario each input face image is coming from), (ii) generation and usage of 2D normalized correlation coefficients, (iii) prediction of eye localization, (iv) computation of summation range filters for accurate pupil detection, and an (v) eye glasses classifier on facial images using support vector machines (to determine when subjects are wearing glasses or not). Our proposed approach is compared against state-of-the-art academic and commercial eye detection algorithms, including the (a) Viola and Jones Adaboost method, (b) one of the latest academic eye detection algorithms proposed by Valenti and Gevers (IEEE TPAMI), and (c) the eye detection approach that is available as part of the G8 commercial face recognition software (package provided by L1 systems). Experimental results demonstrate that our proposed approach outperforms all other approaches when applied on various challenging face datasets, including IR face images captured behind tinted glass or at long ranges up to about 350 feet away, day or night. We also show the benefits of our approach to face normalization and, as a result, face recognition improvement performance in terms of rank-1 identification rates. This is an important achievement that has practical value for forensic tool operators who may have to manually localize the eye centers of all face images available in a dataset, before further face image preprocessing and face-based matching algorithms can be applied.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Notes

  1. http://www.mathworks.com/help/vision/ref/vision.cascadeobjectdetectorclass.html.

  2. http://staff.science.uva.nl/~rvalenti/index.php?content=EyeAPI.

References

  1. Selinger, A., Scolinsky, D.A.: Face recognition in the dark. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshop, Washinton, DC (2004)

  2. Ao, M., Yi, D., Lei, Z., Li, S.Z.: Handbook of remote biometrics for surveillance and security. In: Advances in Computer Vision and Pattern Recognition Series, Springer, New York (2009)

  3. Bourlai, T., Kalka, N., Cao, D., Decann, B., Jafri, Z., Nicolo, F., Whitelam, C., Zou, J., Adjeroh, D., Cukic, B., Dawson, J., Hornak, L., Ross, A., Schmid, N.A.: Ascertaining human identity in night environment. In: Distributed Video Sensor Networks, pp. 451–467. Springer, New York (2011)

  4. Ice, J., Narang, N., Whitelam, C., Kalka, N., Hornak, L., Dawson, J., Bourlai, T.: SWIR imaging for facial image capture through tinted materials. In: SPIE (Vol. 8353), Infrared Technology and Applications XXXVIII, Baltimore (2012)

  5. Bourlai, T., Cukic, B.: Multi-spectral face recognition: identification of people in difficult environments. In: IEEE International Conference on Intelligence and Security Informatics (ISI), Washington, DC (2012)

  6. Grother, P., Quinn, G., Phillips, P.J.: Report on the evaluation of 2D still-image face recognition algorithms. In: National Institute of Standards and Technology interagency/internal report (NISTIR) 7709 (2010)

  7. Chang, K., Bowyer, K., Flynn, P.: Face recognition using 2D and 3D facial data. In: Proceedings of Workshop on Multimodal User Authentication, pp. 25–32. Santa Barbara, CA (2003)

  8. Kong, S., Heo, J., Abidi, B., Paik, J., Abidi, M.: Recent advances in visual and infrared face recognition: a review. In: Proceedings of Computer Vision and Image Understanding, pp. 103–135 (2005)

  9. Yoshitomi, Y., Miyaura, T., Tomita, S., Kimura, S.: Face identification using thermal image processing. In: Proceedings of IEEE Workshop on Robot and Human Communication, pp. 374–379 (1997)

  10. Mndez, H., Martn, C.S., Kittler, J., Plasencia, Y., Garca-Reyes, E. Face recognition with LWIR imagery using local binary patterns. In: Advances in Biometrics, pp. 327–336. Springer, Berlin Heidelberg (2009)

  11. Valenti, R., Gevers, T.: Accurate eye center location through invariant isocentric patterns. In: Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1785–1798 (2012)

  12. Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  13. Shafi, M., Chung, P.: A hybrid method for eyes detection in facial images. In: Proceedings of World Academy of Science, Engineering and Technology (2008)

  14. Dowdall, J., Pavlidis, I., Bebis, G.: Face detection in the near-ir spectrum. In: Proceedings of Image and Vision Computing, pp. 565–578. New Zealand (2003)

  15. Feng, G., Yuen, P.: Multi-cues eye detection on gray intensity image. Pattern Recognit. 34(5), 1033–1046 (2001)

    Article  MATH  Google Scholar 

  16. Wang, P., Green, M.B., Ji, Q., Wayman, J.: Automatic eye detection and its validation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 164–171. San Diego (2005)

  17. Asteriadis, S., Nikolaidis, N., Hajdu, A., Pitas, I.: An eye detection algorithm using pixel to edge information. In: Proceedings of International Symposium on Communications, Control and Signal Processing, Marrakech, Morocco (2006)

  18. Park, C.W., Kwak, J.M., Park, H., Moon, Y.S.: An effective method for eye detection based on texture information. In: Proceedings of International Conference on Communications and Information Technology, pp. 586–589. Washington, DC (2007)

  19. Xingming Z., Huangyuan, Z.: An illumination independent eye detection algorithm. In: Proceedings of International Conference on Pattern Recognition, pp. 392–395. Hong Kong (2006)

  20. Whitelam, C., Jafri, Z., Bourlai, T.: Multispectral eye detection: a preliminary study. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 209–212. Istanbul (2010)

  21. Bourlai, T., Whitelam, C., Kakadiaris, I.: Pupil detection under lighting and pose variations in the visible and active infrared bands. In: Proceedings of International Workshop on Information Forensics and Security, pp. 1–6. Foz do Iguacu, Brazil (2011)

  22. Zhu, Z., Ji, Q., Fujimura, K., Lee, K.: Combing Kalman fltering and mean shift for real time eye tracking under active IR illumination. In: Proceedings of International Conference on Pattern Recognition, pp. 318–324. Quebec (2002)

  23. Bourlai, T., Jafri, Z.: Eye detection in the Middle-Wave Infrared Spectrum: towards recognition in the dark. In: Proceedings of International Workshop on Information Forensics and Security, Foz do Iguacu, Brazil (2011)

  24. Song, J., Chi, Z., Liu, J.: A robust eye detection method using combined binary edge and intensity information. Pattern Recognit. 39(6), 1110–1125 (2006)

    Article  MATH  Google Scholar 

  25. Hsu, R., Abdel-Mottaleb, M., Jain, A.: Face detection in color images. In: Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 696–706 (2002)

  26. Chiang, C., Tai, W., Yang, M., Huang, Y., Huang, C.: A novel method for detecting lips, eyes and faces in real time. Real-Time Imaging 9(4), 277–287 (2003)

    Article  Google Scholar 

  27. Sha, M., Chung, P.: Eyes extraction from facial images using edge density. In: Proceedings of 7th IEEE International Conference on Cybernetic Intelligent Systems (2008)

  28. Lu, H., Zhang, W., Yang, D.: Eye detection based on rectangle features and pixel-pattern-based texture features. In: Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems, pp. 746–749. Xiamen, China (2007)

  29. Khairosfaizal, W.M.K.W.M., Nor’aini, A.J.: Eyes detection in facial images using circular hough transform. In: Proceedings 5th International Colloquium on Signal Processing & Its Applications, pp. 238–242 (2009)

  30. Efraty, B., Huang, C., Shah, S.K., Kakadiaris, I.A.: Facial landmark detection in uncontrolled conditions. In: International Joint Conference on Biometrics (IJCB) 11–13 Oct, pp. 1–8 (2011)

  31. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection, In: Proceedings of IEEE Computer Vision and Pattern Recognition, (CVPR), pp. 886–893 (2005)

  32. Jobson, D., Rahman, Z., Woodell, G.: Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 6(3), 451–462 (1997)

    Article  Google Scholar 

  33. Rahman, Z., Jobson, D., Woodell, G.: Multi-scale retinex for color image enhancement. Proceedings of International Conference on Image Processing 3, 1003–1006 (1996)

    Article  Google Scholar 

  34. Li, S., Yi, D., Lei, Z., Liao, S.: The CASIA NIR-VIS 2.0 face database. In: Proceedings of IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 348–353 (2013)

  35. Toderici, G., Evangelopoulos, G., Fang, T., Kakadiaris, I.: UHDB11 database for 3D–2D face recognition. In: Image and Video Technology, pp. 73–86. Springer, Berlin (2014)

  36. Jesorsky, O., Kirchbergand, K.J., Frischholz, R.: Robust face detection using the Hausdorff distance. In: Audio- and Video-Based Biometric Person Authentication, pp. 90–95 (1992)

  37. Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

    Article  Google Scholar 

Download references

Acknowledgments

This work was sponsored in part through a Grant from the Office of Naval Research (N00014-08-1-0895), “Distribution A—Approved for Unlimited Distribution”. We are also grateful to all faculty and students who assisted us with this work, especially Jerin Young and Jason Ice for their assistance in manually annotating all images used in this study as well as the University of Houston for providing the visible face datasets. Also, the authors would also like to acknowledge West Virginia High Technology Consortium (WVHTC) foundation, and specifically Dr. Brian Lemoff, for assembling and providing the long-distance dataset.

Author information

Authors and Affiliations

  1. MILab LCSEE, West Virginia University, 395 Evansdale Drive, Morgantown, WV, 26506, USA

    Cameron Whitelam & Thirimachos Bourlai

Authors
  1. Cameron Whitelam
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Thirimachos Bourlai
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Cameron Whitelam.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Whitelam, C., Bourlai, T. On designing an unconstrained tri-band pupil detection system for human identification. Machine Vision and Applications 26, 1007–1025 (2015). https://doi.org/10.1007/s00138-015-0700-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0700-3

Keywords