Abstract
This paper presents a novel approach to depth camera based single-/multi-person eye localization for human-machine interactions. Intensity and depth image frames of a single depth camera are used as system input. Foreground objects are segmented respectively from the depth image by using a novel object segmentation technique based on 2-D histogram with Otsu’s method. Contour analysis with ellipse fitting is performed to locate the potential face region on the detected object. Finally, an eye localization algorithm based on a predefined eye template and geometric features is applied on the extracted facial sub-images, which is a hybrid solution combining appearance and feature based eye detection methods using SVM classification to gain robustness. Our goal is to realize a low-cost and robust machine vision system which is insensitive to low spatial resolution for eye detection and tracking based applications, e.g., driver drowsiness detection, autostereoscopic display for gaming/home/office use. The experimental results of the current work with ARTTS 3-D TOF database and with our own Kinect image database demonstrate that the average eye localization rate per face is more than 92% despite of illumination change, head pose, facial expression and spectacles. The performance can be further improved with the integration of an effective tracking algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Otsu, N.: A Threshold Selection Method from Gray-Level Histograms. IEEE Trans. SMC 9, 62–66 (1979)
Haker, M., Böhme, M., Martinetz, T., Barth, E.: Geometric Invariants for Facial Feature Tracking with 3D TOF Cameras. In: ISSCS, pp. 1–4 (2007)
Kories, R., Zimmermann, G.: Eine Familie von nichtlinearen Operatoren zur robusten Auswertung von Bildfolgen. In: Ausgewählte Verfahren der Mustererkennung und Bildverarbeitung, pp. 96–119 (1989)
Harris, C., Stephens, M.: A combined corner and edge detector. In: Proc. 4th Alvey Vision Conference, pp. 147–151 (1988)
Smith, S.M., Brady, J.M.: SUSAN–A New Approach to Low Level Image Processing. International Journal of Computer Vision 23(1), 45–78 (1997)
Bleiweiss, A., Werman, M.: Fusing TOF Depth and Color for Real-Time Segmentation and Tracking. In: Kolb, A., Koch, R. (eds.) Dyn3D 2009. LNCS, vol. 5742, pp. 58–69. Springer, Heidelberg (2009)
Gokturk, S.B., Yalcin, H., Bamji, C.: A Time-Of-Flight Depth Sensor - System Description, Issues and Solutions. In: Conf. on CVPRW, p. 35 (2004)
Elkhalili, O.: Entwicklung von optischen 3D CMOS-Bildsensoren auf der Basis der Pulslaufzeitmessung. Dissertation, University of Duisburg-Essen (2005)
Akman, O., Bayramoǧlu, N., Alatan, A.A., Jonker, P.: Utilization of Spatial Information for Point Cloud Segmentation. In: 3DTV Conf., pp. 1–4 (2010)
Parvizi, E., Wu, Q.M.J.: Real-Time Approach for Adaptive Object Segmentation in Time-of-Flight Sensors. In: 20th IEEE Int. Conf. on TAI, vol. 1, p. 236 (2008)
Merchán, P., Adán, A., Salamanca, S., Cerrada, C.: 3D Complex Scenes Segmentation from a Single Range Image Using Virtual Exploration. In: Garijo, F.J., Riquelme, J.-C., Toro, M. (eds.) IBERAMIA 2002. LNCS (LNAI), vol. 2527, pp. 923–932. Springer, Heidelberg (2002)
Fitzgibbon, A.W., Pilu, M., Fisher, R.B.: Direct Least Squares Fitting of Ellipses. In: Proc. of the 13th Int. Conf. on Pattern Recognition, vol. 1, pp. 253–257 (1996)
Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features. In: Conf. on CVPR, pp. I-511–I-518 (2001)
Ren, J.F., Jiang, X.D.: Fast Eye Localization Based on Pixel Differences. In: 16th IEEE Int. Conf. on Image Processing, p. 2733–2736 (2009)
Kith, V., El-Sharkawy, M., Bergeson-Dana, T., El-Ramly, S., Elnoubi, S.: A Feature and Appearance Based Method for Eye Detection on Gray Intensity Face Images. In: Int. Conf. on Computer Engineering and Systems, p. 41 (2008)
Kim, H., Lee, J.H., Kee, S.C.: A Fast Eye Localization Method for Face Recognition. In: 13th IEEE Int. Workshop on RHIC, pp. 241–245 (2004)
Yang, F., Dai, J.W., Liu, D.: A Novel Eye Localization Method Based on Spectral Residual Model. In: 7th World Congress on ICA, p. 6773 (2008)
Rurainsky, J., Eisert, P.: Eye Center Localization Using Adaptive Templates. In: Conf. on CVPRW, p. 67 (2004)
Besl, P.J., Jain, R.C.: Segmentation Through Variable-Order Surface Fitting. IEEE Trans. PAMI 10, 167 (1988)
ElSabrouty, M., Hamdy, A., Fawky, A., Khalil, S.: Drowsy Driver Assistant System. In: IEEE Int. Symposium on SPIT, pp. 176–180 (2009)
Yang, J.C., Wu, C.S., Hsiao, C.H., Tsai, R.Y., Hung, Y.P.: Evaluation of an Eye Tracking Technology for 3D Display Applications. In: 3DTV Conf., p. 345 (2008)
ARTTS 3D-TOF Database, http://www.artts.eu/publications/3d_tof_db
OpenKinect Project, http://openkinect.org/wiki/Main_Page
PrimeSense Product Specification, http://www.primesense.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, L., Xu, Y., König, A. (2011). Robust Depth Camera Based Eye Localization for Human-Machine Interactions. In: König, A., Dengel, A., Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based and Intelligent Information and Engineering Systems. KES 2011. Lecture Notes in Computer Science(), vol 6881. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23851-2_44
Download citation
DOI: https://doi.org/10.1007/978-3-642-23851-2_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23850-5
Online ISBN: 978-3-642-23851-2
eBook Packages: Computer ScienceComputer Science (R0)