Abstract
Drowsy driver alert systems have been developed to minimize and prevent car accidents. Existing vision-based systems are usually restricted to using visual cues, depend on tedious parameter tuning, or cannot work under general conditions. One additional crucial issue is the lack of public datasets that can be used to evaluate the performance of different methods. In this paper, we introduce a novel hierarchical temporal Deep Belief Network (HTDBN) method for drowsy detection. Our scheme first extracts high-level facial and head feature representations and then use them to recognize drowsiness-related symptoms. Two continuous-hidden Markov models are constructed on top of the DBNs. These are used to model and capture the interactive relations between eyes, mouth and head motions. We also collect a large comprehensive dataset containing various ethnicities, genders, lighting conditions and driving scenarios in pursuit of wide variations of driver videos. Experimental results demonstrate the feasibility of the proposed HTDBN framework in detecting drowsiness based on different visual cues.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bergasa, L., Nuevo, J., Sotelo, M., Barea, R., Lopez, M.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Syst. 7, 63–77 (2006)
World Health Organization: Global status report on road safety 2013: supporting a decade of action: summary. World Health Organization (2013)
Wheaton, G., Shults, R.: Drowsy driving and risk behaviors 10 states and Puerto Rico. Online article (2014)
National Sleep Foundation: Drowsy driving reduction act of 2015 (2014)
Colic, A., Marques, O., Furht, B.: Driver Drowsiness Detection: Systems and Solutions. Springer, Heidelberg (2014)
Mercedes-Benz: Attention assist: drowsiness-detection system warns drivers to prevent them falling asleep momentarily. Online article (2008)
Teyeb, I., Jemai, O., Zaied, M., Ben Amar, C.: A drowsy driver detection system based on a new method of head posture estimation. In: Corchado, E., Lozano, J.A., Quintián, H., Yin, H. (eds.) IDEAL 2014. LNCS, vol. 8669, pp. 362–369. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10840-7_44
Qiang, J., Lan, P., Looney, C.: A probabilistic framework for modeling and real-time monitoring human fatigue. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 36, 862–875 (2006)
Lucey, P., Cohn, J., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2010)
Wu, D., Shao, L.: Leveraging hierarchical parametric networks for skeletal joints based action segmentation and recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 724–731 (2014)
Yang, G., Lin, Y., Bhattacharya, P.: A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Inf. Sci. 180, 1942–1954 (2010)
Ji, Q., Zhu, Z., Lan, P.: Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Trans. Veh. Technol. 53, 1052–1068 (2004)
Mohamed, A., Dahl, G., Hinton, G.: Acoustic modeling using deep belief networks. IEEE Trans. Audio Speech Lang. Process. 20, 14–22 (2012)
Dasgupta, A., George, A., Happy, S., Routray, A.: A vision-based system for monitoring the loss of attention in automotive drivers. IEEE Trans. Intell. Transp. Syst. 14, 1825–1838 (2013)
Alioua, N., Amine, A., Rziza, M.: Drivers fatigue detection based on yawning extraction. Int. J. Veh. Technol. 2014 (2014)
Rezaei, M., Klette, R.: Look at the driver, look at the road: no distraction! No accident! In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 129–136 (2014)
Smith, P., Shah, M., da Vitoria Lobo, N.: Determining driver visual attention with one camera. IEEE Trans. Intell. Transp. Syst. 4, 205–218 (2003)
Eskandarian, A., Sayed, R.: Analysis of driver impairment, fatigue, and drowsiness and an unobtrusive vehicle-based detection scheme. In: Proceeding of International Conference on Traffic Accidents (2005)
Taylor, G., Hinton, G., Roweis, S.: Modeling human motion using binary latent variables. In: Neural Information Processing Systems, pp. 1345–1352 (2006)
Hinton, G., Osindero, S.: A fast learning algorithm for deep belief nets. Neural Comput. 18 (2006)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)
Xiong, X., de la Torre, F.: Supervised descent method and its application to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)
DeMenthon, F., Davis, L.: Model-based object pose in 25 lines of code. Int. J. Comput. Vis. 15, 123–141 (1995)
Heo, J., Savvides, M.: Gender and ethnicity specific generic elastic models from a single 2D image for novel 2D pose face synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2341–2350 (2012)
Yang, X., Tian, Y.: Eigenjoints-based action recognition using naive-bayes-nearest-neighbor. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 14–19 (2012)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates Inc, Red Hook (2012)
Hinton, G.E.: A practical guide to training restricted Boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, 2nd edn, pp. 599–619. Springer, Heidelberg (2012). doi:10.1007/978-3-642-35289-8_32
Freund, Y., Haussler, D.: Unsupervised learning of distributions on binary vectors using two layer networks. Technical report, University of California at Santa Cruz, Santa Cruz, CA, USA (1994)
Yang, L., Widjaja, B., Prasad, R.: Application of hidden Markov models for signature verification. Pattern Recogn. 28, 161–170 (1995)
Devijver, P.A.: Baum’s forward-backward algorithm revisited. Pattern Recogn. Lett. 3, 369–373 (1985)
Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., Hariri, B.: YawDD: a yawning detection dataset. In: Proceedings of the 5th ACM Multimedia Systems Conference, pp. 24–28. ACM (2014)
Omidyeganeh, M., Shirmohammadi, S., Abtahi, S., Khurshid, A., Farhan, M., Scharcanski, J., Hariri, B., Laroche, D., Martel, L.: Yawning detection using embedded smart cameras. IEEE Trans. Instrum. Meas. 65, 570–582 (2016)
Zhang, W., Murphey, Y.L., Wang, T., Xu, Q.: Driver yawning detection based on deep convolutional neural learning and robust nose tracking. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2015)
Acknowledgement
The authors would like to thank Qualcomm Technologies Inc. for supporting this research work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Weng, CH., Lai, YH., Lai, SH. (2017). Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network. In: Chen, CS., Lu, J., Ma, KK. (eds) Computer Vision – ACCV 2016 Workshops. ACCV 2016. Lecture Notes in Computer Science(), vol 10118. Springer, Cham. https://doi.org/10.1007/978-3-319-54526-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-54526-4_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54525-7
Online ISBN: 978-3-319-54526-4
eBook Packages: Computer ScienceComputer Science (R0)