Abstract
Driver’s drowsiness is one of the major causes of increase in the number of road accidents. Therefore, design and implementation of a real-time driver’s drowsiness detection system are considered as a crucial component of the Advanced Driver Assistance System (ADAS). Along with other physiological parameters, yawn is often considered as one of the most important indicators of drowsiness in human. Thus, in this work we propose an efficient and nonintrusive system for yawn detection targeted toward real-time drowsiness detection. The proposed algorithmic pipeline consists of a face and facial landmark detector for face detection and landmark localization, a scheme for extracting feature named mouth aspect ratio (MAR) characterizing the state of the mouth (open/close) in each frame, and a classifier to classify the state of the mouth in a temporal window of some fixed number of frames. The performance of the proposed approach has been validated on a manually annotated dataset extracted from the widely used yawn detection dataset called YawDD. The proposed approach has achieved an accuracy of 99.25% along with F1 score of 98.00% and runs at 30 frames per second (FPS).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
National Center for Statistics and Analysis.: Fatal motor vehicle crashes: Overview. (Traffic Safety Facts Research Note. Report No. DOT HS 812 456). National Highway Traffic Safety Administration, Washington, DC (Oct 2017)
Ji, Q., Zhu, Z., Lan, P.: Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Trans. Veh. Technol. 53(4), 1052–1068 (2004)
Wang, T., Shi, P.: Yawning detection for determining driver drowsiness. In: Proceedings of 2005 IEEE International Workshop on VLSI Design and Video Technology, 2005, pp. 373–376 (May 2005)
Rongben, W., Lie, G., Bingliang, T., Lisheng, J., Monitoring mouth movement for driver fatigue or distraction with one camera. In: Proceedings of the 7th International IEEE Conference on Intelligent Transportation Systems (Oct 2004)
Lu, Y., Wang, Z.: Detecting driver yawning in successive images. In: 2007 1st International Conference on Bioinformatics and Biomedical Engineering, pp. 581–583 (July 2007)
Fan, X., Yin, B.C., Sun, Y.F.: Yawning detection for monitoring driver fatigue. In: 2007 International Conference on Machine Learning and Cybernetics, Vol. 2, pp. 664–668. IEEE (Aug 2007)
Medeiros, R.S., Scharcanski, J., Wong, A.: Multi-scale stochastic color texture models for skin region segmentation and gesture detection. In: 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–4. IEEE (July 2013)
Li, L., Chen, Y., Li, Z.: Yawning detection for monitoring driver fatigue based on two cameras. In: 2009 12th International IEEE Conference on Intelligent Transportation Systems, pp. 1–6. IEEE (Oct 2009)
Bouvier, C., Benoit, A., Caplier, A., Coulon, P.Y.: Open or closed mouth state detection: static supervised classification based on log-polar signature. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 1093–1102. Springer, Berlin (Oct 2008)
Wei, B., Lu, X., Zhang, C., Wu, X.: Efficient detection of eye blinking and yawn based on facial video utilizing IPPG technique. In: 2016 International Forum on Mechanical, Control and Automation (IFMCA 2016). Atlantis Press (Mar 2017)
Alioua, N., Amine, A., Rziza, M.: Driver’s fatigue detection based on yawning extraction. Int. J. Veh. Technol. (2014)
Omidyeganeh, M., Shirmohammadi, S., Abtahi, S., Khurshid, A., Farhan, M., Scharcanski, J., Hariri, B., Laroche, D., Martel, L.: Yawning detection using embedded smart cameras. IEEE Trans. Instrum. Meas. 65(3), 570–582 (2016)
Jie, Z., Mahmoud, M., Stafford-Fraser, Q., Robinson, P., Dias, E., Skrypchuk, L.: Analysis of yawning behaviour in spontaneous expressions of drowsy drivers. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 571–576. IEEE (May 2018)
Zhang, W., Su, J.: Driver yawning detection based on long short-term memory networks. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–5. IEEE (Nov 2017)
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1867–1874 (2014)
Cech, J., Soukupova, T.: Real-time eye blink detection using facial landmarks. 21st Comput. Vis. Winter Work (2016)
Huang, G.B.: What are extreme learning machines? Filling the gap between Frank Rosenblatt’s dream and John von Neumann’s puzzle. Cognitive. Comput. 7(3), 263–278 (2015)
Vashisth, S., Saurav, S.: Histogram of oriented gradients based reduced feature for traffic sign recognition. In: 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 2206–2212. IEEE (Sep 2018)
Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus). Preprint at arXiv:1511.07289 (2015)
Hsu, C.W., Chang, C.C., Lin, C.J.: A practical guide to support vector classification (2003)
Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., Hariri, B.: YawDD: a yawning detection dataset. In: Proceedings of the 5th ACM Multimedia Systems Conference, pp. 24–28. ACM (Mar 2014)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Akusok, A., Björk, K.M., Miche, Y., Lendasse, A.: High-performance extreme learning machines: a complete toolbox for big data applications. IEEE Access 3, 1011–1025 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Saurav, S., Kasliwal, M., Agrawal, R., Singh, S., Saini, R. (2021). Real-Time Yawn Extraction for Driver’s Drowsiness Detection. In: Bhateja, V., Peng, SL., Satapathy, S.C., Zhang, YD. (eds) Evolution in Computational Intelligence. Advances in Intelligent Systems and Computing, vol 1176. Springer, Singapore. https://doi.org/10.1007/978-981-15-5788-0_52
Download citation
DOI: https://doi.org/10.1007/978-981-15-5788-0_52
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-5787-3
Online ISBN: 978-981-15-5788-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)