Skip to main content
SpringerLink
Log in

Driver drowsiness detection using modified deep learning architecture

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

This paper proposes a non-invasive approach to detect driver drowsiness. The facial features are used for detecting the driver’s drowsiness. The mouth and eye regions are extracted from the video frame. These extracted regions are applied on hybrid deep learning model for drowsiness detection. A hybrid deep learning model is proposed by incorporating both modified InceptionV3 and long short-term memory (LSTM) network. InceptionV3 is modified by adding global average pooling layer for spatial robustness and dropout technique to prevent overfitting on training data. The proposed hybrid model is compared with convolutional neural network, IncpetionV3, and LSTM over NTHU-DDD dataset. The proposed model performs better than the other model in terms of performance measures. The proposed model is able to detect driver fatigue effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. World Health Organization (2015) ; World Health Organization: Geneva, Switzerland, 2015

  2. Distracted Driving-Motor Vehicle Safety-CDC Injury Center. https://www.cdc.gov/motorvehiclesafety/distracteddriving/

  3. Johnson T(2018) 2017 Traffic Safety Culture Index. https://aaafoundation.org/2017-traffic-safety-culture-index/

  4. Ed-doughmi Y, Idrissi N(2019) Driver Fatigue Detection using Recurrent Neural Networks. In: Proceedings of 2nd International Conference on Networking, Information Systems & Security, Rabat, Morocco, NY, USA, pp. 44

  5. Chirra VRR, Uyyala SR, Kolli VKK (2019) Deep CNN: A machine learning approach for driver drowsiness detection based on eye state. Reve d’Intelligence Artificielle 33(6):461–466

    Article  Google Scholar 

  6. Summala H (2007) Towards understanding motivational and emotional factors in driver behaviour: Comfort through satisficing. Modelling Driver Behaviour in Automotive Environments. Springer, Berlin/Heidelberg, Germany, pp 189–207

    Chapter  Google Scholar 

  7. Igasaki T, Nagasawa K, Murayama N, Hu Z(2015) Drowsiness estimation under driving environment by heart rate variability and/or breathing rate variability with logistic regression analysis. In: International Conference on Biomedical Engineering and Informatics (BMEI), pp. 189–193

  8. Ramzan M, Khan HU, Awan SM, Ismail A, Ilyas M, Mahmood A (2019) A Survey on State-of-the-Art Drowsiness Detection Techniques. IEEE Access

  9. Borghini G, Astolfi L, Vecchiato G, Mattia D, Babiloni F (2014) Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev 44:58–75

    Article  Google Scholar 

  10. Kulathumani A, Soua R, Karray F, Kamel MS (2017) Recent trends in driver safety monitoring systems: state of the art and challenges. IEEE Trans Veh Technol 66(6):4550–4563

    Article  Google Scholar 

  11. LeCun Y, Bengio Y(1995) Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361

  12. Lawrence S, Giles CL, Tsoi AC, Back AD (1997) Face recognition: A convolutional neural-network approach. IEEE Trans Neural Networks 8(1):98–113

    Article  Google Scholar 

  13. Majdi MS, Ram S, Gill JT, Rodr ́ıguez JJ(2018) Drive-net: Convolutional network for driver distraction detection. In: IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 1–4

  14. Sajjanhar A, Wu Z, Wen Q(2018) Deep learning models for facial expression recognition. In: Digital Image Computing: Techniques and Applications, pp. 1–6

  15. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z(2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826

  16. Hochreiter S, Schmidhuber J (1997) Long short-term memory. 9:1735–1780Neural computation8

  17. Rengasamy D, Morvan HP, Figueredo GP(2018) Deep learning approaches to aircraft maintenance, repair and overhaul: a review. In: International Conference on Intelligent Transportation Systems, pp. 150–156

  18. Omidyeganeh M, Javadtalab A, Shirmohammadi S(2011) Intelligent driver drowsiness detection through fusion of yawning and eye closure. IEEE International Conference on Virtual Environments Human-Computer Interfaces and Measurement Systems Proceedings, pp. 1–6

  19. Li G, Lee B, Chung W (2015) Smartwatch-Based Wearable EEG System for Driver Drowsiness Detection. IEEE Sens J 15(12):7169–7180

    Article  Google Scholar 

  20. You F, Li Y-H, Huang L, Chen K, Zhang R-H, Xu J-M (2017) Monitoring drivers’ sleepy status at night based on machine vision. Multimedia Tools and Applications 76(13):14869–14886

    Article  Google Scholar 

  21. Massoz Q, Langohr T, François C, Verly JG(2016) The ULg multimodality drowsiness database (called DROZY) and examples of use. IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–7

  22. Zhang Y, Hua C (2015) Driver fatigue recognition based on facial expression analysis using local binary patterns. Optik 126(23):4501–4505

    Article  Google Scholar 

  23. Jabbar R, Al-Khalifa K, Kharbeche M, Alhajyaseen W, Jafari M, Jiang S (2018) Real-time Driver Drowsiness Detection for Android Application Using Deep Neural Networks Techniques. Procedia Comput Sci 130:400–407

    Article  Google Scholar 

  24. Shi W, Li J, Yang Y (2020) Face fatigue detection method based on MTCNN and machine vision. Advances in Intelligent Systems and Computing. Huainan, China, pp 233–240

    Google Scholar 

  25. Zhao Z, Zhou N, Zhang L, Yan H, Xu Y, Zhang Z (2020) Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN, 7251280 edn. Computational Intelligence and Neuroscience

  26. Gwak J, Hirao A, Shino M (2020) An investigation of early detection of driver drowsiness using ensemble machine learning based on hybrid sensing. Appl Sci 10(8):2890

    Article  Google Scholar 

  27. Kepesiova Z, Ciganek J, Kozak S(2020) Driver drowsiness detection using convolutional neural networks. In: 2020 Cybernetics & Informatics (K&I)

  28. Sathasivam S, Mahamad AK, Saon S, Sidek A, Som MM, Ameen HA(2020) Drowsiness detection system using eye aspect ratio technique. In 2020 IEEE Student Conference on Research and Development (SCOReD)

  29. Savas BK, Becerikli Y (2020) Real time driver fatigue detection system based on multi-task ConNN. IEEE Access 8:12491–12498

    Article  Google Scholar 

  30. Chen W, Huang H, Peng S et al (2021) YOLO-face: a real-time face detector. Visual Computers 37:805–813

    Article  Google Scholar 

  31. Sinha A, Aneesh RP, Gopal SK(2021) Drowsiness Detection System Using Deep Learning. International conference on Bio Signals, Images, and Instrumentation, Chennai, India

  32. Rajkar A, Kulkarni N, Raut A (2022) Driver Drowsiness Detection Using Deep Learning. In: Iyer B, Ghosh D, Balas VE (eds) Applied Information Processing Systems. Advances in Intelligent Systems and Computing, vol 1354. Springer, Singapore

    Google Scholar 

  33. Ed-Doughmi Y, Idrissi N, Hbali Y (2020) Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network. J Imaging 6(3):8

    Article  Google Scholar 

  34. Faraji F, Lotfi F, Khorramdel J, Najafi A, Ghaffari A(2021) Drowsiness Detection Based On Driver Temporal Behavior Using a New Developed Dataset. ArXiv:2104.00125

  35. Mase JM, Chapman P, Figueredo GP, Torres MT(2020) A Hybrid Deep Learning Approach for Driver Distraction Detection. International Conference on Information and Communication Technology Convergence, Jeju, Korea (South)

  36. Computer Vision Lab, National Tsuing Hua University. Driver Drowsiness Detection Dataset (2016) Available online: http://cv.cs.nthu.edu.tw/php/callforpaper/datasets/DDD/

  37. Park S, Pan F, Kang S, Yoo CD(2016) Driver drowsiness detection system based on feature representation learning using various deep networks. In: Proceedings of the Computer Vision – ACCV 2016 Workshops, vol. 10118, pp.154–164

  38. Yarlagadda V, Koolagudi SG, Kumar M, Donepudi S(2020) Driver drowsiness detection using facial parameters and RNNs with LSTM. In: India Council International Conference (INDICON), New Delhi

  39. Rohila VS, Kumar V, Barnwal KK (2021) Distracted Driver Detection System Using Deep Learning Technique. Handbook of Research on Machine Learning Techniques for Pattern Recognition and Information Security

  40. Kumar PJ (2018) Multilayer Perceptron Neural Network Based Immersive VR System for Cognitive Computer Gaming. Progress in Advanced Computing and Intelligent Engineering. Springer, Berlin/Heidelberg, Germany, pp 91–102

    Chapter  Google Scholar 

  41. Mbouna RO, Kong SG, Chun MG (2013) Visual analysis of eye state and head pose for driver alertness monitoring. IEEE Trans Intell Transp Syst 14:1462–1469

    Article  Google Scholar 

  42. Omidyeganeh M, Shirmohammadi S, Abtahi S, Khurshid A, Farhan M, Scharcanski J, Hariri B, Laroche D, Martel L (2016) Yawning detection using embedded smart cameras. IEEE Trans Instrum Meas 65:570–582

    Article  Google Scholar 

  43. Weng CH, Lai YH, Lai SH(2016) Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network. In Proceedings of the Asian Conference on Computer Vision, Taipei, Taiwan, pp. 117–133

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, National Institute of Technology, Hamirpur, Himachal Pradesh, India

    Vijay Kumar, Shivam Sharma &  Ranjeet

Authors
  1. Vijay Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shivam Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ranjeet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vijay Kumar.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, V., Sharma, S. & Ranjeet Driver drowsiness detection using modified deep learning architecture. Evol. Intel. 16, 1907–1916 (2023). https://doi.org/10.1007/s12065-022-00743-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-022-00743-w

Keywords

Search

Navigation