Skip to main content
SpringerLink
Log in

Joint Shape and Local Appearance Features for Real-Time Driver Drowsiness Detection

  • Conference paper
  • First Online:
Computer Vision – ACCV 2016 Workshops (ACCV 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10118))

Included in the following conference series:

Abstract

In this paper, we propose a framework to detect driver drowsiness from video sequences for an advanced driver assistance system. Our method extracts the effective facial descriptors to describe the drowsiness based on face alignment, and classifies the driver facial states via random forest (RF), finally short-term voting and long-term correlation are applied to output smooth results with long-term memory. In particular, the proposed descriptors can encode both shape and local appearance by the located facial landmarks, and utilize the information from multiple frames to enhance the reliability. The classification and alignment based on RF structure are very efficient for drowsiness detection. Our system can obtain 94% accuracy on our F-DDD dataset and 88.18% accuracy on the evaluating set of NTHU-DDD dataset, meanwhile, the implementation achieves 22 FPS for 640 \(\times \!\) 480 videos.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. World Health Organization: Global Status Report on Road Safety 2013: Supporting a Decade of Action: Summary. World Health Organization, Geneva (2013)

    Google Scholar 

  2. Colic, A., Marques, O., Furht, B.: Driver Drowsiness Detection - Systems and Solutions. Springer Briefs in Computer Science. Springer, Heidelberg (2014)

    Google Scholar 

  3. Wang, J., Gong, Y.: Recognition of multiple drivers’ emotional state. In: ICPR, 8–11 December 2008, pp. 1–4 (2008)

    Google Scholar 

  4. Smith, P., Shah, M., da Vitoria Lobo, N.: Determining driver visual attention with one camera. IEEE Trans. Intell. Transp. Syst. 4, 205–218 (2003)

    Article  Google Scholar 

  5. Rezaei, M., Klette, R.: Look at the driver, look at the road: no distraction! no accident! In: CVPR, 23–28 June 2014, pp. 129–136 (2014)

    Google Scholar 

  6. Liu, W., Song, C., Wang, Y.: Facial expression recognition based on discriminative dictionary learning. In: ICPR, 11–15 November 2012, pp. 1839–1842 (2012)

    Google Scholar 

  7. Chew, S.W., Lucey, S., Lucey, P., Sridharan, S., Conn, J.F.: Improved facial expression recognition via uni-hyperplane classification. In: CVPR, 16–21 June 2012, pp. 2554–2561 (2012)

    Google Scholar 

  8. Liu, P., Han, S., Meng, Z., Tong, Y.: Facial expression recognition via a boosted deep belief network. In: CVPR, 23–28 June 2014, pp. 1805–1812 (2014)

    Google Scholar 

  9. Sun, Y., Yin, L.: Facial expression recognition based on 3D dynamic range model sequences. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5303, pp. 58–71. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88688-4_5

    Chapter  Google Scholar 

  10. Drira, H., Amor, B.B., Daoudi, M., Srivastava, A., Berretti, S.: 3D dynamic expression recognition based on a novel deformation vector field and random forest. In: ICPR, 11–15 November 2012, pp. 1104–1107 (2012)

    Google Scholar 

  11. Wang, Z., Wang, S., Ji, Q.: Capturing complex spatio-temporal relations among facial muscles for facial expression recognition. In: CVPR, 23–28 June 2013, pp. 3422–3429 (2013)

    Google Scholar 

  12. Jung, H., Lee, S., Yim, J., Park, S., Kim, J.: Joint fine-tuning in deep neural networks for facial expression recognition. In: ICCV, 7–13 December 2015, pp. 2983–2991 (2015)

    Google Scholar 

  13. Geng, X., Xia, Y.: Head pose estimation based on multivariate label distribution. In: CVPR, 23–28 June 2014, pp. 1837–1842 (2014)

    Google Scholar 

  14. Demirkus, M., Precup, D., Clark, J.J., Arbel, T.: Probabilistic temporal head pose estimation using a hierarchical graphical model. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 328–344. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10590-1_22

    Google Scholar 

  15. Shi, T., Liang, M., Hu, X.: A reverse hierarchy model for predicting eye fixations. In: CVPR, 23–28 June 2014, pp. 2822–2829 (2014)

    Google Scholar 

  16. Burgos-Artizzu, X.P., Perona, P., Dollár, P.: Robust face landmark estimation under occlusion. In: ICCV, 1–8 December 2013, pp. 1513–1520 (2013)

    Google Scholar 

  17. Wu, Y., Ji, Q.: Robust facial landmark detection under significant head poses and occlusion. In: ICCV 2015, 7–13 December 2015, pp. 3658–3666 (2015)

    Google Scholar 

  18. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: CVPR, 16–21 June 2012, pp. 2879–2886 (2012)

    Google Scholar 

  19. Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 FPS via regressing local binary features. In: CVPR, 23–28 June 2014, pp. 1685–1692 (2014)

    Google Scholar 

  20. Taini, M., Zhao, G., Li, S.Z., Pietikäinen, M.: Facial expression recognition from near-infrared video sequences. In: ICPR, 8–11 December 2008, pp. 1–4 (2008)

    Google Scholar 

  21. Shirakata, T., Tanida, K., Nishiyama, J., Hirata, Y.: Detect the imperceptible drowsiness. SAE Int. J. Passeng. Cars - Electron. Electr. Syst. 3, 98–108 (2010)

    Article  Google Scholar 

  22. Nakamura, T., Maejima, A., Morishima, S.: Detection of driver’s drowsy facial expression. In: 2nd IAPR Asian Conference on Pattern Recognition, ACPR 2013, Naha, Japan, 5–8 November 2013, pp. 749–753 (2013)

    Google Scholar 

  23. Akrout, B., Mahdi, W.: Spatio-temporal features for the automatic control of driver drowsiness state and lack of concentration. Mach. Vis. Appl. 26, 1–13 (2015)

    Article  Google Scholar 

  24. Criminisi, A., Shotton, J., Konukoglu, E.: Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found. Trends Comput. Graph. Vis. 7, 81–227 (2012)

    Article  MATH  Google Scholar 

  25. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, BMVC 2014, Nottingham, UK, 1–5 September 2014 (2014)

    Google Scholar 

  26. Chen, D., Yuan, Z., Wu, Y., Zhang, G., Zheng, N.: Constructing adaptive complex cells for robust visual tracking. In: Proceedings of the IEEE ICCV, pp. 1113–1120 (2013)

    Google Scholar 

  27. Chen, D., Yuan, Z., Hua, G., Wu, Y., Zheng, N.: Description-discrimination collaborative tracking. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 345–360. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10590-1_23

    Google Scholar 

  28. Chen, D., Yuan, Z., Hua, G., Wang, J., Zheng, N.: Multi-timescale collaborative tracking. IEEE TPAMI (2016). doi:10.1109/TPAMI.2016.2539956

Download references

Acknowledgement

This work was supported by the National Basic Research Program of China (No. 2015CB351703), the National Key Research and Development Program of China (No. 2016YFB1001001), and the National Natural Science Foundation of China (No. 61573280).

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China

    Jie Lyu & Zejian Yuan

  2. Shenzhen Forward Innovation Digital Technology Co. Ltd., Shenzhen, China

    Hui Zhang

Authors
  1. Jie Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zejian Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jie Lyu , Hui Zhang or Zejian Yuan .

Editor information

Editors and Affiliations

  1. Institute of Information Science, Academia Sinica, Taipei, Taiwan

    Chu-Song Chen

  2. Tsinghua University , Beijing, China

    Jiwen Lu

  3. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore

    Kai-Kuang Ma

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Lyu, J., Zhang, H., Yuan, Z. (2017). Joint Shape and Local Appearance Features for Real-Time Driver Drowsiness Detection. In: Chen, CS., Lu, J., Ma, KK. (eds) Computer Vision – ACCV 2016 Workshops. ACCV 2016. Lecture Notes in Computer Science(), vol 10118. Springer, Cham. https://doi.org/10.1007/978-3-319-54526-4_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54526-4_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54525-7

  • Online ISBN: 978-3-319-54526-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation