Skip to main content
SpringerLink
Log in

MDAD: A Multimodal and Multiview in-Vehicle Driver Action Dataset

  • Conference paper
  • First Online:
Computer Analysis of Images and Patterns (CAIP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11678))

Included in the following conference series:

Abstract

“Driver’s distraction is deadly!”. Due to its crucial role in saving lives, driver action recognition is an important and trending topic in the field of computer vision. However, a very limited number of public datasets are available to validate proposed methods. This paper introduces a new public, well structured and extensive dataset, named Multiview and multimodal in-vehicle Driver Action Dataset (MDAD). MDAD consists of two temporally synchronised data modalities from side and frontal views. These modalities include RGB and depth data from different Kinect cameras. Many subjects with various body sizes, gender and ages are asked to perform 16 in-vehicle actions in several weather conditions. Each subject drives the vehicle on multiple trip routes in Sousse, Tunisia, at different times to describe a large range of head rotations, changes in lighting conditions and some occlusions. Our recorded dataset provides researchers with a testbed to develop new algorithms across multiple modalities and views under different illumination conditions. To demonstrate the utility of our dataset, we analyze driver action recognition results from each modality and every view independently, and then we combine modalities and views. This public dataset is of benefit to research activities for humans driver action analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://sites.google.com/site/benkhalifaanouar1/6-datasets.

References

  1. Ameur, S., Khalifa, A.B., Bouhlel, M.S.: A comprehensive leap motion database for hand gesture recognition. In: 2016 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), pp. 514–519, December 2016. https://doi.org/10.1109/SETIT.2016.7939924

  2. Billah, T., Rahman, S.M.M., Ahmad, M.O., Swamy, M.N.S.: Recognizing distractions for assistive driving by tracking body parts. IEEE Trans. Circ. Syst. Video Technol. 29(4), 1048–1062 (2019). https://doi.org/10.1109/TCSVT.2018.2818407

    Article  Google Scholar 

  3. Borghi, G., Frigieri, E., Vezzani, R., Cucchiara, R.: Hands on the wheel: a dataset for driver hand detection and tracking. In: 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), pp. 564–570, May 2018. https://doi.org/10.1109/FG.2018.00090

  4. Botta, M., Cancelliere, R., Ghignone, L., Tango, F., Gallinari, P., Luison, C.: Real-time detection of driver distraction: random projections for pseudo-inversion-based neural training. Knowl. Inf. Syst. 1–16 (2019). https://doi.org/10.1007/s10115-019-01339-0

    Article  Google Scholar 

  5. Chebli, K., Khalifa, A.B.: Pedestrian detection based on background compensation with block-matching algorithm. In: 2018 15th International Multi-Conference on Systems, Signals Devices (SSD), pp. 497–501, March 2018. https://doi.org/10.1109/SSD.2018.8570499

  6. Craye, C., Karray, F.: Driver distraction detection and recognition using RGB-D sensor. arXiv preprint arXiv:1502.00250 (2015)

  7. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893, June 2005. https://doi.org/10.1109/CVPR.2005.177

  8. Danafar, S., Gheissari, N.: Action recognition for surveillance applications using optic flow and SVM. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007. LNCS, vol. 4844, pp. 457–466. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76390-1_45

    Chapter  Google Scholar 

  9. Eraqi, H.M., Abouelnaga, Y., Saad, M.H., Moustafa, M.N.: Driver distraction identification with an ensemble of convolutional neural networks. J. Adv. Transp. 2019, 1–12 (2019)

    Article  Google Scholar 

  10. State Farm: state farm distracted driver detection. https://www.kaggle.com/c/state-farm-distracted-driver-detection/

  11. Filtness, A., et al.: Safety implications of co-locating road signs: a driving simulator investigation. Transp. Res. Part F: Traffic Psychol. Behav. 47, 187–198 (2017). https://doi.org/10.1016/j.trf.2017.04.007

    Article  Google Scholar 

  12. Gillmeier, K., Schuettke, T., Diederichs, F., Miteva, G., Spath, D.: Combined driver distraction and intention algorithm for maneuver prediction and collision avoidance. In: 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), pp. 1–6, September 2018. https://doi.org/10.1109/ICVES.2018.8519520

  13. Jafarnejad, S., Castignani, G., Engel, T.: Non-intrusive distracted driving detection based on driving sensing data. In: VEHITS, pp. 178–186 (2018)

    Google Scholar 

  14. Jegham, I., Ben Khalifa, A.: Pedestrian detection in poor weather conditions using moving camera. In: 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), pp. 358–362, October 2017. https://doi.org/10.1109/AICCSA.2017.35

  15. Jegham, I., Ben Khalifa, A., Alouani, I., Mahjoub, M.A.: Safe driving: driver action recognition using SURF keypoints. In: 2018 30th International Conference on Microelectronics (ICM), pp. 60–63, December 2018. https://doi.org/10.1109/ICM.2018.8704009

  16. Knapik, M., Cyganek, B.: Driver’s fatigue recognition based on yawn detection in thermal images. Neurocomputing 338, 274–292 (2019). https://doi.org/10.1016/j.neucom.2019.02.014

    Article  Google Scholar 

  17. Laptev, I., Lindeberg, T.: Space-time interest points. In: Proceedings Ninth IEEE International Conference on Computer Vision, vol. 1, pp. 432–439, October 2003. https://doi.org/10.1109/ICCV.2003.1238378

  18. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, June 2008. https://doi.org/10.1109/CVPR.2008.4587756

  19. Lee, J.D., Young, K.L., Regan, M.A.: Defining driver distraction. In: Driver Distraction: Theory Effects and Mitigation, vol. 13, no. 4, pp. 31–40 (2008)

    Chapter  Google Scholar 

  20. Lejmi, W., Ben Khalifa, A., Mahjoub, M.A.: Fusion strategies for recognition of violence actions. In: 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), pp. 178–183, October 2017. https://doi.org/10.1109/AICCSA.2017.193

  21. Lejmi, W., Mahjoub, M.A., Ben Khalifa, A.: Event detection in video sequences: challenges and perspectives. In: 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pp. 682–690, July 2017. https://doi.org/10.1109/FSKD.2017.8393354

  22. Li, N., Busso, C.: Analysis of facial features of drivers under cognitive and visual distractions. In: 2013 IEEE International Conference on Multimedia and Expo (ICME), July 2013. https://doi.org/10.1109/ICME.2013.6607575

  23. McNally, W., Wong, A., McPhee, J.: STAR-Net: action recognition using spatio-temporal activation reprojection. arXiv preprint arXiv:1902.10024 (2019)

  24. Microsoft: Kinect for windows. http://www.microsoft.com/en-us/kinectforwindows/

  25. Mimouna, A., Khalifa, A.B., Ben Amara, N.E.: Human action recognition using triaxial accelerometer data: selective approach. In: 2018 15th International Multi-Conference on Systems, Signals Devices (SSD), pp. 491–496, March 2018. https://doi.org/10.1109/SSD.2018.8570429

  26. Qin, L., Li, Z.R., Chen, Z., Bill, M.A., Noyce, D.A.: Understanding driver distractions in fatal crashes: an exploratory empirical analysis. J. Saf. Res. 69, 23–31 (2019)

    Article  Google Scholar 

  27. Rangesh, A., Ohn-Bar, E., Trivedi, M.M.: Hidden hands: tracking hands with an occlusion aware tracker. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1224–1231, June 2016. https://doi.org/10.1109/CVPRW.2016.155

  28. Regan, M.A., Hallett, C., Gordon, C.P.: Driver distraction and driver inattention: definition, relationship and taxonomy. Accid. Anal. Prev. 43(5), 1771–1781 (2011). https://doi.org/10.1016/j.aap.2011.04.008

    Article  Google Scholar 

  29. Reyes-Muñoz, A., Domingo, M., López-Trinidad, M., Delgado, J.: Integration of body sensor networks and vehicular ad-hoc networks for traffic safety. Sensors 16(1), 107 (2016)

    Article  Google Scholar 

  30. Wang, H., Ullah, M.M., Klaser, A., Laptev, I., Schmid, C.: Evaluation of local spatio-temporal features for action recognition. In: British Machine Vision Conference, BMVC 2009, p. 124–1. BMVA Press (2009)

    Google Scholar 

  31. WHO: distracted driving. https://www.who.int/violence-injury-prevention/publications/road-traffic/distracted-driving-en.pdf

  32. WHO: The top 10 causes of death. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death

  33. Zhao, C., Zhang, B., Lian, J., He, J., Lin, T., Zhang, X.: Classification of driving postures by support vector machines. In: 2011 Sixth International Conference on Image and Graphics, pp. 926–930, August 2011. https://doi.org/10.1109/ICIG.2011.184

Download references

Author information

Authors and Affiliations

  1. Institut supérieur de l’informatique et des techniques de communication, LATIS-Laboratory of Advanced Technology and Intelligent Systems, Université de Sousse, 4011, Sousse, Tunisia

    Imen Jegham

  2. Ecole Nationale d’Ingénieurs de Sousse, LATIS-Laboratory of Advanced Technology and Intelligent Systems, Université de Sousse, 4023, Sousse, Tunisia

    Anouar Ben Khalifa & Mohamed Ali Mahjoub

  3. IEMN-DOAE, Université polytechnique Hauts-de-France, Valenciennes, France

    Ihsen Alouani

Authors
  1. Imen Jegham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anouar Ben Khalifa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ihsen Alouani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Ali Mahjoub
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imen Jegham .

Editor information

Editors and Affiliations

  1. Department of Computer and Electrical Engineering and Applied Mathematics, University of Salerno, Fisciano (SA), Italy

    Mario Vento

  2. Department of Computer and Electrical Engineering and Applied Mathematics, University of Salerno, Fisciano (SA), Italy

    Gennaro Percannella

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jegham, I., Ben Khalifa, A., Alouani, I., Mahjoub, M.A. (2019). MDAD: A Multimodal and Multiview in-Vehicle Driver Action Dataset. In: Vento, M., Percannella, G. (eds) Computer Analysis of Images and Patterns. CAIP 2019. Lecture Notes in Computer Science(), vol 11678. Springer, Cham. https://doi.org/10.1007/978-3-030-29888-3_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29888-3_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29887-6

  • Online ISBN: 978-3-030-29888-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation