Skip to main content

A Driver Abnormal Behavior Detection Method Based on Improved YOLOv7 and OpenPose

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14090))

Included in the following conference series:

  • 906 Accesses

Abstract

The current driver’s abnormal behavior detection is interfered with by complex backgrounds, body self-obscuring, and other factors leading to low accuracy of small object detection and large error of human node detection. In this paper, we propose a driver abnormal behavior detection method based on improved YOLOv7 and OpenPose. Firstly, we add an attention mechanism to the backbone network of YOLOv7, increase the feature information contained in the shallow feature map, introduce Wise-IoU loss function to improve the detection accuracy of small and long objects (e.g., cigarettes, water glasses). Secondly, we calculate the IoU values between the driver’s hand and the confidence frame of small objects to detect the abnormal behavior of driver-object interaction (e.g., smoking, drinking). Thirdly, we improve the OpenPose network model by replacing convolutional kernels and adjusting the connection of convolutional kernels to achieve two-dimensional node detection of the driver’s driving posture. Finally, we use the FastDTW algorithm, and calculate the similarity of the driver’s two-dimensional nodal information to detect the abnormal posture of the driver. The experimental results show that the method in this paper has high accuracy of small object detection and low nodal error, and can effectively detect abnormal behaviors such as smoking, drinking and abnormal posture of drivers during driving.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhang, H., Zhuang, X., Zheng, J.: Optimization of YOLO network for human anomalous behavior detection. Comput. Eng. Appl. 59(7), 242–249 (2023)

    Google Scholar 

  2. Bao, G., Xi, X., Zhang, H.: A review of human abnormal behavior detection based on deep learning. Ind. Control Comput. 35(5), 102–103+106 (2022),

    Google Scholar 

  3. Tawari, A., Martin, S., Trivedi, M.: Continuous head movement estimator for driver assistance: issues, algorithms, and on-road evaluations. IEEE Trans. Intell. Transp. Syst. 15(2), 818–830 (2014)

    Article  Google Scholar 

  4. Diaz-Chito, K., Hernández-Sabaté, A., López, A.: A reduced feature set for driver head pose estimation. Appl. Soft Comput. 45(3), 98–107 (2016)

    Article  Google Scholar 

  5. Diaz-Chito, K., Del Rincon, M., Hernández-Sabaté, A.: Continuous head pose estimation using manifold subspace embedding and multivariate regression. IEEE Access 6(18), 325–334 (2018)

    Google Scholar 

  6. Borghi, G., Fabbri, M., Vezzani, R.: Face-from-depth for head pose estimation on depth images. IEEE Trans. Pattern Anal. Mach. Intell. 42(3), 596–609 (2018)

    Article  Google Scholar 

  7. Hu, T., Jha, S., Busso, C.: Robust driver head pose estimation in naturalistic conditions from point-cloud data. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1176–1182. IEEE Press (2020)

    Google Scholar 

  8. Liu, Y., Lasang, P., Pranata, S.: Driver pose estimation using recurrent lightweight network and virtual data augmented transfer learning. IEEE Trans. Intell. Transp. Syst. 20(10), 3818–3831 (2019)

    Article  Google Scholar 

  9. Deng, X., Zhang, Y., Yang, S.: Joint hand detection and rotation estimation using CNN. IEEE Trans. Image Process. 27(4), 1888–1900 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  10. Wang, Q., Zhang, G., Yu, S.: 2D hand detection using multi-feature skin model supervised cascaded CNN. J. Signal Process. Syst. 91(10), 1105–1113 (2019)

    Google Scholar 

  11. Xia, Y., Yan, S., Zhang, B.: Combination of ACF detector and multi-task CNN for hand detection. In: 2016 IEEE 13th International Conference on Signal Processing (ICSP), pp. 601–606. IEEE Press (2016)

    Google Scholar 

  12. Yan, S., Xia, Y., Smith, S.: Multiscale convolutional neural networks for hand detection. Appl. Comput. Intell. Soft Comput., 9830641–9830654 (2017)

    Google Scholar 

  13. Yuen, K., Trivedi, M.: Looking at hands in autonomous vehicles: a convnet approach using part affinity fields. IEEE Trans. Intell. Veh. 5(3), 361–371 (2019)

    Article  Google Scholar 

  14. Chen, N., Man, Y., Ning, W.: A deep learning-based approach for monitoring abnormal pilot driving behavior. J. Saf. Environ. 22(1), 249–255 (2022)

    Google Scholar 

  15. Wang, C., Bochkovskiy, A., Liao, H.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv, 2207, 02696 (2022)

    Google Scholar 

  16. Cao, Z., Simon, T., Wei, S.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)

    Google Scholar 

  17. Wang, L., Zhao, T., Wang, W.: Pedestrian detection and tracking algorithm with pose change robustness. Comput. Eng. Des. 43(10), 2877–2881 (2022)

    Google Scholar 

  18. Xiong, M., Li, J., Xiong, J.: Pedestrian fall detection method based on optical flow reconstruction and deep pose features. Sci. Technol. Eng. 22(35), 15688–15696 (2022)

    Google Scholar 

  19. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  20. Girshick, R.: Fast R-CnN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

    Google Scholar 

  21. Dotse, J., Nicolson, R., Rowe, R.: Behavioral influences on driver crash risks in Ghana: a qualitative study of commercial passenger drivers. Traffic Inj. Prev. 20(2), 134–139 (2019)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Funding Project of Beijing Social Science Foundation (No. 19YTC043).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yan Hu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cai, X., Zhou, S., Yao, J., Cheng, P., Hu, Y. (2023). A Driver Abnormal Behavior Detection Method Based on Improved YOLOv7 and OpenPose. In: Huang, DS., Premaratne, P., Jin, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science(), vol 14090. Springer, Singapore. https://doi.org/10.1007/978-981-99-4761-4_21

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-4761-4_21

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-4760-7

  • Online ISBN: 978-981-99-4761-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics