Skip to main content
Log in

Attention-based global context network for driving maneuvers prediction

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Driving maneuvers prediction is one of the most challenging tasks in modern Advanced Driver Assistance System. Such predictions can improve driving safety by alerting the driver to the danger of unsafe or risky traffic situations. In this research, we presents a novel Attention-based Global Context Network (AGCNet) for driving maneuvers prediction from multiple modality data, including front view video data and driver physiological signals. Firstly, with Global Context block, the AGCNet has an ability of modeling the long-range dependency contextual features from multi-modal data with lightweight computation. Secondly, the Channel-wise Attention is introduced in AGCNet to focus on valuable features. Finally, a custom-built Dual attention-based Long Short-Term Memory (DaLSTM) network is designed to learn co-occurrence features and predict driving maneuvers. Specially, the DaLSTM can employ attention mechanisms over heterogeneous features and time steps simultaneously. The experimental results show that the AGCNet is capable of learning the latent features of driving maneuvers and achieving significantly better performance than other advanced models on a real-world driving dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Gao, J., Murphey, Y.L., Yi, J.G., et al.: A data-driven lane-changing behavior detection system based on sequence learning. Transportmetr. B Transp. Dyn. 10(1), 831–848 (2020)

    Article  Google Scholar 

  2. Gao, J., Murphey, Y.L., Zhu, H.H.: Personalized detection of lane changing behavior using multisensor data fusion. Computing 101(12), 1837–1860 (2019)

    Article  MathSciNet  Google Scholar 

  3. Zyner, A., Worrall, S., Ward, J., et al.: Long short term memory for driver intent prediction. In: IEEE Intelligent Vehicles Symposium, pp. 1484–1489. IEEE (2017)

  4. Peng, X., Murphey, Y.L., Liu, R., et al.: Driving maneuver early detection via sequence learning from vehicle signals and video images. Pattern Recogn. 103, 107276 (2020)

    Article  Google Scholar 

  5. Kim, H.T., Song, B., Lee, H., et al.: Multiple vehicle recognition based on radar and vision sensor fusion for lane change assistance. J. Inst. Control 21(2), 121–129 (2015)

    Google Scholar 

  6. Ohn-Bar, E., Tawari, A., Martin, S., Trivedi, M.M.: Predicting driver maneuvers by learning holistic features. In: IEEE Intelligent Vehicles Symposium Proceedings, pp. 719–724 (2014)

  7. Deng, Q., Wang, J., Hillebrand, K., et al.: Prediction performance of lane changing behaviors: a study of combining environmental and eye-tracking data in a driving simulator. IEEE Trans. Intell. Transp. Syst. 21, 3561–3570 (2019)

    Article  Google Scholar 

  8. Deng, Q., Wang, J., Soffker, D.: Prediction of human driver behaviors based on an improved HMM approach. In: IEEE Intelligent Vehicles Symposium, pp. 2066–2071. IEEE (2018)

  9. Tran, D., Sheng, W., Liu, L. et al.: A hidden Markov model based driver intention prediction system. In: IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems, pp. 115–120 (2015)

  10. Ma, X., Ma, Z., Zhu, X., et al.: Driver Behavior Classification under Cut-In Scenarios Using Support Vector Machine Based on Naturalistic Driving Data. SAE Technical Paper (2019)

  11. Dou, Y., Yan, F., Feng, D.: Lane changing prediction at highway lane drops using support vector machine and artificial neural network classifiers. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 901–906 (2016)

  12. Leonhardt, V., Wanielik, G.: Recognition of lane change intentions fusing features of driving situation, driver behavior, and vehicle movement by means of neural networks. In: Advanced Microsystems for Automotive Applications, pp. 59–69. Springer (2018)

  13. Peng, J., Guo, Y., Fu, R., Yuan, W., et al.: Multi-parameter prediction of drivers’ lane changing behaviour with neural network model. Appl. Ergon. 50, 207–217 (2015)

    Article  Google Scholar 

  14. Gao, J., Murphey, Y.L., Zhu, H.H.: Multivariate time series prediction of lane changing behavior using deep neural network. Appl. Intell. 48(10), 3523–3537 (2018)

    Article  Google Scholar 

  15. Scheel, O., Nagaraja, N.S., Schwarz, L., et al.: Attention-based lane change prediction. arXiv:1903.01246 (2019)

  16. Murphey, Y.L., Kochhar, D.S., Xie, Y.Q.: Driver workload in an autonomous vehicle. SAE Technical Paper (2019)

  17. Zyner, A., Worrall, S., Nebot, E.: Naturalistic driver intention and path prediction using recurrent neural networks. In: IEEE Transactions on Intelligent Transportation Systems (2019)

  18. Chen, Y., Dong, C., Palanisamy, P., et al.: Attention-based hierarchical deep reinforcement learning for lane change behaviors in autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

  19. Lu, C., Hu, F., Cao, D., et al.: Transfer learning for driver model adaptation in lane-changing scenarios using manifold alignment. In: IEEE Transactions on Intelligent Transportation Systems (2019)

  20. Xie, D.F., Fang, Z.Z., Jia, B., et al.: A data-driven lane-changing model based on deep learning. Transp. Res. Part C Emerg. Technol. 106, 41–60 (2019)

    Article  Google Scholar 

  21. Mnih, V., Heess, N., Graves, A.: Recurrent models of visual attention. Adv. Neural Inf. Process. Syst. 2014, 2204–2212 (2014)

    Google Scholar 

  22. Yang, Z., He, X., Gao, J., et al.: Stacked attention networks for image question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 21–29 (2016)

  23. Chu, X., Yang, W., Ouyang, W., et al.: Multi-context attention for human pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1831–1840 (2017)

  24. Zhang, H., Dana, K., Shi, J., et al.: Context encoding for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7151–7160 (2018)

  25. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  26. Cao, Y., Xu, J., Lin, S., et al.: GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond. arXiv:1904.11492 (2019)

  27. Huang, Z., Lv, C., Xing, Y., et al.: Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding. IEEE Sens. J. 21, 11781–11790 (2020)

    Article  Google Scholar 

  28. Wang, X., Girshick, R., Gupta, A., et al.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

  29. Chen, L., Zhang, H., Xiao, J., et al.: Sca-cnn: spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5659–5667 (2017)

  30. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  31. Zhang, X., Sun, J., Qi, X., et al.: Simultaneous modeling of car-following and lane-changing behaviors using deep learning. Transp. Res. Part C Emerg. Technol. 104, 287–304 (2019)

    Article  Google Scholar 

  32. Ramanishka, V., Chen, Y., Misu, T., Saenko, K.: Toward driving scene understanding: a dataset for learning driver behavior and causal reasoning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7699–7707 (2018)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jun Gao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

By the Scientific Research Project of Hubei Provincial Department of Education (B2021055).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, J., Yi, J. & Murphey, Y.L. Attention-based global context network for driving maneuvers prediction. Machine Vision and Applications 33, 53 (2022). https://doi.org/10.1007/s00138-022-01305-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-022-01305-x

Keywords

Navigation