Abstract
Driving maneuvers prediction is one of the most challenging tasks in modern Advanced Driver Assistance System. Such predictions can improve driving safety by alerting the driver to the danger of unsafe or risky traffic situations. In this research, we presents a novel Attention-based Global Context Network (AGCNet) for driving maneuvers prediction from multiple modality data, including front view video data and driver physiological signals. Firstly, with Global Context block, the AGCNet has an ability of modeling the long-range dependency contextual features from multi-modal data with lightweight computation. Secondly, the Channel-wise Attention is introduced in AGCNet to focus on valuable features. Finally, a custom-built Dual attention-based Long Short-Term Memory (DaLSTM) network is designed to learn co-occurrence features and predict driving maneuvers. Specially, the DaLSTM can employ attention mechanisms over heterogeneous features and time steps simultaneously. The experimental results show that the AGCNet is capable of learning the latent features of driving maneuvers and achieving significantly better performance than other advanced models on a real-world driving dataset.
Similar content being viewed by others
References
Gao, J., Murphey, Y.L., Yi, J.G., et al.: A data-driven lane-changing behavior detection system based on sequence learning. Transportmetr. B Transp. Dyn. 10(1), 831–848 (2020)
Gao, J., Murphey, Y.L., Zhu, H.H.: Personalized detection of lane changing behavior using multisensor data fusion. Computing 101(12), 1837–1860 (2019)
Zyner, A., Worrall, S., Ward, J., et al.: Long short term memory for driver intent prediction. In: IEEE Intelligent Vehicles Symposium, pp. 1484–1489. IEEE (2017)
Peng, X., Murphey, Y.L., Liu, R., et al.: Driving maneuver early detection via sequence learning from vehicle signals and video images. Pattern Recogn. 103, 107276 (2020)
Kim, H.T., Song, B., Lee, H., et al.: Multiple vehicle recognition based on radar and vision sensor fusion for lane change assistance. J. Inst. Control 21(2), 121–129 (2015)
Ohn-Bar, E., Tawari, A., Martin, S., Trivedi, M.M.: Predicting driver maneuvers by learning holistic features. In: IEEE Intelligent Vehicles Symposium Proceedings, pp. 719–724 (2014)
Deng, Q., Wang, J., Hillebrand, K., et al.: Prediction performance of lane changing behaviors: a study of combining environmental and eye-tracking data in a driving simulator. IEEE Trans. Intell. Transp. Syst. 21, 3561–3570 (2019)
Deng, Q., Wang, J., Soffker, D.: Prediction of human driver behaviors based on an improved HMM approach. In: IEEE Intelligent Vehicles Symposium, pp. 2066–2071. IEEE (2018)
Tran, D., Sheng, W., Liu, L. et al.: A hidden Markov model based driver intention prediction system. In: IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems, pp. 115–120 (2015)
Ma, X., Ma, Z., Zhu, X., et al.: Driver Behavior Classification under Cut-In Scenarios Using Support Vector Machine Based on Naturalistic Driving Data. SAE Technical Paper (2019)
Dou, Y., Yan, F., Feng, D.: Lane changing prediction at highway lane drops using support vector machine and artificial neural network classifiers. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 901–906 (2016)
Leonhardt, V., Wanielik, G.: Recognition of lane change intentions fusing features of driving situation, driver behavior, and vehicle movement by means of neural networks. In: Advanced Microsystems for Automotive Applications, pp. 59–69. Springer (2018)
Peng, J., Guo, Y., Fu, R., Yuan, W., et al.: Multi-parameter prediction of drivers’ lane changing behaviour with neural network model. Appl. Ergon. 50, 207–217 (2015)
Gao, J., Murphey, Y.L., Zhu, H.H.: Multivariate time series prediction of lane changing behavior using deep neural network. Appl. Intell. 48(10), 3523–3537 (2018)
Scheel, O., Nagaraja, N.S., Schwarz, L., et al.: Attention-based lane change prediction. arXiv:1903.01246 (2019)
Murphey, Y.L., Kochhar, D.S., Xie, Y.Q.: Driver workload in an autonomous vehicle. SAE Technical Paper (2019)
Zyner, A., Worrall, S., Nebot, E.: Naturalistic driver intention and path prediction using recurrent neural networks. In: IEEE Transactions on Intelligent Transportation Systems (2019)
Chen, Y., Dong, C., Palanisamy, P., et al.: Attention-based hierarchical deep reinforcement learning for lane change behaviors in autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Lu, C., Hu, F., Cao, D., et al.: Transfer learning for driver model adaptation in lane-changing scenarios using manifold alignment. In: IEEE Transactions on Intelligent Transportation Systems (2019)
Xie, D.F., Fang, Z.Z., Jia, B., et al.: A data-driven lane-changing model based on deep learning. Transp. Res. Part C Emerg. Technol. 106, 41–60 (2019)
Mnih, V., Heess, N., Graves, A.: Recurrent models of visual attention. Adv. Neural Inf. Process. Syst. 2014, 2204–2212 (2014)
Yang, Z., He, X., Gao, J., et al.: Stacked attention networks for image question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 21–29 (2016)
Chu, X., Yang, W., Ouyang, W., et al.: Multi-context attention for human pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1831–1840 (2017)
Zhang, H., Dana, K., Shi, J., et al.: Context encoding for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7151–7160 (2018)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Cao, Y., Xu, J., Lin, S., et al.: GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond. arXiv:1904.11492 (2019)
Huang, Z., Lv, C., Xing, Y., et al.: Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding. IEEE Sens. J. 21, 11781–11790 (2020)
Wang, X., Girshick, R., Gupta, A., et al.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)
Chen, L., Zhang, H., Xiao, J., et al.: Sca-cnn: spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5659–5667 (2017)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Zhang, X., Sun, J., Qi, X., et al.: Simultaneous modeling of car-following and lane-changing behaviors using deep learning. Transp. Res. Part C Emerg. Technol. 104, 287–304 (2019)
Ramanishka, V., Chen, Y., Misu, T., Saenko, K.: Toward driving scene understanding: a dataset for learning driver behavior and causal reasoning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7699–7707 (2018)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
By the Scientific Research Project of Hubei Provincial Department of Education (B2021055).
Rights and permissions
About this article
Cite this article
Gao, J., Yi, J. & Murphey, Y.L. Attention-based global context network for driving maneuvers prediction. Machine Vision and Applications 33, 53 (2022). https://doi.org/10.1007/s00138-022-01305-x
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00138-022-01305-x