Skip to main content
SpringerLink
Log in

Multivariate time series prediction of lane changing behavior using deep neural network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Many real world pattern classification problems involve the process and analysis of multiple variables in temporal domain. This type of problem is referred to as Multivariate Time Series (MTS) problem. It remains a challenging problem due to the nature of time series data: high dimensionality, large data size and updating continuously. In this paper, we use three types of physiological signals from the driver to predict lane changes before the event actually occurs. These are the electrocardiogram (ECG), galvanic skin response (GSR), and respiration rate (RR) and were determined, in prior studies, to best reflect a driver’s response to the driving environment. A novel Group-wise Convolutional Neural Network, MTS-GCNN model is proposed for MTS pattern classification. In our MTS-GCNN model, we present a new structure learning algorithm in training stage. The algorithm exploits the covariance structure over multiple time series to partition input volume into groups, then learns the MTS-GCNN structure explicitly by clustering input sequences with spectral clustering. Different from other feature-based classification approaches, our MTS-GCNN can select and extract the suitable internal structure to generate temporal and spatial features automatically by using convolution and down-sample operations. The experimental results showed that, in comparison to other state-of-the-art models, our MTS-GCNN performs significantly better in terms of prediction accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Kumar P, Perrollaz M, Lefevre S et al. (2013) Learning-based approach for online lane change intention prediction. In: Intelligent vehicles symposium. IEEE, pp 797–802

  2. Morris B, Doshi A, Trivedi M (2011) Lane change intent prediction for driver assistance: on-road design and evaluation. In: Intelligent vehicles symposium. IEEE, pp 895–901

  3. Bevly D et al. (2016) Lane change and merge maneuvers for connected and automated vehicles: a survey. IEEE Trans Intell Veh 1(1):105–120

    Article  Google Scholar 

  4. Kim I H, Bong J H, Park J et al. (2017) Prediction of drivers intention of lane change by augmenting sensor information using machine learning techniques. Sensors 17(6):1350

    Article  Google Scholar 

  5. Butakov V A, Ioannou P (2015) Personalized driver/vehicle lane change models for ADAS. IEEE Trans Veh Technol 64(10):4422–4431

    Article  Google Scholar 

  6. Krizhevsky A, Sutskever I, Hinton G E (2012) ImageNet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105

  7. Ji S, Xu W, Yang M et al. (2013) 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231

    Article  Google Scholar 

  8. He K, Zhang X, Ren S et al. (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  9. Ren S, He K, Girshick R et al. (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  10. Tran D, Sheng W, Liu L et al. (2015) A hidden Markov model based driver intention prediction system. In: IEEE International conference on cyber technology in automation, control, and intelligent systems. IEEE, pp 115–120

  11. Jin L, Hou H, Jiang Y (2012) Driver intention recognition based on continuous hidden Markov model. In: International conference on transportation, mechanical, and electrical engineering. IEEE, pp 739–742

  12. Zheng Y, Hansen J H L (2017) Lane-change detection from steering signal using spectral segmentation and learning-based classification. IEEE Trans Intell Veh 2(1):14–24

    Article  Google Scholar 

  13. Ramyar S, Homaifar A, Karimoddini A et al. (2016) Identification of anomalies in lane change behavior using one-class SVM. In: IEEE International conference on systems, man, and cybernetics, pp 4405–4410

  14. Hou Y, Edara P, Sun C (2014) Modeling mandatory lane changing using Bayes classifier and decision trees. IEEE Trans Intell Transport Syst 15(2):647–655

    Article  Google Scholar 

  15. Schubert R, Schulze K, Wanielik G (2010) Situation assessment for automatic lane-change maneuvers. IEEE Trans Intell Transport Syst 11(3):607–616

    Article  Google Scholar 

  16. Kasper D, Weidl G, Dang T et al. (2012) Object-oriented Bayesian networks for detection of lane change maneuvers. In: Intelligent vehicles symposium. IEEE, pp 673–678

  17. Ulbrich S, Maurer M (2015) Situation assessment in tactical lane change behavior planning for automated vehicles. In: IEEE International conference on intelligent transportation systems, pp 975–981

  18. Yan F, Eilers M, Ldtke A et al. (2016) Developing a model of driver’s uncertainty in lane change situations for trustworthy lane change decision aid systems. In: Intelligent vehicles symposium. IEEE, pp 406–411

  19. Peng J, Guo Y, Fu R, Yuan W, Wang C (2015) Multiparameter prediction of drivers lane-changing behaviour with neural network model. Appl Ergon 50:207–217

    Article  Google Scholar 

  20. Tomar R S, Verma S (2013) Lane change trajectory prediction using artificial neural network. Int J Veh Safe 6(3):213–234

    Article  Google Scholar 

  21. Leonhardt V, Wanielik G (2017) Feature evaluation for lane change prediction based on driving situation and driver behavior. In: IEEE 20th International conference on information fusion, pp 1–7

  22. Li J, Mei X, Prokhorov D et al. (2017) Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans Neural Netw Learn Syst 28(3):690–703

    Article  Google Scholar 

  23. Olabiyi O, Martinson E, Chintalapudi V et al. (2017) Driver action prediction using deep (bidirectional) recurrent neural network. arXiv:http://arXiv.org/abs/1706.02257

  24. lexandru G, Tejaswi K, Smita V et al. (2016) DeepLanes: End-to-end lane position estimation using deep neural networks. In: IEEE Conference on computer vision and pattern recognition, pp 38–45

  25. Kin J, Lee M (2014) Robust lane detection based on convolutional neural network and random sample consensus. In: International Conference On Neural Information Processing, pp 454–461

  26. Woo H et al. (2017) Lane-change detection based on vehicle-trajectory prediction. IEEE Robot Autom Lett 2(2):1109–1116

    Article  Google Scholar 

  27. Yao W, Zhao H, Bonnifait P, Zha H (2013) Lane change trajectory prediction by using recorded human driving data. In: Intelligent vehicles symposium. IEEE, pp 430–436

  28. Nilsson J, Brannstrom M, Coelingh E et al. (2017) Lane change maneuvers for automated vehicles. IEEE Trans Intell Transport Syst 18(5):1087–1096

    Article  Google Scholar 

  29. Satzoda R K, Trivedi M M (2015) Drive analysis using vehicle dynamics and vision-based lane semantics. IEEE Trans Intell Transport Syst 16(1):9–18

    Article  Google Scholar 

  30. Xu G, Liu L, Ou Y et al. (2012) Dynamic modeling of driver control strategy of lane-change behavior and trajectory planning for collision prediction. IEEE Trans Intell Transport Syst 13(3):1138–1155

    Article  Google Scholar 

  31. Murphey Y, Kochhar D, Watta P, Wang X et al. (2015) Driver lane change prediction using physiological measures. SAE Int J Transport Safe. 3(2):118–125

    Google Scholar 

  32. Bach F R, Jordan M I (2004) Learning spectral clustering. Proc Adv Neural Inf Process Syst 16:305–312

    Google Scholar 

  33. Hajinoroozi M, Mao Z, Jung T P et al. (2016) EEG-based prediction of driver’s cognitive performance by deep convolutional neural network. Signal Process Image Commun 47:549–555

    Article  Google Scholar 

  34. Yang J B, Nguyen M N, San P P, Li X L, Krishnaswamy S h (2015) Deep convolutional neural networks on multichannel time series for human activity recognition. In: Proceedings of the international joint conference on artificial intelligence, pp 25–31

  35. Malhotra P, Vig L, Shroff G, Agarwal P (2015) Long short term memory networks for anomaly detection in time series. In: Proceedings of European symposium on artificial neural networks, p 89

  36. Wakasugi T (2005) A study on warning timing for lane change decision aid systems based on driver’s lane change maneuver. In: 19th International technical conference on the enhanced safety of vehicles

Download references

Acknowledgements

This research is supported in part by a University Research Grant from Ford Motor Company.

Author information

Authors and Affiliations

  1. School of Logistics Engineering, Wuhan University of Technology, Wuhan, 430063, China

    Jun Gao & Honghui Zhu

  2. Department of Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI, 48128, USA

    Jun Gao & Yi Lu Murphey

Authors
  1. Jun Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Lu Murphey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Honghui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Gao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, J., Murphey, Y.L. & Zhu, H. Multivariate time series prediction of lane changing behavior using deep neural network. Appl Intell 48, 3523–3537 (2018). https://doi.org/10.1007/s10489-018-1163-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1163-9

Keywords

Search

Navigation