Skip to main content
SpringerLink
Log in

Trajectory prediction of vehicles turning at intersections using deep neural networks

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, an early prediction of vehicle trajectories and turning movements are investigated using traffic cameras. A vision-based tracking system is developed to monitor intersection videos and collect vehicle trajectories with their labels known as turning movements. Firstly, two intersection videos are monitored for 2 h, and collected trajectories with their labels are used to train deep neural networks and obtain the turning models for the prediction task. Deep neural networks are further investigated on a third intersection with different video settings. The future 2 s evaluation of trajectories shows the success of long short-term memory networks to early predict the turning movements with more than 92% accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. LeóCano, J.A., Kovaceva, J., Lindman, M., Brännström, M.: Automatic incident detection and classification at intersections. In: 21st International Conference on Enhanced Safety of Vehicles, ESV. Paper 09–0234 (2009)

  2. Shirazi, M.S., Morris, B.T.: Looking at intersections: a survey of intersection monitoring, behavior and safety analysis of recent studies. IEEE Trans. Intell. Transp. Syst. 18, 4–24 (2017)

    Article  Google Scholar 

  3. Shirazi, M.S., Morris, B.: Observing behaviors at intersections: a review of recent studies and developments. In: IEEE Intelligent Vehicles Symposium (IV), pp. 1258–1263 (2015)

  4. Kumar, P., Ranganath, S., Weimin, H., Sangupta, K.: Framework for real-time behavior interpretation from traffic video. IEEE Trans. Intell. Transp. Syst. 6, 43–53 (2005)

    Article  Google Scholar 

  5. Wu, J., Cui, Z., Zhao, P., Chen, J.: Traffic Vehicle Behavior Prediction Using Hidden Markov Models, pp. 383–390. Springer, Berlin (2012)

    Google Scholar 

  6. Ma, X., Dai, Z., He, Z., Wang, Y.: Learning traffic as images: a deep convolution neural network for large-scale transportation network speed prediction. CoRR (2017). arXiv:1701.04245

  7. Shirazi, M.S., Morris, B.T.: Vision-based turning movement monitoring: count, speed and waiting time estimation. IEEE Intell. Transp. Syst. Mag. 8, 23–34 (2016)

    Article  Google Scholar 

  8. Viti, F., Hoogendoorn, S.P., van Zuylen, H.J., Wilmink, I.R., van Arem, B.: Speed and acceleration distributions at a traffic signal analyzed from microscopic real and simulated data (2008)

  9. Kafer, E., Hermes, C., Wohler, C., Ritter, H., Kummert, F.: Recognition of situation classes at road intersections. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3960–3965 (2010)

  10. Graf, R., Deusch, H., Seeliger, F., Fritzsche, M., Dietmayer, K.: A learning concept for behaviour prediction at intersections. In: IEEE Intelligent Vehicles Symposium (IV), Dearborn, Michigan, pp. 939–945 (2014)

  11. Tran, Q., Firll, J.: Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression. In: IEEE Intelligent Vehicles Symposium (IV), Dearborn, Michigan, pp. 918–923 (2014)

  12. Hulnhagen, T., Dengler, I., Tamke, A., Dang, T., Breuel, G.: Maneuver recognition using probabilistic finite-state machines and fuzzy logic. In: Proceedings of IEEE Intelligent Vehicles Symposium, San Diego, USA, pp. 65–70 (2010)

  13. Jain, A., Koppula, H.S., Raghavan, B., Soh, S., Saxena, A.: Car that knows before you do: anticipating maneuvers via learning temporal driving models. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3182–3190 (2015)

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR (2015). arXiv:1512.03385

  15. Srivastava, N., Mansimov, E., Salakhutdinov, R.: Unsupervised learning of video representations using lstms. CoRR (2015). arXiv:1502.04681

  16. Liu, J., Wang, G., Duan, L., Hu, P., Kot, A.C.: Skeleton based human action recognition with global context-aware attention LSTM networks. CoRR (2017). arXiv:1707.05740

  17. Liu, J., Shahroudy, A., Xu, D., Kot, A.C., Wang, G.: Skeleton-based action recognition using spatio-temporal LSTM network with trust gates. CoRR (2017). arXiv:1706.08276

  18. Veeriah, V., Zhuang, N., Qi, G.: Differential recurrent neural networks for action recognition. CoRR (2015). arXiv:1504.06678

  19. Kim, B., Kang, C.M., Lee, S., Chae, H., Kim, J., Chung, C.C., Choi, J.W.: Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network. CoRR (2017). arXiv:1704.07049

  20. Khosroshahi, A., Ohn-Bar, E., Trivedi, M.M.: Surround vehicles trajectory analysis with recurrent neural networks. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), pp. 2267–2272 (2016)

  21. Lan, X., Ma, A.J., Yuen, P.C.: Multi-cue visual tracking using robust feature-level fusion based on joint sparse representation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1194–1201 (2014)

  22. Lan, X., Zhang, S., Yuen, P.C.: Robust joint discriminative feature learning for visual tracking. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. IJCAI’16, pp. 3403–3410. AAAI Press (2016)

  23. Lan, X., Yuen, P.C., Chellappa, R.: Robust mil-based feature template learning for object tracking. In: AAAI Conference on Artificial Intelligence (2017)

  24. Pernici, F., Bimbo, A.D.: Object tracking by oversampling local features. IEEE Trans. Pattern Anal. Mach. Intell. 36, 2538–2551 (2014)

    Article  Google Scholar 

  25. Lan, X., Zhang, S., Yuen, P.C., Chellappa, R.: Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans. Image Process. 27, 2022–2037 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  26. Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, p. 252 (1999)

  27. Shirazi, M.S., Morris, B.: Vision-based turning movement counting at intersections by cooperating zone and trajectory comparison modules. In: IEEE Intelligent Transportation Systems Conference (ITSC), Qingdao, China, pp. 3100–3105 (2014)

  28. Akoz, O., Karsligil, M.: Traffic event classification at intersections based on the severity of abnormality. Mach. Vis. Appl. 25, 613–632 (2014)

    Article  Google Scholar 

  29. Morris, B., Trivedi, M.: Learning trajectory patterns by clustering: experimental studies and comparative evaluation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 312–319 (2009)

  30. Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 157–166 (1994)

    Article  Google Scholar 

  31. Lv, Y., Duan, Y., Kang, W., Li, Z., Wang, F.Y.: Traffic flow prediction with big data: a deep learning approach. IEEE Trans. Intell. Transp. Syst. 16, 865–873 (2015)

    Google Scholar 

  32. Shirazi, M.S., Morris, B.T.: Investigation of safety analysis methods using computer vision techniques. J. Electron. Imaging 26, 051404 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering and Computer Science, Cleveland State University, 2121 Euclid Avenue, Cleveland, OH, 44115, USA

    Mohammad Shokrolah Shirazi

  2. Electrical and Computer Engineering, University of Nevada, 4505 S Maryland Parkway Box 454026, Las Vegas, NV, 89154-4026, USA

    Brendan Tran Morris

Authors
  1. Mohammad Shokrolah Shirazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brendan Tran Morris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Shokrolah Shirazi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shokrolah Shirazi, M., Morris, B.T. Trajectory prediction of vehicles turning at intersections using deep neural networks. Machine Vision and Applications 30, 1097–1109 (2019). https://doi.org/10.1007/s00138-019-01040-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-019-01040-w

Keywords

Search

Navigation