Abstract
The accurate trajectory prediction is a crucial task for the autonomous vehicles that help to plan and fast decision making capability of the system to reach their destination in the complex road scenario with abiding by the traffic rules. For this, autonomous vehicles should have more attention to their goal without affecting the other’s task and maintain their safety from road accidents. With this motivation, we proposed a multi-scale graph-transformer-based attention mechanism that provides the interaction between the road agents with different time instances, because from time to time, few new agents may enter the frame scene, and few may leave the frame scene. Each dynamic obstacles trajectory can be defined as state sequences within an interval of time, where spatial coordinates of dynamic obstacles represented by the each state under the world coordinate frame. We have presented graph-based Multi-scale spatial features with transformer network that achieves significant prediction results compared to other existing methods, and we provide an in-depth analysis of the trained weights for different highways scenarios with transformer and the Long-Short Term Memory. We evaluate our model with three publicly available datasets and achieve state-of-the-art performances as presented in the manuscript. The performance balance is more in favour of our model for sparser datasets compared to the dense datasets.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11370-022-00422-w/MediaObjects/11370_2022_422_Fig8_HTML.png)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Chandra R et al (2020) Forecasting trajectory and behavior of road-agents using spectral clustering in graph-LSTMs. IEEE Robot Autom Lett 5(3):4882–4890
Xiong W, Wu L, Alleva F, Droppo J, Huang X, Stolcke A (2018) The microsoft 2017 conversational speech recognition system. In: ICASSP, International conference on acoustics, speech, and signal processing, proceedings vol 2018-April, no. August, pp 5934–5938
Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL HLT 2019—2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol.—Proc. Conf., vol 1, no Mlm, pp 4171–4186
Fragkiadaki K, Levine S, Felsen P, Malik J (2015) Recurrent network models for human dynamics. In: Proceedings of the IEEE international conference on computer vision, vol 2015 Inter, pp 4346–4354
Vaswani A et al (2017) Attention is all you need. Adv Neural Inf Process Syst 2017:5999–6009
Luong MT, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. In: Conf. Proc. - EMNLP 2015 Conf. Empir. Methods Nat. Lang. Process., pp 1412–1421
Plastiras G, Kyrkou C, Theocharides T (2018) Efficient convnet-based object detection for unmanned aerial vehicles by selective tile processing. In: ACM international conference proceeding series
Papageorgiou C, Poggio T (2000) Trainable system for object detection. Int J Comput Vis 38(1):15–33
Lin J, Koch L, Kurowski M, Gehrt JJ, Abel D, Zweigel R (2020) Environment perception and object tracking for autonomous vehicles in a harbor scenario. In: 2020 IEEE 23rd Int. Conf. Intell. Transp. Syst. ITSC 2020, no. 4
Rangesh A, Trivedi MM (2019) No blind spots: full-surround multi-object tracking for autonomous vehicles using cameras and LiDARs. IEEE Trans Intell Veh 4(4):588–599
Zhang P, Ouyang W, Zhang P, Xue J, Zheng N (2019) SR-LSTM: State refinement for lstm towards pedestrian trajectory prediction. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, vol 2019-June, pp 12077–12086
Ivanovic B, Pavone M (2019) The trajectron: probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs. In: Proceedings of the IEEE international conference on computer vision, vol 2019-Octob, pp 2375–2384
Fisac JF, Bronstein E, Stefansson E, Sadigh D, Sastry SS, Dragan AD (2019) Hierarchical game-theoretic planning for autonomous vehicles. In: Proceedings of international conference on robotics and automation, vol 2019-May, pp 9590–9596
Liu C, Lee S, Varnhagen S, Tseng HE (2017) Path planning for autonomous vehicles using model predictive control. IEEE Intell Veh Symp Proc 5:174–179
Wu M et al (2020) Visual tracking with multiview trajectory prediction. IEEE Trans Image Process 29:8355–8367
Huynh M, Alaghband G (2019) Trajectory prediction by coupling scene-LSTM with human movement LSTM. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol 11844 LNCS, pp 244–259
Dönmez E, Kocamaz AF (2020) Design of mobile robot control infrastructure based on decision trees and adaptive potential area methods. Iran J Sci Technol Trans Electr Eng 44(1):431–448
Dirik M, Kocamaz AF, Dönmez E (2020) Visual servoing based control methods for nonholonomic mobile robot. J Eng Res 8(2):95–113
Okumuş F, Dönmez E, Kocamaz AF (2020) A cloudware architecture for collaboration of multiple agvs in indoor logistics: case study in fabric manufacturing enterprises. Electron 9(12):1–24
Dönmez E, Kocamaz AF, Dirik M (2018) A vision-based real-time mobile robot controller design based on Gaussian function for indoor environment. Arab J Sci Eng 43(12):7127–7142
Gupta A, Johnson J, Fei-Fei L, Savarese S, Alahi A (2018) Social GAN: socially acceptable trajectories with generative adversarial networks. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2255–2264
Yu C, Ma X, Ren J, Zhao H, Yi S (2020) Spatio-temporal graph transformer networks for pedestrian trajectory prediction
Lee D, Gu Y, Hoang J, Marchetti-Bowick M (2019) Joint interaction and trajectory prediction for autonomous driving using graph neural networks, no. NeurIPS
Sun J, Jiang Q, Lu C (2020) Recursive social behavior graph for trajectory prediction, pp 660–669
Morzy M (2007) Mining frequent trajectories of moving objects for location prediction. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol 4571 LNAI, pp 667–680
Monreale A, Pinelli F, Trasarti R, Giannotti F (2009) WhereNext: a location predictor on trajectory pattern mining. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 637–645
Won JI, Kim SW, Baek JH, Lee J (2009) Trajectory clustering in road network environment. In: 2009 IEEE Symp. Comput. Intell. Data Mining, CIDM 2009 - Proc., pp 299–305
Roh GP, Hwang SW (2010) NNCluster: an efficient clustering algorithm for road network trajectories. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol 5982 LNCS, no. PART 2, pp 47–61
Han B, Liu L, Omiecinski E (2015) Road-network aware trajectory clustering: Integrating locality, flow, and density. IEEE Trans Mob Comput 14(2):416–429
Chen M, Liu Y, Yu X (2015) Predicting next locations with object clustering and trajectory clustering. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol 9078, pp 344–356
Ashbrook D, Starner T (2003) Using GPS to learn significant locations and predict movement across multiple users. Pers Ubiquitous Comput 7(5):275–286
Lv Q, Qiao Y, Ansari N, Liu J, Yang J (2017) Individual mobility prediction at points of interest. IEEE Trans Veh Technol 66(6):5204–5216
Ishikawa Y. From Indexed Spatio-Temporal Datasets. Development, pp 9–16
Gambs S, Killijian MO, Del Prado Cortez MN (2012) Next place prediction using mobility Markov chains. In: Proc. 1st Work. Meas. Privacy, Mobility, MPM’12, pp 0–5
Chandra R, Bhattacharya U, Bera A, Di Manocha R (2019) Traphic: trajectory prediction in dense and heterogeneous traffic using weighted interactions. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, vol 2019, pp 8475–8484
Carrasco S, Llorca DF, Sotelo MÁ (2021) SCOUT: Socially-COnsistent and UndersTandable graph attention network for trajectory prediction of vehicles and VRUs
Gao J et al (2020) VectorNet: encoding HD maps and agent dynamics from vectorized representation. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 11522–11530
Kim B et al (2021) LaPred: lane-aware prediction of multi-modal future trajectories of dynamic agents
Park SH et al (2020) Diverse and admissible trajectory forecasting through multimodal context understanding, Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol 12356 LNCS, pp 282–298
Fang L, Jiang Q, Shi J, Zhou B (2020) TPNet: trajectory proposal network for motion prediction. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 6796–6805
Luo C, Sun L, Dabiri D, Yuille A (2020) Probabilistic multi-modal trajectory prediction with lane attention for autonomous vehicles. IEEE Int Conf Intell Robot Syst 56:2370–2376
He H, Dai H, Wang N (2020) UST: unifying spatio-temporal context for trajectory prediction in autonomous driving. IEEE Int Conf Intell Robot Syst 56:5962–5969
Wang J et al (2021) “F-NET: fusion neural network for vehicle trajectory prediction in autonomous driving, vol 1. Peking University, Beijing, pp 4095–4099
Becker S, Hug R, Hübner W, Arens M (2018) An evaluation of trajectory prediction approaches and notes on the TrajNet Benchmark
Becker S, Hug R, Hübner W, Arens M (2019) RED: a simple but effective baseline predictor for the TrajNet benchmark. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol 11131 LNCS, pp 138–153
Scholler C, Aravantinos V, Lay F, Knoll A (2020) What the constant velocity model can teach us about pedestrian motion prediction. IEEE Robot Autom Lett 5(2):1696–1703
Hammond DK, Vandergheynst P, Gribonval R (2011) Wavelets on graphs via spectral graph theory. Appl Comput Harmon Anal 30(2):129–150
Badue C et al (2017) 1901.04407-1
Ma Y, Zhu X, Zhang S, Yang R, Wang W, Manocha D (2009) TrafficPredict: trajectory prediction for heterogeneous traffic-agents, no. Kalman 1960
Chen L (2020) One thousand and one hours: self-driving motion prediction dataset, no. CoRL 2020, pp 1–10
Chang M et al. Argoverse : 3D tracking and forecasting with rich maps
Li X, Ying X, Chuah MC (2019) GRIP++: enhanced graph-based interaction-aware trajectory prediction for autonomous driving
Li X, Ying X, Chuah MC (2019) GRIP: graph-based interaction-aware trajectory prediction, pp 3960–3966
Julka S, Sowrirajan V, Schloetterer J, Granitzer M (2021) Conditional generative adversarial networks for speed control in trajectory simulation
Kim G, Kim D, Ahn Y, Huh K (2021) Hybrid approach for vehicle trajectory prediction using weighted integration of multiple models. IEEE Access 56:1
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Singh, D., Srivastava, R. Multi-scale graph-transformer network for trajectory prediction of the autonomous vehicles. Intel Serv Robotics 15, 307–320 (2022). https://doi.org/10.1007/s11370-022-00422-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11370-022-00422-w