Skip to main content
SpringerLink
Log in

CIRAN: extracting crowd interaction with residual attention network for pedestrian trajectory prediction

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

This paper proposes a new deep learning network based on the spatial attention mechanism—crowd interaction with residual attention network (CIRAN), which combines the position and velocity information of neighbor pedestrians for trajectory prediction. It adaptively selects the most effective areas of the scene by using the residual attention module to obtain more accurate and reasonable pedestrian trajectories. Therefore, the accuracy of prediction can be improved. In addition, the velocity encoding module is introduced to transform the coordinate based pedestrian social interaction process into the spatial grid based pedestrian social interaction process. Based on two public data, ETH and UCY, this paper obtains the most advanced experimental results up to now, and these results show the validity of the proposed CIRAN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Song Z, Wu K, Shao J (2020) Destination prediction using deep echo state network. Neurocomputing 406:343–353

    Article  Google Scholar 

  2. Cheng Y, Sun L, Liu C et al (2020) Towards efficient human robot collaboration with robust plan recognition and trajectory prediction. IEEE Robot Autom Lett 5(2):2602–2609

    Article  Google Scholar 

  3. Eiffert S, Li K, Shan M et al (2020) Probabilistic crowd GAN: multimodal pedestrian trajectory prediction using a graph vehicle-pedestrian attention network. IEEE Robot Autom Lett 5(4):5026–5033

    Article  Google Scholar 

  4. Li X, Lu L, Yu J et al (2013) A method of abnormal pedestrian behavior detection based on the trajectory model. In: International conference on transportation engineering

  5. Alahi A, Goel K, Ramanathan V et al (2016) Social LSTM: human trajectory prediction in crowded spaces. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE

  6. Chung J, Gulcehre C, Cho KH et al (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. http://arxiv.org/abs/1412.3555

  7. Li Y (2019) Which way are you going? imitative decision learning for path forecasting in dynamic scenes. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  8. Pellegrini S, Ess A, Schindler K et al (2009) You'll never walk alone: modeling social behavior for multi-target tracking. In: IEEE international conference on computer vision. IEEE

  9. Gupta A, Johnson J, Li FF et al (2018) Social GAN: socially acceptable trajectories with generative adversarial networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  10. Pellegrini S, Ess A, Gool LV (2010) Improving data association by joint modeling of pedestrian trajectories and groupings. In: Proceedings of the 11th European conference on Computer vision: part I. Springer, Berlin

  11. Lealtaixe L, Fenzi M, Kuznetsova A et al (2014) Learning an image-based motion context for multiple people tracking. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE

  12. Hewing L, Arcari E, Fröhlich et al (2019) On simulation and trajectory prediction with gaussian process dynamics. http://arxiv.org/abs/1912.109002019.

  13. J Dong, Zhu P, Ferrari S (2020) Oriented pedestrian social interaction modeling and inference. In: 2020 American control conference (ACC)

  14. Liang J, Lu J, Niebles J C et al. Peeking into the future: predicting future person activities and locations in videos. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  15. Zhang P, Ouyang W, Zhang P et al (2020) SR-LSTM: state refinement for LSTM towards pedestrian trajectory prediction. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  16. Xu Y, Piao Z, Gao S (2018) Encoding crowd interaction with deep neural network for pedestrian trajectory prediction. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  17. Li F, Gui Z, Zhang Z et al (2020) A hierarchical temporal attention-based LSTM encoder-decoder model for individual mobility prediction. Neurocomputing 403(25):153–166

    Article  Google Scholar 

  18. Kosaraju V, Sadeghian A, Martín-Martín R et al (2019) Social-BiGAT: multimodal trajectory forecasting using bicycle-GAN and graph attention networks. http://arxiv.org/abs/1907.03395

  19. Zhang L, She Q, Guo P (2019) Stochastic trajectory prediction with social graph network. http://arxiv.org/abs/1907.10233

  20. X Chen, Liu S, Xu Z et al (2021) SCSG attention: a self-centered star graph with attention for pedestrian trajectory prediction

  21. Zhou Y, Wu H, Cheng H et al (2021) Social graph convolutional LSTM for pedestrian trajectory prediction. IET Intell Transport Syst 15(3):396–405

    Article  Google Scholar 

  22. Chen K, Song X, Ren X (2021) Modeling social interaction and intention for pedestrian trajectory prediction. Phys A Stat Mech Appl 570(9):125790

    Article  Google Scholar 

  23. Li J, Wei Y, Liang X et al (2016) Attentive contexts for object detection. IEEE Trans Multimed 19(5):944–954

    Article  Google Scholar 

  24. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. http://arxiv.org/abs/1706.03762

  25. Wang Y, Wang S, Tang J et al (2016) Hierarchical attention network for action recognition in videos. http://arxiv.org/abs/1607.06416

  26. Xu K, Ba J, Kiros R et al (2015) Show, attend and tell: neural image caption generation with visual attention. Comput Sci 2048–2057

  27. Fernando T, Denman S, Sridharan S et al (2017) Soft + hardwired attention: an LSTM framework for human trajectory prediction and abnormal event detection. Neural Netw 108:466–478

    Article  Google Scholar 

  28. Hang S, Dong Y, Zhu J et al (2016) Crowd scene understanding with coherent recurrent neural networks. In: IJCAI'16: proceedings of the twenty-fifth international joint conference on artificial intelligence

  29. Sadeghian A, Kosaraju V, Sadeghian A et al (2020) SoPhie: an attentive GAN for predicting paths compliant to social and physical constraints. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  30. Su H, Zhu J, Dong Y et al (2017) Forecast the plausible paths in crowd scenes. In: Twenty-sixth international joint conference on artificial intelligence

  31. Vemula A, Muelling K, Oh J (2018) Social attention: modeling attention in human crowds. In: 2018 IEEE international conference on robotics and automation (ICRA)

  32. Hao X, Du QH, Reynolds M (2020) A location-velocity-temporal attention LSTM model for pedestrian trajectory prediction. IEEE Access 8:44576–44589

    Article  Google Scholar 

  33. Li X, Liu Y, Wang K et al (2020) A recurrent attention and interaction model for pedestrian trajectory prediction. IEEE/CAA J Autom Sin 7(5):1361–1370

    Google Scholar 

  34. He K, Zhang X, Ren S et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  35. Wang F, Jiang M, Qian C et al (2017) Residual attention network for image classification. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE

  36. Mohamed A, Qian K, Elhoseiny M et al (2020) Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  37. Yang B, Yan G, Wang P et al (2020) TPPO: a novel trajectory predictor with pseudo oracle. http://arxiv.org/abs/2002.01852

  38. Amirian J, Hayet JB, Pettre J (2019) Social ways: learning multi-modal distributions of pedestrian trajectories with GANs. http://arxiv.org/abs/1904.09507

  39. Amirian J, Hayet JB, Pettre J (2019) Social ways: learning multi-modal distributions of pedestrian trajectories with GANs. 1

  40. Huang Y, Bi H, Li Z et al (2019) STGAT: modeling spatial-temporal interactions for human trajectory prediction. In: 2019 international conference in computer vision

  41. Fang L, Jiang Q, Shi J et al (2020) TPNet: trajectory proposal network for motion prediction. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  42. Shuai Y, Li H, Wang X (2015) Understanding pedestrian behaviors from stationary crowd groups. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). IEEE

Download references

Acknowledgements

This work is supported by the Natural Science Foundation of Tianjin City (Grants 18JCYBJC85100), Humanities and Social Science Fund of Ministry of Education of China (Grant 19YJA630046) and Scientific Research Plan Project of Tianjin Municipal Education Commission (Grant No. 2017KJ237).

Author information

Authors and Affiliations

  1. College of Science and Technology, Tianjin University of Finance and Economics, Tianjin, China

    Shang Liu, Xiaoyu Chen & Hao Chen

Authors
  1. Shang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shang Liu.

Ethics declarations

Conflict of interest

These no potential competing interests in our paper. And all authors have seen the manuscript and approved to submit to your journal. We confirm that the content of the manuscript has not been published or submitted for publication elsewhere.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Chen, X. & Chen, H. CIRAN: extracting crowd interaction with residual attention network for pedestrian trajectory prediction. Int. J. Mach. Learn. & Cyber. 13, 2649–2662 (2022). https://doi.org/10.1007/s13042-022-01548-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01548-0

Keywords

Search

Navigation