Skip to main content
Springer Nature Link
Log in

Triple GNN: A Pedestrian-Scene-Object Joint Model for Pedestrian Trajectory Prediction

  • Conference paper
  • First Online:
Artificial Intelligence (CICAI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13604))

Included in the following conference series:

Abstract

Pedestrian trajectory prediction is an amazing but challenging task for vision guided applications, including autonomous driving, intelligent surveillance system, etc. Practically, the trajectory is a result of interaction among pedestrian’s surrounding people, scenes and related objects, which can be represented by a triple. In previous works, limited interactions have been exploited, such as pedestrian-pedestrian and pedestrian-object. These works are facing challenges when comprehensive interactions in natural scenes are involved. In this paper, we propose a triple graph neural network (Triple GNN) where interactions among pedestrians, scenes and objects are all taken into the prediction of pedestrian trajectory. Based on that, spatial relation is exploited to describe the mutual interaction among triple elements, and a two-stage optimization scheme is proposed on weights of the interaction aggregation for better relation exploitation and prediction. Furthermore, temporal relation is also exploited for compact representation and effective computation of future trajectories based on the spatial relation. Our method is verified via ETH and UCY datasets and achieves the state-of-the-art performance.

This work was supported in part by the National Key Research and Development Program of China under Grant 2020YFB2103501, in part by the National Natural Science Foundation of China under Grant 61971203.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Doctor strange. https://marvelcinematicuniverse.fandom.com/wiki/Doctor_Strange_(film)

  2. Al-Mallah, R., Quintero, A., Farooq, B.: Prediction of traffic flow via connected vehicles. IEEE Trans. Mobile Comput. 21, 264–277 (2020). https://doi.org/10.1109/TMC.2020.3006713

    Article  Google Scholar 

  3. Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: human trajectory prediction in crowded spaces. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 961–971 (2016)

    Google Scholar 

  4. Bai, H., Cai, S., Ye, N., Hsu, D., Lee, W.S.: Intention-aware online POMDP planning for autonomous driving in a crowd. In: IEEE International Conference on Robotics and Automation, pp. 454–460. IEEE (2015)

    Google Scholar 

  5. Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)

  6. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: European Conference on Computer Vision, pp. 801–818 (2018)

    Google Scholar 

  7. Choi, W., Savarese, S.: Understanding collective activities of people from videos. IEEE Trans. Pattern Anal. Mach. Intell. 36(6), 1242–1257 (2013)

    Article  Google Scholar 

  8. Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., Alahi, A.: Social GAN: socially acceptable trajectories with generative adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2255–2264 (2018)

    Google Scholar 

  9. Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282 (1995)

    Article  Google Scholar 

  10. Hou, J., Wu, X., Wang, R., Luo, J., Jia, Y.: Confidence-guided self refinement for action prediction in untrimmed videos. IEEE Trans. Image Process. 29, 6017–6031 (2020)

    Article  MATH  Google Scholar 

  11. Huang, Y., Bi, H., Li, Z., Mao, T., Wang, Z.: STGAT: modeling spatial-temporal interactions for human trajectory prediction. In: IEEE International Conference on Computer Vision, pp. 6272–6281 (2019)

    Google Scholar 

  12. Ke, Q., Bennamoun, M., Rahmani, H., An, S., Sohel, F., Boussaid, F.: Learning latent global network for skeleton-based action prediction. IEEE Trans. Image Process. 29, 959–970 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  13. Keller, C.G., Gavrila, D.M.: Will the pedestrian cross? a study on pedestrian path prediction. IEEE Trans. Intell. Transp. Syst. 15(2), 494–506 (2014)

    Article  Google Scholar 

  14. Kong, Y., Tao, Z., Fu, Y.: Adversarial action prediction networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(3), 539–553 (2020)

    Article  Google Scholar 

  15. Leal-Taixé, L., Fenzi, M., Kuznetsova, A., Rosenhahn, B., Savarese, S.: Learning an image-based motion context for multiple people tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3542–3549 (2014)

    Google Scholar 

  16. Leal-Taixé, L., Pons-Moll, G., Rosenhahn, B.: Everybody needs somebody: modeling social and grouping behavior on a linear programming multiple people tracker. In: IEEE International Conference on Computer Vision Workshops, pp. 120–127. IEEE (2011)

    Google Scholar 

  17. Lerner, A., Chrysanthou, Y., Lischinski, D.: Crowds by example. In: Computer Graphics Forum, vol. 26, pp. 655–664. Wiley Online Library (2007)

    Google Scholar 

  18. Li, B., Tian, J., Zhang, Z., Feng, H., Li, X.: Multitask non-autoregressive model for human motion prediction. IEEE Trans. Image Process. 30, 2562–2574 (2021)

    Article  Google Scholar 

  19. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942. IEEE (2009)

    Google Scholar 

  20. Mohamed, A., Qian, K., Elhoseiny, M., Claudel, C.: Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 14424–14432 (2020)

    Google Scholar 

  21. Ng, Y.B., Fernando, B.: Forecasting future action sequences with attention: a new approach to weakly supervised action forecasting. IEEE Trans. Image Process. 29, 8880–8891 (2020)

    Article  MATH  Google Scholar 

  22. Pellegrini, S., Ess, A., Van Gool, L.: Improving data association by joint modeling of pedestrian trajectories and groupings. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 452–465. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_33

    Chapter  Google Scholar 

  23. Raksincharoensak, P., Hasegawa, T., Nagai, M.: Motion planning and control of autonomous driving intelligence system based on risk potential optimization framework. Int. J. Autom. Eng. 7(AVEC14), 53–60 (2016)

    Article  Google Scholar 

  24. Riofrıo-Luzcando, D., Ramırez, J., Berrocal-Lobo, M.: Predicting student actions in a procedural training environment. IEEE Trans. Learn. Technol. 10(4), 463–474 (2017)

    Article  Google Scholar 

  25. Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Rezatofighi, H., Savarese, S.: Sophie: an attentive GAN for predicting paths compliant to social and physical constraints. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1349–1358 (2019)

    Google Scholar 

  26. Salzmann, T., Ivanovic, B., Chakravarty, P., Pavone, M.: Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12363, pp. 683–700. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58523-5_40

    Chapter  Google Scholar 

  27. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  28. Wang, C., Cai, S., Tan, G.: GraphTCN: spatio-temporal interaction modeling for human trajectory prediction. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3450–3459 (2021)

    Google Scholar 

  29. Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1345–1352. IEEE (2011)

    Google Scholar 

  30. Yasuno, M., Yasuda, N., Aoki, M.: Pedestrian detection and tracking in far infrared images. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, p. 125. IEEE (2004)

    Google Scholar 

  31. Yu, C., Ma, X., Ren, J., Zhao, H., Yi, S.: Spatio-temporal graph transformer networks for pedestrian trajectory prediction. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 507–523. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_30

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China

    Xinshengzi Huang, Qiong Liu & You Yang

Authors
  1. Xinshengzi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. You Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to You Yang .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Lu Fang

  2. Xiaomi Inc., Beijing, China

    Daniel Povey

  3. Shanghai Jiao Tong University, Shanghai, China

    Guangtao Zhai

  4. JD Explore Academy, Beijing, China

    Tao Mei

  5. Chinese Academy of Sciences, Beijing, China

    Ruiping Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, X., Liu, Q., Yang, Y. (2022). Triple GNN: A Pedestrian-Scene-Object Joint Model for Pedestrian Trajectory Prediction. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13604. Springer, Cham. https://doi.org/10.1007/978-3-031-20497-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20497-5_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20496-8

  • Online ISBN: 978-3-031-20497-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics