Skip to main content
Springer Nature Link
Log in

Pedestrian Trajectory Prediction Using a Social Pyramid

  • Conference paper
  • First Online:
PRICAI 2019: Trends in Artificial Intelligence (PRICAI 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11671))

Included in the following conference series:

Abstract

Understanding and forecasting human movement paths are vital for a wide range of real world applications. It is not an easy task to generate plausible future paths as the scenes and human movement patterns are often very complex. In this paper, we propose a social pyramid based prediction method (SPP), which includes two encoders to capture motion and social information. Specifically, we design a social pyramid map structure for the Social encoder, which can differentiate the influence of other pedestrians in nearby areas or remote areas based on their spatial locations. For the Motion encoder, a mixing attention mechanism is proposed to combine the location coordinates and velocity vectors. The two encoded features are then merged and passed to the decoder which generates future paths of pedestrians. Our extensive experimental results demonstrate competitive prediction performance from our method compared to state-of-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Li, F.F., Savarese, S.: Social LSTM: human trajectory prediction in crowded spaces. In: CVPR, pp. 961–971, June 2016

    Google Scholar 

  2. Bartoli, F., Lisanti, G., Ballan, L., Del Bimbo, A.: Context-aware trajectory prediction. arXiv preprint arXiv:1705.02503 (2017)

  3. Bhattacharyya, A., Fritz, M., Schiele, B.: Long-term on-board prediction of people in traffic scenes under uncertainty. In: CVPR, June 2018

    Google Scholar 

  4. Chen, M., Ding, G., Zhao, S., Chen, H., Liu, Q., Han, J.: Reference based LSTM for image captioning. In: AAAI, pp. 3981–3987 (2017)

    Google Scholar 

  5. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)

  6. Deo, N., Trivedi, M.M.: Convolutional social pooling for vehicle trajectory prediction. In: CVPR, June 2018

    Google Scholar 

  7. Fernando, T., Denman, S., Sridharan, S., Fookes, C.: Soft+ hardwired attention: an LSTM framework for human trajectory prediction and abnormal event detection. arXiv preprint arXiv:1702.05552 (2017)

  8. Fernando, T., Denman, S., Sridharan, S., Fookes, C.: Tracking by prediction: a deep generative model for mutli-person localisation and tracking. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1122–1132. IEEE (2018)

    Google Scholar 

  9. Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)

  10. Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neural networks. In: ICML, vol. 14, pp. 1764–1772 (2014)

    Google Scholar 

  11. Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., Alahi, A.: Social GAN: socially acceptable trajectories with generative adversarial networks. In: CVPR, June 2018

    Google Scholar 

  12. Hasan, I., Setti, F., Tsesmelis, T., Del Bue, A., Galasso, F., Cristani, M.: MX-LSTM: mixing tracklets and vislets to jointly forecast trajectories and head poses. In: CVPR, June 2018

    Google Scholar 

  13. Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282 (1995)

    Article  Google Scholar 

  14. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  15. Kim, K., Lee, D., Essa, I.: Gaussian process regression flow for analysis of motion trajectories. In: ICCV, pp. 1164–1171. IEEE (2011)

    Google Scholar 

  16. Kim, S., et al.: BRVO: predicting pedestrian trajectories using velocity-space reasoning. Int. J. Robot. Res. 34(2), 201–217 (2015)

    Article  Google Scholar 

  17. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  18. Lee, N., Choi, W., Vernaza, P., Choy, C.B., Torr, P.H.S., Chandraker, M.: DESIRE: distant future prediction in dynamic scenes with interacting agents. In: CVPR (2017)

    Google Scholar 

  19. Li, S., Li, W., Cook, C., Zhu, C., Gao, Y.: Independently Recurrent Neural Network (IndRNN): building a longer and deeper RNN. In: CVPR, June 2018

    Google Scholar 

  20. Li, Y.: A deep spatiotemporal perspective for understanding crowd behavior. IEEE Trans. Multimed., 1–8 (2018). https://doi.org/10.1109/TMM.2018.2834873

    Article  Google Scholar 

  21. Li, Y.: Pedestrian path forecasting in crowd: a deep spatio-temporal perspective. In: Proceedings of the ACM on Multimedia Conference, pp. 235–243. ACM (2017)

    Google Scholar 

  22. Liu, J., Wang, G., Hu, P., Duan, L.Y., Kot, A.C.: Global context-aware attention LSTM networks for 3D action recognition. In: CVPR, pp. 1647–1656 (2017)

    Google Scholar 

  23. Lv, J., Li, Q., Sun, Q., Wang, X.: T-CONV: a convolutional neural network for multi-scale taxi trajectory prediction. In: 2018 IEEE International Conference on Big Data and Smart Computing (BigComp), pp. 82–89. IEEE (2018)

    Google Scholar 

  24. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: CVPR, pp. 935–942. IEEE (2009)

    Google Scholar 

  25. Nikhil, N., Morris, B.T.: Convolutional neural network for trajectory prediction. arXiv preprint arXiv:1809.00696 (2018)

  26. Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: modeling social behavior for multi-target tracking. In: ICCV, pp. 261–268. IEEE (2009)

    Google Scholar 

  27. Ren, J.S., et al.: Look, listen and learn - a multimodal LSTM for speaker identification. In: AAAI, pp. 3581–3587 (2016)

    Google Scholar 

  28. Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Savarese, S.: SoPhie: an attentive gan for predicting paths compliant to social and physical constraints. arXiv preprint arXiv:1806.01482 (2018)

  29. Su, H., Dong, Y., Zhu, J., Ling, H., Zhang, B.: Crowd scene understanding with coherent recurrent neural networks. In: IJCAI, pp. 3469–3476 (2016)

    Google Scholar 

  30. Su, H., Zhu, J., Dong, Y., Zhang, B.: Forecast the plausible paths in crowd scenes. In: IJCAI, pp. 2772–2778 (2017)

    Google Scholar 

  31. Sun, L., Yan, Z., Mellado, S.M., Hanheide, M., Duckett, T.: 3DOF pedestrian trajectory prediction learned from long-term autonomous mobile robot deployment data. arXiv preprint arXiv:1710.00126 (2017)

  32. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: NIPS, pp. 3104–3112 (2014)

    Google Scholar 

  33. Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  34. Vemula, A., Muelling, K., Oh, J.: Social attention: modeling attention in human crowds. In: ICRA, pp. 1–7, May 2018. https://doi.org/10.1109/ICRA.2018.8460504

  35. Vemula, A., Muelling, K., Oh, J.: Modeling cooperative navigation in dense human crowds. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1685–1692. IEEE (2017)

    Google Scholar 

  36. Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 283–298 (2008)

    Article  Google Scholar 

  37. Xie, D., Todorovic, S., Zhu, S.C.: Inferring “dark matter” and “dark energy” from videos. In: ICCV, December 2013

    Google Scholar 

  38. Xu, Y., Piao, Z., Gao, S.: Encoding crowd interaction with deep neural network for pedestrian trajectory prediction. In: CVPR, June 2018

    Google Scholar 

  39. Xue, H., Huynh, D., Reynolds, M.: Bi-Prediction: pedestrian trajectory prediction based on bidirectional LSTM classification. In: International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 307–314 (2017)

    Google Scholar 

  40. Xue, H., Huynh, D.Q., Reynolds, M.: SS-LSTM: a hierarchical LSTM model for pedestrian trajectory prediction. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1186–1194. IEEE (2018)

    Google Scholar 

  41. Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: CVPR, pp. 1345–1352. IEEE (2011)

    Google Scholar 

  42. Yi, S., Li, H., Wang, X.: Understanding pedestrian behaviors from stationary crowd groups. In: CVPR, pp. 3488–3496 (2015)

    Google Scholar 

  43. Yi, S., Li, H., Wang, X.: Pedestrian behavior understanding and prediction with deep neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 263–279. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_16

    Chapter  Google Scholar 

  44. Zhou, B., Wang, X., Tang, X.: Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: CVPR, pp. 2871–2878. IEEE (2012)

    Google Scholar 

  45. Zhu, W., et al.: Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks. In: AAAI, pp. 3697–3703 (2016)

    Google Scholar 

  46. Zou, H., Su, H., Song, S., Zhu, J.: Understanding human behaviors in crowds by imitating the decision-making process. In: AAAI (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Western Australia, Perth, Australia

    Hao Xue, Du Q. Huynh & Mark Reynolds

Authors
  1. Hao Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Du Q. Huynh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark Reynolds
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Xue .

Editor information

Editors and Affiliations

  1. Department of Computing, Macquarie University, Sydney, NSW, Australia

    Abhaya C. Nayak

  2. RIKEN Center for Integrative Medical Sciences, Yokohama, Japan

    Alok Sharma

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xue, H., Huynh, D.Q., Reynolds, M. (2019). Pedestrian Trajectory Prediction Using a Social Pyramid. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11671. Springer, Cham. https://doi.org/10.1007/978-3-030-29911-8_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29911-8_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29910-1

  • Online ISBN: 978-3-030-29911-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics