Abstract
In this work, we use Recurrent Neural Networks (RNNs) in form of Gated Recurrent Unit (GRU) networks to forecast trajectories of vulnerable road users (VRUs), such as pedestrians and cyclists, in road traffic utilizing the past trajectory and 3D poses as input. The 3D poses represent the postures and movements of limbs and torso and contain early indicators for the transition between motion types, e.g. wait, start, move, and stop. VRUs often only become visible from the perspective of an approaching vehicle shortly before dangerous situations occur. Therefore, a network architecture is required which is able to forecast trajectories after short time periods and is able to improve the forecasts in case of longer observations. This motivates us to use GRU networks, which are able to use time series of varying duration as inputs, and to investigate the effects of different observation periods on the forecasting results. Our approach is able to make reasonable forecasts even for short observation periods. The use of poses improves the forecasting accuracy, especially for short observation periods compared to a solely head trajectory based approach. Different motion types benefit to different extent from the use of poses and longer observation periods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: human trajectory prediction in crowded spaces. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 961–971. Las Vegas (2016)
Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302–1310. Honolulu (2017)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734. ACL (2014)
Fang, Z., López, A.M.: Is the pedestrian going to cross? answering by 2D pose estimation. In: IEEE Intelligent Vehicles Symposium (IV), Changshu, pp. 1271–1276 (2018)
Fang, Z., López, A.M.: Intention recognition of pedestrians and cyclists by 2D pose estimation. IEEE Trans. Intell. Transp. Syst. 21, 1–11 (2019)
Fang, Z., Vázquez, D., López, A.M.: On-board detection of pedestrian intentions. Sensors 17(10), 2193 (2017)
Goldhammer, M.: Selbstlernende Algorithmen zur videobasierten Absichtserkennung von Fußgängern. Ph.D. thesis, University of Kassel (2016)
Hubert, A., Zernetsch, S., Doll, K., Sick, B.: Cyclists’ starting behavior at intersections. In: IEEE Intelligent Vehicles Symposium (IV), pp. 1071–1077. Los Angeles (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (ICLR), San Diego (2015)
Kitani, K.M., Ziebart, B.D., Bagnell, J.A., Hebert, M.: Activity forecasting. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 201–214. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33765-9_15
Kress, V., Jung, J., Zernetsch, S., Doll, K., Sick, B.: Human pose estimation in real traffic scenes. In: IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, pp. 518–523 (2018)
Kress, V., Jung, J., Zernetsch, S., Doll, K., Sick, B.: Pose based start intention detection of cyclists. In: IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, pp. 2381–2386 (2019)
Kress, V., Zernetsch, S., Doll, K., Sick, B.: Pose based trajectory forecast of vulnerable road users. In: IEEE Symposium Series on Computational Intelligence (SSCI), Xiamen, pp. 1200–1207 (2019)
Lee, N., Choi, W., Vernaza, P., Choy, C.B., Torr, P.H.S., Chandraker, M.: DESIRE: distant future prediction in dynamic scenes with interacting agents. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2165–2174 (2017)
Pool, E.A.I., Kooij, J.F.P., Gavrila, D.M.: Context-based cyclist path prediction using Recurrent Neural Networks. In: IEEE Intelligent Vehicles Symposium (IV), pp. 824–830 (2019)
Quintero, R., Parra, I., Fernández-Llorca, D., Sotelo, M.A.: Pedestrian path, pose, and intention prediction through gaussian process dynamical models and pedestrian activity recognition. IEEE Trans. Intell. Transp. Syst. 20(5), 1803–1814 (2019)
Radwan, N., Valada, A., Burgard, W.: Multimodal interaction-aware motion prediction for autonomous street crossing. arXiv:1808.06887 (2019)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)
Schneemann, F., Heinemann, P.: Context-based detection of pedestrian crossing intention for autonomous driving in urban environments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, pp. 2243–2248 (2016)
Tome, D., Russell, C., Agapito, L.: Lifting from the deep: Convolutional 3d pose estimation from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5689–5698. Honolulu (2017)
Wang, S., et al.: Leverage of limb detection in pose estimation for vulnerable road users. In: IEEE Intelligent Transportation Systems Conference (ITSC), pp. 528–534 (2019)
Werbos, P.J.: Backpropagation through time: what it does and how to do it. Proc. IEEE 78(10), 1550–1560 (1990)
Acknowledgment
This work was supported by “Zentrum Digitalisierung.Bayern”. In addition, the work is backed by the project DeCoInt\(^2\), supported by the German Research Foundation (DFG) within the priority program SPP 1835: “Kooperativ interagierende Automobile”, grant numbers DO 1186/1-2 and SI 674/11-2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kress, V., Zernetsch, S., Doll, K., Sick, B. (2021). Pose Based Trajectory Forecast of Vulnerable Road Users Using Recurrent Neural Networks. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12661. Springer, Cham. https://doi.org/10.1007/978-3-030-68763-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-68763-2_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68762-5
Online ISBN: 978-3-030-68763-2
eBook Packages: Computer ScienceComputer Science (R0)
