Abstract
The full-scale deployment of autonomous driving demands successful interaction with pedestrians and other vulnerable road users, which requires an understanding of their dynamic behavior and intention. Current research achieves this by estimating pedestrian’s trajectory mainly based on the gait and movement information in the past as well as other relevant scene information. However, the autonomous vehicles still struggle with such interactions since the visual features alone may not supply subtle details required to attain a superior understanding. The decision-making ability of the system can improve by incorporating human knowledge to guide the vision-based algorithms. In this paper, we adopt a novel approach to retrieve human knowledge from the natural text descriptions about the pedestrian-vehicle encounters, which is crucial to anticipate the pedestrian intention and is difficult for computer vision (CV) algorithms to capture automatically. We applied natural language processing (NLP) techniques on the aggregated description from different annotators to generate a temporal knowledge graph, which can achieve the changes of intention and the corresponding reasoning processes in a better resolution. In future work, we plan to show that in combination with video processing algorithms, the knowledge graph has the potential to aid the decision-making process to be more accurate by passively integrating the reasoning ability of humans.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Global status report on road safety 2018: summary (2018). https://www.who.int/violence_injury_prevention/road_safety_status/2018/en/
Antol, S., et al.: VQA: visual question answering. CoRR abs/1505.00468 (2015). http://arxiv.org/abs/1505.00468
Ezzati Amini, R., Katrakazas, C., Antoniou, C.: Negotiation and decision-making for a pedestrian roadway crossing: a literature review. Sustainability 11, 6713 (2019). https://doi.org/10.3390/su11236713
Faerber, B.: Communication and Communication Problems Between Autonomous Vehicles and Human Drivers, pp. 125–144 (2016). https://doi.org/10.1007/978-3-662-48847-8_7
Gupta, S., Vasardani, M., Winter, S.: Negotiation between vehicles and pedestrians for the right of way at intersections. IEEE Trans. Intell. Transp. Syst. 20(3), 888–899 (2019). https://doi.org/10.1109/TITS.2018.2836957
Hagenzieker, M., et al.: Interactions between cyclists and automated vehicles: results of a photo experiment*. J. Transp. Saf. Secur. 1–22 (2019). https://doi.org/10.1080/19439962.2019.1591556
Honnibal, M., Johnson, M.: An improved non-monotonic transition system for dependency parsing. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 1373–1378. Association for Computational Linguistics, September 2015. https://aclweb.org/anthology/D/D15/D15-1162
Honnibal, M., Montani, I.: spaCy 2: natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing (2017, to appear)
Müller, L., Risto, M., Emmenegger, C.: The social behavior of autonomous vehicles, pp. 686–689 (2016). https://doi.org/10.1145/2968219.2968561
Oxley, J., Fildes, B., Ihsen, E., Charlton, J., Day, R.: Differences in traffic judgements between young and old adult pedestrians. Accid. Anal. Prev. 29(6), 839 – 847 (1997). https://doi.org/10.1016/S0001-4575(97)00053-5. http://www.sciencedirect.com/science/article/pii/S0001457597000535
Quintero Mínguez, R., Alonso, I., Ferníndez-Llorca, D., Sotelo, M.: Pedestrian path, pose, and intention prediction through gaussian process dynamical models and pedestrian activity recognition. IEEE Trans. Intell. Transp. Syste. 1–12 (2018). https://doi.org/10.1109/TITS.2018.2836305
Rasouli, A., Kotseruba, Y., Tsotsos, J.: Understanding pedestrian behavior in complex traffic scenes. IEEE Trans. Intell. Veh. 1 (2017). https://doi.org/10.1109/TIV.2017.2788193
Razmi Rad, S., Homem de Almeida Correia, G., Hagenzieker, M.: Pedestrians’ road crossing behaviour in front of automated vehicles: results from a pedestrian simulation experiment using agent-based modelling. Transp. Res. Part F Traffic Psychol. Behav. 69 (2020). https://doi.org/10.1016/j.trf.2020.01.014
Schmidt, S., Faerber, B.: Pedestrians at the kerb - recognising the action intentions of humans. Transp. Res. Part F-Traffic Psychol. Behav. 12, 300–310 (2009). https://doi.org/10.1016/j.trf.2009.02.003
Sherony, R., Tian, R., Chien, S., Fu, L., Chen, Y., Takahashi, H.: Pedestrian/bicyclist limb motion analysis from 110-car TASI video data for autonomous emergency braking testing surrogate development. SAE Int. J. Transp. Saf. 4(1), 113–120 (2016)
Tian, R., et al.: Pilot study on pedestrian step frequency in naturalistic driving environment. In: 2013 IEEE Intelligent Vehicles Symposium (IV), pp. 1215–1220. IEEE (2013)
Tian, R., Li, L., Yang, K., Chien, S., Chen, Y., Sherony, R.: Estimation of the vehicle-pedestrian encounter/conflict risk on the road based on TASI 110-car naturalistic driving data collection. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, pp. 623–629. IEEE (2014)
Volz, B., Behrendt, K., Mielenz, H., Gilitschenski, I., Siegwart, R., Nieto, J.: A data-driven approach for pedestrian intention estimation, pp. 2607–2612 (2016). https://doi.org/10.1109/ITSC.2016.7795975
Wolf, I.: The interaction between humans and autonomous agents. In: Maurer, M., Gerdes, J.C., Lenz, B., Winner, H. (eds.) Autonomous Driving, pp. 103–124. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-48847-8_6
Yang, X., Tang, K., Zhang, H., Cai, J.: Auto-encoding scene graphs for image captioning. CoRR abs/1812.02378 (2018). http://arxiv.org/abs/1812.02378
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Elahi, M.F., Luo, X., Tian, R. (2020). A Framework for Modeling Knowledge Graphs via Processing Natural Descriptions of Vehicle-Pedestrian Interactions. In: Stephanidis, C., Duffy, V.G., Streitz, N., Konomi, S., Krömker, H. (eds) HCI International 2020 – Late Breaking Papers: Digital Human Modeling and Ergonomics, Mobility and Intelligent Environments. HCII 2020. Lecture Notes in Computer Science(), vol 12429. Springer, Cham. https://doi.org/10.1007/978-3-030-59987-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-59987-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59986-7
Online ISBN: 978-3-030-59987-4
eBook Packages: Computer ScienceComputer Science (R0)