Abstract
A key goal of intelligent vehicles is to provide a safer and more efficient method of transportation. One important aspect of intelligent vehicles is to understand the road scene using vehicle-mounted camera images. Perceiving the level of driving risk of a given road scene enables intelligent vehicles to drive more efficiently without compromising on safety. Existing road scene understanding methods, however, do not explicitly nor holistically model this notion of driving risk. This paper proposes a new perspective on scene risk perception by modeling end-to-end road scene affordance using a weakly supervised classifier. A subset of images from BDD100k dataset was relabeled to evaluate the proposed model. Experimental results show that the proposed model is able to correctly classify three different levels of risk. Further, saliency maps were used to demonstrate that the proposed model is capable of visually reasoning about the underlying causes of its decision. By understanding risk holistically, the proposed method is intended to be complementary to existing advanced driver assistance systems and autonomous vehicles.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bojarski, M., et al.: End to end learning for self-driving cars. http://arxiv.org/abs/1604.07316
Chen, C., Seff, A., Kornhauser, A., Xiao, J.: DeepDriving: learning affordance for direct perception in autonomous driving. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2722–2730. IEEE. https://doi.org/10.1109/ICCV.2015.312, http://ieeexplore.ieee.org/document/7410669/
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR09 (2009)
Fridman, L., Brown, D.E., Kindelsberger, J., Angell, L., Mehler, B., Reimer, B.: Human side of tesla autopilot: Exploration of functional vigilance in real-world human-machine collaboration
Gibson, J.J.: The Ecological Approach to Visual Perception: classic edition. Psychology Press (2014)
Hassanin, M., Khan, S., Tahtali, M.: Visual affordance and function understanding: a survey (2018). arXiv preprint, arXiv:1807.06775
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385
Jeong, S.G., Kim, J., Kim, S., Min, J.: End-to-end learning of image based lane-change decision. http://arxiv.org/abs/1706.08211
Joo, J., Li, W., Steen, F.F., Zhu, S.C.: Visual persuasion: inferring communicative intents of images. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 216–223. IEEE. https://doi.org/10.1109/CVPR.2014.35, http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6909429
Kokubun, M., Konishi, H., Higuchi, K., Kurahashi, T., Umemura, Y., Nishi, H.: Assessment of drivers’ risk perception using a simulator. R&D Rev. Toyota CRDL 39(2), 9–15 (2004)
Lin, S.C., et al.: The architectural implications of autonomous driving: constraints and acceleration. In: Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 751–766. ACM (2018)
Lüddecke, T., Wörgötter, F.: Scene affordance: inferring actions from household scenes. In: Workshop on Action and Anticipation for Visual Learning, European Conference on Computer Vision (2016)
Oliva, A.: Gist of the scene. In: Neurobiology of attention, pp. 251–256. Elsevier (2005)
Ramanishka, V., Chen, Y.T., Misu, T., Saenko, K.: Toward driving scene understanding: a dataset for learning driver behavior and causal reasoning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7699–7707 (2018)
Rasouli, A., Kotseruba, I., Tsotsos, J.K.: Are they going to cross? A benchmark dataset and baseline for pedestrian crosswalk behavior. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 206–213. IEEE. https://doi.org/10.1109/ICCVW.2017.33, http://ieeexplore.ieee.org/document/8265243/
Rolison, J.J., Regev, S., Moutari, S., Feeney, A.: What are the factors that contribute to road accidents? an assessment of law enforcement views, ordinary drivers’ opinions, and road accident records. Accid. Anal. Prev. 115, 11–24 (2018)
Sauer, A., Savinov, N., Geiger, A.: Conditional affordance learning for driving in urban environments. arXiv preprint arXiv:1806.06498 (2018)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: Visual explanations from deep networks via gradient-based localization. http://arxiv.org/abs/1610.02391
Siam, M., Gamal, M., Abdel-Razek, M., Yogamani, S., Jagersand, M., Zhang, H.: A comparative study of real-time semantic segmentation for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 587–597 (2018)
Sjöberg, L., Moen, B.E., Rundmo, T.: Explaining risk perception. an evaluation of the psychometric paradigm in risk perception research. Rotunde publikasjoner Rotunde 84, 55–76 (2004)
Wang, H.L., Cheong, L.F.: Affective understanding in film. IEEE Trans. Circuits Syst. Video Technol. 16(6), 689–704 (2006)
Wang, S., Fidler, S., Urtasun, R.: Holistic 3d scene understanding from a single geo-tagged image. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3964–3972. IEEE. https://doi.org/10.1109/CVPR.2015.7299022, http://ieeexplore.ieee.org/document/7299022/
Xu, H., Gao, Y., Yu, F., Darrell, T.: End-to-end learning of driving models from large-scale video datasets. http://arxiv.org/abs/1612.01079
Yu, F., et al.: BDD100K: A diverse driving video database with scalable annotation tooling. CoRR abs/1805.04687 (2018). http://arxiv.org/abs/1805.04687
Zellers, R., Bisk, Y., Farhadi, A., Choi, Y.: From recognition to cognition: visual commonsense reasoning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6720–6731 (2019)
Čermák, J., Angelova, A.: Learning with proxy supervision for end-to-end visual learning. In: 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 1–6 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Withanawasam, J., Javanmardi, E., Wong, K., Javanmardi, M., Kamijo, S. (2020). Road Scene Risk Perception for Intelligent Vehicles Using End-to-End Affordance Learning and Visual Reasoning. In: Palaiahnakote, S., Sanniti di Baja, G., Wang, L., Yan, W. (eds) Pattern Recognition. ACPR 2019. Lecture Notes in Computer Science(), vol 12046. Springer, Cham. https://doi.org/10.1007/978-3-030-41404-7_60
Download citation
DOI: https://doi.org/10.1007/978-3-030-41404-7_60
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41403-0
Online ISBN: 978-3-030-41404-7
eBook Packages: Computer ScienceComputer Science (R0)