skip to main content
10.1145/3397536.3422258acmconferencesArticle/Chapter ViewAbstractPublication PagesgisConference Proceedingsconference-collections
research-article

Cycling-Net: A Deep Learning Approach to Predicting Cyclist Behaviors from Geo-Referenced Egocentric Video Data

Published:13 November 2020Publication History

ABSTRACT

Cycling, as a green transportation mode, provides an environmentally friendly transportation choice for short-distance traveling. However, cyclists are also getting involved in fatal accidents more frequently in recent years. Thus, understanding and modeling their road behaviors is crucial in helping improving road safety laws and infrastructures. Traditionally, people understand road user behavior using either purely spatial trajectory data, or videos from fixed surveillance camera through tracking or predicting their paths. However, these data only cover limited areas and do not provide information from the cyclist's field of view. In this paper, we take advantage of geo-referenced egocentric video data collected from the handlebar cameras of cyclists to learn how to predict their behaviors. This approach is technically more challenging, because both the observer and objects in the scene might be moving, and there are strong temporal dependencies in both the behaviors of cyclists and the video scenes. We propose Cycling-Net, a novel deep learning model that tracks different types of objects in consecutive scenes and learns the relationship between the movement of these objects and the behavior of the cyclist. Experiment results on a naturalistic trip dataset show the Cycling-Net is effective in behavior prediction and outperforms a baseline model.

References

  1. Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. 2016. Social lstm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 961--971.Google ScholarGoogle ScholarCross RefCross Ref
  2. P Chao, Judson S Matthias, and Mary R Anderson. 1978. Cyclist Behavior at Signalized Intersections. J. Transportation Research Board 683 (1978), 34--39.Google ScholarGoogle Scholar
  3. Marco Dozza and Andre Fernandez. 2013. Understanding bicycle dynamics and cyclist behavior from naturalistic field data. IEEE Trans. on Intelligent Transportation Systems 15, 1 (2013), 376--384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ariane Ghekiere, Benedicte Deforche, Lieze Mertens, Ilse De Bourdeaudhuij, Peter Clarys, Bas de Geus, Greet Cardon, Jack Nasar, Jo Salmon, and Jelle Van Cauwenberg. 2015. Creating cycling-friendly environments for children: which micro-scale factors are most important? An experimental study using manipulated photographs. PloS one 10, 12 (2015).Google ScholarGoogle Scholar
  5. Cara J Hamann and Corinne Peek-Asa. 2017. Beyond GPS: Improved study of bicycling exposure through added use of video data. J. Transport & Health 4 (2017), 363--372.Google ScholarGoogle ScholarCross RefCross Ref
  6. M Anne Harris, Conor CO Reynolds, Meghan Winters, Mary Chipman, Peter A Cripton, Michael D Cusimano, and Kay Teschke. 2011. The Bicyclists' Injuries and the Cycling Environment study: a protocol to tackle methodological issues facing studies of bicycling safety. Injury Prevention 17, 5 (2011), e6-e6.Google ScholarGoogle ScholarCross RefCross Ref
  7. Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick. 2017. Mask R-CNN. In The IEEE International Conference on Computer Vision (ICCV).Google ScholarGoogle Scholar
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735--1780. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ling Huang, Jianping Wu, Feng You, Zhihan Lv, and Houbing Song. 2016. Cyclist social force model at unsignalized intersections with heterogeneous traffic. IEEE Trans. on Industrial Informatics 13, 2 (2016), 782--792.Google ScholarGoogle ScholarCross RefCross Ref
  10. Christian Juhra, Britta Wieskoetter, K Chu, L Trost, U Weiss, M Messerschmidt, A Malczyk, M Heckwolf, and M Raschke. 2012. Bicycle accidents-Do we only see the tip of the iceberg?: A prospective multi-centre study in a large German city combining medical and police data. Injury 43, 12 (2012), 2026--2034.Google ScholarGoogle ScholarCross RefCross Ref
  11. Sigal Kaplan, Konstantinos Vavatsoulas, and Carlo Giacomo Prato. 2014. Aggravating and mitigating factors associated with cyclist injury severity in Denmark. J. Safety Research 50 (2014), 75--82.Google ScholarGoogle ScholarCross RefCross Ref
  12. Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander G Hauptmann, and Li Fei-Fei. 2019. Peeking into the future: Predicting future person activities and locations in videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5725--5734.Google ScholarGoogle ScholarCross RefCross Ref
  13. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 740--755.Google ScholarGoogle ScholarCross RefCross Ref
  14. Huang Ling and Jianping Wu. 2004. A study on cyclist behavior at signalized intersections. IEEE Trans. on Intelligent Transportation Systems 5, 4 (2004), 293--299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jianming Lv, Qing Li, Qinghui Sun, and Xintong Wang. 2018. T-CONV: A convolutional neural network for multi-scale taxi trajectory prediction. In Proceedings of the IEEE International Conference on Big Data and Smart Computing. 82--89.Google ScholarGoogle ScholarCross RefCross Ref
  16. Xiaoliang Ma and Ding Luo. 2016. Modeling cyclist acceleration process for bicycle traffic simulation using naturalistic data. Transportation Research Part F: Traffic Psychology and Behaviour 40 (2016), 130--144.Google ScholarGoogle ScholarCross RefCross Ref
  17. Tomas Mikolov, Martin KarafiÃąt, Lukas Burget, Jan Cernocká, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Proceedings of the 11th Annual Conference of the International Speech Communication Association. 1045--1048.Google ScholarGoogle ScholarCross RefCross Ref
  18. Anton Milan, Laura Leal-Taixé, Ian Reid, Stefan Roth, and Konrad Schindler. 2016. MOT16: A benchmark for multi-object tracking. (2016). arXiv:1603.00831Google ScholarGoogle Scholar
  19. Debbie A Niemeier. 1996. Longitudinal analysis of bicycle count variability: Results and modeling implications. J. Transportation Engineering 122, 3 (1996), 200--206.Google ScholarGoogle ScholarCross RefCross Ref
  20. U.S. Department of Transportation. 2020. Bicycle Safety Guide and Countermeasure Selection System. Retrieved Jun 23, 2020 from http://www.pedbikesafe.org/bikesafe/countermeasures.cfmGoogle ScholarGoogle Scholar
  21. Felipe E Pedroso, Federico Angriman, Alexandra L Bellows, and Kathryn Taylor. 2016. Bicycle use and cyclist safety following BostonâĂŹs bicycle infrastructure expansion, 2009-2012. American J. Public Health 106, 12 (2016), 2171--2177.Google ScholarGoogle ScholarCross RefCross Ref
  22. Hyun Soo Park, Jyh-Jing Hwang, Yedong Niu, and Jianbo Shi. 2016. Egocentric future localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4697--4705.Google ScholarGoogle ScholarCross RefCross Ref
  23. Shan Su, Jung Pyo Hong, Jianbo Shi, and Hyun Soo Park. 2017. Predicting behaviors of basketball players from first person videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1501--1510.Google ScholarGoogle ScholarCross RefCross Ref
  24. ShiJie Sun, Naveed Akhtar, HuanSheng Song, Ajmal S Mian, and Mubarak Shah. 2019. Deep affinity network for multiple object tracking. IEEE Trans. on Pattern Analysis and Machine Intelligence (2019).Google ScholarGoogle Scholar
  25. Hao Wu, Ziyang Chen, Weiwei Sun, Baihua Zheng, and Wei Wang. 2017. Modeling trajectories with recurrent neural networks. In Proceedings of the International Joint Conferences on Artificial Intelligence (IJCAI). 3083âÃŞ3090. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Yanyu Xu, Zhixin Piao, and Shenghua Gao. 2018. Encoding crowd interaction with deep neural network for pedestrian trajectory prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5275--5284.Google ScholarGoogle ScholarCross RefCross Ref
  27. Takuma Yagi, Karttikeya Mangalam, Ryo Yonetani, and Yoichi Sato. 2018. Future person localization in first-person videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7593--7602.Google ScholarGoogle ScholarCross RefCross Ref
  28. Yu Yao, Mingze Xu, Chiho Choi, David J Crandall, Ella M Atkins, and Behzad Dariush. 2019. Egocentric vision-based future vehicle localization for intelligent driving assistance systems. In Proceedings of the IEEE Conference on Robotics and Automation (ICRA). 9711--9717.Google ScholarGoogle ScholarCross RefCross Ref
  29. Shuai Yi, Hongsheng Li, and Xiaogang Wang. 2015. Understanding pedestrian behaviors from stationary crowd groups. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3488--3496.Google ScholarGoogle ScholarCross RefCross Ref
  30. Shuai Yi, Hongsheng Li, and Xiaogang Wang. 2016. Pedestrian behavior understanding and prediction with deep neural networks. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 263--279.Google ScholarGoogle ScholarCross RefCross Ref
  31. Zhuoning Yuan, Xun Zhou, and Tianbao Yang. 2018. Hetero-convlstm: A deep learning approach to traffic accident prediction on heterogeneous spatio-temporal data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 984--992. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Cycling-Net: A Deep Learning Approach to Predicting Cyclist Behaviors from Geo-Referenced Egocentric Video Data

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        SIGSPATIAL '20: Proceedings of the 28th International Conference on Advances in Geographic Information Systems
        November 2020
        687 pages
        ISBN:9781450380195
        DOI:10.1145/3397536

        Copyright © 2020 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 13 November 2020

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed limited

        Acceptance Rates

        Overall Acceptance Rate220of1,116submissions,20%

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader