Skip to main content
SpringerLink
Log in

Moving vehicle detection and tracking at roundabouts using deep learning with trajectory union

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The number of vehicles and turning movements at roundabouts provide important information for planning, design and operational analysis of roundabouts. The visual data collected through video cameras make it possible to determine such information via computer-based methods. In this work, a method for detecting, counting, and tracking vehicles in roundabout videos is proposed. There are two main contributions of the method, (i) only the moving vehicles are considered for tracking (moving vehicle detection) and (ii) the vehicle tracks output by the object tracking algorithms are processed to reduce the false track rate (trajectory union). The vehicle detection is performed using YOLOv4, and vehicle tracking throughout the video is accomplished by either Kalman filter or DeepSORT algorithm. The output of the proposed method is compared with both manual counting results and benchmark tracking results where the entry/exit matrix is generated using only YOLOv4 and an object tracker. In a 20-min video with 297 vehicles, absolute error reached by the proposed method is 14 vehicles which corresponds to normalized absolute error percentage of 1.571%. In the same video, the same error metrics obtained by the benchmark method are 33 vehicles and 3.704%. This error tends to increase together with the rate of vehicles in the video or number of legs in the roundabout. However, the rate of increment in the error is much lower than the rate of increment in the number of vehicles in the video. In addition, lower error rates are obtained when DeepSORT is used as the vehicle tracker.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. "https://github.com/AlexeyAB/darknet," Github Repository, Accessed: 07 June 2021.

  2. Abdelwahab MA (2019) "Accurate Vehicle Counting Approach Based on Deep Neural Networks," Proceedings of 2019 International Conference on Innovative Trends in Computer Engineering (Itce 2019), pp. 1–5

  3. Ammour N, Alhichri H, Bazi Y, Benjdira B, Alajlan N, Zuair M (2017) Deep learning approach for car detection in UAV imagery. Remote Sensing 9:312

    Article  Google Scholar 

  4. Artesea G (2007) "Detecting and tracking vehicles in a roundabout," in International Archives of Photogrammetry and Remote Sensing and Spatial Geoinformation Sciences, pp. 6–13.

  5. Awang S, Azmi NMAN (2017) “Vehicle Counting System Based on Vehicle Type Classification Using Deep Learning Method,” in IT Convergence and Security. Singapore 2018:52–59

    Google Scholar 

  6. Bassani M, Mussone L (2020) Experimental analysis of operational data for roundabouts through advanced image processing. J Traffic Transp Eng-English Edition 7:482–497

    Article  Google Scholar 

  7. Bochkovskiy A, Wang C.-Y, and Liao H.-YM (2020) "Yolov4: Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934

  8. Bui N, Yi H, and Cho J (2020) A vehicle counts by class framework using distinguished regions tracking at multiple intersections," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 578–579

  9. Bui K.-HN, Yi H, Jung H, and Cho J (2020) "Video-Based Traffic Flow Analysis for Turning Volume Estimation at Signalized Intersections," in Intelligent Information and Database Systems, Cham, pp. 152–162.

  10. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) “End-to-End Object Detection with Transformers,” in Computer Vision – ECCV. Cham 2020:213–229

    Google Scholar 

  11. Carranza-García M, Torres-Mateo J, Lara-Benítez P, García-Gutiérrez J (2021) On the Performance of One-Stage and Two-Stage Object Detectors in Autonomous Vehicles Using Camera Data. Remote Sensing 13:89

    Article  Google Scholar 

  12. Dai Z, Song H, Wang X, Fang Y, Yun X, Zhang Z et al (2019) Video-Based Vehicle Counting Framework. IEEE Access 7:64460–64470

    Article  Google Scholar 

  13. Dinh H, Tang H (2017) Development of a tracking-based system for automated traffic data collection for roundabouts. J Mod Transp 25:12–23

    Article  Google Scholar 

  14. Fernandez-Sanjurjo M, Bosquet B, Mucientes M, Brea VM (2019) Real-time visual detection and tracking system for traffic monitoring. Eng Appl Artif Intell 85:410–420

    Article  Google Scholar 

  15. Fernández-Sanjurjo M, Mucientes M, and Brea VM (2019) "Real-Time Traffic Monitoring with Occlusion Handling," in Pattern Recognition and Image Analysis, Cham, pp. 273–284.

  16. H. P. M, Thomas A, N. J. S, Gopi VP, and Palanisamy P (2021) Pixel matching search algorithm for counting moving vehicle in highway traffic videos, Multimed Tools Appl 80:3153–3172

  17. Huang L, Xu W, Liu S, Pandey V, Juri NR (2017) “Enabling versatile analysis of large scale traffic video data with deep learning and HiveQL,” in. IEEE International Conference on Big Data (Big Data) 2017:1153–1162

    Article  Google Scholar 

  18. Ismail K, Sayed T, Saunier N (2010) “Camera calibration for urban traffic scenes: Practical issues and a robust approach,” presented at the 89th Annual Meeting of the Transportation Research Board. DC, Washington

    Google Scholar 

  19. Kastrinaki V, Zervakis M, Kalaitzakis K (2003) A survey of video processing techniques for traffic applications. Image Vis Comput 21:359–381

    Article  Google Scholar 

  20. Khalkhali MB, Vahedian A, and Yazdi HS (2021) "Situation Assessment-Augmented Interactive Kalman Filter for Multi-Vehicle Tracking," IEEE Transactions on Intelligent Transportation Systems

  21. Khan MA, Ectors W, Bellemans T, Ruichek Y, Janssens D, Wets G (2018) Unmanned aerial vehicle-based traffic analysis: A case study to analyze traffic streams at urban roundabouts. Procedia Comput Sci 130:636–643

    Article  Google Scholar 

  22. Lu X, Wang W, Ma C, Shen J, Shao L and Porikli F (2019) "See more, know more: Unsupervised video object segmentation with co-attention siamese networks," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3623–3632

  23. Lu X, Wang W, Shen J, Crandall D, and Luo J (2020) "Zero-shot video object segmentation with co-attention siamese networks," IEEE transactions on pattern analysis and machine intelligence

  24. Lu X, Wang W, Shen J, TaiY.-W , Crandall DJ, and Hoi SC (2020) "Learning video object segmentation from unlabeled videos," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8960–8970

  25. Mohamed E and El-Sallab A (2021) "MODETR: Moving Object Detection with Transformers," arXiv preprint arXiv:2106.11422 2021

  26. Mussone L, Matteucci M, Bassani M, and Rizzi D (2011) "Traffic Analysis in Roundabout Intersections by Image Processing," IFAC Proceedings Volumes 44:14922–14927

  27. Mussone L, Matteucci M, Bassani M, and Rizzi D (2011) A comparative analysis of vehicular flow image surveys for roundabout performance evaluation

  28. Mussone L, Matteucci M, Bassani M, Rizzi D (2013) An innovative method for the analysis of vehicle movements in roundabouts based on image processing. J Adv Transp 47:581–594

    Google Scholar 

  29. Ning X, Li W, and Tian W (2018) Deep Adaptive Update of Discriminant KCF for Visual Tracking," in International Conference on Neural Information Processing, pp. 441–451

  30. Ning X, Li W, Tian W, Xu C, Xu J (2019) Adaptive template update of discriminant KCF for visual tracking. CAAI Trans Intell Syst 14:121–126

    Google Scholar 

  31. Pan Z, Liu S, Fu W (2017) A review of visual moving target tracking. Multimed Tool Appl 76:16989–17018

    Article  Google Scholar 

  32. Redmon J, Divvala S, Girshick R, and Farhadi A (2016) You only look once: Unified, real-time object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779–788

  33. SenthilMurugan A, Suganya Devi K, Sivaranjani A, Srinivasan P (2018) A study on various methods used for video summarization and moving object detection for video surveillance applications. Multimed Tools Appl 77:23273–23290

    Article  Google Scholar 

  34. St-Aubin P, Saunier N, Miranda-Moreno L (2015) Large-scale automated proactive road safety analysis using video data. Transp Res Part C: Emerg Technol 58:363–379

    Article  Google Scholar 

  35. Sudha D and Priyadarshini J (2020) "An intelligent multiple vehicle detection and tracking using modified vibe algorithm and deep learning algorithm," Soft Computing

  36. Sun M, Wang Y, Li T, Lv J, Wu J (2017) Vehicle counting in crowded scenes with multi-channel and multi-task convolutional neural networks. J Vis Commun Image Represent 49:412–419

    Article  Google Scholar 

  37. Tian Z, Shen C, Chen H, and He T (2019) "FCOS: Fully convolutional one-stage object detection," in Proceedings of the IEEE/CVF international conference on computer vision, pp. 9627–9636

  38. Unzueta L, Nieto M, Cortes A, Barandiaran J, Otaegui O, Sanchez P (2012) Adaptive Multicue Background Subtraction for Robust Vehicle Counting and Classification. IEEE Trans Intell Transp Syst 13:527–540

    Article  Google Scholar 

  39. Wang J, Simeonova S, and Shahbazi M (2019) "Orientation- and Scale-Invariant Multi-Vehicle Detection and Tracking from Unmanned Aerial Videos," Remote Sensing, vol. 11

  40. Wojke N, Bewley A, Paulus D (2017) “Simple online and realtime tracking with a deep association metric,” in. IEEE international conference on image processing (ICIP) 2017:3645–3649

    Google Scholar 

  41. Wu YJ, Lian FL, and Chang TH (2006) "Traffic monitoring and vehicle tracking using roadside cameras," 2006 IEEE International Conference on Systems, Man and Cybernetics, Vols 1–6, Proceedings, pp. 4631–4636

  42. Xin J, Du X, and Zhang J (2017) "Deep Learning for Robust Outdoor Vehicle Visual Tracking," 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 613–618

  43. Yang K, Hu X, Bergasa LM, Romera E, Huang X, Sun D et al (2019) “Can we pass beyond the field of view? panoramic annular semantic segmentation for real-world surrounding perception,” in. IEEE Intelligent Vehicles Symposium (IV) 2019:446–453

    Article  Google Scholar 

  44. Yang K, Hu X, Bergasa LM, Romera E, Wang K (2019) Pass: Panoramic annular semantic segmentation. IEEE Trans Intell Transp Syst 21:4171–4185

    Article  Google Scholar 

  45. Yang K, Hu X, Chen H, Xiang K, Wang K, Stiefelhagen R (2020) “Ds-pass: Detail-sensitive panoramic annular semantic segmentation through swaftnet for surrounding sensing,” in. IEEE Intelligent Vehicles Symposium (IV) 2020:457–464

    Google Scholar 

  46. Yang K, Hu X, Stiefelhagen R (2021) Is context-aware CNN ready for the surroundings? panoramic semantic segmentation in the wild. IEEE Trans Image Process 30:1866–1881

    Article  Google Scholar 

  47. Zhang Z, Liu K, Gao F, Li X, Wang G (2016) “Vision-based vehicle detecting and counting for traffic flow analysis,” in. International Joint Conference on Neural Networks (IJCNN) 2016:2267–2273

    Article  Google Scholar 

  48. Zhang J, Sun J, Wang J, and Yue X-G (2020) Visual object tracking based on residual network and cascaded correlation filters," Journal of Ambient Intelligence and Humanized Computing, pp. 1–14

  49. Zhang YJ, Wang J, and Yang X (2017) "Real-time vehicle detection and tracking in video based on faster R-CNN," 2nd Annual International Conference on Information System and Artificial Intelligence (Isai2017), vol. 887

  50. Zhang S, Wu G, Costeira JP, and Moura JM (2017) "Fcn-rlstm: Deep spatio-temporal neural networks for vehicle counting in city cameras," in Proceedings of the IEEE international conference on computer vision, pp. 3667–3676

  51. Zhou W, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Dokuz Eylül University, 35390, İzmir, Turkey

    Ercan Avşar

  2. Civil Engineering Department, Dokuz Eylül University, 35390, İzmir, Turkey

    Yağmur Özinal Avşar

Authors
  1. Ercan Avşar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yağmur Özinal Avşar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ercan Avşar.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Avşar, E., Avşar, Y.Ö. Moving vehicle detection and tracking at roundabouts using deep learning with trajectory union. Multimed Tools Appl 81, 6653–6680 (2022). https://doi.org/10.1007/s11042-021-11804-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11804-0

Keywords

Search

Navigation