Skip to main content
SpringerLink
Log in

Vehicle detection and traffic density estimation using ensemble of deep learning models

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Traffic density estimation can be used for controlling traffic light signals to provide effective traffic management. It can be done in two steps: vehicle recognition and counting. Deep learning (DL) technologies are being explored more and more as CNN grows in popularity. In this study, initially, data was collected from various open-source libraries that is FLIR, KITTI, and MB7500. Vehicles in the images are labelled in six different classes. To deal with an imbalanced dataset, data augmentation techniques were applied. Then, a model based on an ensemble of the faster region-based convolutional neural networks (Faster R-CNN) and Single-shot detector (SSD) were trained on finally processed datasets. The results of the proposed model were compared with base estimators of the FLIR dataset (Thermal and RGB images separately), MB7500, and KITTI dataset. Experimental results depict that the highest mAP obtained was 94% by the proposed Ensemble on FLIR thermal dataset which was 34% better than SSD and 6% from the Faster R-CNN model. Overall, the proposed ensemble achieves better and more promising results as compared to base estimators. Experimental results also show that detection with thermal images was better than visible images. In addition, three algorithms were compared for estimated density and the proposed model shows significant potential for traffic density estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. AlZu’bi S, Jararweh Y (2020) Data fusion in autonomous vehicles research, literature tracing from imaginary idea to smart surrounding community. In: Fifth International Conference on Fog and Mobile Edge Computing (FMEC), pp 306–311. https://doi.org/10.1109/FMEC49853.2020.9144916

  2. AlZu’bi S, Al-Qatawneh S, Alsmirat M (2018) Transferable HMM trained matrices for accelerating statistical segmentation time. In: Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS), pp 172–176. https://doi.org/10.1109/SNAMS.2018.8554487

  3. AlZu’bi S, Hawashin B, Mujahed M, Jararweh Y, Gupta BB (2019) An efficient employment of internet of multimedia things in smart and future agriculture. Multimed Tools Appl 78(20):29581–29605. https://doi.org/10.1007/s11042-019-7367-0

    Article  Google Scholar 

  4. AlZu’bi S, Shehab M, Al-Ayyoub M, Jararweh Y, Gupta B (2020) Parallel implementation for 3D medical volume fuzzy segmentation. Pattern Recogn Lett 130:312–318. https://doi.org/10.1016/j.patrec.2018.07.026

    Article  Google Scholar 

  5. Al-Zu’bi S, Hawashin B, Mughaid A, Baker T (2020) Efficient 3D medical image segmentation algorithm over a secured multimedia network. Multimed Tools Appl 80(11):16887–16905. https://doi.org/10.1007/s11042-020-09160-6

    Article  Google Scholar 

  6. Aqel D, Al-Zubi S, Mughaid A, Jararweh Y (2021) Extreme learning machine for plant diseases classification: a sustainable approach for smart agriculture. Clust Comput 25(3:2007–2020. https://doi.org/10.1007/s10586-021-03397-y

  7. Arrospide J, Salgado L (2013) Log-Gabor filters for image-based vehicle verification. IEEE Trans Image Process 22(6):2286–2295. https://doi.org/10.1109/tip.2013.2249080

    Article  MathSciNet  MATH  Google Scholar 

  8. Azimi SM (2018) ShuffleDet: real-time vehicle detection network in on-board embedded UAV imagery. In: European Conference on Computer Vision ECCV. Computer vision – ECCV 2018 workshops Lect Notes Comput Sci 2019:88–99. https://doi.org/10.1007/978-3-030-11012-3_7

  9. Biswas D, Su H, Wang C, Stevanovic A, Wang W (2019) An automatic traffic density estimation using Single Shot Detection (SSD) and MobileNet-SSD. Phys Chem Earth, Parts 110:176–184. https://doi.org/10.1016/J.PCE.2018.12.001

    Article  Google Scholar 

  10. Bodla N, Singh B, Chellappa R, Davis LS (2017) Soft-nms – improving object detection with one line of code. In: Proceedings of the IEEE international conference on computer vision, pp 5561–5569

  11. Cai Z, Vasconcelos N (2018) Cascade r-cnn: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154–6162

  12. Chan YM, Huang SS, Fu LC, Hsiao PY, Lo MF (2012) Vehicle detection and tracking under various lighting. IET Intell Transp Syst 6:1–8

    Article  Google Scholar 

  13. Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. Adv Neural Inf Process Syst 30

  14. Chowdhury PN, Shivakumara P, Pal U, Lu T, Blumenstein M (2020) A new augmentation-based method for text detection in night and day license plate images. Multimed Tools Appl 79(43–44):33303–33330. https://doi.org/10.1007/s11042-020-09681-0

    Article  Google Scholar 

  15. Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. Adv Neural Inf Process Syst 29

  16. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05). Ieee, vol 1, pp 886–893

  17. Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019) Centernet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 6569–6578

  18. Fidler S, Mottaghi R, Yuille A, Urtasun R (2013) Bottom-up segmentation for top-down detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3294–3301

  19. Gao Y, Lee HJ (2015) Moving car detection and model recognition based on deep learning. Adv Sci Technol Lett 90 57–61

  20. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

  21. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  22. Harsha SS, Anne KR (2016) Gaussian mixture model and deep neural network based vehicle detection and classification. (IJACSA). Int J Adv Comput Sci Appl 7(9):17–25

    Google Scholar 

  23. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

  24. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  25. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer, Cham, pp 630–645

  26. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

  27. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  28. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–456

  29. Jagannathan P, Rajkumar S, Frnda J, Divakarachari PB, Subramani P (2021) Moving vehicle detection and classification using Gaussian mixture model and ensemble deep learning technique. Wirel Commun Mob Comput 2021:1–15. https://doi.org/10.1155/2021/5590894

    Article  Google Scholar 

  30. Jamiya SS, Esther Rani P (2021) LittleYOLO-SPP: A delicate real-time vehicle detection algorithm. Optik 225:165818. https://doi.org/10.1016/j.ijleo.2020.165818

    Article  Google Scholar 

  31. John V, Mita S, Zheng L, Qi B (2015) Pedestrian detection in thermal images using adaptive fuzzy C-means clustering and convolutional neural networks. In: 2015 14th IAPR International Conference on Machine Vision Applications (MVA). IEEE, pp 246–249

  32. Kembhavi A, Harwood D, Davis LS (Jun. 2011) Vehicle detection using partial least squares. IEEE Trans Pattern Anal Mach Intell 33(6):1250–1265. https://doi.org/10.1109/TPAMI.2010.182

    Article  Google Scholar 

  33. Keserwani P, Dhankhar A, Saini R, Roy PP (2021) Quadbox: quadrilateral bounding box based scene text detection using vector regression. IEEE Access 9:36802–36818. https://doi.org/10.1109/ACCESS.2021.3063030

    Article  Google Scholar 

  34. Kleban J, Xie X, Ma W-Y (2008) Spatial pyramid mining for logo detection in natural scenes. In: 2008 IEEE International Conference on Multimedia and Expo. IEEE, pp 1077–1080

  35. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25

  36. Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV), pp 734–750

  37. Lee JT, Chung Y (2017) Deep learning-based vehicle classification using an ensemble of local expert and global networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 47–52. https://doi.org/10.1109/CVPRW.2017.127

  38. Lienhart R, Maydt J (2002) An extended set of haar-like features for rapid object detection. In: Proceedings international conference on image processing. IEEE, vol 1, pp I–I

  39. Lin T-Y, Dollàr P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection, in: CVPR

  40. Lin T-Y, Goyal P, Girshick R, He K, Dollàr P (2017) Focal loss for dense object detection, in: ICCV

  41. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector, in: ECCV

  42. Liu W, Luo Z, Li S (2018) Improving deep ensemble vehicle classification by using selected adversarial samples. Knowl-Based Syst 160:167–175. https://doi.org/10.1016/j.knosys.2018.06.035

    Article  Google Scholar 

  43. Lowe DG (1999) Object recognition from local scale-invariant features, in: ICCV

  44. Nam Y, Nam Y-C (2018) Vehicle classification based on images from visible light and thermal cameras. EURASIP J Image Vid Process 2018. https://doi.org/10.1186/s13640-018-0245-2

  45. Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: European conference on computer vision. Springer, Cham, pp 483–499

  46. Oliveira DC, Wehrmeister MA (2018) Using deep learning and low cost RGB and thermal cameras to detect pedestrians in aerial images captured by multirotor UAV. Sensors (Basel) 18(7):2244. https://doi.org/10.3390/s18072244

    Article  Google Scholar 

  47. Ozkurt C, Camci F (Dec. 2009) Automatic traffic density estimation and vehicle classification for traffic surveillance systems using neural networks. Math Comput Appl 14(3):187–196. https://doi.org/10.3390/mca14030187

    Article  Google Scholar 

  48. Ranjeeth Kumar C, Anuradha R (2020) Feature selection and classification methods for vehicle tracking and detection. J Ambient Intell Humaniz Comput 12(3):4269–4279. https://doi.org/10.1007/s12652-020-01824-3

    Article  Google Scholar 

  49. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger, in: CVPR

  50. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection, in: CVPR

  51. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks, in: NeurIPS

  52. Rezaee H, Aghagolzadeh A, Seyedarabi MH, Zu’bi SA (2011) “Tracking and occlusion handling in multi-sensor networks by particle filter,” in IEEE GCC Conference and Exhibition (GCC), pp. 397–400, https://doi.org/10.1109/IEEEGCC.2011.5752541.

  53. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, in: arXiv preprint arXiv:1409.1556

  54. Suryanto, Kim D-H, Kim H-K, Ko S-J (2011) Spatial color histogram based center voting method for subsequent object tracking and segmentation. Image Vis Comput 29(12):850–860. https://doi.org/10.1016/j.imavis.2011.09.008

    Article  Google Scholar 

  55. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions, in: CVPR

  56. Uijlings JR, Van De Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. IJCV 104:154–171

    Article  Google Scholar 

  57. Wu X, Sahoo D, Hoi SCH (2019) Recent advances in deep learning for object detection. Elsevier

  58. Xie S, Girshick R, Dollàr P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks, in: CVPR

  59. Yang Y, Song H, Sun S, Zhang W, Chen Y, Rakal L, Fang Y (2021) A fast and effective video vehicle detection method leveraging feature fusion and proposal temporal link. J Real-Time Image Proc 18(4):1261–1274. https://doi.org/10.1007/s11554-021-01121-y

    Article  Google Scholar 

  60. Zhang B (2013) Reliable classification of vehicle types based on Cascade classifier ensembles. IEEE Trans Intell Transp Syst 14(1):322–332. https://doi.org/10.1109/tits.2012.2213814

    Article  Google Scholar 

  61. Zhang S, Wen L, Bian X, Lei Z, Li SZ (2018) Single-shot refinement neural network for object detection, in: CVPR

  62. Zhou Y, Nejati H, Do TT, Cheung NM, Cheah L (2016) Image-based vehicle analysis using deep neural network: a systematic study. IEEE international conference on digital signal processing (DSP). IEEE, Beijing

    Google Scholar 

  63. Zhou X, Wang D, Krähenbühl P (2019) Objects as points, in: arXiv preprint arXiv:1904.07850

  64. Zhu J, Li X, Jin P, Xu Q, Sun Z, Song X (2020) MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance. Sensors 21(1):27. https://doi.org/10.3390/s21010027

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Lovely Professional University, Phagwara, Punjab, India

    Usha Mittal & Priyanka Chawla

Authors
  1. Usha Mittal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Priyanka Chawla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Priyanka Chawla.

Ethics declarations

Conflict of interest

There is no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mittal, U., Chawla, P. Vehicle detection and traffic density estimation using ensemble of deep learning models. Multimed Tools Appl 82, 10397–10419 (2023). https://doi.org/10.1007/s11042-022-13659-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13659-5

Keywords

Search

Navigation