Skip to main content
SpringerLink
Log in

EnsembleNet: a hybrid approach for vehicle detection and estimation of traffic density based on faster R-CNN and YOLO models

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Due to static traffic management regulations on roadways, traffic flow may become congested as it has been growing on roads. Estimating traffic density impacts intelligent transportation systems as it helps build efficient traffic management. Vehicle recognition and counting are two main steps to estimate traffic density. Vehicle identification systems can use motion, handcrafted features, and convolutional neural network (CNN)-based methods. The utilization of deep learning technologies is increasing daily with the popularity of CNN. Different classification and detection models have been developed using transfer learning. In this study, data are collected from several open-source libraries, including MB7500, KITTI, and FLIR. Image annotation has been done to classify vehicles into different categories. Various data augmentation methods are implemented to increase the dataset size and to reduce class imbalance problem. Image quality has been enhanced by performing the sharpening process. Then, a hybrid model of Faster R-CNN and YOLO using majority voting classifier has been trained on processed data. The proposed model’s findings have been compared with its base estimators on the collected datasets. The proposed model has demonstrated detection accuracy of up to 98%, whereas YOLO and Faster R-CNN provide 95.8 and 97.5%, respectively. Additionally, compared to YOLO and Faster R-CNN, experimental results show that the proposed model performs better at estimating traffic density. Hence, the proposed approach can effectively enhance road traffic management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

Data Availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: ECCV

  2. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR

  3. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR

  4. Mittal U, Srivastava S, Chawla P (2019) Review of different techniques for object detection using deep learning. In: Proceedings of the third international conference on advanced informatics for computing research - ICAICR ’19, pp. 1–8, https://doi.org/10.1145/3339311.3339357

  5. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: CVPR

  6. Lowe DG (1999) Object recognition from local scale-invariant features. In: ICCV

  7. Lienhart R, Maydt J (2002) An extended set of haar-like features for rapid object detection. In: International conference on image processing

  8. Fidler S, Mottaghi R, Yuille A, Urtasun R (2013) Bottom-up segmentation for top-down detection. In: CVPR

  9. Kleban J, Xie X, Ma W-Y (2008) Spatial pyramid mining for logo detection in natural scenes. In: Multimedia and Expo, 2008 IEEE international conference on

  10. Girshick R (2015) Fast r-cnn, In: ICCV

  11. Uijlings JR, Van De Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. In: IJCV

  12. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: NeurIPS

  13. Dai J, Li Y, He K, Sun J (2016) R-fcn: Object detection via region-based fully convolutional networks. In: NeurIPS

  14. Lin T-Y, Doll´ar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: CVPR

  15. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: CVPR

  16. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg A C (2016) SSD: Single shot multibox detector, In: ECCV

  17. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: CVPR

  18. Lin T-Y, Goyal P, Girshick R, He K, Doll´ar P (2017) Focal loss for dense object detection. In: ICCV

  19. Law H, Deng J (2018) Cornernet: Detecting objects as paired keypoints. In: ECCV

  20. Zhang Z, Xu C, Feng W (2017) Road vehicle detection and classification based on Deep Neural Network. In: Proceedings of the 7th IEEE international conference on software engineering and service science, Aug. 26–28, IEEE Xplore Press: Beijing, China, pp. 675–678. https://doi.org/10.1109/ICSESS.2016.7883158

  21. Harsha SS, Anne KR (2016) Gaussian mixture model and deep neural Network based Vehicle Detection and Classification. (IJACSA) Int J Adv Comput Sci Appli, Vol. 7(9): pp. 17–25

  22. Zhou Y, Nejati H, Do T T, Cheung NM, Cheah L (2016) Image-based Vehicle Analysis using Deep Neural Network: A Systematic Study. In: IEEE international conference on digital signal processing (DSP). Beijing, China: IEEE

  23. Gao Y, Lee HJ (2015) Moving car detection and model recognition based on deep learning. Adv Sci Technol Lett, pp. 57–61

  24. Chan YM, Huang SS, Fu LC, Hsiao PY, Lo MF (2012) Vehicle detection and tracking under various lighting. IET Intell Trans Sys 6:1–8

    Article  Google Scholar 

  25. Berg A, Ahlberg J, Felsberg M (2015) A thermal object tracking benchmark. In: 12th IEEE international conference on advanced video and signal based surveillance (AVSS) (pp. 1–7). Karlsruhe, Germany: IEEE

  26. Mittal U, Srivastava S, Chawla P (2019) Object detection and classification from thermal images using region based convolutional neural network. J Comput Sci 15(7):961–971. https://doi.org/10.3844/jcssp.2019.961.971

    Article  Google Scholar 

  27. Rodin CD, Lima LN, Andrade FA, Haddad DB, Johansen TA, Storvold R (2018) Object classification in thermal images using convolutional neural networks for search and rescue missions with unmanned aerial systems. Int Joint Conf Neural Netw (IJCNN) 2018:1–8

    Google Scholar 

  28. Nam Y, Nam Y-C (2018) Vehicle classification based on images from visible light and thermal cameras. EURASIP J Image Video Process. https://doi.org/10.1186/s13640-018-0245-2

    Article  Google Scholar 

  29. Moranduzzo T, Melgani F (2014) Detecting cars in UAV images with a catalog-based approach. IEEE Trans Geosci Remote Sens 52(10):6356–6367

    Article  Google Scholar 

  30. Chen Y-L, Chen T-S, Huang T-W, Yin L-C, Wang S-Y, Chiueh T-C (2013) Intelligent Urban video surveillance system for automatic vehicle detection and tracking in clouds. In: IEEE 27th international conference on advanced information networking and applications (AINA) (pp. 814–821). Barcelona, Spain: IEEE

  31. Tuermer S, Kurz F, Reinartz P, Stilla U (2013) Airborne vehicle detection in dense Urban areas using HoG features and disparity maps. IEEE J Selected Topics Appl Earth Observ Remote Sens 6(6):2327–2337. https://doi.org/10.1109/JSTARS.2013.2242846

    Article  Google Scholar 

  32. Prabha C, Shah I (2016) Study of vehicular traffic using hybrid deep neural network. Int J Innov Res Comput Commun Eng pp. 4334–4338.

  33. Chen Z, Ellis T, Velastin SA (2012). Vehicle detection, tracking and classification in Urban Traffic. In: 15th international IEEE conference on intelligent transportation systems (pp. 951–956). Anchorage, Alaska, USA,: IEEE

  34. He D, Lang C, Feng S, Du X, Zhang C (2015). Vehicle detection and classification based on convolutional neural network. In: Proceedings of the 7th international conference on internet multimedia computing and service, (pp. 1–5). Zhangjiajie, Hunan, China

  35. Chen Z, Ellis T, Velastin SA (2011) Vehicle type categorization: a comparison of classification schemes. In: Proceedings of the 14th international IEEE conference on intelligent transportation systems washington, DC, USA, pp: 74–79. https://doi.org/10.1109/ITSC.2011.6083075

  36. Vijayaraghavan V, Laavanya M (2019) Vehicle classification and detection using deep learning. Int J Eng Adv Technol (IJEAT) 9(15):24–28

    Google Scholar 

  37. Ma R, Zhang Z, Dong Y, Pan Y (2020) deep learning based vehicle detection and classification methodology using strain sensors under bridge deck. Sensors 20(18):5051. https://doi.org/10.3390/s20185051

    Article  Google Scholar 

  38. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: NeurIPS

  39. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: CVPR

  40. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: arXiv preprint arXiv:1409.1556

  41. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: ECCV, Springer

  42. Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: ECCV

  43. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: CVPR, 2017

  44. Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. In: NeurIPS, 2017, pp. 4467–4475

  45. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications, in: arXiv preprint arXiv:1704.04861

  46. Cai Z, Vasconcelos N (2018) Cascade r-cnn: Delving into high quality object detection. In: CVPR

  47. Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019) Centernet: Keypoint triplets for object detection, in: arXiv preprint arXiv:1904.08189

  48. Maithani M (2020). EfficientDet: Guide to state of the art object detection model. Retrieved from https://analyticsindiamag.com/efficientdet/

  49. Kaur R, Talwar M, (2016) Automated vehicle detection and classification with probabilistic neural network. IJARIIT

  50. Bhartee AK, Srivastava KM, Sharma T (2017) Object Identification using thermal image processing. Int. J. Eng. Sci. Computing

  51. Mittal U, Potnuru R, Chawla P (2020) Vehicle detection and classification using improved faster region based convolution neural network. In: 2020 8th International conference on reliability, infocom technologies and optimization (Trends and Future Directions) (ICRITO), 2020, pp. 511–514, https://doi.org/10.1109/ICRITO48877.2020.9197805

  52. Oliveira DC, Wehrmeister MA (2018) Using deep learning and low cost RGB and thermal cameras to detect pedestrians in aerial images captured by multirotor UAV. Sensors (Basel) 18(7):2244. https://doi.org/10.3390/s18072244

    Article  Google Scholar 

  53. Sun W, Zhang G, Zhang X, Zhang X, Ge N (2020) Fine-grained vehicle type classification using lightweight convolutional neural network with feature optimization and joint learning strategy. Multimed Tools Appl 80(20):30803–30816. https://doi.org/10.1007/s11042-020-09171-3

    Article  Google Scholar 

  54. Ranjeeth Kumar C, Anuradha R (2020) RETRACTED ARTICLE: Feature selection and classification methods for vehicle tracking and detection. J Ambient Intell Human Comput 12(3):4269–4279. https://doi.org/10.1007/s12652-020-01824-3

    Article  Google Scholar 

  55. Wang X, Chen X, Wang Y (2020) Small vehicle classification in the wild using generative adversarial network. Neural Comput Appl 33:5369–5379. https://doi.org/10.1007/s00521-020-05331-6

    Article  Google Scholar 

  56. Shvai N, Hasnat A, Meicler A, Nakib A (2020) Accurate classification for automatic vehicle-type recognition based on ensemble classifiers. IEEE Trans Intell Transp Syst 21(3):1288–1297. https://doi.org/10.1109/tits.2019.2906821

    Article  Google Scholar 

  57. Awang S, Azmi NMAN, Rahman MdA (2020) Vehicle type classification using an enhanced sparse-filtered convolutional neural network with layer-skipping strategy. IEEE Access 8:14265–14277. https://doi.org/10.1109/access.2019.2963486

    Article  Google Scholar 

  58. Grents A, Varkentin V, Goryaev N (2020) Determining vehicle speed based on video using convolutional neural network. Transportation Research Procedia 50:192–200. https://doi.org/10.1016/j.trpro.2020.10.024

    Article  Google Scholar 

  59. Zhu J, Li X, Jin P, Xu Q, Sun Z, Song X (2020) MME-YOLO: Multi-sensor multi-level enhanced YOLO for robust vehicle detection in traffic surveillance. Sensors 21(1):27. https://doi.org/10.3390/s21010027

    Article  Google Scholar 

  60. Jagannathan P, Rajkumar S, Frnda J, Divakarachari PS (2021) Moving vehicle detection and classification using gaussian mixture model and ensemble deep learning technique. Wireless Commun Mobile Comput 2021:1–15. https://doi.org/10.1155/2021/5590894

    Article  Google Scholar 

  61. Hu X, Wei Z, Zhou W, (2021) A video streaming vehicle detection algorithm based on YOLOv4. In: 5th advanced information technology, electronic and automation control conference (IAEAC), pp. 2081–2086, https://doi.org/10.1109/IAEAC50856.2021.9390613.

  62. Yang Y et al (2021) A fast and effective video vehicle detection method leveraging feature fusion and proposal temporal link. J Real-Time Image Proc 18(4):1261–1274. https://doi.org/10.1007/s11554-021-01121-y

    Article  Google Scholar 

  63. Sri Jamiya S, Esther Rani P (2021) LittleYOLO-SPP: A delicate real-time vehicle detection algorithm. Optik 225:165818. https://doi.org/10.1016/j.ijleo.2020.165818

    Article  Google Scholar 

  64. Wang X (2022) Vehicle image detection method using deep learning in UAV video. Comput Intell Neurosci 2022:1–10. https://doi.org/10.1155/2022/8202535

    Article  Google Scholar 

Download references

Funding

No funding support for this work.

Author information

Authors and Affiliations

  1. School of Computer Science & Engineering, Lovely Professional University, Phagwara, India

    Usha Mittal

  2. Department of Computer Science & Engineering, National Institute of Technology Warangal, Telangana, India

    Priyanka Chawla

  3. School of Computer Science, University of Petroleum and Energy Studies, Dehradun, India

    Rajeev Tiwari

Authors
  1. Usha Mittal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Priyanka Chawla
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajeev Tiwari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Priyanka Chawla.

Ethics declarations

Conflicts of interest

We declare no any conflict of interest. This manuscript is submitted only in this journal and is not in parallel submitted or in review at any other venue.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mittal, U., Chawla, P. & Tiwari, R. EnsembleNet: a hybrid approach for vehicle detection and estimation of traffic density based on faster R-CNN and YOLO models. Neural Comput & Applic 35, 4755–4774 (2023). https://doi.org/10.1007/s00521-022-07940-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07940-9

Keywords

Search

Navigation