Skip to main content
Springer Nature Link
Log in

Traffic sign detection based on multi-scale feature extraction and cascade feature fusion

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Existing algorithms have difficulty in solving the two tasks of localization and classification simultaneously when performing traffic sign detection on realistic images of complex traffic scenes. In order to solve the above problems, a new road traffic sign dataset is created, and based on the YOLOv4 algorithm, for the complexity of realistic traffic scene images and the large variation in the size of traffic signs in the images, the multi-scale feature extraction module, cascade feature fusion module and attention mechanism module are designed to improve the algorithm’s ability to locate and classify traffic signs simultaneously. Experimental results on the newly created dataset show that the improved algorithm achieves a mean average precision of 84.44%, which is higher than several major CNN-based object detection algorithms for the same type of task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

All data generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Saponara S (2013) Real-time color/shape-based traffic signs acquisition and recognition system. In: Real-time image and video processing 2013, Burlingame, CA, USA, Feb 6-7, vol 8656, p 86560

  2. Khan JF, Bhuiyan SMA, Adhami RR (2011) Image segmentation and shape analysis for road-sign detection. IEEE Trans Intell Transp Syst 12(1):83–96

    Article  Google Scholar 

  3. Liang M, Yuan M, Hu X, Li J, Liu H (2013) Traffic sign detection by ROI extraction and histogram features-based recognition. In: The 2013 International Joint Conference on Neural Networks, IJCNN, 2013 Dallas, TX, USA, Aug 4-9, pp 1–8

  4. Fleyeh H, Biswas R, Davami E (2013) Traffic sign detection based on adaboost color segmentation and svm classification. In: Eurocon, pp 2005–2010

  5. Maldonado-Bascon S, Lafuente-Arroyo S, Gil-Jimenez P, Gomez-Moreno H, Lopez-Ferreras F (2007) Road-sign detection and recognition based on support vector machines. IEEE Trans Intell Transp Syst 8(2):264–278

    Article  MATH  Google Scholar 

  6. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp 580–587

  7. Girshick RB (2015) Fast R-CNN. In: 2015 IEEE international conference on computer vision, ICCV 2015, Santiago, Chile, Dec 7-13, pp 1440–1448

  8. Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  9. Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C, Berg AC (2016) SSD: single shot multibox detector. In: Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I, vol 9905, pp 21–37

  10. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 779–788

  11. Qiao K, Gu H, Liu J, Liu P (2017) Optimization of traffic sign detection and classification based on faster r-cnn. In: 2017 International Conference on Computer Technology, Electronics and Communication (ICCTEC), pp 608–611

  12. Li J, Wang Z (2019) Real-time traffic sign recognition based on efficient cnns in the wild. IEEE Trans Intell Transp Syst 20(3):975–984

    Article  Google Scholar 

  13. Rajendran SP, Shine L, Pradeep R, Vijayaraghavan S (2019) Real-time traffic sign recognition using yolov3 based detector. In: 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), pp 1–7

  14. Everingham M, Gool LV, Williams CKI, Winn JM, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  15. Lin T, Maire M, Belongie SJ, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. In: Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, Proceedings, Part V, vol 8693, pp 740–755

  16. Liu Z, Du J, Tian F, Wen J (2019) Mr-cnn: a multi-scale region-based convolutional neural network for small traffic sign recognition. IEEE Access 7:57120–57128

    Article  Google Scholar 

  17. Tabernik D, Skočaj D (2020) Deep learning for large-scale traffic-sign detection and recognition. IEEE Trans Intell Transp Syst 21(4):1427–1440

    Article  Google Scholar 

  18. Lee HS, Kim K (2018) Simultaneous traffic sign detection and boundary estimation using convolutional neural network. IEEE Trans Intell Transp Syst 19(5):1652–1663

    Article  Google Scholar 

  19. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Computer Science

  20. Szegedy C, Liu W, Jia Y, Sermanet P, Reed SE, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, pp 1–9

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, pp 770–778

  22. Stallkamp J, Schlipsing M, Salmen J, Igel C (2011) The german traffic sign recognition benchmark: a multi-class classification competition. In: International Joint Conference on Neural Networks

  23. Stallkamp J, Schlipsing M, Salmen J et al (2012) Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw 32:323–332

    Article  Google Scholar 

  24. Uijlings JRR, Sande KEAVD, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171

    Article  Google Scholar 

  25. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. CoRR: https://arxiv.org/abs/1804.02767

  26. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  27. Lin T, Dollár P, Girshick RB, He K, Hariharan B, Belongie SJ (2017) Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, pp 936–944

  28. Bochkovskiy A, Wang C, Liao HM (2020) Yolov4: optimal speed and accuracy of object detection. CoRR https://arxiv.org/abs/2004.10934

  29. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, pp 8759–8768

  30. Zhu Z, Liang D, Zhang S, Huang X, Li B, Hu S (2016) Traffic-sign detection and classification in the wild. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, pp 2110–2118

  31. Zhang J, Huang M, Jin X, Li X (2017) A real-time chinese traffic sign detection algorithm based on modified yolov2. Algorithms 10(4):127

    Article  MathSciNet  MATH  Google Scholar 

  32. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, Feb 4-9, San Francisco, California, USA, pp 4278–4284

  33. Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023

    Article  Google Scholar 

  34. Misra D (2019) Mish: a self regularized non-monotonic neural activation function. CoRR https://arxiv.org/abs/1908.08681

  35. Liu F, Qian Y, Li H, Wang Y, Zhang H (2021) Caffnet: channel attention and feature fusion network for multi-target traffic sign detection. Int J Pattern Recognit Artif Intell 35(7):2152008–1215200820

    Article  Google Scholar 

  36. Ren K, Huang L, Fan C, Han H, Deng H (2021) Real-time traffic sign detection network using ds-detnet and lite fusion FPN. J Real Time Image Process 18(6):2181–2191

    Article  Google Scholar 

  37. Serna CG, Ruichek Y (2020) Traffic signs detection and classification for european urban environments. IEEE Trans Intell Transp Syst 21(10):4388–4399

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the Anhui Provincial Key R &D Program of China under Grant 202004a05020040, in part by the Intelligent Network and New Energy Vehicle Special Project of Intelligent Manufacturing Institute of HFUT under Grant IMIWL2019003, in part by the National Key Research and Development Program of China under Grant 2018YFC0604404, and in part by the Fundamental Research Funds for the Central Universities under Grant PA2021GDGP0061.

Author information

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, 230009, China

    Yongliang Zhang, Yang Lu, Wuqiang Zhu, Xing Wei & Zhen Wei

  2. Anhui Mine IOT and Security Monitoring Technology Key Laboratory, Hefei, 230088, China

    Yang Lu & Zhen Wei

  3. Engineering Research Center of Safety Critical Industrial Measurement and Control Technology, Ministry of Education, Hefei University of Technology, Hefei, 230009, China

    Yang Lu, Xing Wei & Zhen Wei

  4. Intelligent Manufacturing Institute of HeFei University of Technology, Hefei, 230009, China

    Xing Wei

Authors
  1. Yongliang Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yang Lu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wuqiang Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xing Wei
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Zhen Wei
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Zhen Wei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Lu, Y., Zhu, W. et al. Traffic sign detection based on multi-scale feature extraction and cascade feature fusion. J Supercomput 79, 2137–2152 (2023). https://doi.org/10.1007/s11227-022-04670-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04670-6

Keywords