Skip to main content
SpringerLink
Log in

MTSDet: multi-scale traffic sign detection with attention and path aggregation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

To solve the problem that existing traffic signs are not easily detected leading to low detection performance due to their small sizes and external factors such as weather conditions, this paper proposes a traffic sign detection method, MTSDet (Multi-scale Traffic Sign Detection with attention and path aggregation), which focuses on the multi-scale detection problem and effectively improves the detection performance. First, the method efficiently extracts semantic features by introducing the Attention Mechanism Network(AMNet), and then feeds the multi-scale semantic features into Path Aggregation Feature Pyramid Network(PAFPN) for multi-scale feature fusion to obtain multi-scale advanced semantic features. Finally, the multi-scale advanced semantic feature map is deformable interest pooled to effectively enhance the multi-scale object detection modeling capability. In this paper, the above method is validated by two classical datasets, German traffic sign detection dataset and Chinese traffic sign detection dataset, which achieve 92.9% and 94.3% mAP, respectively, and have obvious detection accuracy improvement when compared with other classical advanced algorithms, effectively proving the superiority and generalization of the algorithm in this paper. Code is available at https://github.com/why529913/MTSDet

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Malik, Khurshid J, Ahmad SN (2007) Road sign detection and recognition using colour segmentation. In: Shape analysis and template matching, 2007 international conference on machine learning and cybernetics, pp 3556–3560, DOI https://doi.org/10.1109/ICMLC.2007.4370763, (to appear in print)

  2. Gao XW, Podladchikova L, Shaposhnikov D, et al. (2006) Recognition of traffic signs based on their colour and shape features extracted using human vision models[J]. J Vis Commun Image Represent 17 (4):675–685

    Article  Google Scholar 

  3. Ellahyani A, Ansari ME, Jaafari IE (2016) Traffic sign detection and recognition based on random forests[J]. Appl Soft Comput 46:805–815

    Article  Google Scholar 

  4. Ren S, He K, Girshick R, et al. (2015) Faster r-cnn: towards real-time object detection with region proposal networks[J]. Adv Neural Inform Process Syst 28:91–99

    Google Scholar 

  5. He K, Gkioxari G, Dollár P et al (2017) Mask r-cnn[C]. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  6. Cai Z, Vasconcelos N (2018) Cascade r-cnn. In: Delving into high quality object detection. Proceedings of the IEEE conference on computer vision and pattern recognition

  7. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement, arXiv:1804.02767

  8. Lin T-Y, et al. (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision

  9. Liu W, et al. (2016) Ssd: single shot multibox detector European conference on computer vision. Springer, Cham

    Google Scholar 

  10. Zhu Z, et al. (2016) Traffic-sign detection and classification in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  11. Liu Z, Li D, Ge SS, Tian F (2020) Small traffic sign detection from large image. Appl Intell 50:1–13

    Article  Google Scholar 

  12. Song S, Que Z, Hou J, Du S, Song Y (2019) An efficient convolutional neural network for small traffic sign detection. J Syst Archit 97:269–277

    Article  Google Scholar 

  13. Liu Z, Du J, Tian F, Wen J (2019) MR-CNN: a multi-scale region-based convolutional neural network for small traffic sign recognition. IEEE Access 7:57120–57128

    Article  Google Scholar 

  14. Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ’squeeze & excitation’in fully convolutional networks. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham

  15. Woo S, et al. (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV)

  16. Gao Z, et al. (2019) Global second-order pooling convolutional networks

  17. Xu H (2020) J Zhang. Adaptive aggregation network for efficient stereo matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Aanet

    Google Scholar 

  18. Zhang J, et al. (2020) A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754

    Article  Google Scholar 

  19. Liu F, Qian Y, Li H, et al. (2021) CAFFNet: channel attention and feature fusion network for multi-target traffic sign detection[J]. International Journal of Pattern Recognition and Artificial Intelligence

  20. Sang J, Wu Z, Guo P, et al. (2018) An improved YOLOv2 for vehicle detection[J]. Sensors 18(12):4272

    Article  Google Scholar 

  21. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. Computer Vision and Pattern Recognition (CVPR). IEEE, Salt Lake City, pp 126–134

    Google Scholar 

  22. Liu W, et al. (2016) SSD: single shot multibox detector. In: European conf. computer vision ECCV. Springer, Cham, pp 21–37

  23. Ren S, et al. (2015) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  24. Lin TY, et al. (2017) Focal loss for dense object detection. In: Proc. IEEE Int. conf. computer vision ICCV, Venice, pp 2980–2988

  25. Zhang J, et al. (2020) A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754

    Article  Google Scholar 

  26. Sun K, Xiao B, Liu D, et al. (2019) Deep high-resolution representation learning for human pose estimation[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). arXiv

  27. Carion N, Massa F, Synnaeve G et al (2020) End-to-end object detection with transformers[M]

  28. Pang J, Chen K, Shi J et al (2020) Libra R-CNN: towards balanced learning for object detection[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  29. Wu Y, Chen Y, Yuan L et al (2020) Rethinking classification and localization for object detection[C]. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE

  30. Zhu X, Cheng D, Zhang Z et al (2019) An empirical study of spatial attention mechanisms in deep networks[C]. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 6688–6697

  31. Fan BB, Yang H (2021) Multi-scale traffic sign detection model with attention[J]. Proceedings of the Institution of Mechanical Engineers Part D: Journal of Automobile Engineering 235(2-3):708–720

    Google Scholar 

  32. Lopez-Montiel M, Orozco-Rosas U, Sánchez-Adame M et al (2021) Evaluation method of deep learning-based embedded systems for traffic sign detection[J]. IEEE Access

  33. Shen L, You L, Peng B, et al. (2021) Group multi-scale attention pyramid network for traffic sign detection[J]. Neurocomputing 452:1–14

    Article  Google Scholar 

  34. Liu Y, Peng J, Xue JH, et al. (2021) TSingNet: scale-aware and context-rich feature learning for traffic sign detection and recognition in the wild[J]. Neurocomputing 447:10–22

    Article  Google Scholar 

  35. Ahmed S, Kamal U, Hasan M K (2021) DFR-TSD: a deep learning based framework for robust traffic sign detection under challenging weather conditions[J]. IEEE Transactions on Intelligent Transportation Systems

  36. Sudha M (2021) Traffic sign detection and recognition using RGSM and a novel feature extraction method[J]. Peer-to-Peer Networking and Applications, 1–12

  37. Zhang J, Xie Z, Sun J, et al. (2020) A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection[J]. IEEE Access 8:29742–29754

    Article  Google Scholar 

  38. Liu Z, Shen C, Fan X, et al. (2020) Scale-aware limited deformable convolutional neural networks for traffic sign detection and classification[J]. IET Intell Transp Syst 14(12):1712–1722

    Article  Google Scholar 

  39. Liu Z, Li D, Ge SS, et al. (2020) Small traffic sign detection from large image[J]. Appl Intell 50(1):1–13

    Article  Google Scholar 

  40. Santos DC, da Silva FA, Pereira DR, et al. (2020) Real-time traffic sign detection and recognition using CNN[J]. IEEE Lat Am Trans 18(03):522–529

    Article  Google Scholar 

  41. Hechri A, Mtibaa A (2020) Two-stage traffic sign detection and recognition based on SVM and convolutional neural networks[J]. IET Image Process 14(5):939–946

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Science Foundation of China under Grant U1803261. Funded by the National Natural Science Foundation of China (61966035). the Funds for Creative Research Groups of Higher Education of Xinjiang Uygur Autonomous Region under Grant No.XJEDU2017T002. Autonomous Region Graduate Innovation Project (XJ2019G072). Tianshan Innovation Team Plan Project of Xinjiang Uygur Autonomous Region under Grant No. 202101642.

Author information

Authors and Affiliations

  1. Software College, Xinjiang University, Urumqi, 830000, China

    Hongyang Wei, Qianqian Zhang, Yurong Qian, Zheng Xu & Jingjing Han

  2. Key Laboratory of Signal Detection and Processing in Xinjiang Uygur Autonomous Region, Urumqi, 830000, China

    Hongyang Wei, Qianqian Zhang, Yurong Qian, Zheng Xu & Jingjing Han

  3. Key Laboratory of Software Engineering, Xinjiang University, Urumqi, 830000, China

    Hongyang Wei, Qianqian Zhang, Yurong Qian, Zheng Xu & Jingjing Han

Authors
  1. Hongyang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qianqian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yurong Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jingjing Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yurong Qian.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, H., Zhang, Q., Qian, Y. et al. MTSDet: multi-scale traffic sign detection with attention and path aggregation. Appl Intell 53, 238–250 (2023). https://doi.org/10.1007/s10489-022-03459-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03459-7

Keywords

Search

Navigation