Skip to main content
SpringerLink
Log in

Traffic sign detection based on improved faster R-CNN for autonomous driving

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The timely and accurate identification of traffic signs plays a significant role in realizing the autonomous driving of vehicles. However, the size of traffic signs accounts for a low proportion of the input picture, which increases the difficulty of detection. This paper proposes an improved faster R-CNN traffic sign detection method. ResNet50-D feature extractor, attention-guided context feature pyramid network (ACFPN), and AutoAugment technology are designed for the faster R-CNN model. ResNet50-D is selected as the backbone network to obtain more characteristic information. ACFPN is performed to decrease the loss of contextual information. And data augmentation and transfer learning are adopted to make model training more convenient and time-saving. To prove the availability of the proposed method, we compare it with mainstream approaches (SSD, YOLOv3, RetinaNet, cascade R-CNN, FCOS, and CornerNet-Squeeze) and state-of-the-art methods. Experimental results on the CCTSDB dataset show that the improved faster R-CNN achieves the frames per second of 29.8 and the mean average precision of 99.5%, which is superior to the state-of-the-art methods and more suitable for traffic sign detection. Moreover, the proposed model is extended to the Tsinghua-Tencent 100 K (TT100K) dataset and also achieves a competitive detection result.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Luo H, Yang Y, Tong B, Wu F, Fan B (2018) Traffic sign recognition using a multi-task convolutional neural network. IEEE Trans Intell Transp Syst 19(4):1100–1111. https://doi.org/10.1109/TITS.2017.2714691

    Article  Google Scholar 

  2. Yang Y, Luo H, Xu H, Wu F (2016) Towards real-time traffic sign detection and classification. IEEE Trans Intell Transp Syst 17(7):2022–2031. https://doi.org/10.1109/TITS.2015.2482461

    Article  Google Scholar 

  3. Gudigar A, Chokkadi S, Raghavendra U, Acharya UR (2016) Multiple thresholding and subspace based approach for detection and recognition of traffic sign. Multimed Tools Appl 76(5):6973–6991. https://doi.org/10.1007/s11042-016-3321-6

    Article  Google Scholar 

  4. Xu X, Jin J, Zhang S, Zhang L, Pu S, Chen Z (2019) Smart data driven traffic sign detection method based on adaptive color threshold and shape symmetry. Futur Gener Comput Syst 94:381–391. https://doi.org/10.1016/j.future.2018.11.027

    Article  Google Scholar 

  5. Liu W et al (2016) SSD: single shot multibox detector. In: 14th European Conference on Computer Vision (ECCV), vol 9905, pp 21–37. https://doi.org/10.1007/978-3-319-46448-0_2

  6. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/tpami.2016.2577031

    Article  Google Scholar 

  7. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 779–788. https://doi.org/10.1109/CVPR.2016.91

  8. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). arxiv:1804.02767

  9. Lin T, Goyal P, Girshick R, He K, Dollár P (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 42(2):318–327. https://doi.org/10.1109/tpami.2018.2858826

    Article  Google Scholar 

  10. Cai Z, Vasconcelos N (2021) Cascade R-CNN: high quality object detection and instance segmentation. IEEE Trans Pattern Anal Mach Intell 43(5):1483–1498. https://doi.org/10.1109/tpami.2019.2956516

    Article  Google Scholar 

  11. Tian Z, Shen C, Chen H, He T (2020) FCOS: a simple and strong anchor-free object detector. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/tpami.2020.3032166

    Article  Google Scholar 

  12. Law H, Teng Y, Russakovsky O, Deng J (2019) CornerNet-Lite: efficient keypoint based object detection. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). arxiv:1904.08900

  13. Yao Z, He D, Chen Y, Liu B, Miao J et al (2020) Inspection of exterior substance on high-speed train bottom based on improved deep learning method. Measurement. https://doi.org/10.1016/j.measurement.2020.108013

    Article  Google Scholar 

  14. He D, Yao Z, Jiang Z, Chen Y, Deng J, Xiang W (2019) Detection of foreign matter on high-speed train underbody based on deep learning. IEEE Access 7:183838–183846. https://doi.org/10.1109/ACCESS.2019.2960439

    Article  Google Scholar 

  15. Zheng X, Zheng S, Kong Y, Chen J (2021) Recent advances in surface defect inspection of industrial products using deep learning techniques. Int J Adv Manuf Technol 113(1):35–58. https://doi.org/10.1007/s00170-021-06592-8

    Article  Google Scholar 

  16. He D, Zou Z, Chen Y, Liu B, Yao X, Shan S (2021) Obstacle detection of rail transit based on deep learning. Measurement. https://doi.org/10.1016/j.measurement.2021.109241

    Article  Google Scholar 

  17. Zhong Y, Zheng Z, Ma A, Lu X, Zhang L (2020) Color: cycling, offline learning, and online representation framework for airport and airplane detection using GF-2 satellite images. IEEE Trans Geosci Remote Sens 58(12):8438–8449. https://doi.org/10.1109/TGRS.2020.2987907

    Article  Google Scholar 

  18. Chen B, Miao X (2020) Distribution line pole detection and counting based on YOLO using UAV inspection line video. J Electr Eng Technol 15(1):441–448. https://doi.org/10.1007/s42835-019-00268-w

    Article  MathSciNet  Google Scholar 

  19. Polat Ö, Güngen C (2021) Classification of brain tumors from MR images using deep transfer learning. J Supercomput 77(7):7236–7252. https://doi.org/10.1007/s11227-020-03572-9

    Article  Google Scholar 

  20. Yoneda K, Kuramoto A, Suganuma N, Asaka T, Aldibaja M, Yanase R (2020) Robust traffic light and arrow detection using digital map with spatial prior information for automated driving. Sensors. https://doi.org/10.3390/s20041181

    Article  Google Scholar 

  21. Pon A, Adrienko O, Harakeh A, Waslander SL (2018) A hierarchical deep architecture and mini-batch selection method for joint traffic sign and light detection. In: 2018 15th Conference on Computer and Robot Vision (CRV). https://doi.org/10.1109/CRV.2018.00024

  22. Wang H, Li Y, Wang S (2020) Fast pedestrian detection with attention-enhanced multi-scale rpn and soft-cascaded decision trees. IEEE Trans Intell Transp Syst 21(12):5086–5093. https://doi.org/10.1109/TITS.2019.2948398

    Article  Google Scholar 

  23. Cai Y, Liu Z, Sun X, Chen L, Wang H, Zhang Y (2017) Vehicle detection based on deep dual-vehicle deformable part models. J Sens 2017:1–10. https://doi.org/10.1155/2017/5627281

    Article  Google Scholar 

  24. Hu X et al (2019) SINet: a scale-insensitive convolutional neural network for fast vehicle detection. IEEE Trans Intell Transp Syst 20(3):1010–1019. https://doi.org/10.1109/TITS.2018.2838132

    Article  Google Scholar 

  25. Tang J, Li S, Liu P (2021) A review of lane detection methods based on deep learning. Pattern Recogn. https://doi.org/10.1016/j.patcog.2020.107623

    Article  Google Scholar 

  26. Kausar A, Jamil A, Nida N, Yousaf MH (2020) Two-wheeled vehicle detection using two-step and single-step deep learning models. Arab J Sci Eng 45(12):10755–10773. https://doi.org/10.1007/s13369-020-04837-4

    Article  Google Scholar 

  27. Lee HS, Kim K (2018) Simultaneous traffic sign detection and boundary estimation using convolutional neural network. IEEE Trans Intell Transp Syst 19(5):1652–1663. https://doi.org/10.1109/tits.2018.2801560

    Article  Google Scholar 

  28. Yang T, Long X, Sangaiah AK, Zheng Z, Tong C (2018) Deep detection network for real-life traffic sign in vehicular networks. Comput Netw 136:95–104. https://doi.org/10.1016/j.comnet.2018.02.026

    Article  Google Scholar 

  29. Arcos-García Á, Álvarez-García JA, Soria-Morillo LM (2018) Evaluation of deep neural networks for traffic sign detection systems. Neurocomputing 316:332–344. https://doi.org/10.1016/j.neucom.2018.08.009

    Article  Google Scholar 

  30. Song S, Que Z, Hou J, Du S, Song Y (2019) An efficient convolutional neural network for small traffic sign detection. J Syst Archit 97:269–277. https://doi.org/10.1016/j.sysarc.2019.01.012

    Article  Google Scholar 

  31. Li J, Wang Z (2019) Real-time traffic sign recognition based on efficient CNNs in the Wild. IEEE Trans Intell Transp Syst 20(3):975–984. https://doi.org/10.1109/TITS.2018.2843815

    Article  Google Scholar 

  32. Tian Y, Gelernter J, Wang X, Li J, Yu Y (2019) Traffic sign detection using a multi-scale recurrent attention network. IEEE Trans Intell Transp Syst 20(12):4466–4475. https://doi.org/10.1109/tits.2018.2886283

    Article  Google Scholar 

  33. Zhang J, Wang W, Lu C, Wang J, Sangaiah AK (2020) Lightweight deep network for traffic sign classification. Ann Telecommun 75(7–8):369–379. https://doi.org/10.1007/s12243-019-00731-9

    Article  Google Scholar 

  34. Wan J et al (2020) An efficient small traffic sign detection method based on YOLOv3. J Signal Process Syst Signal Image Video Technol. https://doi.org/10.1007/s11265-020-01614-2

    Article  Google Scholar 

  35. Tabernik D, Skocaj D (2020) Deep learning for large-scale traffic-sign detection and recognition. IEEE Trans Intell Transp Syst 21(4):1427–1440. https://doi.org/10.1109/TITS.2019.2913588

    Article  Google Scholar 

  36. Gamez Serna C, Ruichek Y (2020) Traffic signs detection and classification for European urban environments. IEEE Trans Intell Transp Syst 21(10):4388–4399. https://doi.org/10.1109/TITS.2019.2941081

    Article  Google Scholar 

  37. Haque WA, Arefin S, Shihavuddin ASM, Hasan MA (2021) DeepThin: a novel lightweight CNN architecture for traffic sign recognition without GPU requirements. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2020.114481

    Article  Google Scholar 

  38. Liu Z, Qi M, Shen C, Fang Y, Zhao X (2021) Cascade saccade machine learning network with hierarchical classes for traffic sign detection. Sustain Cities Soc. https://doi.org/10.1016/j.scs.2020.102700

    Article  Google Scholar 

  39. Tang Q, Cao G, Jo KH (2021) Integrated feature pyramid network with feature aggregation for traffic sign detection. IEEE Access 9:117784–117794. https://doi.org/10.1109/access.2021.3106350

    Article  Google Scholar 

  40. Shen L, You L, Peng B, Zhang C (2021) Group multi-scale attention pyramid network for traffic sign detection. Neurocomputing 452:1–14. https://doi.org/10.1016/j.neucom.2021.04.083

    Article  Google Scholar 

  41. Liu Y, Peng J, Xue J-H, Chen Y, Fu Z-H (2021) TSingNet: Scale-aware and context-rich feature learning for traffic sign detection and recognition in the wild. Neurocomputing 447:10–22. https://doi.org/10.1016/j.neucom.2021.03.049

    Article  Google Scholar 

  42. Cao J, Chen Q, Guo J, Shi R (2020) Attention-guided context feature pyramid network for object detection. arXiv:2005.11475

  43. Zhang J, Huang M, Jin X, Li X (2017) A real-time Chinese traffic sign detection algorithm based on modified YOLOv2. Algorithms. https://doi.org/10.3390/a10040127

    Article  MATH  Google Scholar 

  44. Zhu Z, Liang D, Zhang S, Huang X, Li B, Hu S (2016) Traffic-sign detection and classification in the wild. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2016.232

  45. Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV (2019) AutoAugment: learning augmentation strategies from data. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 113–123. https://doi.org/10.1109/CVPR.2019.00020

  46. Li C, Chen Z, Wu QMJ, Liu C (2019) Deep saliency with channel-wise hierarchical feature responses for traffic sign detection. IEEE Trans Intell Transp Syst 20(7):2497–2509. https://doi.org/10.1109/tits.2018.2867183

    Article  Google Scholar 

  47. Zhang J, Xie Z, Sun J, Zou X, Wang J (2020) A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754. https://doi.org/10.1109/access.2020.2972338

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China [Grant No. 2018YFC0808004], Fundamental Research Funds for the Central Universities [Project No. 2018CDXYJX0019], and project supported by Graduate Scientific Research and Innovation Foundation of Chongqing, China [Grant No. CYB19009].

Funding

Funding was provided by National Key Research and Development Program of China (2018YFC0808004), Fundamental Research Funds for the Central Universities (2018CDXYJX0019), and Graduate Scientific Research and Innovation Foundation of Chongqing, China (CYB19009).

Author information

Authors and Affiliations

  1. State Key Laboratory of Mechanical Transmissions, Chongqing University, No. 174 Shazheng Street, Shapingba District, Chongqing City, 400044, China

    Xiaomei Li, Zhijiang Xie, Xiong Deng, Yanxue Wu & Yangjun Pi

  2. College of Mechanical and Vehicle Engineering, Chongqing University, No. 174 Shazheng Street, Shapingba District, Chongqing City, 400044, China

    Xiaomei Li, Zhijiang Xie, Xiong Deng, Yanxue Wu & Yangjun Pi

Authors
  1. Xiaomei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhijiang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiong Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanxue Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yangjun Pi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yangjun Pi.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Xie, Z., Deng, X. et al. Traffic sign detection based on improved faster R-CNN for autonomous driving. J Supercomput 78, 7982–8002 (2022). https://doi.org/10.1007/s11227-021-04230-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-04230-4

Keywords

Search

Navigation