Skip to main content
SpringerLink
Log in

Real-time traffic sign detection and classification towards real traffic scene

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper we propose a real-time traffic sign recognition algorithm which is robust to the small-sized objects and can identify all traffic sign categories. Specifically, we present a two-level detection framework which consists of the region proposal module(RPM) which is responsible for locating the objects and the classification module(CM) which aims to classify the located objects. In addition, to solve the problem of insufficient samples, we present an effective data augmentation method based on traffic sign logo to generate enough training data. The experiments are conducted in TT100k, and the results show the superiority of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bahlmann C, Zhu Y, Ramesh V, Pellkofer M, Koehler T (2005) A system for traffic sign detection, tracking, and recognition using color, shape, and motion information. IEEE Intell Vehicles Symposium: 255–260

  2. Bell S, Zitnick CL, Bala K, Girshick RB (2016) Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks. Compu Vis Pattern Recogn: 2874–2883

  3. Berkaya SK, Gunduz H, Ozsen O, Akinlar C, Gunal S (2016) On circular traffic sign detection and recognition. Expert Sys Appl 48:67–75

    Article  Google Scholar 

  4. Bloice M, Roth PM, Holzinger A (2019) Biomedical image augmentation using augmentor. Bioinformatics 35(21):4522–4524

    Article  Google Scholar 

  5. Chen W, An J, Li R, Fu L, Xie G, Bhuiyan ZA, Li K (2018) A novel fuzzy deep-learning approach to traffic flow prediction with uncertain spatial–temporal data features. Futur Gener Comput Syst 89:78–88

    Article  Google Scholar 

  6. Chen Y, Yuan J, Li Z, Wu Y, Nouioua M, Xie G (2019) Person re-identification based on re-ranking with expanded k-reciprocal nearest neighbors. J Vis Commun Image Represent 58:486–494

    Article  Google Scholar 

  7. Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. Neural Inform Process Sys: 379–387

  8. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. Comput Vis Pattern Recogn 1:886–893

    Google Scholar 

  9. Devries T, Taylor GW (2017) Improved regularization of convolutional neural networks with cutout. Comput Vis Pattern Recogn

  10. Felzenszwalb PF, Girshick RB, Mcallester DA, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  11. Gidaris S, Komodakis N (2015) Object detection via a multi-region and semantic segmentation-aware cnn model. Int Conf Comput Vis: 1134–1142

  12. Girshick RB (2015) Fast r-cnn. Int Conf Comput Vis: 1440–1448

  13. Girshick RB, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. Comput Vis Pattern Recogn: 580–587

  14. Goebel R, Chander A, Holzinger K, Lecue F, Akata Z, Stumpf S, Kieseberg P, Holzinger A (2018) Explainable ai: the new 42?. In: International cross-domain conference for machine learning and knowledge extraction. Springer, pp 295–303

  15. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37 (9):1904–1916

    Article  Google Scholar 

  16. Holzinger A, Kieseberg P, Weippl E, Tjoa AM (2018) Current advances, trends and challenges of machine learning and knowledge extraction: from machine learning to explainable ai. In: International cross-domain conference for machine learning and knowledge extraction. Springer, pp 1–8

  17. Houben S, Stallkamp J, Salmen J, Schlipsing M, Igel C (2013) Detection of traffic signs in real-world images: The german traffic sign detection benchmark. Int Joint Conf Neural Netw: 1–8

  18. Jin J, Fu K, Zhang C (2014) Traffic sign recognition with hinge loss trained convolutional neural networks. IEEE Trans Intell Transp Syst 15(5):1991–2000

    Article  Google Scholar 

  19. La Escalera AD, Moreno L, Salichs MA, Armingol JM (1997) Road traffic sign detection and classification. IEEE Trans Ind Electron 44(6):848–859

    Article  Google Scholar 

  20. Li J, Liang X, Wei Y, Xu T, Feng J, Yan S (2017) Perceptual generative adversarial networks for small object detection. Comput Vis Pattern Recogn: 1951–1959

  21. Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C, Berg AC (2016) Ssd: single shot multibox detector. European Conf Comput Vis: 21–37

  22. Meng Z, Fan X, Chen X, Chen M, Tong Y (2017) Detecting small signs from large images. Inform Reuse Integrat: 217–224

  23. Nai K, Li Z, Li G, Wang S (2018) Robust object tracking via local sparse appearance model. IEEE Trans Image Process 27(10):4958–4970

    Article  MathSciNet  Google Scholar 

  24. Nai K, Xiao D, Li Z, Jiang S, Gu Y (2019) Multi-pattern correlation tracking. Knowledge Based Systems 104789:181

    Google Scholar 

  25. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. Comput Vis Pattern Recogn: 6517–6525

  26. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. Comput Vis Pattern Recogn

  27. Redmon J, Divvala SK, Girshick RB, Farhadi A (2016) You only look once: unified, real-time object detection. Comput Vis Pattern Recogn: 779–788

  28. Ren S, He K, Girshick RB, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  29. Salti S, Petrelli A, Tombari F, Fioraio N, Stefano LD (2015) Traffic sign detection via interest region extraction. Pattern Recogn 48(4):1039–1049

    Article  Google Scholar 

  30. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, Lecun Y (2014) Overfeat: integrated recognition, localization and detection using convolutional networks. Int Conf Learn Represent

  31. Shen Z, Liu Z, Li J, Jiang Y, Chen Y, Xue X (2017) Dsod: learning deeply supervised object detectors from scratch. Int Conf Comput Vis: 1937–1945

  32. Stallkamp J, Schlipsing M, Salmen J, Igel C (2012) 2012 special issue: man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw 32:323–332

    Article  Google Scholar 

  33. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  34. Yang Y, Luo H, Xu H, Wu F (2016) Towards real-time traffic sign detection and classification. IEEE Trans Intell Transp Syst 17(7):2022–2031

    Article  Google Scholar 

  35. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2017) Random erasing data augmentation. Comput Vis Pattern Recogn

  36. Zhu Y, Zhang C, Zhou D, Wang X, Bai X, Liu W (2016) Traffic sign detection and recognition using fully convolutional network guided proposals. Neurocomputing 214:758–766

    Article  Google Scholar 

  37. Zhu Z, Liang D, Zhang S, Huang X, Li B, Hu S (2016) Traffic-sign detection and classification in the wild. Comput Vis Pattern Recogn: 2110–2118

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (No.61672215), National Key R&D Program of China (No. 2018YFB1308604) and Hunan Science and Technology Innovation Project (No. 2017XK2102).

Author information

Authors and Affiliations

  1. College of Computer Science and Electronic Engineering Hunan University, Changsha, China

    Yiqiang Wu, Zhiyong Li, Ying Chen, Ke Nai & Jin Yuan

  2. Key Laboratory for Embedded and Network Computing of Hunan Province, Changsha, China

    Yiqiang Wu, Zhiyong Li, Ying Chen, Ke Nai & Jin Yuan

Authors
  1. Yiqiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiyong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ke Nai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jin Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiyong Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Li, Z., Chen, Y. et al. Real-time traffic sign detection and classification towards real traffic scene. Multimed Tools Appl 79, 18201–18219 (2020). https://doi.org/10.1007/s11042-020-08722-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08722-y

Keywords

Search

Navigation