Skip to main content
SpringerLink
Log in

Traffic sign detection and recognition based on pyramidal convolutional networks

  • Multi-Source Data Understanding (MSDU)
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

With the development of driverless technology, we are in dire need of a method to understand traffic scenes. However, it is still a difficult task to detect traffic signs because of the tiny scale of signs in real-world images. In complex scenarios, some traffic signs could be very elusive due to the awful weather and lighting conditions. To implement a more comprehensive detection and recognition system, we develop a two-stage network. At the region proposal stage, we adopt a deep feature pyramid architecture with lateral connections, which makes the semantic feature of small object more sensitive. At the classification stage, densely connected convolutional network is used to strengthen the feature transmission and multiplexed, which leads to more accurate classification with less number of parameters. We test on GTSDB detection benchmark, as well as the challenging Tsinghua-Tencent 100K benchmark which is pretty difficult for most traditional networks. Experiments show that our proposed method achieves a very great performance and surpasses the other state-of-the-art methods. Implementation source code is available at https://github.com/derderking/Traffic-Sign.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Google Scholar 

  2. Bell S, Zitnick CL, Bala K, Girshick RB (2016) Inside–outside net: detecting objects in context with skip pooling and recurrent neural networks. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp 2874–2883

  3. Cabrera MD, Cerri P, Medici P (2015) Robust real-time traffic light detection and distance estimation using a single camera. Expert Syst Appl 42(8):3911–3923

    Google Scholar 

  4. Carlet J, Abayowa B (2017) Fast vehicle detection in aerial imagery. CoRR arXiv:abs/1709.08666

  5. Chen X, Kundu K, Zhu Y, Ma H, Fidler S, Urtasun R (2018) 3D object proposals using stereo imagery for accurate object class detection. IEEE Trans Pattern Anal Mach Intell 40(5):1259–1272

    Google Scholar 

  6. Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In: Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, 5–10 Dec 2016, Barcelona, Spain, pp 379–387

  7. Ellahyani A, Ansari ME, Jaafari IE (2016) Traffic sign detection and recognition based on random forests. Appl Soft Comput 46:805–815

    Google Scholar 

  8. de la Escalera A, Moreno L, Salichs MA, Armingol JM (1997) Road traffic sign detection and classification. IEEE Trans Ind Electron 44(6):848–859

    Google Scholar 

  9. Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35(8):1915–1929

    Google Scholar 

  10. Girshick RB (2015) Fast R-CNN. In: 2015 IEEE international conference on computer vision, ICCV 2015, Santiago, Chile, 7–13 Dec 2015, pp 1440–1448

  11. Hariharan B, Arbeláez PA, Girshick RB, Malik J (2015) Hypercolumns for object segmentation and fine-grained localization. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, pp 447–456

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp 770–778

  13. Houben S, Stallkamp J, Salmen J, Schlipsing M, Igel C (2013) Detection of traffic signs in real-world images: the German traffic sign detection benchmark. In: The 2013 international joint conference on neural networks, IJCNN 2013, Dallas, TX, USA, 4–9 Aug 2013, pp 1–8

  14. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp 2261–2269

  15. Jin J, Fu K, Zhang C (2014) Traffic sign recognition with hinge loss trained convolutional neural networks. IEEE Trans Intell Transp Syst 15(5):1991–2000

    Google Scholar 

  16. Kuang X, Fu W, Yang L (2018) Real-time detection and recognition of road traffic signs using MSER and random forests. Int J Online Eng 14(03):34–51

    Google Scholar 

  17. Kuo W, Lin C (2007) Two-stage road sign detection and recognition. In: Proceedings of the 2007 IEEE international conference on multimedia and expo, ICME 2007, 2–5 July 2007, Beijing, China, pp 1427–1430

  18. Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, pp 5325–5334

  19. Liang M, Yuan M, Hu X, Li J, Liu H (2013) Traffic sign detection by ROI extraction and histogram features-based recognition. In: The 2013 international joint conference on neural networks, IJCNN 2013, Dallas, TX, USA, 4–9 Aug 2013, pp 1–8

  20. Liang Z, Shao J, Zhang D, Gao L (2018) Small object detection using deep feature pyramid networks. In: Advances in multimedia information processing—PCM 2018—19th Pacific-Rim conference on multimedia, Hefei, China, 21–22 Sept 2018, Proceedings, Part III, pp 554–564

    Google Scholar 

  21. Lin T, Dollár P, Girshick RB, He K, Hariharan B, Belongie SJ (2017) Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp 936–944

  22. Lin T, Goyal P, Girshick RB, He K, Dollár P (2017) Focal loss for dense object detection. In: IEEE international conference on computer vision, ICCV 2017, Venice, Italy, 22–29 Oct 2017, pp 2999–3007

  23. Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C, Berg AC (2016) SSD: single shot multibox detector. In: Computer vision—ECCV 2016—14th European conference, Amsterdam, The Netherlands, 11–14 Oct 2016, Proceedings, Part I, pp 21–37

  24. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, pp 3431–3440

  25. Marcu A (2016) A local–global approach to semantic segmentation in aerial images. CoRR arXiv:abs/1607.05620

  26. Pinheiro PO, Lin T, Collobert R, Dollár P (2016) Learning to refine object segments. In: Computer vision—ECCV 2016—14th European conference, Amsterdam, The Netherlands, 11–14 Oct 2016, Proceedings, Part I, pp 75–91

  27. Ranjan A, Black MJ (2017) Optical flow estimation using a spatial pyramid network. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp 2720–2729

  28. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp 6517–6525

  29. Ren S, He K, Girshick RB, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, 7–12 Dec 2015, Montreal, Quebec, Canada, pp 91–99

  30. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention—MICCAI 2015—18th international conference Munich, Germany, 5–9 Oct 2015, Proceedings, Part III, pp 234–241

    Google Scholar 

  31. Ruta A, Li Y, Liu X (2007) Towards real-time traffic sign recognition by class-specific discriminative features. In: Proceedings of the British machine vision conference 2007, University of Warwick, UK, 10–13 Sept 2007, pp 1–10

  32. Sakai Y, Lu H, Tan JK, Kim H (2019) Recognition of surrounding environment from electric wheelchair videos based on modified YOLOv2. Future Gener Comput Syst 92:157–161

    Google Scholar 

  33. Salti S, Petrelli A, Tombari F, Fioraio N, di Stefano L (2013) A traffic sign detection pipeline based on interest region extraction. In: The 2013 international joint conference on neural networks, IJCNN 2013, Dallas, TX, USA, 4–9 Aug 2013, pp 1–7

  34. Sermanet P, Kavukcuoglu K, Chintala S, LeCun Y (2013) Pedestrian detection with unsupervised multi-stage feature learning. In: 2013 IEEE conference on computer vision and pattern recognition, Portland, OR, USA, 23–28 June 2013, pp 3626–3633

  35. Sermanet P, LeCun Y (2011) Traffic sign recognition with multi-scale convolutional networks. In: The 2011 international joint conference on neural networks, IJCNN 2011, San Jose, CA, USA, July 31—August 5 2011, pp 2809–2813

  36. Stallkamp J, Schlipsing M, Salmen J, Igel C (2012) Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw 32:323–332

    Google Scholar 

  37. Wang W, Sun S, Jiang M, Yan Y, Chen X (2017) Traffic lights detection and recognition based on multi-feature fusion. Multimedia Tools Appl 76(13):14829–14846

    Google Scholar 

  38. Xie S, Tu Z (2017) Holistically-nested edge detection. Int J Comput Vis 125(1–3):3–18

    MathSciNet  Google Scholar 

  39. Yang F, Choi W, Lin Y (2016) Exploit all the layers: fast and accurate CNN object detector with scale dependent pooling and cascaded rejection classifiers. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp 2129–2137

  40. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multi-task cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503. https://doi.org/10.1109/LSP.2016.2603342

    Google Scholar 

  41. Zheng W, Zhu X, Wen G, Zhu Y, Yu H, Gan J (2018) Unsupervised feature selection by self-paced learning regularization. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec

    Google Scholar 

  42. Zheng W, Zhu X, Zhu Y, Hu R, Lei C (2018) Dynamic graph learning for spectral feature selection. Multimedia Tools Appl 77(22):29739–29755

    Google Scholar 

  43. Zhu X, Zhang S, He W, Lei C, Yang L, Zhu P (2018) Multi-view spectral clustering. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2018.2873378

    Google Scholar 

  44. Zhu X, Zhang S, Li Y, Zhang J, Yang L, Fang Y (2018) Low-rank sparse subspace for spectral clustering. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2018.2858782

    Google Scholar 

  45. Zhu Z, Liang D, Zhang S, Huang X, Li B, Hu S (2016) Traffic-sign detection and classification in the wild. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp 2110–2118

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grants Nos. 61672133, 61832001, 61632007).

Author information

Authors and Affiliations

  1. Center for Future Media, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China

    Zhenwen Liang, Jie Shao, Dongyang Zhang & Lianli Gao

Authors
  1. Zhenwen Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lianli Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Shao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Z., Shao, J., Zhang, D. et al. Traffic sign detection and recognition based on pyramidal convolutional networks. Neural Comput & Applic 32, 6533–6543 (2020). https://doi.org/10.1007/s00521-019-04086-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04086-z

Keywords

Search

Navigation