Skip to main content
SpringerLink
Log in

ADM-Net: attentional-deconvolution module-based net for noise-coupled traffic sign recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Convolutional Neural Networks (CNNs) have become primary technologies in computer vision systems across multiple fields. Its central characteristic is to slide filters on input images and repeats the same procedures to obtain the image’s robust features. However, conventional CNNs struggle to classify objects when the input images are contaminated by unavoidable external noises such as missing information, blur, or illumination. This paper proposes an attentional-deconvolution module (ADM)-based net(ADM-Net) in which ADMs, convolutional-pooling, and a fully convolutional network (FCN) are applied to improve classification under such harsh conditions. The structure of ADM includes an attention layer, deconvolution layer and max-pooling. The attention layer and convolutional pooling help the proposed network maintain key features through convolution procedures under noise-coupled environments. The deconvolution layers and fully convolutional structure have advantages in providing additional information from upscale feature maps and enabling the network to store local pixel information. The ADM-Net was demonstrated on the German traffic sign recognition benchmark with different noise cases comparing densenet, multi-scale CNN, a committee of CNN, hierarchical CNN, and a multi-column deep neural network. Demonstrations of ADM-Net achieve the highest records in different cases such as 1) blur and missing information case: 86.637%, 2) missing information and illumination case: 92.329%, and 3) blur, missing information, and illumination case: 80.221%. Training datasets for ADM-Net have limited conditions, the proposed network demonstrates its robustness effectively under noise-coupled environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. Rectified Linear Unit

  2. Convolution layer with zero-padding

References

  1. Arcos-Garcia A, Alvarez-Garcia JA, Soria-Morillo LM (2012) Deep neural network for traffic sign recognition systems: an analysis of spatial transformers and stochastic optimisation methods. Neural Netw 99:158–165

    Article  Google Scholar 

  2. Bangquan X, Xiong WX (2019) Real-time embedded traffic sign recognition using efficient convolutional neural network. IEEE Access 7:53330–53346

    Article  Google Scholar 

  3. Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. Neural Netw TricksTrade:437–478

  4. Bi Q, Qin K, Zhang H, Li Z, Xu K (2020) RADC-Net: a residual attention based convolution network for aerial scene classification. Neurocomputing 377:345–359

    Article  Google Scholar 

  5. Cheng G, Li R, Lang C, Han J (2021) Task-wise attention guided part complementary learning for few-shot image classification. Sci China Inf Sci 64(2):1–14

    Article  Google Scholar 

  6. Cheng G, Yang C, Yao X, Guo L, Han J (2018) When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs. IEEE Geosci Remote Sens Lett 56(5):2811–2821

    Article  Google Scholar 

  7. Chu X, Yang W, Ouyang W, Ma C, Yuille AL, Wang X (2017) Multi-context attention for human pose estimation. 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp 5669–5678

  8. Chung JH, Kim DW, Kang TK, Lim MT (2020) Traffic sign recognition in harsh environment using attention based convolutional pooling neural network. Neural Process Lett 51:2551–2573

    Article  Google Scholar 

  9. Ciresan D, Meier U, Masci J, Schmidhuber J (2011) A committee of neural networks for traffic sign classification. The 2011 International Joint Conference on Neural Networks, pp 1918–1921

  10. Ciresan D, Meier U, Masci J, Schmidhuber J (2012) Multi-column deep neural network for traffic sign classification. Neural Netw 32:333–338

    Article  Google Scholar 

  11. Ding X, Guo Y, Ding G, Han J (2019) ACNet: strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks. Proceedings of the IEEE/CVF, International Conference on Computer Vision (ICCV), pp 1911–1920

  12. Du W, Wang Y, Qiao Y (2018) Recurrent spatial-temporal attention network for action recognition in videos. IEEE Trans Image Process 27(3):1347–1360

    Article  MathSciNet  MATH  Google Scholar 

  13. Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35 (8):1915–1929

    Article  Google Scholar 

  14. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. Proceedings of the IEEE/CVF, Conference on Computer Vision and Pattern Recognition (CVPR), pp 3146–3154

  15. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. Proc IEEE Conf Comput Vis Pattern Recognit:580–587

  16. Gudigar A, Chokkadi S, Raghavendra U, Acharya UR (2019) An efficient traffic sign recognition based on graph embedding features. Neural Comput Appl 31(2):395–407

    Article  Google Scholar 

  17. Hamker FH (2004) Predictions of a model of spatial attention using sum-and max-pooling functions. Neurocomputing 56:329–343

    Article  Google Scholar 

  18. Haque WA, Arefin S, Shihavuddin ASM, Hasan MA (2021) DeepThin: a novel lightweight CNN architecture for traffic sign recognition without GPU requirements. Expert Syst Appl 168:114481

  19. Hechri A, Mtibaa A (2020) Two-stage traffic sign detection and recognition based on SVM and convolutional neural networks. IET Image Process 14 (5):939–946

    Article  Google Scholar 

  20. Hong IP, Hwang YB, Kim DY (2019) Efficient deep learning of image denoising using patch complexity local divide and deep conquer. Pattern Recogn 96:106945

  21. Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440

  22. Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu W (2019) CCNet: criss-cross attention for semantic segmentation. Proceedings of the IEEE/CVF, International Conference on Computer Vision (ICCV), pp 603–612

  23. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. Int Conf Mach Learn:448–456

  24. Ji Y, Zhang H, Jie Z, Ma L, Wu QMJ (2020) CASNet: a cross-attention siamese network for video salient object detection. IEEE Trans Neural Netw Learn Syst:1–15

  25. Jin J, Fu K, Zhang C (2014) Traffic sign recognition with hinge loss trained convolutional neural networks. IEEE Trans Intell Trans Syst 15:1991–2000

    Article  Google Scholar 

  26. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems. [Online]. Available: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf, vol 25. Curran Associates, Inc., pp 1097–1105

  27. Lateef F, Ruichek Y (2019) Survey on semantic segmentation using deep learning techniques. Neurocomputing 338:321–348

    Article  Google Scholar 

  28. Li X, Jie Z, Feng J, Liu C, Yan S (2018) Learning with rethinking: recurrently improving convolutional neural networks through feedback. Pattern Recogn 79:183–194

    Article  Google Scholar 

  29. Liu C, Chang F, Chen Z, Liu D (2016) Fast traffic sign recognition via high-contrast region extraction and extended sparse representation. IEEE Trans Intell Trans Syst 17(1):79–92

    Article  Google Scholar 

  30. Liu Z, Du J, Tian F, Wen J (2019) MR-CNN: a multi-scale region-based convolutional neural network for small traffic sign recognition. IEEE Access 7:57120–57128

    Article  Google Scholar 

  31. Liu J, Wang Y, Li Y, Fu J, Li J, Lu H (2018) Collaborative deconvolutional neural networks for joint depth estimation and semantic segmentation. IEEE Trans Neural Netw Learn Syst 29(11):5655–5666

    Article  MathSciNet  Google Scholar 

  32. Liu D, Wen B, Jiao J, Liu X, Wang Z, Huang TS (2020) Connecting image denoising and high-level vision tasks via deep learning. IEEE Trans Image Process 29:3695–3706

    Article  Google Scholar 

  33. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440

  34. Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. Proceedings of the IEEE/CVF, Conference on Computer Vision and Pattern Recognition (CVPR), pp 3623–3632

  35. Lu X, Wang W, Shen J, Crandall DJ, Gool LV (2021) Segmenting objects from relational visual data. IEEE Trans Pattern Anal Mach Intell

  36. Lu X, Wang Y, Zhou X, Zhang Z, Ling Z (2017) Traffic sign recognition via multi-modal tree-structure embedded multi-task learning. IEEE Trans Intell Trans Syst 18(4):960–972

    Article  Google Scholar 

  37. Luo H, Yang Y, Tong B, Wu F, Fan B (2018) Traffic sign recognition using a multi-task convolutional neural network. IEEE Trans Intell Trans Syst 19(4):1100–1111

    Article  Google Scholar 

  38. Mao T, Zhang Y, Ruan Y, Gao H, Zhou H, Li D (2018) Feature learning and process monitoring of injection molding using convolution-deconvolution auto encoders. Comput Chem Eng 118:77–90

    Article  Google Scholar 

  39. Mhalla A, Chateau T, Amara NEB (2019) Spatio-temporal object detection by deep learning: video-interlacing to improve multi-object tracking. Image Vis Comput 88:120–131

    Article  Google Scholar 

  40. Mogelmose A, Trivedi MM, Moeslund TB (2012) Vision-based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans Intell Trans Syst 13(4):1484–1497

    Article  Google Scholar 

  41. Noh HW, Hong SH, Han BH (2015) Learning deconvolution network for semantic segmentation. Proceedings of the IEEE international conference on computer vision, pp 1520–1528

  42. Noord N, Postma E (2017) Learning scale-variant and scale-invariant features for deep image classification. Pattern Recogn 61:583–592

    Article  Google Scholar 

  43. Pang Y, Sun M, Jiang X, Li X (2018) Convolution in convolution for network in network. IEEE Trans Neural Netw Learn Syst 29(5):1587–1597

    Article  MathSciNet  Google Scholar 

  44. Saedi SI, Khosravi H (2020) A deep neural network approach towards real-time on-branch fruit recognition for precision horticulture. Expert Syst Appl 159:345–359

    Article  Google Scholar 

  45. Scherer D, Muller A, Behnke S (2010) Evaluation of pooling operations in convolutional architectures for object recognition. Int Conf Artif Neural Netw, pp 92–101

  46. Sermanet P, LeCun Y (2011) Traffic sign recognition with multi-scale convolutional networks, Neural Networks (IJCNN). The 2011 International Joint Conference on, pp 2809–2813

  47. Sharma S, Kiros R, Salakhutdinov R (2016) Action recognition using visual attention, arXiv:1511.04119

  48. Shen J, Ropbertson N (2021) BBAS: Towards large scale effective ensemble adversarial attacks against deep neural network learning. Inf Sci 569:469–478

    Article  Google Scholar 

  49. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, arXiv:1409.1556

  50. Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: the all convolutional net, arXiv:1412.6806

  51. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  52. Stallkamp J, Schlipsing M, Salmen J, lgel C (2012) Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw 32:323–332

    Article  Google Scholar 

  53. Stollenga M, Masci J, Gomez F, Schmidhuber J (2014) Design of stabilizing state feedback for delay systems via convex optimization. In: Advances in neural information processing systems, pp 3545–3553

  54. Sun M, Song Z, Jiang X, Pan J, Pang Y (2017) Learning pooling for convolutional neural network. Neurocomputing 224:96–104

    Article  Google Scholar 

  55. Szegedy C, Vanhoucke V, Ioffe S, Shlens J (2015) Going deeper with convolutions. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–9

  56. Tabernik D, Skocaj D (2020) Deep learning for large-scale traffic-sign detection and recognition. IEEE Trans Intell Trans Syst 21(4):1427–1440

    Article  Google Scholar 

  57. Timofte R, Zimmermann K, Gool LV (2021) Multi-view traffic sign detection, recognition, and 3D localisation. Mach Vis Appl 25(3):633–647

    Article  Google Scholar 

  58. Vidnerova P, Neruda R (2020) Vulnerability of classifiers to evolutionary generated adversarial examples. Neural Netw 127:168–181

    Article  MATH  Google Scholar 

  59. Wang W, Lu X, Shen J, Crandall DJ, Shao L (2019) Zero-shot video object segmentation via attentive graph neural networks. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp 9236–9245

  60. Wickramasinghe CS, Amarasinghe K, Manic M (2019) Deep self-organizing maps for unsupervised image classification. IEEE Trans Ind Informat 15 (11):5837–5845

    Article  Google Scholar 

  61. Wojna Z, Gorban A, Lee DS, Murphy K, Yu Q, Li Y, Ibarz J (2017) Attention-based extraction of structured information from street view imagery, arXiv:1704.03549

  62. Wong A, Shafiee MJ, Jules MS (2018) Micronnet: a highly compact deep convolutional neural network architecture for real-time embedded traffic sign classification. IEEE Access 6:59 803–59 810

    Article  Google Scholar 

  63. Yan Z, Feng Y, Cheng C, Fu J, Zhou X, Yuan J (2018) Extensive exploration of comprehensive vehicle attributes using D-CNN with weighted multi-attribute strategy. IET Intell Transp Syst 12(3):186–193

    Article  Google Scholar 

  64. Yang S, Deng B, Wang J, Li H, Lu M, Che Y, Wei X, Loparo KA (2020) Scalable digital neuromorphic architecture for large-scale biophysically meaningful neural network with multi-compartment neurons. IEEE Trans Neural Netw Learn Syst 31(1):148–162

    Article  Google Scholar 

  65. Yang S, Deng B, Wang J, Liu C, Li H, Lin Q, Fietkiewicz C, Loparo KA (2019) Design of hidden-property-based variable universe fuzzy control for movement disorders and its efficient reconfigurable implementation. IEEE Trans Fuzzy Syst 27(2):304–318

    Article  Google Scholar 

  66. Yang S, Gao T, Wang J, Deng B, Lansdell B, Linares-Barranco B (2021) Efficient spike-driven learning with dendritic event-based processing. Front Neurosci 15:97

    Google Scholar 

  67. Yang S, Wang J, Deng B, Azghadi MR, Linares-Barranco B (2021) Neuromorphic context-dependent learning framework with fault-tolerant spike routing. IEEE Trans Neural Netw Learn Syst

  68. Yang S, Wei X, Deng B, Liu C, Li H, Wang J (2018) Efficient digital implementation of a conductance-based globus pallidus neuron and the dynamics analysis. Physica A: Stat Mech Appl 494:484–502

    Article  MathSciNet  Google Scholar 

  69. Yuan Y, Xiong Z, Wang Q (2019) VSSA-NET: vertical Spatial sequence attention network for traffic sign detection. IEEE Trans Image Process 28 (7):3423–3434

    Article  MathSciNet  MATH  Google Scholar 

  70. Zeiler MD, Fergus R (2013) Stochastic pooling for regularization of deep convolutional neural networks, arXiv:1301.3557, pp 2278–2324

  71. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1D1A1B01016071 and NRF-2017R1D1A1B03031467).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Korea University, Seoul, Republic of Korea

    Jun Ho Chung & Myo Taeg Lim

  2. Department of Digital Electronics, Inha Technical College, Incheon, Korea

    Dong Won Kim

  3. Department of Human Intelligence and Robot Engineering, Sangmyung University, Cheonan, Republic of Korea

    Tae Koo Kang

Authors
  1. Jun Ho Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong Won Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tae Koo Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Myo Taeg Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Dong Won Kim or Myo Taeg Lim.

Ethics declarations

Conflict of Interests

The authors declare that there is no conflict of interest regarding the publication of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chung, J.H., Kim, D.W., Kang, T.K. et al. ADM-Net: attentional-deconvolution module-based net for noise-coupled traffic sign recognition. Multimed Tools Appl 81, 23373–23397 (2022). https://doi.org/10.1007/s11042-022-12219-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12219-1

Keywords

Search

Navigation