Skip to main content
SpringerLink
Log in

An axially decomposed self-attention network for the precise segmentation of surface defects on printed circuit boards

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The self-attention mechanism has been widely used to capture long-range relationships in various computer-vision tasks and is designed to update the representation of each pixel using a weighted sum of the features of all pixels in an image. However, it is computationally expensive, due to its potentially large matrix multiplication. It also does not make full use of position information, although this is crucial for modeling position-dependent interactions. To deal with these problems, the commonly used dot-product similarity is replaced by a position-aware similarity in this paper, which is introduced as a metric to evaluate the correlation between any two spatial positions. Then, on the basis of an axial decomposition operation, two concise and lightweight variants of self-attention are carefully constructed in sequence, namely the axial attention module and the complete decomposition module. The former decomposes only the first matrix multiplication of the self-attention mechanism, but the latter decomposes the entire process. Detailed experiments conducted on a real-world dataset of printed circuit board surface defects demonstrate the effectiveness and efficiency of the two variants. Their performance is comparable to that of state-of-the-art methods, and their computational costs is lower, which suggests that they could be widely utilized in various industrial inspection tasks based on computer vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The PCB surface-defect dataset is publicly available at https://github.com/youtang1993/MeiweiPCB.

  2. https://pytorch.org.

  3. https://github.com/open-mmlab/mmsegmentation.

References

  1. Amiruddin AAAM, Zabiri H, Taqvi SAA et al (2020) Neural network applications in fault diagnosis and detection: an overview of implementations in engineering-related systems. Neural Comput Appl 32(2):447–472. https://doi.org/10.1007/s00521-018-3911-5

    Article  Google Scholar 

  2. Bu H, Wang J, Huang X (2009) Fabric defect detection based on multiple fractal features and support vector data description. Eng Appl Artif Intell 22(2):224–235. https://doi.org/10.1016/j.engappai.2008.05.006

    Article  Google Scholar 

  3. Buades A, Coll B, Morel J (2005) A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR 2005), 20-26 June 2005, San Diego, CA, USA. IEEE Computer Society, pp 60–65, https://doi.org/10.1109/CVPR.2005.38

  4. Cao Y, Xu J, Lin S, et al (2019) GCNet: Non-local networks meet squeeze-excitation networks and beyond. In: 2019 IEEE/CVF international conference on computer vision workshops, ICCV workshops 2019, Seoul, Korea (South), October 27-28, 2019. IEEE, pp 1971–1980, https://doi.org/10.1109/ICCVW.2019.00246

  5. Chen H, Hu Q, Zhai B et al (2020) A robust weakly supervised learning of deep conv-nets for surface defect inspection. Neural Comput Appl 32(15):11,229-11,244. https://doi.org/10.1007/s00521-020-04819-5

    Article  Google Scholar 

  6. Chen L, Papandreou G, Kokkinos I et al (2018) DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  7. Chen L, Zhu Y, Papandreou G, et al (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part VII, Lecture Notes in Computer Science, vol 11211. Springer, pp 833–851, https://doi.org/10.1007/978-3-030-01234-2_49

  8. Chen LC, Papandreou G, Schroff F, et al (2017) Rethinking atrous convolution for semantic image segmentation. CoRR arXiv:1706.05587

  9. Deng J, Dong W, Socher R, et al (2009) ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA. IEEE Computer Society, pp 248–255, https://doi.org/10.1109/CVPR.2009.5206848

  10. Du W, Shen H, Fu J et al (2021) Automated detection of defects with low semantic information in X-ray images based on deep learning. J Intell Manuf 32:141–156. https://doi.org/10.1007/s10845-020-01566-1

    Article  Google Scholar 

  11. Fu J, Liu J, Tian H, et al (2019) Dual attention network for scene segmentation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). Computer vision foundation / IEEE, pp 3141–3149, https://doi.org/10.1109/CVPR.2019.00326

  12. Gao Y, Li X, Wang XV et al (2021) A review on recent advances in vision-based defect recognition towards industrial intelligence. J Manuf Syst 1:1–1. https://doi.org/10.1016/J.JMSY.2021.05.008

    Article  Google Scholar 

  13. Girshick RB (2015) Fast R-CNN. In: 2015 IEEE International conference on computer vision, ICCV 2015, Santiago, Chile, December 7-13, 2015. IEEE Computer Society, pp 1440–1448, https://doi.org/10.1109/ICCV.2015.169

  14. Gryllias K, Antoniadis I (2012) A support vector machine approach based on physical model training for rolling element bearing fault detection in industrial environments. Engineering Applications of Artificial Intelligence 25(2):326–344. https://doi.org/10.1016/j.engappai.2011.09.010, special Section: Local Search Algorithms for Real-World Scheduling and Planning

  15. He J, Deng Z, Qiao Y (2019) Dynamic multi-scale filters for semantic segmentation. In: 2019 IEEE/CVF international conference on computer vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, pp 3561–3571, https://doi.org/10.1109/ICCV.2019.00366

  16. He J, Deng Z, Zhou L, et al (2019) Adaptive pyramid context network for semantic segmentation. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, pp 7519–7528, https://doi.org/10.1109/CVPR.2019.00770

  17. He K, Zhang X, Ren S, et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part III, Lecture Notes in Computer Science, vol 8691. Springer, pp 346–361, https://doi.org/10.1007/978-3-319-10578-9/23

  18. He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, pp 770–778, https://doi.org/10.1109/CVPR.2016.90

  19. He T, Zhang Z, Zhang H, et al (2019) Bag of tricks for image classification with convolutional neural networks. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, pp 558–567, https://doi.org/10.1109/CVPR.2019.00065

  20. Huang Z, Wang X, Wei Y et al (2020) CCNet: Criss-cross attention for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 1:603–612. https://doi.org/10.1109/TPAMI.2020.3007032

    Article  Google Scholar 

  21. Kim B, Yuvaraj N, SriPreethaa KR et al (2021) Surface crack detection using deep learning with shallow CNN architecture for enhanced computation. Neural Comput Appl 33(15):9289–9305. https://doi.org/10.1007/s00521-021-05690-8

    Article  Google Scholar 

  22. Kirillov A, Wu Y, He K, et al (2020) PointRend: Image segmentation as rendering. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE, pp 9796–9805, https://doi.org/10.1109/CVPR42600.2020.00982

  23. LeCun Y, Boser BE, Denker JS et al (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551. https://doi.org/10.1162/neco.1989.1.4.541

    Article  Google Scholar 

  24. Li X, Zhong Z, Wu J, et al (2019) Expectation-maximization attention networks for semantic segmentation. In: 2019 IEEE/CVF international conference on computer vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, pp 9166–9175, https://doi.org/10.1109/ICCV.2019.00926

  25. Luo Q, Fang X, Liu L et al (2020) Automated visual defect detection for flat steel surface: A survey. IEEE Trans Instrum Meas 69(3):626–644. https://doi.org/10.1109/TIM.2019.2963555

    Article  Google Scholar 

  26. Luo W, Zhang H, Li J et al (2020) Learning semantically enhanced feature for fine-grained image classification. IEEE Signal Process Lett 27:1545–1549. https://doi.org/10.1109/LSP.2020.3020227

    Article  Google Scholar 

  27. Maniat M, Camp CV, Kashani A (2021) Deep learning-based visual crack detection using google street view images. Neural Computing and Applications pp 1–18. https://doi.org/10.1007/S00521-021-06098-0

  28. Ming W, Shen F, Li X et al (2020) A comprehensive review of defect detection in 3C glass components. Measurement 158(107):722. https://doi.org/10.1016/j.measurement.2020.107722

    Article  Google Scholar 

  29. Ooi MPL, Sok HK, Kuang YC et al (2013) Defect cluster recognition system for fabricated semiconductor wafers. Eng Appl Artif Intell 26(3):1029–1043. https://doi.org/10.1016/j.engappai.2012.03.016

    Article  Google Scholar 

  30. Santoro A, Raposo D, Barrett DGT, et al (2017) A simple neural network module for relational reasoning. In: Advances in neural information processing systems 30: Annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 4967–4976

  31. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651. https://doi.org/10.1109/TPAMI.2016.2572683

    Article  Google Scholar 

  32. Shrivastava A, Gupta A, Girshick RB (2016) Training region-based object detectors with online hard example mining. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, pp 761–769, https://doi.org/10.1109/CVPR.2016.89

  33. Ting H, Liu Y, Yabin Y et al (2020) Application of deep convolutional neural network on feature extraction and detection of wood defects. Measurement 152(107):357. https://doi.org/10.1016/j.measurement.2019.107357

    Article  Google Scholar 

  34. Tsai DM, Rivera Molina DE (2019) Morphology-based defect detection in machined surfaces with circular tool-mark patterns. Measurement 134:209–217. https://doi.org/10.1016/j.measurement.2018.10.079

    Article  Google Scholar 

  35. Vaswani A, Shazeer N, Parmar N, et al (2017) Attention is all you need. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 5998–6008

  36. Wang X, Girshick RB, Gupta A, et al (2018) Non-local neural networks. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. IEEE Computer Society, pp 7794–7803, https://doi.org/10.1109/CVPR.2018.00813

  37. Wang Z, Zhu D (2019) An accurate detection method for surface defects of complex components based on support vector machine and spreading algorithm. Measurement 147(106):886. https://doi.org/10.1016/J.MEASUREMENT.2019.106886

    Article  Google Scholar 

  38. Wei X, Yang Z, Liu Y et al (2019) Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study. Eng Appl Artif Intell 80:66–81. https://doi.org/10.1016/j.engappai.2019.01.008

    Article  Google Scholar 

  39. Xiao T, Liu Y, Zhou B, et al (2018) Unified perceptual parsing for scene understanding. In: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part V, Lecture Notes in Computer Science, vol 11209. Springer, pp 432–448, https://doi.org/10.1007/978-3-030-01228-1/26

  40. Xing J, Jia M (2021) A convolutional neural network-based method for workpiece surface defect detection. Measurement 176(109):185. https://doi.org/10.1016/J.MEASUREMENT.2021.109185

    Article  Google Scholar 

  41. Yin M, Yao Z, Cao Y, et al (2020) Disentangled non-local neural networks. In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XV, Lecture Notes in Computer Science, vol 12360. Springer, pp 191–207, https://doi.org/10.1007/978-3-030-58555-6/12

  42. Yu C, Wang J, Peng C, et al (2018) Learning a discriminative feature network for semantic segmentation. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. IEEE Computer Society, pp 1857–1866, https://doi.org/10.1109/CVPR.2018.00199

  43. Yuan Y, Chen X, Wang J (2020) Object-contextual representations for semantic segmentation. In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VI, Lecture Notes in Computer Science, vol 12351. Springer, pp 173–190, https://doi.org/10.1007/978-3-030-58539-6/11

  44. Zhang H, Dana KJ, Shi J, et al (2018) Context encoding for semantic segmentation. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. IEEE Computer Society, pp 7151–7160, https://doi.org/10.1109/CVPR.2018.00747

  45. Zhang H, Jiang L, Li C (2021) CS-ResNet: Cost-sensitive residual convolutional neural network for PCB cosmetic defect detection. Expert Syst Appl 185(1):115673. https://doi.org/10.1016/j.eswa.2021.115673

    Article  Google Scholar 

  46. Zhang K, Wang W, Lv Z et al (2021) Computer vision detection of foreign objects in coal processing using attention CNN. Eng Appl Artif Intell 102(104):242. https://doi.org/10.1016/j.engappai.2021.104242

    Article  Google Scholar 

  47. Zhao H, Shi J, Qi X, et al (2017) Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, pp 6230–6239, https://doi.org/10.1109/CVPR.2017.660

  48. Zhao H, Zhang Y, Liu S, et al (2018) PSANet: Point-wise spatial attention network for scene parsing. In: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part IX, Lecture Notes in Computer Science, vol 11213. Springer, pp 270–286, https://doi.org/10.1007/978-3-030-01240-3/17

  49. Zhou T, Zhang J, Su H et al (2021) EDDs: a series of efficient defect detectors for fabric quality inspection. Measurement 172(108):885. https://doi.org/10.1016/j.measurement.2020.108885

    Article  Google Scholar 

  50. Zhu Z, Xu M, Bai S, et al (2019) Asymmetric non-local neural networks for semantic segmentation. In: 2019 IEEE/CVF International conference on computer vision (ICCV). IEEE, pp 593–602, https://doi.org/10.1109/ICCV.2019.00068

Download references

Acknowledgements

This work was partially supported by the Key Areas Research and Development Program of Guangdong Province under Grant 2018B010109007 and the Key-Area Research and Development Program of Guangzhou under Grant 202007030004.

Author information

Authors and Affiliations

  1. Sun Yat-Sen University, Guangzhou, 510006, China

    Danqing Kang, Yu Han & Jianhuang Lai

  2. Guangdong Province Key Laboratory of Information Security Technology, Guangzhou, 510006, China

    Jianhuang Lai

  3. Guangdong Provincial Key Laboratory of Fire Science and Technology, Guangzhou, 510006, China

    Yu Han

  4. Intelligence Eyes Co.,Ltd, Guangzhou, 510006, China

    Junyong Zhu

Authors
  1. Danqing Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianhuang Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianhuang Lai.

Ethics declarations

Conflicts of interest

These have no potential competing interests in this paper. We confirm that the content of the manuscript has not been published or submitted for publication elsewhere. All authors have seen the manuscript and approved to submit to your journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 26832 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, D., Han, Y., Zhu, J. et al. An axially decomposed self-attention network for the precise segmentation of surface defects on printed circuit boards. Neural Comput & Applic 34, 13697–13712 (2022). https://doi.org/10.1007/s00521-022-07192-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07192-7

Keywords

Search

Navigation