Skip to main content
SpringerLink
Log in

A deep learning-based approach for the automated surface inspection of copper clad laminate images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Surface quality inspection and control are extremely important for electronic manufacturing. The use of machine vision technology to automatically detect the defects of products has become an indispensable means for better quality control. A machine vision-based surface quality inspection system is usually composed of two processes: image acquisition and automatic defect detection. In this paper, we propose a deep learning-based approach for the defect detection of Copper Clad Laminate (CCL) images acquired from an industrial CCL production line. In the proposed approach, a new convolutional neural network (CNN) that realizes fast defect detection while maintaining high accuracy is designed. Our proposed approach makes four contributions. First, we introduce the depthwise separable convolution to reduce the calculation time. Second, we improve the squeeze-and-excitation block to improve network performance. Third, we introduce the squeeze-and-expand mechanism to reduce the computation cost. Fourth, we employ a smoother activation function (Mish) to allow improved information flow. The proposed network is compared with the benchmark CNNs (including Inception, ResNet and MobileNet). The experimental results show that compared with the benchmark networks, our proposed network has achieved the best results regarding the accuracy and suboptimal results in terms of the speed compared with the benchmark networks. Therefore, our proposed method has been integrated into an industrial CCL production line as a guideline for online defective product rejection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Bo T, Jianyi K, Shiqian W, et al. (2017) Review of machine vision surface defect detection. Chinese Journal of Image and Graphics 22:1640–1663

    Google Scholar 

  2. Li S, Jing Y, Zheng W (2018) Review of the development and application of defect detection technology. Journal of Automation 15:55–58

    Google Scholar 

  3. Huangpeng Q, Zhang H, Zeng X, et al. (2018) Automatic visual defect detection using texture prior and low-rank representation. IEEE Access 6:37965–37976

    Article  Google Scholar 

  4. Ojala T, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recogn 29:51–59

    Article  Google Scholar 

  5. Liu K, Wang H, Chen H, et al. (2017) Steel surface defect detection using a new Haar-Weibull-Variance model in unsupervised manner. IEEE Transactions on Instrumentation Measurement, pp 1–12

  6. Luo Q, Sun Y, Li P, et al. (2018) Generalized completed local binary patterns for time-efficient steel surface defect classification. IEEE Transactions on Automation Science Engineering, pp 1–13

  7. Sun X, Gu J, Tang S, et al. (2018) Research progress of visual inspection technology of steel Products-A review. Appl Sci 8:11

    Google Scholar 

  8. Li Y, Zhao W, Pan J, et al. (2017) Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Transactions on Automation Science Engineering 14:1256–1264

    Article  Google Scholar 

  9. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  10. Altman NS (1992) An introduction to kernel and Nearest-Neighbor nonparametric regression. Am Stat 46:175–185

    MathSciNet  Google Scholar 

  11. Breiman L, Friedman J, Stone C, et al. (1984) Classification and regression trees. CRC Press

  12. Yuan ZC, Zhang Z-T, Su H, et al. (2018) Vision-based defect detection for mobile phone cover glass using deep neural networks. Int J Precision Eng Manufac 19:801–810

    Article  Google Scholar 

  13. Jang C, Yun S, Hwang H, et al. (2018) A defect inspection method for machine vision using defect probability image with deep convolutional neural network. In: The 14th Asian conference on computer vision, Perth, Australia, pp 2–6

  14. Ren R, Hung T, Tan KC (2018) A generic deep learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics 48:929–940

    Article  Google Scholar 

  15. Zheng X, Wang H, Chen J, Zheng S, Kong Y (2020) A generic semi-supervised deep learning-based approach for automated surface inspection. IEEE Access 8:114088–114099

    Article  Google Scholar 

  16. Soukup D, Mork H (2014) Convolutional neural networks for steel surface defect detection from photometric stereo images. Advanced in Visual Computing, Berlin, Germany, pp 668–677

  17. Je KP, Kwon BK, Park J-H, et al. (2016) Machine learning-based imaging system for surface defect inspection. Int J Precision Eng Manufac Green Technol 3:303–310

    Article  Google Scholar 

  18. Wang T, Chen Y, Qiao M, et al. (2018) A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, Berlin, Germany, pp 3465–3471

  19. Caggiano A, Zhang J, Alfieri V , et al. (2019) Machine learning-based image processing for on-line defect recognition in additive manufacturing. CIRP Annals-Manufacturing Technology

  20. Michalski P, Ruszczak B, Tomaszewski M (2018) Convolutional neural networks implementations for computer vision. In: The 3rd international scientific conference on brain-computer interfaces, Opole, Poland, pp 13–14

  21. Rongsheng L, Ang W, Tengda Z (2018) Review of automatic optical (visual) detection technology and its application in defect detection. Journal of Optics 38:23–58

    Google Scholar 

  22. Howard AG, Zhu M, Bo C, et al. (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861

  23. Sandler M, Howard A, Zhu M, et al. (2018) MobileNetV2: inverted residuals and linear bottlenecks. arXiv:1801.04381v3

  24. Hu J, Li S, Albanie S, et al. (2018) Squeeze-and-excitation networks. arXiv:1709.01507

  25. Forrest N, Iandola SH, et al. (2017) Squeezenet: alexnet-level accuracy with 50x fewer parameters and < 0.5mb model. arXiv:1602.07360

  26. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  27. Misra D (2019) Mish: a self regularized non-monotonic neural activation function. arXiv:1908.08681

  28. Sifre L (2014) Rigid-motion scattering for image classification. Dissertation, Polytechnique

  29. Szegedy C, Vanhoucke V, Ioffe S, et al. (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition, Las Vegas, NV, pp 2818–2826

  30. Deng W, Dong R, Socher L , et al. (2009) ImageNet: a large-scale hierarchical image database. In: CVPR

  31. Tan C, Sun F, Kong T, et al. (2018) A survey on deep transfer learning. In: The 27th international conference on artificial neural networks, Rhodes, Greece, pp 4–7

  32. Liu S, Tian G, Xu Y (2019) A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter. Neurocomputing 338:191–206

    Article  Google Scholar 

  33. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

Download references

Acknowledgements

The authors would like to express their appreciation to the developers of the Keras framework and the developers of classical CNNs, including ResNet, MobileNet and Inception.

Author information

Authors and Affiliations

  1. Hangzhou Dianzi University, Hangzhou, China

    Xiaoqing Zheng, Jie Chen, Hongcheng Wang, Song Zheng & Yaguang Kong

Authors
  1. Xiaoqing Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongcheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Song Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaguang Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoqing Zheng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by National Natural Science Foundation of China under grant number U1609212, Zhejiang Provincial Science and Technology Plan under grant number 2019C04021, and Zhejiang Province Public Technology Research Project under grant number LGG20F030002.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, X., Chen, J., Wang, H. et al. A deep learning-based approach for the automated surface inspection of copper clad laminate images. Appl Intell 51, 1262–1279 (2021). https://doi.org/10.1007/s10489-020-01877-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01877-z

Keywords

Search

Navigation