Skip to main content
SpringerLink
Log in

Defect classification on limited labeled samples with multiscale feature fusion and semi-supervised learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Defect inspection is an essential part of ensuring the quality of industrial products. Deep learning has achieved great success in defect inspection when a large number of labeled samples are available. However, it is infeasible to collect and label numerous samples in many manufacturing processes. Meanwhile, deep learning methods cannot conform to the high defect recognition accuracy of strict production requirements when the labeled samples are scarce but varied. This paper proposed a novel convolutional neural network architecture and a semi-supervised learning strategy using soft pseudo labels and a mutual correction classifier to improve the defect inspection accuracy when labeled samples are scarce. The effectiveness of the proposed method is verified on a famous industrial defect inspection benchmark dataset and a practical dataset containing images collected from actual injection molding production lines. The results indicate that the proposed method achieves an accuracy of 99.03% on the benchmark defect dataset, which is approximately 13.2% higher than other methods when the training dataset contains only 45 labeled images and 135 unlabeled samples per category. The best accuracy on the benchmark dataset obtained by the proposed method reaches 99.72%. Besides, an average accuracy of 99.25% is achieved with only 20 labeled samples and 180 unlabeled samples per category in the practical defect inspection task. Visualization methods prove that the performance improvement comes from the proposed multiscale architecture and the semi-supervised learning strategy. The proposed method can be used in practical defect inspection applications of industrial manufacturing, such as steel rolling, welding, and injection molding.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Zheng X, Chen J, Wang H et al (2021) A deep learning-based approach for the automated surface inspection of copper clad laminate images. Appl Intell 51:1262–1279

    Article  Google Scholar 

  2. Wu J, Le J, Xiao Z, et al (2021) Automatic fabric defect detection using a wide-and-light network[J]. Applied Intelligence 51(7):4945–4961

  3. Di H, Ke X, Peng Z, Dongdong Z (2019) Surface defect classification of steels with a new semi-supervised learning method. Opt Lasers Eng 117:40–48

    Article  Google Scholar 

  4. Dong H, Song K, He Y, et al (2019) PGA-Net: Pyramid feature fusion and global context attention network for automated surface defect detection[J]. IEEE Transactions on Industrial Informatics 16(12):7448–7458

  5. Lee H, Ryu K (2020) Dual-Kernel-Based Aggregated Residual Network for Surface Defect Inspection in Injection Molding Processes. Appl Sci 10:8171

    Article  Google Scholar 

  6. Liu B, Huang P, Zeng X, Li Z (2017) Hidden defect recognition based on the improved ensemble empirical decomposition method and pulsed eddy current testing. Ndt E Int 86:175–185

    Article  Google Scholar 

  7. Jian C, Gao J, Ao Y (2017) Automatic surface defect detection for mobile phone screen glass based on machine vision. Appl Soft Comput 52:348–358

    Article  Google Scholar 

  8. Xiao Q, Dai J, Luo J, Fujita H (2019) Multi-view manifold regularized learning-based method for prioritizing candidate disease miRNAs. Knowledge-Based Syst 175:118–129

    Article  Google Scholar 

  9. Hayashi T, Fujita H, Hernandez-Matamoros A (2021) Less complexity one-class classification approach using construction error of convolutional image transformation network. Inf Sci (Ny) 560:217–234

    Article  MathSciNet  Google Scholar 

  10. Ren R, Hung T, Tan KC (2017) A generic deep-learning-based approach for automated surface inspection. IEEE Trans Cybern 48:929–940

    Article  Google Scholar 

  11. Fujita H, Cimr D (2019) Decision support system for arrhythmia prediction using convolutional neural network structure without preprocessing. Appl Intell 49:3383–3391

    Article  Google Scholar 

  12. Lin X, Wang X, Li L (2020) Intelligent detection of edge inconsistency for mechanical workpiece by machine vision with deep learning and variable geometry model. Appl Intell 50:2105–2119

    Article  Google Scholar 

  13. Okaro IA, Jayasinghe S, Sutcliffe C et al (2019) Automatic fault detection for laser powder-bed fusion using semi-supervised machine learning. Addit Manuf 27:42–53

    Google Scholar 

  14. Qi G-J, Luo J (2020) Small data challenges in big data era: A survey of recent progress on unsupervised and semi-supervised methods. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2020.3031898

  15. Wu H, Prasad S (2017) Semi-supervised deep learning using pseudo labels for hyperspectral image classification. IEEE Trans Image Process 27:1259–1270

    Article  MathSciNet  Google Scholar 

  16. Li Z, Ko B, Choi H-J (2019) Naive semi-supervised deep learning using pseudo-label. Peer-to-Peer Netw Appl 12:1358–1368

    Article  Google Scholar 

  17. He Y, Song K, Dong H, Yan Y (2019) Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network. Opt Lasers Eng 122:294–302

    Article  Google Scholar 

  18. Protopapadakis E, Doulamis A, Doulamis N, Maltezos E (2021) Stacked autoencoders driven by semi-supervised learning for building extraction from near infrared remote sensing imagery. Remote Sens 13:371

    Article  Google Scholar 

  19. Zhan Y, Hu D, Wang Y, Yu X (2017) Semisupervised hyperspectral image classification based on generative adversarial networks. IEEE Geosci Remote Sens Lett 15:212–216

    Article  Google Scholar 

  20. Luo Q, Gao B, Woo WL, Yang Y (2019) Temporal and spatial deep learning network for infrared thermal defect detection. NDT E Int 108:102164

  21. Yang L, Wang Z, Gao S (2019) Pipeline magnetic flux leakage image detection algorithm based on multiscale SSD network. IEEE Trans Ind Informatics 16:501–509

    Article  Google Scholar 

  22. Tulbure A-A, Tulbure A-A, Dulf E-H (2021) A review on modern defect detection models using DCNNs–Deep convolutional neural networks. J Adv Res. https://doi.org/10.1016/j.jare.2021.03.015

  23. Lobov SA, Mikhaylov AN, Shamshin M et al (2020) Spatial properties of STDP in a self-learning spiking neural network enable controlling a mobile robot. Front Neurosci 14:88

    Article  Google Scholar 

  24. Yang S, Gao T, Wang J et al (2021) Efficient spike-driven learning with dendritic event-based processing. Front Neurosci 15:97

    Google Scholar 

  25. Gao Y, Gao L, Li X, Yan X (2020) A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robot Comput Integr Manuf 61:101825

  26. Wang F, Zhu L, Li J, et al (2021) Unsupervised soft-label feature selection. Knowledge-Based Syst 219:106847

  27. Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  28. Han D, Liu Q, Fan W (2018) A new image classification method using CNN transfer learning and web data augmentation. Expert Syst Appl 95:43–56

    Article  Google Scholar 

  29. Moradi R, Berangi R, Minaei B (2020) A survey of regularization strategies for deep models. Artif Intell Rev 53:3947–3986

    Article  Google Scholar 

  30. Baek K, Bang D, Shim H (2021) GridMix: Strong regularization through local context mapping. Pattern Recognit 109:107594

  31. Shao S, McAleer S, Yan R, Baldi P (2018) Highly accurate machine fault diagnosis using deep transfer learning. IEEE Trans Ind Informatics 15:2446–2455

    Article  Google Scholar 

  32. Li C, Zhang S, Qin Y, Estupinan E (2020) A systematic review of deep transfer learning for machinery fault diagnosis. Neurocomputing 407:121–135

    Article  Google Scholar 

  33. Deng J, Dong W, Socher R, et al (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp 248–255

  34. Li X, Zhang W, Ding Q (2019) Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction. Reliab Eng Syst Saf 182:208–218

    Article  Google Scholar 

  35. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980

  36. Krueger D, Ballas N, Jastrzebski S, Arpit D, Kanwal MS, Maharaj T, Bengio E, Fischer A, Courville AC (2017) Deep nets don’t learn via memorization. ICLR

  37. Wang Y, Gao L, Gao Y, Li X (2021) A new graph-based semi-supervised method for surface defect classification. Robot Comput Integr Manuf 68:102083

  38. Yang S, Wang J, Hao X et al 2021 BiCoSS: toward large-scale cognition brain with multigranular neuromorphic architecture IEEE Trans Neural Networks Learn Syst 1 15 https://doi.org/10.1109/TNNLS.2020.3045492

  39. Yang S, Wang  J, Deng B et al 2021 Neuromorphic context-dependent learning framework with fault-tolerant spike routing IEEE Trans Neural Networks Learn Syst 1 15 https://doi.org/10.1109/TNNLS.2021.3084250

  40. Yang S, Wang  J, Zhang N et al 2021 CerebelluMorphic: large-scale neuromorphic model and architecture for supervised motor learning IEEE Trans Neural Networks Learn Syst 1 15 https://doi.org/10.1109/TNNLS.2021.3057070

  41. Abadi M, Agarwal A, Barham P, et al (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems[J]. arXiv preprint arXiv:1603.04467

  42. Cui Y, Wu D, Huang J (2020) Optimize TSK fuzzy systems for classification problems: Minibatch gradient descent with uniform regularization and batch normalization[J]. IEEE Transactions on Fuzzy Systems 28(12):3065–3075

  43. Zhang X, Wu D (2019) On the vulnerability of CNN classifiers in EEG-based BCIs. IEEE Trans Neural Syst Rehabil Eng 27:814–825

    Article  Google Scholar 

  44. Szegedy C, Vanhoucke V, Ioffe S, et al (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2818–2826

  45. Chen H, Hu Q, Zhai B et al (2020) A robust weakly supervised learning of deep Conv-Nets for surface defect inspection. Neural Comput Appl 32:11229–11244

    Article  Google Scholar 

  46. Yang S, Deng B, Wang J et al (2019) Scalable digital neuromorphic architecture for large-scale biophysically meaningful neural network with multi-compartment neurons. IEEE Trans neural networks Learn Syst 31:148–162

    Article  Google Scholar 

  47. Yang S, Wang J, Deng B et al (2018) Real-time neuromorphic system for large-scale conductance-based spiking neural networks. IEEE Trans Cybern 49:2490–2503

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge financial support from Key-Area Research and Development Program of Guangdong Province (Grant No. 2019B090918001) and National Program on Key Basic Research Project (Grant No. 2019YFB1704900).

Funding

Key-Area Research and Development Program of Guangdong Province (Grant No. 2019B090918001).

National Program on Key Basic Research Project (Grant No. 2019YFB1704900).

Author information

Authors and Affiliations

  1. State Key Laboratory of Materials Processing and Die & Mould Technology, School of Materials Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China

    Jiahuan Liu, Fei Guo, Yun Zhang, Binkui Hou & Huamin Zhou

Authors
  1. Jiahuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Binkui Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huamin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Binkui Hou.

Ethics declarations

Conflicts of interest/ Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Guo, F., Zhang, Y. et al. Defect classification on limited labeled samples with multiscale feature fusion and semi-supervised learning. Appl Intell 52, 8243–8258 (2022). https://doi.org/10.1007/s10489-021-02917-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02917-y

Keywords

Search

Navigation