Skip to main content
SpringerLink
Log in

Automatic quality inspection in additive manufacturing using semi-supervised deep learning

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Ensuring quality production is essential in additive manufacturing processes such as selective laser sintering (SLS). Deep learning based automated systems may provide a better way for real time monitoring to ensure quality production. These systems are usually trained in a fully supervised manner which require large amount of labeled data. Obtaining labeled training data in large quantities is a tedious and time consuming process. To overcome this, a novel semi-supervised deep learning approach is proposed in this work, which can be trained using both labeled and unlabeled data, and hence, reducing the manual labeling efforts needed to train the system. Experimental results on a SLS powder bed defect detection dataset show that the proposed approach is the new state-of-the-art, and shows its potential as a standalone real time monitoring system for SLS. In this dataset the proposed approach beats the state-of-the-art accuracy of \(98\%\) with only \(25\%\) of the labeled training data compared to other approaches. In addition, an extensive set of experiments were conducted on three additional public defect inspection datasets (NEU steel surface defects, KolektorSDD surface images of plastic electronics commutators, and surface textures) to show the applicability of the proposed approach on other computer integrated manufacturing environments for quality inspection. In all of these datasets the proposed approach beats the state-of-the-art results with relatively small amounts of training data compared to other approaches, which shows the effectiveness of the proposed approach for real time, automated, and accurate quality inspection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Bai, X., Fang, Y., Lin, W., Wang, L., & Ju, B. (2014). Saliency-based defect detection in industrial images by using phase spectrum. IEEE Transactions on Industrial Informatics, 10(4), 2135–2145.

    Article  Google Scholar 

  • Baumgartl, H., Tomas, J., Buettner, R., & Merkel, M. (2020). A deep learning-based model for defect detection in laser-powder bed fusion using in-situ thermographic monitoring. Progress in Additive Manufacturing, 5, 277–285.

    Article  Google Scholar 

  • Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., & Raffel, C. A. (2019). MixMatch: A holistic approach to semi-supervised learning. Advances in Neural Information Processing Systems, 32, 5049–5059.

    Google Scholar 

  • Božič, J., Tabernik, D., & Skočaj, D. (2020). End-to-end training of a two-stage neural network for defect detection. arXiv:2007.07676.

  • Caputo, B., Hayman, E., & Mallikarjuna, P. (2005). Class-specific material categorisation. In IEEE international conference on computer vision (Vol. 2, pp. 1597–1604).

  • Cha, Y.-J., Choi, W., Suh, G., Mahmoudkhani, S., & Büyüköztürk, O. (2018). Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer Aided Civil and Infrastructure Engineering, 33, 731–747.

    Article  Google Scholar 

  • Chen, Y., Peng, X., Kong, L., Dong, G., Remani, A., & Leach, R. (2021). Defect inspection technologies for additive manufacturing. International Journal of Extreme Manufacturing, 3(2), 022002.

    Article  Google Scholar 

  • Di, H., Ke, X., Peng, Z., & Dongdong, Z. (2019). Surface defect classification of steels with a new semi-supervised learning method. Optics and Lasers in Engineering, 117, 40–48.

    Article  Google Scholar 

  • Gao, Y., Gao, L., Li, X., & Yan, X. (2020). A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robotics and Computer-Integrated Manufacturing, 61, 101825.

  • Gobert, C., Reutzel, E. W., Petrich, J., Nassar, A. R., & Phoha, S. (2018). Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. Additive Manufacturing, 21, 517–528.

    Article  Google Scholar 

  • Guo, J., Wang, Q., & Li, Y. (2020). Semi-supervised learning based on convolutional neural network and uncertainty filter for façade defects classification. Computer-Aided Civil and Infrastructure Engineering, 1–17.

  • Hajizadeh, S., Núñez, A., & Tax, D. M. (2016). Semi-supervised rail defect detection from imbalanced image data. In 14th IFAC symposium on control in transportation systems (Vol. 49, No. 3, pp. 78–83).

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016) Deep residual learning for image recognition. In IEEE conference on computer vision and pattern recognition (pp. 770–778).

  • He, Y., Song, K., Dong, H., & Yan, Y. (2019). Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network. Optics and Lasers in Engineering, 122, 294–302.

    Article  Google Scholar 

  • Huang, Y., Qiu, C., Wang, X., Wang, S., & Yuan, K. (2020). A compact convolutional neural network for surface defect inspection. Sensors, 20(7), 1974.

    Article  Google Scholar 

  • Kwon, O., Kim, H. G., Ham, M. J., Kim, W., Kim, G.-H., Cho, J.-H., et al. (2020). A deep neural network for classification of melt-pool images in metal additive manufacturing. Journal of Intelligent Manufacturing, 31(2), 375–386.

    Article  Google Scholar 

  • Kylberg, G. (2011). The Kylberg texture dataset v. 1.0,” Tech. Rep. 35, Centre for Image Analysis, Swedish University of Agricultural Sciences and Uppsala University, Uppsala, Sweden.

  • Laine, S, & Aila, T. (2016). Temporal ensembling for semi-supervised learning. arXiv:1610.02242.

  • Lazebnik, S., Schmid, C., & Ponce, J. (2005). A sparse texture representation using local affine regions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1265–1278.

    Article  Google Scholar 

  • Lee, D.-H., et al. (2013).Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML (Vol. 3, p. 896).

  • Liang, P. (2005). Semi-supervised learning for natural language. PhD thesis, Massachusetts Institute of Technology.

  • Liu, J., Song, K., Feng, M., Yan, Y., Tu, Z., & Zhu, L. (2021). Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection. Optics and Lasers in Engineering, 136, 106324.

    Article  Google Scholar 

  • Liu, Y., Yuan, Y., Balta, C., & Liu, J. (2020). A light-weight deep-learning model with multi-scale features for steel surface defect classification. Materials,13(20).

  • Lv, X., Duan, F., Jiang, J., Fu, X., & Gan, L. (2020). Deep metallic surface defect detection: The new benchmark and detection network. Sensors, 20(6), 1562.

    Article  Google Scholar 

  • Mallik-Goswami, B., & Datta, A. (2000). Detecting defects in fabric with laser-based morphological image processing. Textile Research Journal, 70, 758–762.

    Article  Google Scholar 

  • Mayuravaani, M., & Manivannan, S. (2021) A semi-supervised deep learning approach for the classification of steel surface defects. In International Conference on Information and Automation for Sustainability, pp. 179–184.

  • Mei, S., Yang, H., & Yin, Z. (2018). An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Transactions on Instrumentation and Measurement, 67(6), 1266–1277.

    Article  Google Scholar 

  • Mo, J., Gan, Y., & Yuan, H. (2021). Weighted pseudo labeled data and mutual learning for semi-supervised classification. IEEE Access, 9, 36522–36534.

    Article  Google Scholar 

  • Okaro, I. A., Jayasinghe, S., Sutcliffe, C., Black, K., Paoletti, P., & Green, P. L. (2019). Automatic fault detection for laser powder-bed fusion using semi-supervised machine learning. Additive Manufacturing, 27, 42–53.

    Article  Google Scholar 

  • Ren, R., Hung, T., & Tan, K. C. (2018). A generic deep-learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics, 48(3), 929–940.

    Article  Google Scholar 

  • Ren, Z., Yeh, R., & Schwing, A. (2020). Not all unlabeled data are equal: Learning to weight data in semi-supervised learning. Advances in Neural Information Processing Systems, 33, 21786–21797.

    Google Scholar 

  • Rizve, M. N., Duarte, K., Rawat, Y. S., & Shah, M. (2021) In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. arXiv:2101.06329.

  • Rudolph, M., Wandt, B., & Rosenhahn, B. (2020). Same same but differnet: Semi-supervised defect detection with normalizing flows. arXiv:2008.12577.

  • Sajjadi, M., Javanmardi, M., & Tasdizen, T. (2016). Regularization with stochastic transformations and perturbations for deep semi-supervised learning. Advances in Neural Information Processing Systems, 29, 1171–1179.

    Google Scholar 

  • Scime, L., & Beuth, J. (2018). Anomaly detection and classification in a laser powder bed additive manufacturing process using a trained computer vision algorithm. Additive Manufacturing, 19, 114–126.

    Article  Google Scholar 

  • Scime, L., & Beuth, J. (2018). A multi-scale convolutional neural network for autonomous anomaly detection and classification in a laser powder bed fusion additive manufacturing process. Additive Manufacturing, 24, 273–286.

    Article  Google Scholar 

  • Shi, W., Gong, Y., Ding, C., Tao, Z. M., & Zheng, N. (2018). Transductive semi-supervised deep learning using min-max features. In Proceedings of the European conference on computer vision (pp. 299–315).

  • Shumin, D., Zhoufeng, L., & Chunlei, L. (2011). Adaboost learning for fabric defect detection based on HOG and SVM. In International conference on multimedia technology (pp. 2903–2906).

  • Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., & Raffel, C. (2020). Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv:2001.07685.

  • Song, K., & Yan, Y. (2013). A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science, 285, 858–864.

    Article  Google Scholar 

  • Soukup, D., & Huber-Mörk, R. (2014). Convolutional neural networks for steel surface defect detection from photometric stereo images. International Symposium on Visual Computing, 668–677.

  • Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929–1958.

    Google Scholar 

  • Tabernik, D., Šela, S., Skvarč, J., & Skočaj, D. (2019) Segmentation-based deep-learning approach for surface-defect detection Journal of Intelligent Manufacturing,31.

  • Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., & Liu, C. (2018) A survey on deep transfer learning. arXiv:1808.01974.

  • Tanaka, D., Ikami, D., Yamasaki, T., & Aizawa, K. (2018). Joint optimization framework for learning with noisy labels. In IEEE conference on computer vision and pattern recognition (pp. 5552–5560).

  • Tao, X., Zhang, D., Ma, W., Liu, X., & Xu, D. (2018). Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences, 8, 1575.

    Article  Google Scholar 

  • Van Engelen, J. E., & Hoos, H. H. (2020). A survey on semi-supervised learning. Machine Learning, 109(2), 373–440.

    Article  Google Scholar 

  • Wang, Y., Gao, L., Gao, Y., & Li, X. (2021). A new graph-based semi-supervised method for surface defect classification. Robotics and Computer-Integrated Manufacturing, 68, 102083.

    Article  Google Scholar 

  • Westphal, E., & Seitz, H. (2021). A machine learning method for defect detection and visualization in selective laser sintering based on convolutional neural networks. Additive Manufacturing, 41, 101965.

    Article  Google Scholar 

  • Win, M., Bushroa, A. R., Hassan, M. A., Hilman, N. M., & Ide-Ektessabi, A. (2015). A contrast adjustment thresholding method for surface defect detection based on mesoscopy. IEEE Transactions on Industrial Informatics, 11(3), 642–649.

    Article  Google Scholar 

  • Xiao, L., Lu, M., & Huang, H. (2020). Detection of powder bed defects in selective laser sintering using convolutional neural network. The International Journal of Advanced Manufacturing Technology, 107, 2485–2496.

    Article  Google Scholar 

  • Xu, L., Lv, S., Deng, Y., & Li, X. (2020). A weakly supervised surface defect detection based on convolutional neural network. IEEE Access, 8, 42285–42296.

    Article  Google Scholar 

  • Yi, L., Li, G., & Jiang, M. (2016). An end-to-end steel strip surface defects recognition system based on convolutional neural networks. Steel Research International, 88(2), 1600068.

    Article  Google Scholar 

  • Yuan, B., Giera, B., Guss, G., Matthews, I., & Mcmains, S. (2019) Semi-supervised convolutional neural networks for in-situ video monitoring of selective laser melting. In IEEE winter conference on applications of computer vision, pp. 744–753.

  • Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. arXiv:1710.09412

  • Zhang, Y., Hong, G. S., Ye, D., Zhu, K., & Fuh, J. Y. (2018). Extraction and evaluation of melt pool, plume and spatter information for powder-bed fusion am process monitoring. Materials & Design, 156, 458–469.

    Article  Google Scholar 

  • Zheng, X., Wang, H., Chen, J., Kong, Y., & Zheng, S. (2020). A generic semi-supervised deep learning-based approach for automated surface inspection. IEEE Access, 8, 114088–114099.

    Article  Google Scholar 

  • Zhou, S., Chen, Y., Zhang, D., Xie, J., & Zhou, Y. (2017). Classification of surface defects on steel sheet using convolutional neural networks. Materiali in tehnologije, 51, 123–131.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Science, University of Jaffna, Jaffna, Sri Lanka

    Siyamalan Manivannan

Authors
  1. Siyamalan Manivannan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siyamalan Manivannan.

Ethics declarations

Conflict of interest

The author declares that he has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Manivannan, S. Automatic quality inspection in additive manufacturing using semi-supervised deep learning. J Intell Manuf 34, 3091–3108 (2023). https://doi.org/10.1007/s10845-022-02000-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-02000-4

Keywords

Search

Navigation