Skip to main content
SpringerLink
Log in

Online inspection of narrow overlap weld quality using two-stage convolution neural network image recognition

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In narrow overlap welding, serious defects in the weld will lead to band breakage accident, and the whole hot dip galvanizing unit will be shut down. Laser vision inspection hardware is used to collect real-time image of weld surface, and image defect recognition and evaluation system is developed to real-time detect quality. Firstly, region division is implemented. In view of the characteristics of gray image such as large information, low contrast and blurred edge, an improved image segmentation algorithm is proposed by using image convolution to enhance edge features and combining with integral image, which can quickly and accurately extract the weld edge and divide the region, and the processing time can meet the real-time requirements. Then comparing the effect of traditional method and convolution neural network in identifying defects, VGG16 is used to identify weld defects. In order to ensure real-time performance, a two-stage weld defect recognition is proposed. First, the large defective area is identified, and then, the defect is accurately identified in the area. This method can quickly extract defect areas and complete weld defect classification. Experiments show that the accuracy can reach 97% and average running time takes 3.2 s, meeting the online detection requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Zhang, C.: Resistance welding technology and its application equipment. Electron. Technol. 24(5), 201–203 (2017)

    Google Scholar 

  2. Yang, J., Li, Z., Fan, L., et al.: Weld surface defect detection system based on linear laser scanning. Welding 2, 19–23 (2016)

    Google Scholar 

  3. Fei, Z., Yang, E., Hu, H., et al.: Review of machine vision-based electronic travel aids. In: Automation and Computing (ICAC), 23rd International Conference on. IEEE, 2017, pp. 1–7 (2017)

  4. Singhka, D.K.H., Neogi, N., Mohanta, D.: Surface defect classification of steel strip based on machine vision. In: Computer and Communications Technologies (ICCCT), 2014 International Conference on IEEE, pp. 1–5 (2014)

  5. Zhang, Y., Liu, S., Cao, J., et al.: Wood board image processing based on dual-tree complex wavelet feature selection and compressed sensing. Wood Sci. Technol. 50(2), 1–15 (2016)

    Article  Google Scholar 

  6. Wang, H., Miao, R., Wu, X.S., et al.: Polycrystalline silicon wafer surface color defect inspection based on machine vision. Appl. Mech. Mater. 548–549, 48–52 (2014)

    Google Scholar 

  7. Hao, X., Li, B.: Detection of rail track surface defects based on machine vision. J. Beijing Jiaotong Univ. 39(2), 28 (2015)

    Google Scholar 

  8. He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Comput. Ind. Eng. 128, 290–297 (2019)

    Article  Google Scholar 

  9. Kai, L.: Fingerprint feature fusion method based on local structure of minutiae and LBP. Inf. Technol. 9, 109–112 (2017)

    Google Scholar 

  10. Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 12, 2037–2041 (2006)

    Article  Google Scholar 

  11. Hu, H., Peng, G., Wang, X., Zhou, Z.: Weld defect classification using 1-D LBP feature extraction of ultrasonic signals. Nondestr. Test. Eval. 33(1), 92–108 (2018)

    Article  Google Scholar 

  12. Wang, L., Wang, X.-F.: Identification of weld defects based on rotation-invariant HOG feature. Comput. Syst. Appl. 27(02), 157–162 (2018)

    Google Scholar 

  13. Tamilselvan, P., Wang, P.: Failure diagnosis using deep belief learning based health state classification. Reliab. Eng. Syst. Saf. 115(Jul.), 124–135 (2013)

    Article  Google Scholar 

  14. Bondarenko, A., Borisov, A.: Research on the classification ability of deep belief networks on small and medium datasets. Inf. Technol. Manag. Sci. 16(1), 60–65 (2013)

    Google Scholar 

  15. Krauss, C., Do, X.A., Huck, N.: Deep neural networks, gradient-boosted trees, random forests: statistical arbitrage on the S&P 500. Eur. J. Oper. Res. 259(2), 689–702 (2017)

    Article  Google Scholar 

  16. Nie, Z., Ma, H., Lin, Y.: Structural damage detection with automatic feature-extraction through deep learning. Comput.-Aided Civ. Infrastruct. Eng. 32(12), 1025–1046 (2017)

    Article  Google Scholar 

  17. Donahue, J., Hendricks, L.A., Rohrbach, M., et al.: Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 677–691 (2017a)

    Article  Google Scholar 

  18. He, K., Zhang, X., Ren, S., et al.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)

    Article  Google Scholar 

  19. Liu, Y., Yang, E., Fei, Y., et al.: Automated pixel-level pavement crack detection on 3D Asphalt surfaces using a deep-learning network. Comput.-Aided Civ. Infrastruct. Eng. 32(10), 805–819 (2017)

    Article  Google Scholar 

  20. Wan, X., Wang, Y., Zhao, D., et al.: A comparison of two types of neural network for weld quality prediction in small scale resistance spot welding. Mech. Syst. Signal Process. 93, 634–644 (2017)

    Article  Google Scholar 

  21. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105 (2012)

  22. Rawat, W., Wang, Z.: Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput. 29(9), 2352–2449 (2017). https://doi.org/10.1162/neco_a_00990

  23. Cha, Y.-J., Choi, W., Buyukozturk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017)

    Article  Google Scholar 

  24. Zhiqiang, C., Chuan, L., Sanchez, R.-V.: Gearbox fault identification and classification with convolutional neural networks. Shock Vib. 2015, 1–10 (2015)

    Google Scholar 

  25. Chen, Z., Li, C., Sánchez, R.V.: Multi-layer neural network with deep belief network for gearbox fault diagnosis. J. Vibroeng. 17(5), 2379–2392 (2015)

    Google Scholar 

  26. Gan, M., Wang, C., Zhu, C.: Construction of hierarchical diagnosis network based on deep learning and its application in the fault pattern recognition of rolling element bearings. Mech. Syst. Signal Process. 72, 92–104 (2016)

  27. Lin, Y., Nie, Z., Ma, H.: Structural damage detection with automatic feature-extraction through deep learning. Comput.-Aided Civ. Infrastruct. Eng. 32(12), 1025–1046 (2017)

    Article  Google Scholar 

  28. Donahue, J., Hendricks, L.A., Rohrbach, M., et al.: Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 677–691 (2017b)

    Article  Google Scholar 

  29. Fu, Y., Zhang, Y., Gao, Y., et al.: Machining vibration states monitoring based on image representation using convolutional neural networks. Eng. Appl. Artif. Intell. 65, 240–251 (2017)

    Article  Google Scholar 

  30. Zhang, A., Wang, K.C.P., Li, B., Yang, E., Dai, X., Peng, Y., Fei, Y., Liu, Y., Li, J.Q., Chen, C.: Automated Pixel-level pavement crack detection on 3D Asphalt surfaces using a deep-learning network. Comput.-Aided Civ. Infrastruct. Eng. 32(10), 805–819 (2017)

    Article  Google Scholar 

  31. He, K., Zhang, X., Ren, S., et al.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2014)

    Article  Google Scholar 

  32. Jiao, Z., Jia, G., Cai, Y.: A new approach to oil spill detection that combines deep learning with unmanned aerial vehicles. Comput. Ind. Eng. 135, 1300–1311 (2019)

    Article  Google Scholar 

  33. Cha, Y.-J., Choi, W., Suh, G., et al.: Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput.-Aided Civ. Infrastruct. Eng. 33(4), 1–17 (2018)

    Google Scholar 

  34. Kang, D., Benipal, S.S., Gopal, D.L., Cha, Y.-J.: Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Autom. Constr. 118, 103291 (2020)

    Article  Google Scholar 

  35. Beckman, G.H., Polyzois, D., Cha, Y.-J.: Deep learning-based automatic volumetric damage quantification using depth camera. Autom. Constr. 99(MAR), 114–124 (2018)

    Google Scholar 

  36. Choi, W., Cha, Y.-J.: SDDNet: real-time crack segmentation. IEEE Trans. Ind. Electron. 67(9), 8016–8025 (2020)

    Article  Google Scholar 

  37. Nakama, T.: Theoretical analysis of batch and on-line training for gradient descent learning in neural networks. Neurocomputing 73(1–3), 151–159 (2009)

    Article  Google Scholar 

  38. Lin, G., Shen, W.: Research on convolutional neural network based on improved Relu piecewise activation function. Proc. Comput. Sci. 131, 977–984 (2018)

    Article  Google Scholar 

  39. Li, X.B., Wang, W.Q.: Learning discriminative features via weights-biased softmax loss. Pattern Recogn. 2020, 107405 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the financial support of the National Natural Science foundation, China.

Funding

This work was supported by the National Natural Science Foundation, China (Nos. 51435009, 71971139).

Author information

Authors and Affiliations

  1. School of Naval Architecture, Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Rui Miao

  2. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Zihang Jiang, Yizhou Wu & Yuntian Gao

  3. SJTU – ParisTech Elite Institute of Technology, Shanghai Jiao Tong University, Shanghai, 200240, China

    Qinye Zhou

  4. Institute of Artificial Intelligence, Donghua University, Shanghai, 201620, China

    Jie Zhang

  5. Antai College of Economics and Management, Shanghai Jiao Tong University, Shanghai, 200240, China

    Zhibin Jiang

Authors
  1. Rui Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zihang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qinye Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yizhou Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuntian Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhibin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rui Miao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miao, R., Jiang, Z., Zhou, Q. et al. Online inspection of narrow overlap weld quality using two-stage convolution neural network image recognition. Machine Vision and Applications 32, 27 (2021). https://doi.org/10.1007/s00138-020-01158-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01158-2

Keywords

Search

Navigation