Skip to main content
SpringerLink
Log in

Bi-deformation-UNet: recombination of differential channels for printed surface defect detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Deep learning is frequently recommended for standard defect detection because of its ace accuracy and robustness. Unfortunately, current deep learning methods exist several challenges in detecting printed surface defects with multi-scale textures. Firstly, the existing methods only highlight the texture of defects, but concealed the color information of defects. Secondly, since the subtle defects of printed contained with weak semantic, it is difficult for current multi-scale network to locate the defects. Finally, current metric methods cannot measure the similarity between each of defect under class-imbalanced precisely. Therefore, Bi-Deformation-UNet (Bi-DUNet) is designed for automatic printed surface defect detection. In Bi-DUNet, the template-defect image pairs are first enhanced by our proposed pre-processing module Recombination of the Differential Channels. This module can highlight the texture and maintain the color information simultaneously. Then, the preprocessed image pairs are fed into the Dual-fusion Module (DM) and generated the output features with edge information and contextual information. The DM consists of two branches: the Template Branch and the Defect Branch. The two branches are identical in structure and Multi-channel Edge Attention Module. Besides, an Automatic Dual-margin Metric Loss is proposed to alleviate the situation of class-imbalance when measuring similarity of output features. Moreover, a 2020 Assembly Line Defective Product dataset (ALDP2020) is proposed, which contains 4000 images with different environment styles. Finally, our proposed Bi-DUNet achieves 3.97% higher than the state-of-the-arts in ALDP2020 in mAP50. The code is available at https://github.com/MRziyang/DefectDetection.git.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Li, X., Woo, W.L., Gu, L., Qiu, X.: Quantitative surface crack evaluation based on eddy current pulsed thermography. IEEE Sensors J., pp. 1–1 (2016)

  2. Yuan, X.C., Wu, L.S., Peng, Q.: An improved Otsu method using the weighted object variance for defect detection. Appl. Surf. Sci. 349, 472–484 (2015)

    Article  Google Scholar 

  3. Win, M., Bushroa, A.R., Hassan, M., Nordin, H., Ide-Ektessabi, A.: A contrast adjustment thresholding method for surface defect detection based on mesoscopy. IEEE Trans. Ind. Inform. 11 (2015)

  4. Wakaf, Z., Jalab, H.: Defect detection based on extreme edge of defective region histogram. J. King Saud Univ. - Comput. Inf. Sci. 30 (2016)

  5. Bai, X., Fang, Y., Lin, W., Wang, L., Ju, B.F.: Saliency-based defect detection in industrial images by using phase spectrum. IEEE Trans. Ind. Inform. 10, 2135–2145 (2014)

    Article  Google Scholar 

  6. Borwankar, R., Ludwig, R.: An optical surface inspection and automatic classification technique using the rotated wavelet transform. IEEE Trans. Instrum. Measurement, pp. 1–8 (2018)

  7. Hu, G.-H.: Automated defect detection in textured surfaces using optimal elliptical gabor filters. Optik - Int. J. Light Electron Opt. 126(14), 1331–1340 (2015)

    Article  Google Scholar 

  8. Lin, H., Li, B., Wang, X., Shu, Y., Niu, S.: Automated defect inspection of led chip using deep convolutional neural network. J. Intell. Manuf. 30, 1–10 (2019)

    Article  Google Scholar 

  9. Kim, J., Ko, J., Choi, H., Kim, H.: Printed circuit board defect detection using deep learning via a skip-connected convolutional autoencoder. Sensors (Basel, Switzerland) 21 (2021)

  10. Liu, Z., Liu, S., Li, C., Ding, S., Dong, Y.: Fabric defects detection based on ssd (2018)

  11. Xie, Q., Li, D., Xu, J., Yu, Z., Wang, J.: Automatic detection and classification of sewer defects via hierarchical deep learning. IEEE Trans. Autom. Sci. Eng. 16, 1836–1847 (2019)

    Article  Google Scholar 

  12. Lin, W.Y., Lin, C.Y., Chen, G., Hsu, C.Y.: Steel surface defects detection based on deep learning. In: Advances in physical ergonomics and human factors (2018)

  13. Qiu, Y., Tang, L., Li, B., Niu, S., Niu, T.: Uneven illumination surface defects inspection based on saliency detection and intrinsic image decomposition. IEEE Access 8, 190663–190676 (2020)

    Article  Google Scholar 

  14. Feng, X., Li, J., Hua, Z., Zhang, F.: Low-light image enhancement based on multi-illumination estimation. Appl. Intell. 51, 1–21

  15. Protopapadakis, E., Voulodimos, A., Doulamis, A., Doulamis, N., Stathaki, T.: Automatic crack detection for tunnel inspection using deep learning and heuristic image post-processing. Appl. Intell. 49 (2019)

  16. Du, X., Cheng, Y., Gu, Z.: Change detection: the framework of visual inspection system for railway plug defects. IEEE Access 8(99), 152161–152172 (2020)

    Article  Google Scholar 

  17. Wang, M., Kumar, S.S., Cheng, J.: Automated sewer pipe defect tracking in cctv videos based on defect detection and metric learning. Autom. Constr. 121, 103438 (2021)

    Article  Google Scholar 

  18. Jia, L., Chen, C., Xu, S., Shen, J.: Fabric defect inspection based on lattice segmentation and template statistics. Inf. Sci. 512, 964–984 (2020)

    Article  Google Scholar 

  19. Hu, W., Wang, T., Wang, Y., Chen, Z.Y., Huang, G.: Le-msfe-ddnet: a defect detection network based on low-light enhancement and multi-scale feature extraction. Vis. Comput., pp. 1–15 (2021)

  20. Wang, H., Chen, Z., Sun, L.: Image preprocessing methods to identify micro-cracks of road pavement. Opt. Photonics J. 03, 720–726 (2013)

    Article  Google Scholar 

  21. Jing, J., Chen, S.B., Li, P.: Fabric defect detection based on golden image subtraction. Color. Technol. 133, 26–39 (2017)

    Article  Google Scholar 

  22. Li, B., Wei, W., Ferreira, A., Tan, S.: Rest-net: diverse activation modules and parallel subnets-based cnn for spatial image steganalysis. IEEE Signal Process. Lett. 25, 650–654 (2018)

    Article  Google Scholar 

  23. Zhang, W., Xiong, Q., Shi, W., Chen, S.: Region saliency detection via multi-feature on absorbing markov chain. Vis. Comput. 32(3), 275–287 (2016)

    Article  Google Scholar 

  24. Shankar, K., Zhang, Y., Liu, Y., Wu, L., Chen, C.H.: Hyperparameter tuning deep learning for diabetic retinopathy fundus image classification. IEEE Access 8, 118164–118173 (2020)

    Article  Google Scholar 

  25. Perumal, S., Velmurugan, T.: Preprocessing by contrast enhancement techniques for medical images. Int. J. Pure Appl. Math. 118(18), 3681–3688 (2018)

    Google Scholar 

  26. Chen, S., Li, X.: On image preprocessing methods for underwater image matching. In: 2021 International Conference on Computer Communication and Artificial Intelligence (CCAI), pp. 36–40 (2021)

  27. Amiri, S.A., Hassanpour, H.: A preprocessing approach for image analysis using gamma correction. Int. J. Comput. Appl. 38 (2012)

  28. Haddad, B.M., Karam, L., Ye, J., Patel, N.S., Oberkönig, M.: Multi-feature sparse-based defect detection and classification in semiconductor units. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 754–758 (2016)

  29. Wang, B., Chen, S., Wang, J., Hu, X.: Residual feature pyramid networks for salient object detection. Vis. Comput. 36, 1897–1908 (2019)

    Article  Google Scholar 

  30. He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Trans. Instrum. Meas. 69(99), 1493–1504 (2019)

    Google Scholar 

  31. Cui, L., Jiang, X., Xu, M., Li, W., Lv, P., Zhou, B.: Sddnet: a fast and accurate network for surface defect detection. IEEE Trans. Instrum. Meas. 70, 1–13 (2021)

    Google Scholar 

  32. Wei, B., He, H., Hao, K., Gao, L., Tang, X.S.: Visual interaction networks: a novel bio-inspired computational model for image classification. Neural Netw. 130, 100–110 (2020)

    Article  Google Scholar 

  33. Wang, W., Shen, J.: Deep visual attention prediction. IEEE Trans. Image Process. 27, 2368–2378 (2018)

    Article  MathSciNet  Google Scholar 

  34. Cheng, X., Yu, J.: Retinanet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection. IEEE Trans. Instrum. Meas. 70, 1–11 (2021)

    Google Scholar 

  35. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015)

  36. Wu, S., Wu, Y., Cao, D., Zheng, C.: A fast button surface defect detection method based on Siamese network with imbalanced samples. Multimedia Tools Appl. 78, 34627–34648 (2019)

    Article  Google Scholar 

  37. Lu, S., Wang, H., Zhou, Z.: All-in-one multicategory ramp loss maximum margin of twin spheres support vector machine. Appl. Intell. 49, 1–14 (2019)

    Article  Google Scholar 

  38. Sohn, K.: Improved Deep Metric Learning with Multi-class n-pair Loss Objective. Curran Associates Inc (2016)

  39. Sun, S.j.: Self-attention enhanced cnns with average margin loss for chinese zero pronoun resolution. Appl. Intell. (2021)

  40. Zhao, D., Chen, C., Li, D.: Multi-stage attention and center triplet loss for person re-identication. Appl. Intell., pp. 1–13 (2021)

  41. Lin, T.Y., Goyal, P., Girshick, R.B., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. 42, 318–327 (2020)

    Article  Google Scholar 

  42. Wang, N., Zhao, X., Jiang, Y., Gao, Y.: Iterative metric learning for imbalance data classification (2018)

  43. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2017)

  44. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

  45. Park, T., Zhu, J.Y., Wang, O., Lu, J., Shechtman, E., Efros, A.A., Zhang, R.: Swapping autoencoder for deep image manipulation. arXiv:2007.00653 (2020)

  46. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv:1804.02767 (2018)

  47. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  48. Tian, Z., Shen, C., Chen, H., He, T.: Fcos: Fully convolutional one-stage object detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9626–9635 (2019)

  49. Feng, Z., Guo, L., Huang, D., Li, R.: Electrical insulator defects detection method based on yolov5. In: 2021 IEEE 10th Data Driven Control and Learning Systems Conference (DDCLS), pp. 979–984 (2021)

  50. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.: Ssd: single shot multibox detector, pp. 21–37 (2016)

  51. Luan, C., Cui, R., Sun, L., Lin, Z.: A siamese network utilizing image structural differences for cross-category defect detection. pp. 778–782 (2020). https://doi.org/10.1109/ICIP40778.2020.9191128

  52. Liu, Z., Wang, B., Li, C., Li, B., Liu, X.: Fabric defect detection algorithm based on convolution neural network and low-rank representation. In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pp. 465–470 (2017)

Download references

Acknowledgements

This work was supported in part by the R &D projects in key areas of Guangdong Province under Grant 2019B010153002 , the Science and technology research in key areas in Foshan under Grant 2020001006832, the Guangdong Key Areas R &D Program Project under Grant 2018B010109007, the Guangdong Provincial Key Laboratory of Cyber-Physical System under Grant 2020B1212060069, the National Natural Science Foundation of China Guangdong Joint Fund under Grant U1801263 and U2001201, the Guangzhou R &D Programme in Key Areas of Science and Technology Projects under Grant 202007040006, the Program of Marine Economy Development (Six Marine Industries) Special Foundation of Department of Natural Resources of Guangdong Province under Grant GDNGC [2020]056.

Author information

Authors and Affiliations

  1. School of Computers, Guangdong University of Technology, Guangzhou, 510006, China

    Ziyang Chen, Guoheng Huang & Ying Wang

  2. School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Junhao Qiu

  3. School of Physics and Optoelectronic Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Fan Yang

  4. Department of Computer and Information Science, University of Macau, Macau SAR, 999078, China

    Zhiwen Yu

  5. School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China

    Chi-Man Pun

  6. School of Information Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Wing-Kuen Ling

Authors
  1. Ziyang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoheng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junhao Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhiwen Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chi-Man Pun
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wing-Kuen Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoheng Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Huang, G., Wang, Y. et al. Bi-deformation-UNet: recombination of differential channels for printed surface defect detection. Vis Comput 39, 3995–4013 (2023). https://doi.org/10.1007/s00371-022-02554-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02554-7

Keywords

Search

Navigation