Skip to main content
SpringerLink
Log in

Defect detection on new samples with siamese defect-aware attention network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep learning-based methods have recently shown great promise in the defect detection task. However, current methods rely on large-scale annotated data and are unable to adapt a trained deep learning model to new samples that were not observed during training. To address this issue, we propose a new siamese defect-aware attention network (SDANet) with a template comparison detection strategy that improves the defect detection technique for matching new samples without rapidly collecting new data and retraining the model. In SDANet, the siamese feature pyramid network is used to extract multi-scale features from input and template images, the defect-aware attention module is proposed to obtain inconsistency between input and template features and use it to enhance abnormality in input image features, and the self-calibration module is developed to calibrate the alignment error between the input and template features. SDANet can be used as a plug-in module to enable most existing mainstream detection algorithms to detect defects using not only the features of defects, but also the inconsistency between features of the inspected image and the template image. Extensive experiments on two publicly available industrial defect detection benchmarks highlight the effectiveness of our method. SDANet can be seamlessly integrated into mainstream detection methods and improve the mAP of mainstream detection algorithms on unseen samples by 12% on average which outperforms current state-of-the-art method by 7.7%. It can also improve the performance in seen samples by 4.3% on average. SDANet can be used in general defect detection applications of industrial manufacturing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. These datasets are available on the https://tianchi.aliyun.com/dataset/

References

  1. Jia L, Chen C, Liang J, Hou Z (2017) Fabric defect inspection based on lattice segmentation and gabor filtering. Neurocomputing 238:84–102

    Article  Google Scholar 

  2. Yapi D, Allili M S, Baaziz N (2017) Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Trans Autom Sci Eng 15(3):1014–1026

    Article  Google Scholar 

  3. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  4. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99

    Google Scholar 

  5. Cheng X, Yu J (2020) Retinanet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection. IEEE Trans Instrum Meas 70:1–11

    Google Scholar 

  6. Cai Z, Vasconcelos N (2018) Cascade r-cnn: Delving into high quality object detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 6154–6162

  7. Qiao S, Chen L-C, Yuille A (2021) Detectors: Detecting objects with recursive feature pyramid and switchable atrous convolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 10213–10224

  8. Wu J, Le J, Xiao Z, Zhang F, Geng L, Liu Y, Wang W (2021) Automatic fabric defect detection using a wide-and-light network. Appl Intell 51(7):4945–4961

  9. Zheng X, Chen J, Wang H, Zheng S, Kong Y (2021) A deep learning-based approach for the automated surface inspection of copper clad laminate images. Appl Intell 51(3):1262–1279

    Article  Google Scholar 

  10. Tian R, Shi H, Guo B, Zhu L (2021) Multi-scale object detection for high-speed railway clearance intrusion. Appl Intell, pp 1–16

  11. Dai Z, Yi J, Zhang Y, Zhou B, He L (2020) Fast and accurate cable detection using cnn. Appl Intell 50(12):4688–4707

    Article  Google Scholar 

  12. Li Y, Li J (2021) An end-to-end defect detection method for mobile phone light guide plate via multitask learning. IEEE Trans Instrum Meas 70:1–13

    Google Scholar 

  13. Czimmermann T, Ciuti G, Milazzo M, Chiurazzi M, Roccella S, Oddo C. M, Dario P (2020) Visual-based defect detection and classification approaches for industrial applications—a survey. Sensors 20(5):1459

    Article  Google Scholar 

  14. Wang J, Ma Y, Zhang L, Gao R X, Wu D (2018) Deep learning for smart manufacturing: Methods and applications. J Manuf Syst 48:144–156

    Article  Google Scholar 

  15. Božič J, Tabernik D, Skočaj D (2021) Mixed supervision for surface-defect detection: From weakly to fully supervised learning. Comput Ind 103459:129

    Google Scholar 

  16. Ren R, Hung T, Tan KC (2017) A generic deep-learning-based approach for automated surface inspection. IEEE Trans Cybern 48(3):929–940

    Article  Google Scholar 

  17. Lin X, Wang X, Li L (2020) Intelligent detection of edge inconsistency for mechanical workpiece by machine vision with deep learning and variable geometry model. Appl Intell 50(7):2105–2119

    Article  Google Scholar 

  18. Okaro IA, Jayasinghe S, Sutcliffe C, Black K, Paoletti P, Green PL (2019) Automatic fault detection for laser powder-bed fusion using semi-supervised machine learning. Addit Manuf 27:42–53

    Google Scholar 

  19. Qi G-J, Luo J (2020) Small data challenges in big data era: A survey of recent progress on unsupervised and semi-supervised methods. IEEE Transactions on Pattern Analysis and Machine Intelligence

  20. Pal SK, Pramanik A, Maiti J, Mitra P (2021) Deep learning in multi-object detection and tracking: state of the art. Appl Intell 51(9):6400–6429

    Article  Google Scholar 

  21. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271

  22. Farhadi A, Redmon J (2018) Yolov3: An incremental improvement. In: Computer vision and pattern recognition, vol 1804. Springer, Berlin/Heidelberg, Germany

  23. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision, Springer, pp 21–37

  24. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  25. Bochkovskiy A, Wang C-Y, Liao H-Y M (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934

  26. Tabernik D, Šela S, Skvarč J, Skočaj D (2020) Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 31(3):759–776

    Article  Google Scholar 

  27. Wang Y, Liu M, Zheng P, Yang H, Zou J (2020) A smart surface inspection system using faster r-cnn in cloud-edge computing environment. Adv Eng Inform 101037:43

    Google Scholar 

  28. Cui L, Jiang X, Xu M, Li W, Lv P, Zhou B (2021) Sddnet: A fast and accurate network for surface defect detection. IEEE Trans Instrum Meas 70:1–13

    Google Scholar 

  29. Luo J, Yang Z, Li S, Wu Y (2021) Fpcb surface defect detection: A decoupled two-stage object detection framework. IEEE Trans Instrum Meas 70:1–11

    Google Scholar 

  30. Li Y, Huang H, Xie Q, Yao L, Chen Q (2018) Research on a surface defect detection algorithm based on mobilenet-ssd. Appl Sci 8(9):1678

    Article  Google Scholar 

  31. Liu K, Li Y, Yang J, Liu Y, Yao Y (2020) Generative principal component thermography for enhanced defect detection and analysis. IEEE Trans Instrum Meas 69(10):8261–8269

    Google Scholar 

  32. Yin X, Chen Y, Bouferguene A, Zaman H, Al-Hussein M, Kurach L (2020) A deep learning-based framework for an automated defect detection system for sewer pipes. Autom Construct 109:102967

    Article  Google Scholar 

  33. Jing J, Zhuo D, Zhang H, Liang Y, Zheng M (2020) Fabric defect detection using the improved yolov3 model. J Eng Fibers Fabr 15:1558925020908268

    Google Scholar 

  34. Tulbure A-A, Tulbure A-A, Dulf E-H (2022) A review on modern defect detection models using dcnns–deep convolutional neural networks. J Adv Res 35:33–48

    Article  Google Scholar 

  35. Dong H, Song K, He Y, Xu J, Yan Y, Meng Q (2019) Pga-net: Pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Trans Industr Inform 16(12):7448–7458

    Article  Google Scholar 

  36. Tu Y, Ling Z, Guo S, Wen H (2020) An accurate and real-time surface defects detection method for sawn lumber. IEEE Trans Instrum Meas 70:1–11

    Google Scholar 

  37. Li F, Xi Q (2021) Defectnet: Toward fast and effective defect detection. IEEE Trans Instrum Meas 70:1–9

    Google Scholar 

  38. Li D, Li Y, Xie Q, Wu Y, Yu Z, Wang J (2021) Tiny defect detection in high-resolution aero-engine blade images via a coarse-to-fine framework. IEEE Trans Instrum Meas 70:1–12

    Google Scholar 

  39. Huang Y, Jing J, Wang Z (2021) Fabric defect segmentation method based on deep learning. IEEE Trans Instrum Meas 70:1–15

    Google Scholar 

  40. Xing J, Jia M (2021) A convolutional neural network-based method for workpiece surface defect detection. Measurement 176:109185

    Article  Google Scholar 

  41. Xu Y, Li D, Xie Q, Wu Q, Wang J (2021) Automatic defect detection and segmentation of tunnel surface using modified mask r-cnn. Measurement 178:109316

    Article  Google Scholar 

  42. Zhang J, Su H, Zou W, Gong X, Zhang Z, Shen F (2021) Cadn: a weakly supervised learning-based category-aware object detection network for surface defect detection. Pattern Recogn 109:107571

    Article  Google Scholar 

  43. Zhang G, Pan Y, Zhang L (2021) Semi-supervised learning with gan for automatic defect detection from images. Autom Constr 128:103764

    Article  Google Scholar 

  44. Kang B, Liu Z, Wang X, Yu F, Feng J, Darrell T (2019) Few-shot object detection via feature reweighting. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 8420–8429

  45. Wang X, Huang T, Gonzalez J, Darrell T, Yu F (2020) Frustratingly simple few-shot object detection. In: International conference on machine learning, pp 9919–9928. PMLR

  46. Zheng Y, Wu J, Qin Y, Zhang F, Cui L (2021) Zero-shot instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2593–2602

  47. Wang Q, Zhang X, Chen G, Dai F, Gong Y, Zhu K (2018) Change detection based on faster r-cnn for high-resolution remote sensing images. Remote Sens Lett 9(10):923–932

    Article  Google Scholar 

  48. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  49. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  50. Tianchi: Smart Diagnosis of Cloth Flaw Dataset (2020). https://tianchi.aliyun.com/dataset/dataDetail?dataId=79336. Accessed 2021-04

  51. Tianchi: Smart Diagnosis of Tile Flaw Dataset (2020). https://tianchi.aliyun.com/dataset/dataDetail?dataId=110088. Accessed 2021-04

  52. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  53. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037

    Google Scholar 

Download references

Acknowledgments

The authors would like to acknowledge financial support from the National Natural Science Foundation of China (NSFC) under Grant No. 61672498.

Funding

The National Natural Science Foundation of China (NSFC) under Grant No. 61672498.

Author information

Authors and Affiliations

  1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    Ye Zheng & Li Cui

  2. University of Chinese Academy of Sciences, Beijing, 100190, China

    Ye Zheng

Authors
  1. Ye Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Cui.

Ethics declarations

Conflicts of interest/ Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, Y., Cui, L. Defect detection on new samples with siamese defect-aware attention network. Appl Intell 53, 4563–4578 (2023). https://doi.org/10.1007/s10489-022-03595-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03595-0

Keywords

Search

Navigation