Skip to main content
SpringerLink
Log in

Camera module Lens blemish detection based on neural network interpretability

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Lens blemish detection is an important link in camera module production. Automatic blemish detection for camera module Lens is a challenging task, owing to sparse defect data, fast product update and low contrast between blemish and background. In this paper, A types of lens blemish detection models of camera module, named SA-LensNet, is developed using global average pooling (GAP) and Self-attention Mechanism, based on neural network visualization. The models developed are based on convolutional neural networks (CNN), and a class activation map (CAM) technique is applied to localize blemish regions without using region-level human annotations based on CNN classification network. The model has accuracy of 99% and recall of 98.7% in the module lenses classification (with and without blemish), localizing exact defect regions of blemish as well. Comparative experiments of several methods show that the proposed model has strong robustness and generalization ability for the detection of blemish.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1
Fig.2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Aghdam HH, Heravi EJ (2017) Guide to convolutional neural networks: a practical application to traffic-sign detection and classification. Springer

    Book  Google Scholar 

  2. Alexey Bochkovskiy (2020) Darknet: Open Source Neural Networks in Python, Available online: https://github.com/AlexeyAB/darknet

  3. Bang HT, Park S, Jeon H (2020) Defect identification in composite materials via thermography and deep learning techniques. Compos Struct 246:112405

    Article  Google Scholar 

  4. Bochkovskiy A, Wang CY, Liao H (2020) YOLOv4: Optimal Speed and Accuracy of Object Detection. In: Conference Record of the 2020 IEEE Conference on Computer Vision and Pattern Recognition, USA

  5. Cao Y, Xu J, Lin S et al (2019) GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond, arXiv:1904.11492v1 [cs.CV] 25 Apr 2019

  6. Cha YJ, Choi W, Suh G et al (2018) Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comp Aided Civ Infrastruct Eng 33(9):731–747

    Article  Google Scholar 

  7. Chang F, Liu M, Dong M et al (2019) A mobile vision inspection system for tiny defect detection on smooth car-body surfaces based on deep ensemble learning. Meas Sci Technol 125905:1–10

    Google Scholar 

  8. Chang F, Liu M, Dong M et al (2019) A mobile vision inspection system for tiny defect detection of smooth car-body surface based on deep ensemble learning. Meas Sci Technol. https://doi.org/10.1088/1361-6501/ab1467

    Article  Google Scholar 

  9. Chen S, Hu T, Liu G, Pu Z, Li M et al (2008) Defect classification algorithm for IC photomask based on PCA and SVM. In: 2008 Congress on Image and Signal Processing. IEEE Press, Sanya, 1: 491–496

  10. Chen QL, Wang DX, Lin JG et al (2020) Surface defect detection of stamping and grinding flat parts. Forg Stamp Technol 6:168–174

    Google Scholar 

  11. Chen SH, Kang CH, Perng CB (2020) Detecting and measuring defects in wafer die using GAN and YOLOv3. Appl Sci 10(8725):8725

    Article  Google Scholar 

  12. Chu M, Liu X, Gong R et al (2018) Multi-class classification method using twin support vector machines with multi-information for steel surface defects. Chemom Intell Lab Syst 176:108–118

    Article  Google Scholar 

  13. Di H, Ke X, Peng Z et al (2019) Surface defect classification of steels with a new semi-supervised learning method. Opt Lasers Eng 117:40–48

    Article  Google Scholar 

  14. Fu XW (2014) Research of the Key Technology on Detection System of Lens Defect Based on Image Recognition, Dissertation, Shenyang Ligong University

  15. Fu J, Liu J, Tian H (2019) Dual Attention Network for Scene Segmentation. arXiv:1809.02983v4 [cs.CV] 21 Apr 2019

  16. Gao Y, Gao L, Li X et al (2020) A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robot Comput Integr Manuf 61:101825

    Article  Google Scholar 

  17. Girshick R (2015) Fast R-CNN. In: Conference Record of the 2015 IEEE International Conference on Computer Vision. Santiago, Chile

  18. Global CMOS Image Sensor Market Report 2018, http://www.chlueresearch.com/global-cmos-image-sensor-market-report-2018/

  19. He K, Zhang X, Ren S et al (2014) Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  20. Huang JX, Li D, Ye F et al (2010) Detection of surface defection of solder on flexible printed circuit. Guangxue Jingmi Gongcheng/Opt Precis Eng 18(11):2443–2453

    Google Scholar 

  21. Ketkar N (2017) Convolutional Neural Networks. In: Deep Learning with Python. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-2766-4_5

  22. Kingam DP, Ba L (2015) Adam: a method for stochastic optimization[C]//Published as a conference paper at the 3rd International Conference for Learning Representations. San Diego

  23. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In NIPS’2012

  24. Li A, Yang T, Zhang Y (2007) Preliminary research of surface defect recognition based on machine vision. J Chongqing Univ Posts Telecommun (Nat Sci) 19(4):442–445

    Google Scholar 

  25. Lin H, Li B, Wang X et al (2018) Automated defect inspection of LED chip using deep convolutional neural network. J Intell Manuf 30(6):2525–2534

    Article  Google Scholar 

  26. Liu W, Anguelov D, Erhan D, et al (2016) SSD: single shot multibox detector[M]//Computer Vision-ECCV 2016. Cham: 21–37

  27. Long J, Shelhamer E, Darrell T (2014) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Google Scholar 

  28. Modhej M, Bastanfard A, Teshnehlab M et al (2020) Pattern Separation Network Based on the Hippocampus Activity for Handwritten Recognition. IEEE Access 8:212803–212817

    Article  Google Scholar 

  29. Napoletano P, Piccoli F, Schettini R (2018) Anomaly detection in nanofibrous materials by CNN-based self-similarity. Sensors 18(1):209

    Article  Google Scholar 

  30. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  31. Park JK, Kwon BK, Park JH et al (2016) Machine learning-based imaging system for surface defect inspection. Int J Precis Eng Manuf Green Technol 3(3):303–310

    Article  Google Scholar 

  32. Peng C, Zhang X, Yu G et al (2017) Large Kernel Matters-Improve Semantic Segmentation by Global Convolutional Network, arXiv:1703.02719v1 [cs.CV] 8 Mar 2017

  33. Perkins WA (1986) INSPECTOR: a computer vision system which learn to inspect part. IEEE Trans PAMI 8(5):584–592

    Google Scholar 

  34. Pierre S, David E, Zhang X et al (2014) OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks. arXiv:1312.6229v4 [cs.CV] 24 Feb 2014

  35. Redmon J, Farhadi A (2018) YOLOv3: An Incremental Improvement. In: Conference Record of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA

  36. Redmon J, Divvala S, Girshick R et al (2016) You Only Look Once: Unified, Real-Time Object Detection. In: Conference Record of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA. 779–788

  37. Ren S, He K, Girshick R et al (2017) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  38. Shi XK, Hua M, Cheung EHM et al (2004) Automatic recognition and evaluation of micro-contaminant particles on ultra-smooth optical substrates using image analysis method. Opt Lasers Eng 41(6):901–917

    Article  Google Scholar 

  39. Sun J, Wang P, Luo YK et al (2019) Surface Defects Detection Based on Adaptive Multiscale Image Collection and Convolutional Neural Networks. IEEE Trans Instrum Meas 68(12):4787–4797

    Article  Google Scholar 

  40. Szegedy C, Ioffe S, Vanhoucke V (2017) Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. In: Conference Record of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-2017). San Francisco

  41. Wang Z, Zhu D (2019) An accurate detection method for surface defects of complex components based on support vector machine and spreading algorithm. Measurement 147(8):1–10

    Google Scholar 

  42. Wang H, Zhang Y, Shen HH et al (2017) Review of image enhancement algorithms. Chin J Opt 10(4):438–448

    Article  Google Scholar 

  43. Wang T, Chen Y, Qiao M et al (2018) A fast and robust convolutional neural network-based defect detection model in product quality control. Int J Adv Manuf Technol 94:3465–3471

    Article  Google Scholar 

  44. Wang Y, Wu ZH, Duan XY et al (2018) Design of Gear Defect Detection System Based on Machine Vision. IOP Conf Ser Earth Environ Sci 108:022025

    Article  Google Scholar 

  45. Xi J, Shentu L, Hu J et al (2017) Automated surface inspection for steel products using computer vision approach. Appl Opt 56(2):184

    Article  Google Scholar 

  46. Xu GS (2013) Fabric defect automated detection technology. Appl Mech Mater 325–326:1431–1434

    Article  Google Scholar 

  47. Yu J, Lu X (2016) Wafer map defect detection and recognition using joint local and nonlocal linear discriminant analysis. IEEE Trans Semicond Manuf 29(1):33–43

    Article  Google Scholar 

  48. Yu Z, Wu X, Gu X (2017) Fully convolutional networks for surface defect inspection in industrial environment, ICVS 2017. LNCS 10528:417–426

    Google Scholar 

  49. Yun JP, Kim D, Kim KH et al (2017) Vision-based surface defect inspection for thick steel plates. Opt Eng 56(5):053108

    Article  Google Scholar 

  50. Zhai D, Ding YN, Li T (2014) An Overview of Non-local Image Denoising. Electro Sci Technol 27(1):160–162

    Google Scholar 

  51. Zhang N, Donahue J, Girshick R et al (2014) Part-based R-CNNs for fine-grained category detection. Eur Conf Comput Vis 8689(1):834–849

    Google Scholar 

  52. Zhou X, Wang YN, Zhu Q et al (2016) Research on defect detection method for bottle mouth based on machine vision. J Electron Meas Inst 30(5):702–713

    Google Scholar 

  53. Zhou X, Wang Y, Zhu Q et al (2019) A surface defect detection framework for glass bottle bottom using visual attention model and wavelet transform. IEEE Trans Ind Inf 16(4):2189–2201

    Article  Google Scholar 

  54. Zeiler MD, Fergus R (2014) Visualizing and Understanding Convolutional Networks [OL]. Computer Vision-ECCV 2014. arxiv:1311.2901

  55. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. IEEE Conf Comput Vis Pattern Recogn 2016:2921–2929. https://doi.org/10.1109/CVPR.2016.319

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the support of this research by the Science and Technology Research Program of Chongqing Municipal Education Commission (No. KJQN201801213), and the Science and Technology Research Program of Chongqing Municipal Education Commission (Grant No. KJZD-M201901201).

Author information

Authors and Affiliations

  1. Key Laboratory of Electronics and Information, Chongqing Three Gorges University, Wanzhou, Chongqing, China

    Mei Yang, Jin Wu & Xiaowei Niu

Authors
  1. Mei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaowei Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mei Yang.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, M., Wu, J. & Niu, X. Camera module Lens blemish detection based on neural network interpretability. Multimed Tools Appl 81, 5373–5388 (2022). https://doi.org/10.1007/s11042-021-11716-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11716-z

Keywords

Search

Navigation