Skip to main content
SpringerLink
Log in

Defect detection of photovoltaic glass based on level set map

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Defect detection of photovoltaic (PV) glass products is a challenge due to its complex optical properties and lack of defective samples. Aiming at this problem, a multi-task framework, in which an auxiliary semantic segmentation task based on the generative adversarial network assists the main defective classification task, is proposed. The auxiliary task guides the feature extractor to focus on the contour of the glass. Level set map (LSM), a new representation of contour which is the integration of a mask, is introduced into the framework to further improve the performance. Unlike level set loss, which is an indirect contour constraint on the segmentation loss in the form of a regular term, LSM is a direct annotation parallel with mask. Recovering LSM and mask simultaneously can affect each other positively. Two synthetic PV glass datasets, named SynSmall and SynBig, and two real-world PV glass datasets, named Mask3 and Defect3, are established to validate the proposed method. Two groups of experiments on the glass datasets are designed to inspect the feasibility and performance of the proposed framework from different aspects. Furthermore, a group of experiments involving separate segmentation tasks on SynSmall and Mask3, together with two public datasets TOMNet and Human, are executed to illustrate the performance of LSM. Experimental results show that the proposed framework can improve the accuracy of PV glass defective detection task significantly and LSM can improve the segmentation accuracy by filtering isolated wrongly recovered regions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Yang X, Pang G, Yung N (2005) Robust fabric defect detection and classification using multiple adaptive wavelets. Vis Image Signal Process 152:715–723

    Article  Google Scholar 

  2. Lin S, He Z, Sun L (2019) Defect enhancement generative adversarial network for enlarging data set of microcrack defect. IEEE Access 7:148413–148423

    Article  Google Scholar 

  3. Liu K, Li A, Wen X, Chen H, Yang P (2019) Steel surface defect detection using GAN and one-class classifier. In: International conference on automation and computing, pp 5–7. Chinese Automation and Computing Society in the UK - CACSUK

  4. Lian J, Jia W, Zareapoor M, Zheng Y, Luo R (2020) Deep-learning-based small surface defect detection via an exaggerated local variation-based generative adversarial network. IEEE Trans Ind Inf 16(2):1343–1351

    Article  Google Scholar 

  5. Bing H, Wang J (2020) Detection of PCB surface defects with improved faster-RCNN and feature pyramid network. IEEE Access 8:108335–108345

    Article  Google Scholar 

  6. Feng Y, Chen Z, Wang D, Chen J, Feng Z (2020) DeepWelding: a deep learning enhanced approach to GTAW using multisource sensing images. IEEE Trans Ind Inf 16(1):465–474

    Article  Google Scholar 

  7. Ming W, Shen F, Li X, Zhang Z, Jinguang D, Chen Z, Cao Y (2020) A comprehensive review of defect detection in 3C glass components. Measurement 158(1):1–20

    Google Scholar 

  8. Zhang C, Chen X, Yunjie X, Wei Z, Zhou S (2020) Surface defect detection system of glass. Packag Eng 41(13):216–222

    Google Scholar 

  9. Jian C, Gao J, Ao Y (2017) Automatic surface defect detection for mobile phone screen glass based on machine vision. Appl Soft Comput J 52(3):348–358

    Article  Google Scholar 

  10. Wang C, Huang Y, Zhang X, Shenglin L (2020) A Method of scratch defects detection for curved glass based on machine vision. Tech Autom Appl 39(1):134–139

    Google Scholar 

  11. Zhou X, Wang Y, Zhu Q, Mao J, Xiao C, Xiao L, Zhang H (2020) A surface defect detection framework for glass bottle bottom using visual attention model and wavelet transform. IEEE Trans Ind Inf 16(4):2189–2201

    Article  Google Scholar 

  12. Yao HB, Ping J, Ma GD, Li LW, Gu JN (2013) The system research on automatic defect detection of glasses. Appl Mech Mater 437:362–365

    Article  Google Scholar 

  13. Rosli NS, Fauadi MHFM, Awang NF, Noor AZM (2018) Vision-based defects detection for glass production based on improved image processing method. J Adv Manuf Technol 12(203–212):1–224

    Google Scholar 

  14. Xiong H, Fan C, Zhao S, Ying Yu (2020) Glass surface defect detection method based on multiscale convolution neural network. Comput Integr Manuf Syst 26(4):900–909

    Google Scholar 

  15. Zhang J, Su H, Zou W, Gong X, Zhang Z, Shen F (2020) CADN: a weakly supervised learning-based category-aware object detection network for surface defect detection. Pattern Recognit 109:107571

    Article  Google Scholar 

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE international conference on computer vision, pp 770–778

  17. Tan M, Le QV (2019) EfficientNet: Rethinking model scaling for convolutional neural networks. arXiv:1905.11946, pp 1–10

  18. Poudel RPK, Liwicki S, Cipolla R (2019) Fast-SCNN: fast semantic segmentation network. arXiv:1902.04502, pp 1–17

  19. He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  20. Ren S, He K, Girshick R, Jian S (2015) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  21. Yang Z, Xie X, Zhan Q, Liu G, Cai Q (2019) A neural-network-based framework for cigarette laser code and identification. Neural Comput Appl 32(15):11597–11606

    Article  Google Scholar 

  22. Bochkovskiy A, Wang CY, Mark Liao HY (2020). YOLOv4: optimal speed and accuracy of object detection. arXiv: 2004.10934v1, pp 1–17

  23. Kirillov A, He K, Girshick R, Rother C, Dollar P (2019) Panoptic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 9396–9405

  24. Isola P, Zhu JY, Zhou T, Efros AA (2017). Image-to-image translation with conditional adversarial networks. In: IEEE international conference on computer vision, pp 5967–5976

  25. Kim Y, Kim S, Kim T, Kim C (2019) CNN-based semantic segmentation using level set loss. In: 2019 IEEE winter conference on applications of computer vision (WACV), pp 1752–1760

  26. Peng S, Jiang W, Pi H, Li X, Bao H, Zhou X (2020) Deep snake for real-time instance segmentation. arXiv:2001.01629, pp 1–10

  27. Chen G, Han K, Wong KKY (2018) TOM-Net: learning transparent object matting from a single image. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 9233–9241

  28. Liang Y, Huang H, Cai Z, Hao Z, Feng F (2020) Survey of natural image matting. Appl Res Comput 38(5):1–10

    Google Scholar 

  29. Goodfellow IJ, Pouget-abadie J, Mirza M, Xu B, Warde-farley D (2014) Generative adversarial nets. arXiv: 1406.2661v1, pp 1–9

  30. Mirza M, Osindero S (2014) Conditional generative adversarial nets.pdf. arXiv: 1411.1784v1, pp 1–7

  31. Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier gans. In: International conference on machine learning, ICML 2017, vol 6, pp 4043–4055

  32. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein GAN. arXiv: 1701.07875v3, pp 1–32

  33. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A (2017) Improved training of Wasserstein GANs.pdf. arXiv: 1704.00028v3, pp 1–20

  34. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA (2018) Generative adversarial networks: an overview. IEEE Signal Process Mag 35(1):53–65

    Article  Google Scholar 

  35. Hong Y, Hwang U, Yoo J, Yoon S (2019) How generative adversarial networks and their variants work: an overview. ACM Comput Surv 52(1):1–41

    Article  Google Scholar 

  36. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. arXiv: 1505.04597, pp 1–8

  37. Wang T, Liu M, Zhu J, Liu G, Tao A, Kautz J, Catanzaro B (2018) Video-to-video synthesis. arXiv: 1808.06601v2, pp 1–14

  38. Wang TC, Liu MY, Zhu JY, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional GANs. In: IEEE international conference on computer vision, pp 8798–8807

  39. Zhu JY, Park T, Isola P, Efros AA (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE international conference on computer vision, pp 2242–2251

  40. Yi Z, Zhang H, Tan P, Gong M (2017) DualGAN: unsupervised dual learning for image-to-image translation. In: IEEE international conference on computer vision, pp 2868–2876

  41. Kim T, Cha M, Kim H, Kwon J, Jiwon L (2017) Learning to discover cross-domain relations with generative adversarial networks. arXiv: 1703.05192, pp 1–10

  42. Li B, Yuan X, Shi M (2020) Synthetic data generation based on local-foreground generative adversarial networks for surface defect detection. J Electron Imaging 29(1):1–14

    Article  Google Scholar 

  43. Xinyi L, Junhui M, Haodong Z, Boyu Z, Juntong X (2020) A learning-based approach for surface defect detection using small image datasets. Neurocomputing 408:112–120

    Article  Google Scholar 

  44. Liu J, Wang C, Hai S, Bo D, Tao D (2020) Multistage GAN for fabric defect detection. IEEE Trans Image Process 29:3388–3400

    Article  Google Scholar 

  45. Lyu Y, Han Z, Zhong J, Li C, Liu Z (2020) A generic anomaly detection of catenary support components based on generative adversarial networks. IEEE Trans Instrum Meas 69(5):2439–2448

    Article  Google Scholar 

  46. Wang J, Li X, Yang J (2018) Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: IEEE international conference on computer vision, pp 1788–1797

  47. Bolya D, Zhou C, Xiao F, Lee Y J (2019) YOLACT: Real-time instance segmentation. arXiv:1904.02689, pp 1–12

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (62002053), Natural Science Foundation of Guangdong Province (2021A1515011866), Guangdong Basic and Applied Basic Research Projects (2019A1515111082), Social Welfare Major Project of Zhongshan (2019B2010, 2019B2011, 420S36), Achievement Cultivation Project of Zhongshan Industrial Technology Research Institute (419N26), Science and Technology Foundation of Guangdong Province (2021A0101180005), and Young Innovative Talents Project of Education Department of Guangdong Province (2018KQNCX337, 2019KQNCX186).

Author information

Authors and Affiliations

  1. University of Electronic Science and Technology of China, Zhongshan Institute, Zhongshan, 528402, China

    Shuai Dong, Chen Chen, Yihui Liang & Kun Zou

  2. School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics, Chengdu, 611130, China

    Guisong Liu

Authors
  1. Shuai Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yihui Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kun Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guisong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guisong Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Tables 9 and 10.

Table 9 Comparison of augmentation methods for separate segmentation on glass datasets
Full size table
Table 10 Accuracy of \(\widetilde{\varvec{Y}}_M\) with different auxiliary strength and frequency
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, S., Chen, C., Liang, Y. et al. Defect detection of photovoltaic glass based on level set map. Neural Comput & Applic 34, 10691–10705 (2022). https://doi.org/10.1007/s00521-022-07005-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07005-x

Keywords

Search

Navigation