Skip to main content
SpringerLink
Log in

Deeplab-YOLO: a method for detecting hot-spot defects in infrared image PV panels by combining segmentation and detection

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Aiming at the problem of difficult operation and maintenance of PV power plants in complex backgrounds and combined with image processing technology, a method for detecting hot spot defects in infrared image PV panels that combines segmentation and detection, Deeplab-YOLO, is proposed. In the PV panel segmentation stage, MobileNetV2 was introduced into the Deeplabv3+ model. Empty convolution in the atrous spatial pyramid pooling (ASPP) structure was improved, and established a relationship between layer-level features, the CBAM attention mechanism was combined, which achieved fast and accurate segment of PV panels and avoided false detection of hot spots. In the hot-spot recognition stage, a lightweight MobileNetV3 network was designed to replace the YOLO v5 backbone network, a small defect prediction head was added, and EIOU was used as a loss function, which improved the speed and accuracy of hot-spot detection and enhanced the performance of the YOLO v5 model. The experimental results show that the optimized Deeplabv3+ model and YOLO v5 model improve the accuracy of segmenting PV panels in images and identifying hot-spot defects by 2.61% and 0.7%, respectively, compared with the original model. This proposed method can accurately segment the PV panels and then identify different sizes of hot-spot defects on the PV panels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

Data will be made available on reasonable request.

References

  1. Basit, B.A., Jung, J.-W.: Recent developments and future research recommendations of control strategies for wind and solar pv energy systems. Energy Rep. 8, 14318–14346 (2022)

    Article  Google Scholar 

  2. Lukas, B., Mathis, H., Claudia, B.-L., Tobias, P., Jens, H., Christoph, B., Andreas, M., Ian, M.P.: Anomaly detection in ir images of pv modules using supervised contrastive learning. Progr. Photovolt. Res. Appl. 30(6), 597–614 (2022)

    Article  Google Scholar 

  3. Bolun, D., Yang, R., He, Y., Wang, F., Huang, S.: Nondestructive inspection, testing and evaluation for si-based, thin film and multi-junction solar cells: an overview. Renew. Sustain. Energy Rev. 78, 1117–1151 (2017)

    Article  Google Scholar 

  4. Vijithananda, S.M., Jayatilake, M.L., Hewavithana, B., Gonçalves, T., Rato, L.M., Weerakoon, B.S., Kalupahana, T.D., Silva, A.D., Dissanayake, K.D.: Feature extraction from mri adc images for brain tumor classification using machine learning techniques. Biomed. Eng. Online 21(1), 52 (2022)

    Article  Google Scholar 

  5. Firuzi, K., Vakilian, M., Phung, B.T., Blackburn, T.R.: Partial discharges pattern recognition of transformer defect model by lbp & hog features. IEEE Trans. Power Deliv. 34(2), 542–550 (2018)

    Article  Google Scholar 

  6. Luo, Q., Sun, Y., Li, P., Oluyomi, S., Tian, L., He, Y.: Generalized completed local binary patterns for time-efficient steel surface defect classification. IEEE Trans. Instrum. Meas. 68(3), 667–679 (2018)

    Article  Google Scholar 

  7. Wang, J., Zhao, B., Yao, X.: Pv abnormal shading detection based on convolutional neural network. In: 2020 Chinese Control And Decision Conference (CCDC), pp. 1580–1583. IEEE (2020)

  8. Guo, S., Wang, Z., Lou, Y., Li, X., Lin, H.: Detection method of photovoltaic panel defect based on improved mask r-cnn. J. Internet Technol. 23(2), 397–406 (2022)

    Article  Google Scholar 

  9. Zhang, M., Yin, L.: Solar cell surface defect detection based on improved yolo v5. IEEE Access 10, 80804–80815 (2022)

    Article  Google Scholar 

  10. Ying, Y., Qi, Y., Rong, L., Men, H., Joo, Y.H.: Anchor points based accurate fault locating in large-scale photovoltaic plants via aerial infrared videos. IEEE J. Photovolt. 12(1), 437–443 (2021)

    Article  Google Scholar 

  11. Binyi, S., Chen, H., Zhu, Y., Liu, W., Liu, K.: Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor. IEEE Trans. Instrum. Meas. 68(12), 4675–4688 (2019)

    Article  Google Scholar 

  12. Binyi, S., Chen, H., Chen, P., Bian, G., Liu, K., Liu, W.: Deep learning-based solar-cell manufacturing defect detection with complementary attention network. IEEE Trans. Ind. Inform. 17(6), 4084–4095 (2020)

    Google Scholar 

  13. Binyi, S., Chen, H., Liu, K., Liu, W.: Rcag-net: residual channelwise attention gate network for hot spot defect detection of photovoltaic farms. IEEE Trans. Instrum. Meas. 70, 1–14 (2021)

    Google Scholar 

  14. Zhao, Q., Liu, B., Lyu, S., Wang, C., Zhang, H.: Tph-yolov5++: boosting object detection on drone-captured scenarios with cross-layer asymmetric transformer. Remote Sens. 15(6), 1687 (2023)

    Article  Google Scholar 

  15. Kim, J.-H., Kim, N., Park, Y.W., Won, C.S.: Object detection and classification based on yolo-v5 with improved maritime dataset. J. Mar. Sci. Eng. 10(3), 377 (2022)

    Article  Google Scholar 

  16. Xiao, D., Xie, F.T., Gao, Y., Li, Z.N., Xie, H.F.: A detection method of spangle defects on zinc-coated steel surfaces based on improved YOLO-v5. Int J Adv Manuf Technol. 128, 937–951 (2023)

    Article  Google Scholar 

  17. Sun, W., Wang, R.: Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with dsm. IEEE Geosci. Remote Sens. Lett. 15(3), 474–478 (2018)

    Article  Google Scholar 

  18. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  19. Niu, Z., Liu, W., Zhao, J., Jiang, G.: Deeplab-based spatial feature extraction for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 16(2), 251–255 (2018)

    Article  Google Scholar 

  20. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  21. Gurita, A., Mocanu, I.G.: Image segmentation using encoder-decoder with deformable convolutions. Sensors 21(5), 1570 (2021)

    Article  Google Scholar 

  22. Baheti, B., Innani, S., Gajre, S., Talbar, S.: Semantic scene segmentation in unstructured environment with modified deeplabv3+. Pattern Recogn. Lett. 138, 223–229 (2020)

    Article  Google Scholar 

  23. Xi, D., Qin, Y., Wang, Z.: Attention deeplabv3 model and its application into gear pitting measurement. J. Intell. Fuzzy Syst. 42(4), 3107–3120 (2022)

    Article  Google Scholar 

  24. Kuanqi, G., Yutong, L., Yuwei, Z., Lielie, Q., Nannan, Z., Yingli, C.: Photovoltaic hot spot detection of aerial infrared image based on deep learning. Electron. Meas. Technol. 45(22), 7 (2022)

    Google Scholar 

  25. ChunMing, W., YiQian, S., TiaoJun, W., YaLi, L.: Underwater trash detection algorithm based on improved yolov5s. J. Real-Time Image Proc. 19, 911–920 (2022)

    Article  Google Scholar 

  26. Zhiqiang, H., Ying, S., Hao, G., Juanjuan, L., Sugang, M., Jiulun, F.: M-yolo: an object detector based on global context information for infrared images. J. Real-Time Image Process. 19, 1009–1022 (2022)

    Article  Google Scholar 

  27. Chen, B., Zhang, Z., Liu, N., Tan, Y., Liu, X., Chen, T.: Spatiotemporal convolutional neural network with convolutional block attention module for micro-expression recognition. Information 11(8), 380 (2020)

    Article  Google Scholar 

  28. Zhang, L., Cheng, J., Liu, J., Liu, T., Xiang, D., Yi, S.: Unsupervised ship detection in sar images using superpixels and cspnet. IEEE Geosci. Remote Sens. Lett. 20, 1–5 (2023)

    Article  Google Scholar 

  29. Zhang, Y.-F., Ren, W., Zhang, Z., Jia, Z., Wang, L., Tan, T.: Focal and efficient iou loss for accurate bounding box regression. Neurocomputing 506, 146–157 (2022)

    Article  Google Scholar 

Download references

Funding

This work is supported by the National Natural Science Foundation of China (61563032, 61963025), the Gansu Provincial Science and Technology Program (22YF7GA164, 22CX8GA131) and the Red Willow Outstanding Young Talent Program of Lanzhou University of Technology (Grant Nos. 062001).

Author information

Authors and Affiliations

  1. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou, 730050, China

    Ye Lei, Xiaoye Wang, Aimin An & Haijiao Guan

  2. Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou, 730050, China

    Xiaoye Wang & Aimin An

  3. National Demonstration Center for Experimental Electrical and Control Engineering Education, Lanzhou University of Technology, Lanzhou, 730050, China

    Xiaoye Wang & Aimin An

Authors
  1. Ye Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoye Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aimin An
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haijiao Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Lei Ye and Wang Xiaoye completed the main manuscript documents, Guan Haijiao provided ideas, and An Aimin provided guidance. All authors have reviewed the manuscript.

Corresponding authors

Correspondence to Xiaoye Wang or Aimin An.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, Y., Wang, X., An, A. et al. Deeplab-YOLO: a method for detecting hot-spot defects in infrared image PV panels by combining segmentation and detection. J Real-Time Image Proc 21, 52 (2024). https://doi.org/10.1007/s11554-024-01415-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-024-01415-x

Keywords

Search

Navigation