Abstract
In the initial stage of the establishment of photovoltaic (PV) module production lines or the upgrading of production processes, the available data for some defects are limited. The detection for these data-scarce defects of photovoltaic (PV) modules is a challenging task, because most detectors hardly extract meaningful high-level features from limited information of several samples. To address this challenge, this paper proposes a novel end-to-end meta-learning based few-shot detector (Meta-FSDet). Firstly, a prototype vector extractor (PVE) is proposed, which utilizes locational prior knowledge of data-scarce defects to extract corresponding prototype vectors. Then, a novel saliency highlighting network (SHN), which measures the similarity between extracted prototype vectors from PVE and each spatial-position vector of the query feature map, is proposed to infer a class-agnostic saliency map, in which the defective areas are well highlighted. Next, a remodel region proposal network (RRPN), which further leverages the saliency map from SHN to pixel-wisely reweight the inputted feature map of region proposal network (RPN), is presented to better infer meaningful high-level region-of-interest (RoI) features of data-scarce defects. Finally, these RoI features from RRPN are further processed by downstream classification and regression module to generate final prediction results. Extensive experiments on PV module dataset demonstrate our Meta-FSDet owns superior performance compared to other existing methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akram, M. W., Li, G., Jin, Y., Chen, X., Zhu, C., & Ahmad, A. (2020). Automatic detection of photovoltaic module defects in infrared images with isolated and develop-model transfer deep learning. Solar Energy, 198(1), 175–186. https://doi.org/10.1016/j.solener.2020.01.055
Akram, M. W., Li, G., Jin, Y., Chen, X., Zhu, C., Zhao, X., & Ahmad, A. (2019). CNN based automatic detection of photovoltaic cell defects in electroluminescence images. Energy, 189(15), 116–319.
Balzategui, J., Eciolaza, L., & Maestro-Watson, D. (2021). Anomaly detection and automatic labeling for solar cell quality inspection based on Generative Adversarial Network. Sensors, 21(13), 4361. https://doi.org/10.3390/s21134361
Chen, H., Pang, Y., Hu, Q., & Liu, K. (2020). Solar cell surface defect inspection based on multispectral convolutional neural network. Journal of Intelligent Manufacturing, 31(2), 453–468. https://doi.org/10.1007/s10845-018-1458-z
Chen, H., Wang, Y., Wang, G., & Qiao, Y. (2018). Lstd: A low-shot transfer detector for object detection. In Proceedings of the AAAIConference on Artificial Intelligence (Vol. 32, Issue 1). https://doi.org/10.1609/aaai.v32i1.11716.
Dunderdale, C., Brettenny, W., Clohessy, C., & van Dyk, E. E. (2020). Photovoltaic defect classification through thermal infrared imaging using a machine learning approach. Progress in Photovoltaics: Research and Applications, 28(3), 177–188. https://doi.org/10.1002/pip.3191
Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2), 303–338. https://doi.org/10.1007/s11263-009-0275-4
Fan, Q., Zhuo, W., Tang, C. K., & Tai, Y. W. (2020). Few-shot object detection with attention-RPN and multi-relation detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 4013–4022). https://doi.org/10.1109/CVPR42600.2020.00407.
Han, G., Huang, S., Ma, J., He, Y., & Chang, S. F. (2022). Meta faster r-cnn: Towards accurate few-shot object detection with attentive feature alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 36(1), 780–789. https://doi.org/10.1609/aaai.v36i1.19959
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021). Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (Early Access). https://doi.org/10.1109/TPAMI.2021.3079209
Hsu, C. Y., & Chien, J. C. (2022). Ensemble convolutional neural networks with weighted majority for wafer bin map pattern classification. Journal of Intelligent Manufacturing, 33(3), 831–844. https://doi.org/10.1007/s10845-020-01687-7
Huang, F., Wang, B. W., Li, Q. P., & Zou, J. (2021). Texture surface defect detection of plastic relays with an enhanced feature pyramid network. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01864-2
Kang, B., Liu, Z., Wang, X., Yu, F., Feng, J., & Darrell, T. (2019). Few-shot object detection via feature reweighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 8420–8429). https://doi.org/10.1109/ICCV.2019.00851.
Kim, Y., Kim, T., Youn, B. D., & Ahn, S. H. (2022a). Machining quality monitoring (MQM) in laser-assisted micro-milling of glass using cutting force signals: An image-based deep transfer learning. Journal of Intelligent Manufacturing, 33(6), 1813–1828. https://doi.org/10.1007/s10845-021-01764-5
Kim, T., Lee, J. W., Lee, W. K., & Sohn, S. Y. (2022b). Novel method for detection of mixed-type defect patterns in wafer maps based on a single shot detector algorithm. Journal of Intelligent Manufacturing, 33(6), 1715–1724. https://doi.org/10.1007/s10845-021-01755-6
Li, A., Huang, W., Lan, X., Feng, J., Li, Z., & Wang, L. (2020). Boosting few-shot learning with adaptive margin loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 12576–12584). https://doi.org/10.1109/CVPR42600.2020.01259.
Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 2980–2988). https://doi.org/10.1109/TPAMI.2018.2858826.
Liu, B., Kang, H., Li, H., Hua, G., & Vasconcelos, N. (2020a). Few-shot open-set recognition using meta-learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 8798–8807). https://doi.org/10.1109/CVPR42600.2020a.00882.
Liu, Q., Majumder, O., Achille, A., Ravichandran, A., Bhotika, R., & Soatto, S. (2020b, August). Incremental few-shot meta-learning via indirect discriminant alignment. In European Conference on Computer Vision (ECCV) (pp. 685–701). https://doi.org/10.1007/978-3-030-58571-6_40.
Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 618–626). https://doi.org/10.1007/s11263-019-01228-7.
Su, B., Chen, H., Chen, P., Bian, G., Liu, K., & Liu, W. (2020). Deep learning-based solar-cell manufacturing defect detection with complementary attention network. IEEE Transactions on Industrial Informatics, 17(6), 4084–4095. https://doi.org/10.1109/TII.2020.3008021
Su, B., Chen, H., & Zhou, Z. (2021b). BAF-detector: An efficient CNN-based detector for photovoltaic cell defect detection. IEEE Transactions on Industrial Electronics, 69(3), 3161–3171. https://doi.org/10.1109/TIE.2021.3070507
Su, B., Chen, H., Zhu, Y., Liu, W., & Liu, K. (2019). Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor. IEEE Transactions on Instrumentation and Measurement, 68(12), 4675–4688. https://doi.org/10.1016/j.energy.2019.116319
Su, Y., Tao, F., Jin, J., & Zhang, C. (2021a). Automated overheated region object detection of photovoltaic module with thermography image. IEEE Journal of Photovoltaics, 11(2), 535–544. https://doi.org/10.1109/JPHOTOV.2020.3045680
Sun, B., Li, B., Cai, S., Yuan, Y., & Zhang, C. (2021). FSCE: Few-shot object detection via contrastive proposal encoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 7352–7362).
Tang, W., Yang, Q., Xiong, K., & Yan, W. (2020). Deep learning based automatic defect identification of photovoltaic module using electroluminescence images. Solar Energy, 201(1), 453–460. https://doi.org/10.1016/j.solener.2020.03.049
Wang, Y. X., Ramanan, D., & Hebert, M. (2019). Meta-learning to detect rare objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9925–9934).
Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3–19). https://doi.org/10.1007/978-3-030-01234-2_1.
Wu, X., Sahoo, D., & Hoi, S. (2020). Meta-RCNN: Meta learning for few-shot object detection. In Proceedings of the 28th ACM International Conference on Multimedia (pp. 1679–1687). https://doi.org/10.1145/3394171.3413832.
Yan, X., Chen, Z., Xu, A., Wang, X., Liang, X., & Lin, L. (2019). Meta R-CNN: Towards general solver for instance-level low-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9576–9585). https://doi.org/10.1109/ICCV.2019.00967.
Zhang, G., Cui, K., Wu, R., Lu, S., & Tian, Y. (2021a). PNPDet: Efficient few-shot detection without forgetting via plug-and-play sub-networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 3823–3832). https://doi.org/10.1109/WACV48630.2021a.00387.
Zhang, G., Luo, Z., Cui, K., & Lu, S. (2021b). Meta-detr: Few-shot object detection via unified image-level meta-learning. arXiv:2103.11731, 2(6).
Zhao, Y., Zhan, K., Wang, Z., & Shen, W. (2021). Deep learning-based automatic detection of multitype defects in photovoltaic modules and application in real production line. Progress in Photovoltaics: Research and Applications, 29(4), 471–484. https://doi.org/10.1002/pip.3395
Acknowledgements
This work is supported in part by National Natural Science Foundation of China under Grant 62073117, Grant U21A20482, Grant 62173124 and Natural Science Foundation of Hebei Province under Grant F2019202305.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that none of the authors have any financial or scientific conflicts of interest with regard to the research described in this manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, S., Chen, H., Liu, K. et al. Meta-FSDet: a meta-learning based detector for few-shot defects of photovoltaic modules. J Intell Manuf 34, 3413–3427 (2023). https://doi.org/10.1007/s10845-022-02001-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-022-02001-3