Skip to main content
SpringerLink
Log in

Salient feature fusion convolutional network for multi-class meters detection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Automatic meter reading via deep learning and computer vision have become feasible for ensuring safe and stable substation operation. Meter model classification is a difficult task because of the great similarity in appearance of the different meters. Existing methods mostly focus on detecting and reading individual types of meters, while neglecting to put the classification of multi-class meters on the map. For that, we propose a salient feature fusion convolutional network (SFFCN) for meter model classification and detecting multi-class meters, both of which are crucial steps for further meter readings. The central fusion pyramid network within SFFCN is an improved FPN that efficiently extracts and fuses multi-scale features, thereby enhancing feature saliency and diversity. To improve the accuracy of localization and classification, we introduce a classification-weighted localization attention module (CLAM) to the detection heads. CLAM guides the location branch based on the feature map of the classification branch, mitigating the mismatch between classification confidence and localization accuracy. The experimental results demonstrate that the proposed SFFCN achieves 88.8 mAP on the power meter image dataset and reaches a speed of 68.1 FPS on RTX 3080Ti, effectively improving the detection and classification of multi-class meters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets generated during and/or analyzed during the current study are available from the authors upon reasonable request.

References

  1. Xiu, H., He, J., Zhang, X., Wang, L., Qi, Y.: HRC-mCNNs: a hybrid regression and classification multibranch CNNs for automatic meter reading with smart shell. IEEE Internet Things J. 9(24), 25752–25766 (2022)

    Article  Google Scholar 

  2. He, P., Zuo, L., Zhang, C., Zhang, Z.: A value recognition algorithm for pointer meter based on improved mask-RCNN. In: 2019 9th International Conference on Information Science and Technology (ICIST), pp. 108–113. IEEE (2019)

  3. Fan, Z., Shi, L., Xi, C., Wang, H., Wang, S., Wu, G.: Real time power equipment meter recognition based on deep learning. IEEE Trans. Instrum. Meas. 71, 1–15 (2022)

    Google Scholar 

  4. Tian, Z., Shen, C., Chen, H., He, T.: FCOS: a simple and strong anchor-free object detector. IEEE Trans. Pattern Anal. Mach. Intell. 44, 1922–1933 (2020)

    Google Scholar 

  5. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

  6. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  7. Zheng, Z., Wang, P., Ren, D., Liu, W., Ye, R., Hu, Q., Zuo, W.: Enhancing geometric factors in model learning and inference for object detection and instance segmentation. IEEE Trans. Cybern. 52, 8574–8586 (2021)

    Article  Google Scholar 

  8. Liu, Y., Liu, J., Ke, Y.: A detection and recognition system of pointer meters in substations based on computer vision. Measurement 152, 107333 (2020)

    Article  Google Scholar 

  9. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. Adv. Neural. Inf. Process. Syst. 28, 91–99 (2015)

    Google Scholar 

  10. Zhang, H., Chen, Q., Lei, L., et al.: A YOLOv3-based industrial instrument classification and reading recognition method. Mob. Inf. Syst. 2022 (2022)

  11. Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  12. Deng, G., Huang, T., Lin, B., Liu, H., Yang, R., Jing, W.: Automatic meter reading from UAV inspection photos in the substation by combining YOLOv5s and DeepLabv3+. Sensors 22(18), 7090 (2022)

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  13. Zhu, X., Lyu, S., Wang, X., Zhao, Q.: TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2778–2788 (2021)

  14. Sun, J., Huang, Z., Zhang, Y.: A novel automatic reading method of pointer meters based on deep learning. Neural Comput. Appl. 1–14 (2022)

  15. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: YOLOv4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)

  16. Qiao, G., Yang, M., Wang, H.: A water level measurement approach based on YOLOv5s. Sensors 22(10), 3714 (2022)

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  17. Feng, Q., Huang, L., Sun, Y., Tong, X., Liu, X., Xie, Y., Li, J., Fan, H., Chen, B.: Substation instrumentation target detection based on multi-scale feature fusion. Concurr. Comput. Pract. Exp. 34(23), 7177 (2022)

    Article  Google Scholar 

  18. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: SSD: single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer (2016)

  19. Ji, D., Zhang, W., Zhao, Q., Yang, W.: Correction and pointer reading recognition of circular pointer meter. Meas. Sci. Technol. 34(2), 025406 (2022)

    Article  ADS  Google Scholar 

  20. Liu, F., Pan, L., Gao, R., Zhang, L., Pang, Y., Ning, X., Zhang, H., Liu, K.: A multitask cascading convolutional neural network for high-accuracy pointer meter automatic recognition in outdoor environments. Meas. Sci. Technol. 34(5), 055011 (2023)

    Article  ADS  Google Scholar 

  21. Chen, L., Wu, X., Sun, C., Zou, T., Meng, K., Lou, P.: An intelligent vision recognition method based on deep learning for pointer meters. Meas. Sci. Technol. 34(5), 055410 (2023)

    Article  ADS  Google Scholar 

  22. Lv, Q., Rao, Y., Zeng, S., Huang, C., Cheng, Z.: Small-scale robust digital recognition of meters under unstable and complex conditions. IEEE Trans. Instrum. Meas. 71, 1–13 (2022)

    Google Scholar 

  23. Li, Y., Chen, Y., Wang, N., Zhang, Z.: Scale-aware trident networks for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6054–6063 (2019)

  24. Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8759–8768 (2018)

  25. Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., Lin, D.: Libra R-CNN: towards balanced learning for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 821–830 (2019)

  26. Wang, J., Chen, K., Xu, R., Liu, Z., Loy, C.C., Lin, D.: CARAFE: content-aware reassembly of features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3007–3016 (2019)

  27. Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5693–5703 (2019)

  28. Li, Y., Mao, H., Girshick, R., He, K.: Exploring plain vision transformer backbones for object detection. arXiv preprint arXiv:2203.16527 (2022)

  29. Chen, Q., Wang, Y., Yang, T., Zhang, X., Cheng, J., Sun, J.: You only look one-level feature. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13039–13048 (2021)

  30. Cao, Y., Xu, J., Lin, S., Wei, F., Hu, H.: GCNET: non-local networks meet squeeze-excitation networks and beyond. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 0–0 (2019)

  31. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  32. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

  33. Sun, Z., Dai, M., Leng, X., Lei, Y., Xiong, B., Ji, K., Kuang, G.: An anchor-free detection method for ship targets in high-resolution SAR images. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 14, 7799–7816 (2021)

    Article  ADS  Google Scholar 

  34. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  35. Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., et al.: MMDetection: open MMLab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155 (2019)

  36. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  37. Zhang, S., Chi, C., Yao, Y., Lei, Z., Li, S.Z.: Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9759–9768 (2020)

  38. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

  39. Kong, T., Sun, F., Liu, H., Jiang, Y., Li, L., Shi, J.: FoveaBox: beyound anchor-based object detection. IEEE Trans. Image Process. 29, 7389–7398 (2020)

    Article  ADS  Google Scholar 

  40. Terven, J., Cordova-Esparza, D.: A comprehensive review of YOLO: from YOLOv1 to YOLOv8 and beyond. arXiv preprint arXiv:2304.00501 (2023)

  41. Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: YOLOX: exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 (2021)

  42. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)

Download references

Acknowledgements

This work was supported by Key-Area Research and Development Program of Guangdong Province under Grants (2020B1111010002, 2018B010109001, 2019B020214001).

Author information

Authors and Affiliations

  1. School of Automation Science and Engineering, South China University of Technology, Wushan Street, Guangzhou, 510641, Guangdong, China

    Zhaolin Wang, Lianfang Tian, Qiliang Du & Zhengzheng Sun

  2. Department of Telecommunications and Information Processing, Ghent University, St-Pietersnieuwstraat 41, 9000, Ghent, Belgium

    Wenzhi Liao

Authors
  1. Zhaolin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lianfang Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiliang Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhengzheng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenzhi Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZW conceived and designed the research, conducted experiments, and analyzed the data. LT and QD provided guidance and supervision throughout the project, contributed to the experimental design, and reviewed and edited the manuscript. ZS assisted in data analysis, conducted experiments, and contributed to the interpretation of the results. WL contributed to the literature review, performed data collection, and helped in manuscript preparation. All authors reviewed and approved the final version of the manuscript.

Corresponding authors

Correspondence to Lianfang Tian or Qiliang Du.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest related to this work, including financial, personal or other relationships with other people or organizations that could inappropriately influence their work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Tian, L., Du, Q. et al. Salient feature fusion convolutional network for multi-class meters detection. SIViP 18, 1183–1192 (2024). https://doi.org/10.1007/s11760-023-02721-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02721-w

Keywords

Search

Navigation