Skip to main content

Advertisement

SpringerLink
Log in

A method of detecting defects of smart meter LCD screen based on LSD and deep learning

  • 1171: Real-time 2D/ 3D Image Processing with Deep Learning
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to detect the defects of smart meter liquid crystal display (LCD) screen quickly and accurately, this paper proposes a method based on line segment detector (LSD) and deep learning. Characters of smart meter LCD screen can be divided into two types: ordinary characters and digital tube characters. Convolutional neural network (CNN) is used to locate and identify the ordinary characters. Since there may be certain missing segment in digital tube characters, conventional text detection methods are not applicable, and template matching method is used for detecting them. Compared with the whole screen, the digital tube area is very small, and the relative positions of the digital tube characters of smart meter of most manufacturers are nearly the same, so the detection rate of the template matching method is extremely high and the method can be commonly used. The whole detection process is as follows: First, horizontal straight lines got by LSD are used for tilt correction. Secondly, CNN is used to locate and identify an ordinary character of the meter screen to determine whether it is missing or not. Then, whether there are missing segments in the digital tube area is determined by template matching method. At last, the experimental results show that the accuracy of the method for detecting the character defects of LCD screen is about 96%, and the accuracy of the electric quantity identification function of the method is about 97%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Al Mahmud N (2020) Performance Analysis of Different Acoustic Features based on LSTM for Bangla Speech Recognition[J]. Int J Multimedia Applic 12(4):17–25

    Article  Google Scholar 

  2. Cha YJ, You K, Choi W (2016) Vision-based detection of loosened bolts using the Hough transform and support vector machines. Autom Constr 71:181–188

    Article  Google Scholar 

  3. Chen L, Li S (2018) Improvement Research and Application of Text Recognition Algorithm Based on CRNN[C]. SPML '18 Proceedings of the 2018 International Conference on Signal Processing and Machine Learning

  4. Chunlei R (2013) Research of the energy meter detection system based on machine vision [D].Zhejiang University of Technology

  5. Gioi RGV, Jakubowicz J, Morel JM et al (2010) LSD: a fast line segment detector with a false detection control[J]. IEEE Trans Pattern Analys Mach Intell 32(4):722–732

    Article  Google Scholar 

  6. Graves A, Fernández S, Gomez F (2006) Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks[C]. Proceedings of the 23rd international conference on Machine learning. 369–376

  7. Guiping D, Li D, Yue G (2015) The exterior type consistency inspection system for smart meter[J]. Lect Notes Electr Eng, 197–203

  8. He B, Xiyang O, Xingzhe H (2014) Quality detecting method of electricity meter LCD screen based on image processing[J]. Electr Meas Instrum 51(9):25–28

    Google Scholar 

  9. Hongyi J, Liping Z, Zhangpeng O (2017) Text recognition of natural scene image based on MSER and Tesseract[J]. Comp Knowl Technol, 213–216

  10. Kiranyaz S, Zabihi M, Rad AB, Ince T, Hamila R, Gabbouj M (2020) Real-time phonocardiogram anomaly detection by adaptive 1D convolutional neural networks[J]. Neurocomputing 411:291–301

    Article  Google Scholar 

  11. Lai W-x, Wang W, Huang B-b, Ren H, Shi T-l (2009) Fast Template Matching Based on Local Entropy Difference[C]. 2009 2nd International Congress on Image and Signal Processing, 1–5

  12. Min Y, Xiao B, Dang J, Yue B, Cheng T (2018) Real time detection system for rail surface defects based on machine vision. EURASIP J Image Video Process 3:1–11

    Google Scholar 

  13. Na Y, Zhiyuan Q, Zhongtao L, Sha Z (2014) Rapid Algorithm of Normalized Cross-Correlation Matching[J]. J Inf Eng Univ 15(02):215–218

    Google Scholar 

  14. Neogi N, Mohanta DK, Dutta PK et al (2014) Eur J Image Video Process 50:1–19

    Google Scholar 

  15. Sahani SK, Adhikari G, Das BK (2011) A fast template matching algorithm for aerial object tracking[C]. 2011 International Conference on Image Information Processing, 1–6

  16. Sharma H, Jalal AS (2020) Incorporating external knowledge for image captioning using CNN and LSTM[J]. Mod Phys Lett B 34(28):12

    Article  MathSciNet  Google Scholar 

  17. Sheng W (2019) Design and implementation of quality inspection system for smart meter display defects[D]. University of Electronic Science and Technology of China

  18. Shengcheng W (2018) The machine vision inspection system of electric energy meter LCD screen based on LSD [D]. Zhengzhou University

  19. Shi B, Bai X, Yao C (2016) An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition[J]. IEEE Trans Pattern Anal Mach Intell 39(11):2298–2304

    Article  Google Scholar 

  20. Song R, Cai Q, Wang Z, Su H, Shao X, Li W, Jin P (2018) A novel quality test method of electricity meter LCD Screen based on image processing[J]. J Phys Conf Ser

  21. Tong W (2020) Research on Intelligent Detection Method of Traffic Signs Using Convolutional Neural Network[D].Beijing University of Architecture and Architecture

  22. Weichen L, Duming T (2011) Defect inspection in low-contrast LCD images using hough transform-based nonstationary line detection[J]. IEEE Trans Ind Inform 7(1):136–147

    Article  Google Scholar 

  23. Wojna Z, Gorban AN, Lee DS et al. (2017) Attention-Based Extraction of Structured Information from Street View Imagery[J]. arXiv, 844–850

  24. Xie G, Lin M, Yanhui DAI (2015) Research on LCD screen appearance detection of smart meter in low-contrast[J]. Comput Eng Appl 50(2):247–251

    Google Scholar 

  25. Yang SW, Lin CS, Lin SK, Chiang HT (2014) Automatic defect recognition of TFT array process using gray level co-occurrence matrix[J]. Opt-Int J Light Electron Opt 125:2671–2676

    Article  Google Scholar 

  26. Yang Y, Xie R, Jia W, Chen Z, Yang Y, Xie L, Jiang BX (2021) Accurate and automatic tooth image segmentation model with deep convolutional neural networks and level set method[J]. Neurocomputing 419:108–125

    Article  Google Scholar 

  27. Yongqing C, Zhu X, Wang Y, Guo J, Lianqing C (2014) Research on image recognition of internal thread. Tool Eng 12:77–79

    Google Scholar 

  28. Yu P (2019) A method of detecting defects of electric meter LCD screen based on LSD and CNN [D]. Zhengzhou University

  29. Zhi T, Weilin H, Tong H, Pan H, Yu Q (2016) Detecting text in natural image with connectionist text proposal network[J]. Comput Vis – ECCV 2016, 9912,56–72

Download references

Author information

Authors and Affiliations

  1. College of Electrical Engineering, Zhengzhou University, Zhengzhou, 450001, People’s Republic of China

    Baoying Pei, Yu Peng & Yong Luo

Authors
  1. Baoying Pei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Luo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pei, B., Peng, Y. & Luo, Y. A method of detecting defects of smart meter LCD screen based on LSD and deep learning. Multimed Tools Appl 80, 35955–35972 (2021). https://doi.org/10.1007/s11042-020-10481-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10481-9

Keywords

Search

Navigation