skip to main content
research-article

A Novel Classification Model SA-MPCNN for Power Equipment Defect Text

Published: 12 August 2021 Publication History

Abstract

The text classification of power equipment defect is of great significance to equipment health condition evaluation and power equipment maintenance decisions. Most of the existing classification methods do not sufficiently consider the semantic relation between words in the same sentence and cannot extract deep semantic features. To tackle those problems, this article proposes a novel classification method by combining the self-attention mechanism and multi-channel pyramid convolution neural networks. We utilize the bidirectional gated recurrent unit to model the text sequence and, on this basis, improve self-attention layer to dot multiplication on the forward and backward features to obtain the global attention score. Thereby, effective features are enhanced, invalid features are weakened, and important text representation vectors are obtained. To solve the problem that the shallow network structure cannot extract deep semantic features, we design a multi-channel pyramid convolution network, which first extracts deep text features from the channels of different windows and then fuses the text features of each channel. By comparing with the state-of-the-art methods, the model in this article has better performance in text classification of power equipment defects.

References

[1]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473. Retrieved from https://arxiv.org/abs/1409.0473.
[2]
Vishwanath Bijalwan, Vinay Kumar, Pinki Kumari, and Jordan Pascual. 2014. KNN based machine learning approach for text and document mining. Int. J. Datab. Theory Appl. 7, 1 (2014), 61–70.
[3]
Jindong Chen, Yizhou Hu, Jingping Liu, Yanghua Xiao, and Haiyun Jiang. 2019. Deep short text classification with knowledge powered attention. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6252–6259.
[4]
Yushi Chen, Jelena Ponocko, Nikola Milosevic, Goran Nenadic, and Jovica V. Milanovic. 2016. Towards application of text mining for enhanced power network data analytics—part ii: Offline analysis of textual data. (2016), 1–8.
[5]
Xie Cheng, Guoping Zou, Huifang Wang, and Yongtao Jin. 2016. A new condition assessment method for distribution transformers based on operation data and record text mining technique. In Proceedings of the China International Conference on Electricity Distribution.
[6]
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555. Retrieved from https://arxiv.org/abs/1412.3555.
[7]
Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, and Yoshua Bengio. 2015. Gated feedback recurrent neural networks. In Proceedings of the International Conference on Machine Learning. 2067–2075.
[8]
Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. (2018), 4171–4186.
[9]
Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 6645–6649.
[10]
Qipeng Guo, Xipeng Qiu, Pengfei Liu, Xiangyang Xue, and Zheng Zhang. 2019. Multi-scale self-attention for text classification. arXiv:1912.00544. Retrieved from https://arxiv.org/abs/1912.00544.
[11]
Abdalraouf Hassan and Ausif Mahmood. 2017. Efficient deep learning model for text classification based on recurrent and convolutional layers. In Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA’17). IEEE, 1108–1113.
[12]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neur. Comput. 9, 8 (1997), 1735–1780.
[13]
Wei Jiang and Zhong Jin. 2017. Integrating bidirectional LSTM with inception for text classification. In Proceedings of the 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR’17). IEEE, 870–875.
[14]
Rie Johnson and Tong Zhang. 2017. Deep pyramid convolutional neural networks for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 562–570.
[15]
Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv:1408.5882. Retrieved from https://arxiv.org/abs/1408.5882.
[16]
Kamran Kowsari, Donald E. Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S. Gerber, and Laura E. Barnes. 2017. HDLTex: Hierarchical deep learning for text classification. (2017), 1–25.
[17]
Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In Proceedings of the 29th AAAI Conference on Artificial Intelligence.
[18]
Gaël Letarte, Frédérik Paradis, Philippe Giguère, and François Laviolette. 2018. Importance of self-attention for sentiment analysis. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. 267–275.
[19]
Yuancheng Li, Pan Zhang, and Rong Huang. 2019. Lightweight quantum encryption for secure transmission of power data in smart grid. IEEE Access 7 (2019), 36285–36293.
[20]
Ziquan Liu, Huifang Wang, Jing Cao, and Jian Qiu. 2018. A classification model of power equipment defect texts based on convolutional neural network. Power System Technology (2018), 475–487.
[21]
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv:1508.04025. Retrieved from https://arxiv.org/abs/1508.04025.
[22]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781. Retrieved from https://arxiv.org/abs/1301.3781.
[23]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111–3119.
[24]
Nikolaos Pappas and Andrei Popescu-Belis. 2017. Multilingual hierarchical attention networks for document classification. arXiv:1707.00896. Retrieved from https://arxiv.org/abs/1707.00896.
[25]
Tomas Pranckevičius and Virginijus Marcinkevičius. 2016. Application of logistic regression with part-of-the-speech tagging for multi-class text classification. In Proceedings of the 2016 IEEE 4th Workshop on Advances in Information, Electronic and Electrical Engineering (AIEEE’16). IEEE, 1–5.
[26]
Jian Qiu, Huifang Wang, Gaoliang Ying, Bo Zhang, Guoping Zou, and H. E. Benteng. 2016. Text mining technique and application of lifecycle condition assessment for circuit breaker. Autom. Electr. Power Syst. 40, 6 (2016), 107–112.
[27]
C. Rudin, D. Waltz, R. N. Anderson, A. Boulanger, A. Salleb-Aouissi, M. Chow, H. Dutta, P. N. Gross, B. Huang, and S. Ierome. 2012. Machine learning for the new york city power grid. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2 (2012), 328–345.
[28]
Feng Shen, Xiong Luo, and Yi Chen. 2013. Text classification dimension reduction algorithm for chinese web page based on deep learning. (2013), 451–456.
[29]
Weiwei Shi. 2015. Improving power grid monitoring data quality: An efficient machine learning framework for missing data prediction. In Proceedings of the IEEE International Conference on High Performance Computing and Communications (HPCC’15).
[30]
Mengjiao Song, Xingyu Zhao, Yong Liu, and Zhihong Zhao. 2018. Text sentiment analysis based on convolutional neural network and bidirectional LSTM model. In Proceedings of the International Conference of Pioneering Computer Scientists, Engineers and Educators. Springer, 55–68.
[31]
Haifeng Sun, Zhaoyu Wang, Jianhui Wang, Zhen Huang, NichelleLe Carrington, and Jianxin Liao. 2016. Data-driven power outage detection by social sensors. IEEE Trans. Smart Grid 7, 5 (2016), 2516–2524.
[32]
Haifeng Sun, Zhaoyu Wang, Jianhui Wang, Huang Zhen, Nichelle Le Carrington, and Jianxin Liao. 2016. Data-driven power outage detection by social sensors. IEEE Trans. Smart Grid 7, 5 (2016), 2516–2524.
[33]
Qinting Tang, Jiayu Chen, Hengtong Lu, Yu Du, Kehan Yang, et al. 2019. Full attention-based bi-GRU neural network for news text classification. In Proceedings of the 2019 IEEE 5th International Conference on Computer and Communications (ICCC’19). IEEE, 1970–1974.
[34]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998–6008.
[35]
Hui-fang Wang and Zi-quan Liu. 2019. An error recognition method for power equipment defect records based on knowledge graph technology. Front. Inf. Technol. Electr. Eng. 20, 11 (2019), 1564–1577.
[36]
Shiyao Wang, Minlie Huang, and Zhidong Deng. 2018. Densely connected CNN with multi-scale feature attention for text classification. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’18). 4468–4474.
[37]
Sida I. Wang and Christopher D. Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 90–94.
[38]
Daqian Wei, Bo Wang, Gang Lin, Dichen Liu, Zhaoyang Dong, Hesen Liu, and Yilu Liu. 2017. Research on unstructured text data mining and fault classification based on RNN-LSTM with malfunction inspection report. Energies 10, 3 (2017), 406–.
[39]
Jason Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv:1901.11196. Retrieved from https://arxiv.org/abs/1901.11196.
[40]
Xin Wu, Yi Cai, Qing Li, Jingyun Xu, and Ho-fung Leung. 2018. Combining contextual information by self-attention mechanism in convolutional neural networks for text classification. In Proceedings of the International Conference on Web Information Systems Engineering. Springer, 453–467.
[41]
Zheng Xiao and PiJun Liang. 2016. Chinese sentiment analysis using bidirectional LSTM with word embedding. In Proceedings of the International Conference on Cloud Computing and Security. Springer, 601–610.
[42]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1480–1489.
[43]
Wenpeng Yin, Katharina Kann, Mo Yu, and Hinrich Schütze. 2017. Comparative study of cnn and rnn for natural language processing. arXiv:1702.01923. Retrieved from https://arxiv.org/abs/1702.01923.
[44]
Tom Young, Devamanyu Hazarika, Soujanya Poria, and Erik Cambria. 2018. Recent trends in deep learning based natural language processing. IEEE Comput. Intell. Mag. 13, 3 (2018), 55–75.
[45]
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. arXiv:1509.01626. Retrieved from https://arxiv.org/abs/1509.01626.

Cited By

View all
  • (2024)Enhancing power equipment defect identification through multi-label classification methodsScientific Reports10.1038/s41598-024-71996-x14:1Online publication date: 18-Sep-2024
  • (2023)SFF-Siam: A New Oracle Bone Rejoining Method Based on Siamese NetworkIEEE Computer Graphics and Applications10.1109/MCG.2023.328400043:6(22-32)Online publication date: 14-Jun-2023
  • (2023)A novel circuit breaker fault diagnosis method based on dense residual and attention mechanismIET Generation, Transmission & Distribution10.1049/gtd2.1296217:19(4316-4328)Online publication date: 20-Sep-2023
  • Show More Cited By

Index Terms

  1. A Novel Classification Model SA-MPCNN for Power Equipment Defect Text

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 20, Issue 6
      November 2021
      439 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3476127
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 12 August 2021
      Accepted: 01 April 2021
      Revised: 01 March 2021
      Received: 01 August 2020
      Published in TALLIP Volume 20, Issue 6

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Text classification
      2. defect texts
      3. self-attention mechanism
      4. bidirectional gated recurrent unit
      5. pyramid convolution

      Qualifiers

      • Research-article
      • Refereed

      Funding Sources

      • NSFC
      • Project of Electric Power Research Institute of State Grid Gansu Electric Power Company
      • Shanghai Engineering Research Center on Big Data Management System

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)9
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Enhancing power equipment defect identification through multi-label classification methodsScientific Reports10.1038/s41598-024-71996-x14:1Online publication date: 18-Sep-2024
      • (2023)SFF-Siam: A New Oracle Bone Rejoining Method Based on Siamese NetworkIEEE Computer Graphics and Applications10.1109/MCG.2023.328400043:6(22-32)Online publication date: 14-Jun-2023
      • (2023)A novel circuit breaker fault diagnosis method based on dense residual and attention mechanismIET Generation, Transmission & Distribution10.1049/gtd2.1296217:19(4316-4328)Online publication date: 20-Sep-2023
      • (2022)State Evaluation Method of Distribution Equipment Based on Health Index in Big Data EnvironmentMathematical Problems in Engineering10.1155/2022/53028262022(1-9)Online publication date: 14-Jul-2022
      • (2022)A Meta-Learning Framework for Predicting Power Digital Equipment Defect Texts via Hypergraph Modeling2022 4th International Conference on Frontiers Technology of Information and Computer (ICFTIC)10.1109/ICFTIC57696.2022.10075278(46-51)Online publication date: 2-Dec-2022

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media