Text Classification Based on Graph Convolution Neural Network and Attention Mechanism
Pages 137 - 142
Abstract
Extracting and representing text features is the most important part of text classification. Aiming at the problem of incomplete feature extraction in traditional text classification methods, a text classification model based on graph convolution neural network and attention mechanism is proposed. Firstly, the text is input into BERT (Bi-directional Encoder Representations from Transformers) model to obtain the word vector representation, the context semantic information of the given text is learned by the BiGRU (Bi-directional Gated Recurrent Unit), and the important information is screened by attention mechanism and used as node features. Secondly, the dependency syntax diagram and the corresponding adjacency matrix of the input text are constructed. Thirdly, the GCN (Graph Convolution Neural Network) is used to learn the node features and adjacency matrix. Finally, the obtained text features are input into the classifier for text classification. Experiments on two datasets show that the proposed model achieves a good classification effect, and better accuracy is achieved in comparison with baseline models.
References
[1]
Kim Y. Convolutional neural networks for sentence classification[C]//In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014:1746-1751.
[2]
Gao Z, Feng A, Song X, Target-dependent sentiment classification with BERT[J]. IEEE Access, 2019, 7:1-1.
[3]
Reimers N, Gurevych I. Alternative weighting schemes for ELMo embeddings[J]. arXiv preprint arXiv:1904.02954, 2019.
[4]
Onan A. Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks[J]. Concurrency and Computation:Practice and Experience, 2020, 33(5).
[5]
Wang Y, Huang M, Zhu X, Attention-based LSTM for aspect-level sentiment classification[C]//In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016.
[6]
Li X, Bing L, Lam W, Transformation networks for target-oriented sentiment classification[C]//In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018.
[7]
Zhou J, Cui G, Hu S, Graph neural networks:A review of methods and application[J]. AI Open, 2020, 1:57-81.
[8]
Scarselli F, Gori M, Tsoi A C, The graph neural network model[J]. IEEE transactions on neural networks, 2008, 20(1),61-80.
[9]
Wang H T, Song W, Wang H. Text Classification method based on hybrid model of LSTM and CNN[J]. Journal of Chinese Computer Systems, 2020, 41(06):1163-1168.
[10]
Wang W, Sun Y X, Qi Q J, Text sentiment classification model based on BiGRU-attention neural network[J]. Application Research of Computers, 2019, 36(12): 3558-3564.
[11]
Hassan A, Mahmood A. Convolutional recurrent deep learning model for sentence classification[J]. IEEE Access, 2018, 6(1): 13949-13957.
[12]
Li K Y, Chen Y, Niu S Z. Social e-commerce text classification algorithm based on BERT[J]. Computer Science, 2021, 48(2):87-92.
[13]
Rasmy L, Xiang Y, Xie Z, Med-BERT:pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction[J]. NPJ Digital Medicine, 2021, 4(1):1-13.
[14]
Weng X F, Zhao J H, Jiang C X, Research on sentiment classification of futures predictive texts based on BERT[J/OL]. Computing, 2021. https://doi.org/10.1007/s00607-021-00989-9.
[15]
Chen Z Q, Ju T. Research on tendency analysis of microblog comments based on BERT and BLSTM[J]. Information Studuies:Theory&Application, 2020, 43(8):173-177.
[16]
Li W, Qi F, Tang M, Bidirectional LSTM with self-attention mechanism and multi-channel features for sentiment classification[J]. Neurocomputing, 2020, 387: 63-77.
[17]
Liu J, Yang H, Lv S Q, Attention-based BiGRU-CNN for chinese question classification[J]. Journal of Ambient Intelligence and Humanized Computing, 2019(2).
[18]
Xu B B, Cen K T, Huang J J, A survey on graph convolutional neural network[J]. Chinese Journal of Computers, 2020, 43(5): 755-780.
[19]
Fang A. Short-text question classification based on dependency parsing and attention mechanism[C]//In Proceedings of the 2019 International Conference on Machine Learning and Cybernetics (ICMLC), 2019.
[20]
Luo H, Li T, Liu B, Improving aspect term extraction with bidirectional dependency tree representation[J]. IEEE Trans Audio Speech Lang Process, 2019, 27(7):1201-1212.
[21]
Sun K, Zhang R C. Aspect-level sentiment analysis via convolution over dependency tree[C]//In Proceedings of Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019.
[22]
Wang K, Shen W, Yang Y, Relational graph attention network for aspect-based sentiment analysis[C]//In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
[23]
Yao L, Mao C, Luo Y. Graph convolutional networks for text classification[C]//In proceedings of the AAAI conference on artificial intelligence, 2018.
[24]
Vaswani A, Shazeer N, Parmar N, Attention is all you need[C]//In proceedings of the 31st Conference on Neural Information Processing Systems, 2017.
[25]
Zhu Y, Chen S P. Commentary text sentiment analysis combining convolution neural network and attention[J]. Journal of Chinese Computer Systems, 2020, 41(03):551-557.
Index Terms
- Text Classification Based on Graph Convolution Neural Network and Attention Mechanism
Recommendations
Attention Based Stopword Generation for Neural Network Based Text Processing
Information Integration and Web IntelligenceAbstractStopwords are used to improve the accuracy of document classification and retrieval. We believe that setting appropriate stopwords improves classification accuracy. However, in our preliminary experiments, in document classification tasks using ...
Chinese text classification based on neural network
ISNN'13: Proceedings of the 10th international conference on Advances in Neural Networks - Volume Part IText classification is widely used nowadays. In this paper, we proposed a combination feature reduction method to reduce feature space dimension based on inductive analysis of existing researches. Neural network was then trained and used to classify new ...
Two-channel hierarchical attention mechanism model for short text classification
AbstractText classification plays an important role in information science. In order to address the issues of low classification efficiency, low accuracy, and incomplete text feature extraction in existing classification methods, this work offers a two-...
Comments
Information & Contributors
Information
Published In
September 2022
1221 pages
ISBN:9781450396899
DOI:10.1145/3573942
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 May 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
AIPR 2022
AIPR 2022: 2022 5th International Conference on Artificial Intelligence and Pattern Recognition
September 23 - 25, 2022
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 40Total Downloads
- Downloads (Last 12 months)17
- Downloads (Last 6 weeks)2
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format