Abstract
Text classification is the most fundamental and foundational problem in many natural language processing applications. Recently, the graph-based model (e.g., GNN-based model and GCN-based model) has been applied to this task and achieved excellent performance because of their superior capacity of modeling context from the global perspective. However, a multitude of existing graph-based models constructs a corpus-level graph structure which causes a high memory consumption and overlooks the local contextual information. To address these issues, we present a novel GNN-based model which contains a new model for building a text graph for text classification. The proposed model is called two sliding windows text GNN-based model (TSW-GNN). To be more specific, a unique text-level graph is constructed for each text, which contains a dynamic global window and a local sliding window. The local window slides inside the text to construct local word connections. Additionally, the dynamic global window slides between texts to determine word edge weights, which conquers the limitation of a single local sliding window and provides more abundant global information. We perform extensive experiments on seven benchmark datasets, and the experimental results manifest the amelioration of TSW-GNN over the most advanced models in terms of the classification accuracy.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
References
Zhang L, Jiang L, Li C, Kong G (2016) Two feature weighting approaches for naive Bayes text classifiers. Knowl Based Syst 100:137–144
Yu D, Chen CP, Xu H (2021) Fuzzy swarm control based on sliding-mode strategy with self-organized omnidirectional mobile robots system. IEEE Trans Syst Man Cybern Syst. https://doi.org/10.1109/TSMC.2020.3048733
Barushka A, Hajek P (2020) Spam detection on social networks using cost-sensitive feature selection and ensemble-based regularized deep neural networks. Neural Comput Appl 32(9):4239–4257
Zhu J, Li X, Gao C, Wang Z, Kurths J (2021) Unsupervised community detection in attributed networks based on mutual information maximization. New J Phys 23(11):113016
Chen J, Yan S, Wong K-C (2020) Verbal aggression detection on twitter comments: convolutional neural network for short-text sentiment analysis. Neural Comput Appl 32(15):10809–10818
Li L, Goh T-T, Jin D (2020) How textual quality of online reviews affect classification performance: a case of deep learning sentiment analysis. Neural Comput Appl 32(9):4387–4415
Zhu J, Wang C, Gao C, Zhang F, Wang Z, Li X (2022) Community detection in graph: an embedding method. IEEE Trans Netw Sci Eng 9(2):689–702
Chen L, Jiang L, Li C (2021) Using modified term frequency to improve term weighting for text classification. Eng Appl Artif Intell 101:104215
Jiang L, Li C, Wang S, Zhang L (2016) Deep feature weighting for naive Bayes and its application to text classification. Eng Appl Artif Intell 52:26–39
Forman G (2008) BNS feature scaling: an improved representation over TF-IDF for SVM text classification. In: Proceedings of the international conference on information and knowledge management, pp 263–270
Jiang L, Wang S, Li C, Zhang L (2016) Structure extended multinomial naive Bayes. Inf Sci 329:346–356
Androutsopoulos I, Koutsias J, Chandrinos KV, Paliouras G, Spyropoulos CD (2000) An evaluation of naive Bayesian anti-spam filtering. In: Proceedings of the European conference on machine learning, pp 9–17
Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1746–1751
Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101
Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: Proceedings of the association for the advance of artificial intelligence, pp 7370–7377
Zhang Y, Yu X, Cui Z, Wu S, Wen Z, Wang L (2020) Every document owns its structure: inductive text classification via graph neural networks. In: Proceedings of the annual meeting of the association for computational linguistics, pp 334–339
Huang L, Ma D, Li S, Zhang X, Houfeng W (2019) Text level graph neural network for text classification. In: Proceedings of the conference on empirical methods in natural language processing, pp 3435–3441
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105
Johnson R, Zhang T (2017) Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the annual meeting of the association for computational linguistics, pp 562–570
Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Graves A, Schmidhuber J (2005) Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw 18(5–6):602–610
Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder–decoder approaches. arXiv preprint arXiv:1409.1259
Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075
Zhao J, Zhan Z, Yang Q, Zhang Y, Hu C, Li Z, Zhang L, He Z (2018) Adaptive learning of local semantic and global structure representations for text classification. In: Proceedings of the international conference on computational linguistics, pp 2033–2043
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of the conference of the North American chapter of the association for computational linguistics, pp 2227–2237
Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training
Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
Si C, Chen W, Wang W, Wang L, Tan T (2019) An attention enhanced graph convolutional LSTM network for skeleton-based action recognition. In: Proceedings of the international conference on computer vision and pattern recognition, pp 1227–1236
Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the international conference on knowledge discovery and data mining, pp 974–983
Bian T, Xiao X, Xu T, Zhao P, Huang W, Rong Y, Huang J (2020) Rumor detection on social media with bi-directional graph convolutional networks. In: Proceedings of the association for the advance of artificial intelligence, vol 34, pp 549–556
Peng H, Li J, He Y, Liu Y, Bao M, Wang L, Song Y, Yang Q (2018) Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In: Proceedings of the world wide web conference, pp 1063–1072
Liu X, You X, Zhang X, Wu J, Lv P (2020) Tensor graph convolutional networks for text classification. In: Proceedings of the association for the advance of artificial intelligence, pp 8409–8416
Fan M, Cheng D, Yang F, Luo S, Luo Y, Qian W, Zhou A (2020) Fusing global domain information and local semantic information to classify financial documents. In: Proceedings of the international conference on information and knowledge management, pp 2413–2420
Linmei H, Yang T, Shi C, Ji H, Li X (2019) Heterogeneous graph attention networks for semi-supervised short text classification. In: Proceedings of the conference on empirical methods in natural language processing, pp 4821–4830
Li Y, Tarlow D, Brockschmidt M, Zemel R (2016) Gated graph sequence neural networks. In: Proceedings of the international conference on learning representations, pp 1532–1543
Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the conference on empirical methods in natural language processing, pp 1631–1642
Pang B, Lee L (2005) Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the annual meeting on association for computational linguistics, pp 115–124
Joulin A, Grave É, Bojanowski P, Mikolov T (2017) Bag of tricks for efficient text classification. In: Proceedings of the conference of the European chapter of the association for computational linguistics, pp 427–431
Wu X, Luo Z, Du Z, Wang J, Gao C, Li X (2021) TW-TGNN: TWO windows graph-based model for text classification. In: Proceedings of the international joint conference on neural networks, pp 1–8. IEEE
Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the conference on empirical methods in natural language processing, pp 1532–1543
Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of the international conference on learning representations, pp 3435–3441
Chen L, Jiang L, Li C (2021) Modified DFS-based term weighting scheme for text classification. Expert Syst Appl 168:114438
Zhang L, Jiang L, Li C (2019) A discriminative model selection approach and its application to text classification. Neural Comput Appl 31(4):1173–1187
Lvd M, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579–2605
Acknowledgements
This work was supported by the Key Program for International Science and Technology Cooperation Projects of China (No. 2022YFE0112300), National Natural Science Foundation for Distinguished Young Scholars (No. 62025602), National Natural Science Foundation of China (No. 61976181), Key Technology Research and Development Program of Science and Technology-Scientific and Technological Innovation Team of Shaanxi Province (No. 2020TD-013), Natural Science Basic Research Plan in Shaanxi Province of China (No. 2022JM-325), Fundamental Research Funds for the Central Universities (No. D5000210827).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, X., Wu, X., Luo, Z. et al. Integration of global and local information for text classification. Neural Comput & Applic 35, 2471–2486 (2023). https://doi.org/10.1007/s00521-022-07727-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-022-07727-y