Skip to main content
SpringerLink
Log in

Integration of global and local information for text classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Text classification is the most fundamental and foundational problem in many natural language processing applications. Recently, the graph-based model (e.g., GNN-based model and GCN-based model) has been applied to this task and achieved excellent performance because of their superior capacity of modeling context from the global perspective. However, a multitude of existing graph-based models constructs a corpus-level graph structure which causes a high memory consumption and overlooks the local contextual information. To address these issues, we present a novel GNN-based model which contains a new model for building a text graph for text classification. The proposed model is called two sliding windows text GNN-based model (TSW-GNN). To be more specific, a unique text-level graph is constructed for each text, which contains a dynamic global window and a local sliding window. The local window slides inside the text to construct local word connections. Additionally, the dynamic global window slides between texts to determine word edge weights, which conquers the limitation of a single local sliding window and provides more abundant global information. We perform extensive experiments on seven benchmark datasets, and the experimental results manifest the amelioration of TSW-GNN over the most advanced models in terms of the classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://www.cs.umb.edu/smimarog/textmining/datasets/.

  2. http://disi.unitn.it/moschitti/corpora.htm.

  3. http://www.cs.cornell.edu/people/pabo/movie-review-data/.

  4. https://www.cs.umb.edu/smimarog/textmining/datasets/.

  5. https://github.com/tensorflow/tensor2tensor.

  6. https://github.com/google-research/bert.

  7. http://nlp.stanford.edu/data/glove.6B.zip.

References

  1. Zhang L, Jiang L, Li C, Kong G (2016) Two feature weighting approaches for naive Bayes text classifiers. Knowl Based Syst 100:137–144

    Article  Google Scholar 

  2. Yu D, Chen CP, Xu H (2021) Fuzzy swarm control based on sliding-mode strategy with self-organized omnidirectional mobile robots system. IEEE Trans Syst Man Cybern Syst. https://doi.org/10.1109/TSMC.2020.3048733

    Article  Google Scholar 

  3. Barushka A, Hajek P (2020) Spam detection on social networks using cost-sensitive feature selection and ensemble-based regularized deep neural networks. Neural Comput Appl 32(9):4239–4257

    Article  Google Scholar 

  4. Zhu J, Li X, Gao C, Wang Z, Kurths J (2021) Unsupervised community detection in attributed networks based on mutual information maximization. New J Phys 23(11):113016

    Article  MathSciNet  Google Scholar 

  5. Chen J, Yan S, Wong K-C (2020) Verbal aggression detection on twitter comments: convolutional neural network for short-text sentiment analysis. Neural Comput Appl 32(15):10809–10818

    Article  Google Scholar 

  6. Li L, Goh T-T, Jin D (2020) How textual quality of online reviews affect classification performance: a case of deep learning sentiment analysis. Neural Comput Appl 32(9):4387–4415

    Article  Google Scholar 

  7. Zhu J, Wang C, Gao C, Zhang F, Wang Z, Li X (2022) Community detection in graph: an embedding method. IEEE Trans Netw Sci Eng 9(2):689–702

    Article  MathSciNet  Google Scholar 

  8. Chen L, Jiang L, Li C (2021) Using modified term frequency to improve term weighting for text classification. Eng Appl Artif Intell 101:104215

    Article  Google Scholar 

  9. Jiang L, Li C, Wang S, Zhang L (2016) Deep feature weighting for naive Bayes and its application to text classification. Eng Appl Artif Intell 52:26–39

    Article  Google Scholar 

  10. Forman G (2008) BNS feature scaling: an improved representation over TF-IDF for SVM text classification. In: Proceedings of the international conference on information and knowledge management, pp 263–270

  11. Jiang L, Wang S, Li C, Zhang L (2016) Structure extended multinomial naive Bayes. Inf Sci 329:346–356

    Article  Google Scholar 

  12. Androutsopoulos I, Koutsias J, Chandrinos KV, Paliouras G, Spyropoulos CD (2000) An evaluation of naive Bayesian anti-spam filtering. In: Proceedings of the European conference on machine learning, pp 9–17

  13. Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1746–1751

  14. Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101

  15. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: Proceedings of the association for the advance of artificial intelligence, pp 7370–7377

  16. Zhang Y, Yu X, Cui Z, Wu S, Wen Z, Wang L (2020) Every document owns its structure: inductive text classification via graph neural networks. In: Proceedings of the annual meeting of the association for computational linguistics, pp 334–339

  17. Huang L, Ma D, Li S, Zhang X, Houfeng W (2019) Text level graph neural network for text classification. In: Proceedings of the conference on empirical methods in natural language processing, pp 3435–3441

  18. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  19. Johnson R, Zhang T (2017) Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the annual meeting of the association for computational linguistics, pp 562–570

  20. Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211

    Article  Google Scholar 

  21. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  22. Graves A, Schmidhuber J (2005) Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw 18(5–6):602–610

    Article  Google Scholar 

  23. Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder–decoder approaches. arXiv preprint arXiv:1409.1259

  24. Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075

  25. Zhao J, Zhan Z, Yang Q, Zhang Y, Hu C, Li Z, Zhang L, He Z (2018) Adaptive learning of local semantic and global structure representations for text classification. In: Proceedings of the international conference on computational linguistics, pp 2033–2043

  26. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  27. Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of the conference of the North American chapter of the association for computational linguistics, pp 2227–2237

  28. Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training

  29. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  30. Si C, Chen W, Wang W, Wang L, Tan T (2019) An attention enhanced graph convolutional LSTM network for skeleton-based action recognition. In: Proceedings of the international conference on computer vision and pattern recognition, pp 1227–1236

  31. Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the international conference on knowledge discovery and data mining, pp 974–983

  32. Bian T, Xiao X, Xu T, Zhao P, Huang W, Rong Y, Huang J (2020) Rumor detection on social media with bi-directional graph convolutional networks. In: Proceedings of the association for the advance of artificial intelligence, vol 34, pp 549–556

  33. Peng H, Li J, He Y, Liu Y, Bao M, Wang L, Song Y, Yang Q (2018) Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In: Proceedings of the world wide web conference, pp 1063–1072

  34. Liu X, You X, Zhang X, Wu J, Lv P (2020) Tensor graph convolutional networks for text classification. In: Proceedings of the association for the advance of artificial intelligence, pp 8409–8416

  35. Fan M, Cheng D, Yang F, Luo S, Luo Y, Qian W, Zhou A (2020) Fusing global domain information and local semantic information to classify financial documents. In: Proceedings of the international conference on information and knowledge management, pp 2413–2420

  36. Linmei H, Yang T, Shi C, Ji H, Li X (2019) Heterogeneous graph attention networks for semi-supervised short text classification. In: Proceedings of the conference on empirical methods in natural language processing, pp 4821–4830

  37. Li Y, Tarlow D, Brockschmidt M, Zemel R (2016) Gated graph sequence neural networks. In: Proceedings of the international conference on learning representations, pp 1532–1543

  38. Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the conference on empirical methods in natural language processing, pp 1631–1642

  39. Pang B, Lee L (2005) Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the annual meeting on association for computational linguistics, pp 115–124

  40. Joulin A, Grave É, Bojanowski P, Mikolov T (2017) Bag of tricks for efficient text classification. In: Proceedings of the conference of the European chapter of the association for computational linguistics, pp 427–431

  41. Wu X, Luo Z, Du Z, Wang J, Gao C, Li X (2021) TW-TGNN: TWO windows graph-based model for text classification. In: Proceedings of the international joint conference on neural networks, pp 1–8. IEEE

  42. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the conference on empirical methods in natural language processing, pp 1532–1543

  43. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of the international conference on learning representations, pp 3435–3441

  44. Chen L, Jiang L, Li C (2021) Modified DFS-based term weighting scheme for text classification. Expert Syst Appl 168:114438

    Article  Google Scholar 

  45. Zhang L, Jiang L, Li C (2019) A discriminative model selection approach and its application to text classification. Neural Comput Appl 31(4):1173–1187

    Article  Google Scholar 

  46. Lvd M, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579–2605

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the Key Program for International Science and Technology Cooperation Projects of China (No. 2022YFE0112300), National Natural Science Foundation for Distinguished Young Scholars (No. 62025602), National Natural Science Foundation of China (No. 61976181), Key Technology Research and Development Program of Science and Technology-Scientific and Technological Innovation Team of Shaanxi Province (No. 2020TD-013), Natural Science Basic Research Plan in Shaanxi Province of China (No. 2022JM-325), Fundamental Research Funds for the Central Universities (No. D5000210827).

Author information

Authors and Affiliations

  1. The School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, 710072, Shaanxi, China

    Xianghua Li, Zhen Wang & Chao Gao

  2. The College of Computer and Information Science, Southwest University, Chongqing, Chongqing, 400715, China

    Xinyu Wu & Zheng Luo

  3. School of Public Health, The University of Hong Kong, Hong Kong, China

    Zhanwei Du

Authors
  1. Xianghua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinyu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhanwei Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chao Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao Gao.

Ethics declarations

Conflict of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Wu, X., Luo, Z. et al. Integration of global and local information for text classification. Neural Comput & Applic 35, 2471–2486 (2023). https://doi.org/10.1007/s00521-022-07727-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07727-y

Keywords

Search

Navigation