Skip to main content
SpringerLink
Log in

Local discriminative graph convolutional networks for text classification

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Recently, graph convolutional networks (GCNs) has demonstrated great success in the text classification. However, the GCN only focuses on the fitness between the ground-truth labels and the predicted ones. Indeed, it ignores the local intra-class diversity and local inter-class similarity that is implicitly encoded by the graph, which is an important cue in machine learning field. In this paper, we propose a local discriminative graph convolutional network (LDGCN) to boost the performance of text classification. Different from the text GCN that minimize only the cross entropy loss, our proposed LDGCN is trained by optimizing a new discriminative objective function. So that, in the new LDGCN feature spaces, the texts from the same scene class are mapped closely to each other and the texts of different classes are mapped as farther apart as possible. So as to ensure that the features extracted by GCN have good discriminative ability, achieve the maximum separability of samples. Experimental results demonstrate its superiority against the baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://disi.unitn.it/moschitti/corpora.htm.

  2. https://www.cs.umb.edu/smimarog/textmining/datasets/

  3. http://www.cs.cornell.edu/people/pabo/movie-review-data/.

  4. http://qwone.com/jason/20Newsgroups/.

  5. http://www.nltk.org/.

References

  1. Phan, H.T., Nguyen, N.T., Hwang, D.: Aspect-level sentiment analysis: a survey of graph convolutional network methods. Inf. Fus. 91, 149–172 (2023)

    Article  Google Scholar 

  2. Parlak, B., Uysal, A.K.: A novel filter feature selection method for text classification: extensive feature selector. J. Inf. Sci. 49(1), 59–78 (2023)

    Article  Google Scholar 

  3. Rao, S., Verma, A.K., Bhatia, T.: A review on social spam detection: challenges, open issues, and future directions. Expert Syst. Appl. 186, 115742 (2021)

    Article  Google Scholar 

  4. Chen, Y.: Convolutional neural network for sentence classification. Master’s thesis, University of Waterloo (2015)

  5. Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., Xu, B.: Text classification improved by integrating bidirectional lstm with two-dimensional max pooling. arXiv preprint arXiv:1611.06639 (2016)

  6. Peng, H., Li, J., He, Y., Liu, Y., Bao, M., Wang, L., Song, Y., Yang, Q.: Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In: Proceedings of the 2018 World Wide Web Conference, pp. 1063–1072 (2018)

  7. Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 7370–7377 (2019)

  8. Vashishth, S., Bhandari, M., Yadav, P., Rai, P., Bhattacharyya, C., Talukdar, P.: Incorporating syntactic and semantic information in word embeddings using graph convolutional networks. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3308–3318 (2019)

  9. Liu, X., You, X., Zhang, X., Wu, J., Lv, P.: Tensor graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 8409–8416 (2020)

  10. Ragesh, R., Sellamanickam, S., Iyer, A., Bairi, R., Lingam, V.: Hetegcn: heterogeneous graph convolutional networks for text classification. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 860–868 (2021)

  11. Liu, Y., Guan, R., Giunchiglia, F., Liang, Y., Feng, X.: Deep attention diffusion graph neural networks for text classification. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8142–8152 (2021)

  12. Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. Adv. Neural Inf. Process. Syst. 28 (2015)

  13. Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1556–1566 (2015)

  14. Campos Camunez, V., Jou, B., Giró Nieto, X., Torres Viñals, J., Chang, S.-F.: Skip rnn: learning to skip state updates in recurrent neural networks. In: Sixth International Conference on Learning Representations: Monday April 30-Thursday May 03, 2018, Vancouver Convention Center, Vancouver:[proceedings], pp. 1–17 (2018)

  15. Chang, S., Zhang, Y., Han, W., Yu, M., Guo, X., Tan, W., Cui, X., Witbrock, M., Hasegawa-Johnson, M.A., Huang, T.S.: Dilated recurrent neural networks. Adv. Neural Inf. Process. Syst. 30 (2017)

  16. Yin, W., Schütze, H., Xiang, B., Zhou, B.: Abcnn: attention-based convolutional neural network for modeling sentence pairs. Trans. Assoc. Comput. Linguist. 4, 259–272 (2016)

    Article  Google Scholar 

  17. Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 2873–2879 (2016)

  18. Kenton, J.D.M.-W.C., Toutanova, L.K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)

  19. Oh, S.H., Kang, M., Lee, Y.: Protected health information recognition by fine-tuning a pre-training transformer model. Healthc. Inform. Res. 28(1), 16–24 (2022)

    Article  Google Scholar 

  20. Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., Pei, J., Long, B.: Graph neural networks for natural language processing: a survey. Found. Trends Mach. Learn. 16(2), 119–328 (2023)

    Article  MATH  Google Scholar 

  21. Wu, J., Zhang, C., Liu, Z., Zhang, E., Wilson, S., Zhang, C.: Graphbert: Bridging graph and text for malicious behavior detection on social media. In: 2022 IEEE International Conference on Data Mining (ICDM), pp. 548–557 (2022). IEEE

  22. Yang, Y., Miao, R., Wang, Y., Wang, X.: Contrastive graph convolutional networks with adaptive augmentation for text classification. Inf. Process. Manag. 59(4), 102946 (2022)

    Article  Google Scholar 

  23. Krishnaveni, P., Balasundaram, S.: Generating fuzzy graph based multi-document summary of text based learning materials. Expert Syst. Appl. 214, 119165 (2023)

    Article  Google Scholar 

  24. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations

  25. Cao, Y., Liu, Z., Li, C., Li, J., Chua, T.-S.: Multi-channel graph neural network for entity alignment. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1452–1461 (2019)

  26. Dai, Y., Shou, L., Gong, M., Xia, X., Kang, Z., Xu, Z., Jiang, D.: Graph fusion network for text classification. Knowl. Based Syst. 236, 107659 (2022)

    Article  Google Scholar 

  27. Zhu, X., Zhu, L., Guo, J., Liang, S., Dietze, S.: Gl-gcn: global and local dependency guided graph convolutional networks for aspect-based sentiment classification. Expert Syst. Appl. 186, 115712 (2021)

    Article  Google Scholar 

  28. Jin, D., Song, X., Yu, Z., Liu, Z., Zhang, H., Cheng, Z., Han, J.: Bite-gcn: A new gcn architecture via bidirectional convolution of topology and features on text-rich networks. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 157–165 (2021)

  29. Jin, T., Cao, L., Zhang, B., Sun, X., Deng, C., Ji, R.: Hypergraph induced convolutional manifold networks. In: IJCAI, pp. 2670–2676 (2019)

  30. Deng, Y., Yang, J., Xiang, J., Tong, X.: Gram: Generative radiance manifolds for 3d-aware image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10673–10683 (2022)

  31. Vepakomma, P., Balla, J., Raskar, R.: Privatemail: Supervised manifold learning of deep features with privacy for image retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 8503–8511 (2022)

  32. Sugiyama, M.: Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J. Mach. Learn. Res. 8(5) (2007)

  33. Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on Machine Learning, pp. 1188–1196 (2014). PMLR

  34. Joulin, A., Grave, É., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 427–431 (2017)

  35. Tang, J., Qu, M., Mei, Q.: Pte: Predictive text embedding through large-scale heterogeneous text networks. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1165–1174 (2015)

  36. Shen, D., Wang, G., Wang, W., Min, M.R., Su, Q., Zhang, Y., Li, C., Henao, R., Carin, L.: Baseline needs more love: On simple word-embedding-based models and associated pooling mechanisms. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 440–450 (2018)

  37. Wang, G., Li, C., Wang, W., Zhang, Y., Shen, D., Zhang, X., Henao, R., Carin, L.: Joint embedding of words and labels for text classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2321–2331 (2018)

  38. Liu, T., Zhang, X., Zhou, W., Jia, W.: Neural relation extraction via inner-sentence noise reduction and transfer learning. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2195–2204 (2018)

  39. Huang, L., Ma, D., Li, S., Zhang, X., Wang, H.: Text level graph neural network for text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3444–3450 (2019)

  40. Zhang, C., Zhu, H., Peng, X., Wu, J., Xu, K.: Hierarchical information matters: Text classification via tree based graph neural network. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 950–959 (2022)

  41. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  42. Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program of China (No. 2022YFC3301801), the Fundamental Research Funds for the Central Universities (No. DUT22ZD205).

Author information

Authors and Affiliations

  1. Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, No.2 Linggong Road, Dalian, 116024, Liaoning, China

    Bolin Wang, Yuanyuan Sun, Changrong Min, Zhihao Yang & Hongfei Lin

  2. School of Information Science and Engineering, Henan University of Technology, No. 100 Lianhua Street, Zhengzhou, 450001, Henan, China

    Yonghe Chu

Authors
  1. Bolin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanyuan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yonghe Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Changrong Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhihao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongfei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanyuan Sun.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, B., Sun, Y., Chu, Y. et al. Local discriminative graph convolutional networks for text classification. Multimedia Systems 29, 2363–2373 (2023). https://doi.org/10.1007/s00530-023-01112-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01112-y

Keywords

Search

Navigation