Abstract
Text classification, a fundamental task in natural language processing, has been extensively studied by researchers worldwide. The primary focus of text classification is on extracting effective features from text, as accurate information extraction is crucial for the task. However, the current utilization of text information in text classification is not optimal, and thus, effective extraction of text information remains an important research topic. Graph attention networks (GATs) have gained popularity among researchers due to their excellent performance in various tasks, including text classification. Additionally, previous graph neural network-derived models only address the differences in importance of edges within nodes during the information aggregation process, but ignore the importance differences between different nodes. But, the correlation between nodes also needs to be exploited for a more comprehensive understanding of the text. In this paper, we propose a novel multi-display graph attention network (MDGAT)-based model to address the challenges of inadequate text information capture and higher-order interactions between words. Our approach involves fusing multiple display graphs to capture diverse features of the text, for comprehensive text information representation. Additionally, we introduce a new information aggregation method called multi-step information aggregation, which considers the importance within nodes and the correlation between nodes, leading to improved text representation learning. We validate the performance of our proposed method through extensive experiments on various benchmark datasets. The results demonstrate the superior performance of our approach in text classification tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Minaee, S., Kalchbrenner, N., Cambria, E., et al.: Deep learning-based text classification: a comprehensive review. ACM Comput. Surv. (CSUR) 54(3), 1–40 (2021)
Marelli, M., Bentivogli, L., Baroni, M., Bernardi, R., Menini, S., Zamparelli, R.: Semeval-2014 task 1: evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp. 1–8 (2014)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751 (2014)
Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 2873–2879 (2016)
Tai, K.S., Socher, R., Manning, C.D.: Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks: Association for Computational Linguistics (ACL) (2015). https://doi.org/10.3115/v1/P15-1150
Cho, K., Van Merriënboer, B., Gulcehre, C., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Kipf, T.N., Welling, M.: Graph neural networks: a review of methods and applications. AI Open 1, 57–81 (2020)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2021). https://doi.org/10.1109/TNNLS.2020.2978386
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: NeurIPS (2017)
Brody, S., Alon, U., Yahav, E.: How attentive are graph attention networks? arXiv preprint arXiv:2105.14491 (2021)
Veličković, P., Cucurull, G., Casanova, A., et al.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 1, pp. 7370–7377 (2019)
Huang, L., Ma, D., Li, S., Zhang, X., Wang, H.: Text level graph neural network for text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 3435–3441 (2019)
Sun, K., Zhang, R., Mensah, S., et al.: Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5679–5688 (2019)
Wang, Z., Wang, C., Zhang, H., et al.: Learning dynamic hierarchical topic graph with graph convolutional network for document classification. In: International Conference on Artificial Intelligence and Statistics, pp. 3959–3969. PMLR (2020)
Tang, H., Ji, D., Li, C., et al.: Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6578–6588 (2020)
Ding, K., Wang, J., Li, J., Li, D., Liu, H.: Be more with less: hypergraph attention networks for inductive text classification. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 4927–4936 (2020)
Bai, S., Zhang, F., Torr, P.H.S.: Hypergraph convolution and hypergraph attention. Pattern Recogn. 110, 107637 (2021)
Feng, Y., You, H., Zhang, Z., et al.: Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 1, pp. 3558–3565 (2019)
Lin, Y., Meng, Y., Sun, X., et al.: BertGCN: transductive text classification by combining GCN and BERT. arXiv preprint arXiv:2105.05727 (2021)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics (2019)
Liu, Y., Guan, R., Giunchiglia, F., Liang, Y., Feng, X.: Deep attention diffusion graph neural networks for text classification. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8142–8152 (2021)
Zhang, Z., Zhou, Z., Wang, Y.: SSEGCN: syntactic and semantic enhanced graph convolutional network for aspect-based sentiment analysis. In: Proceedings of the. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4916–4925 (2022)
Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 427–431 (2017)
Shen, D., Wang, G., Wang, W., et al.: Baseline needs more love: on simple word-embedding-based models and associated pooling mechanisms. arXiv preprint arXiv:1805.09843 (2018)
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
Peng, H., Li, J., He, Y., et al.: Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In: Proceedings of the 2018 World Wide Web Conference, pp. 1063–1072 (2018)
Ragesh, R., Sellamanickam, S., Iyer, A., et al.: HeteGCN: heterogeneous graph convolutional networks for text classification. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 860–868 (2021)
Liu, X., You, X., Zhang, X., et al.: Tensor graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 8409–8416 (2020)
Guidotti, E., Ferrara, A.: Text classification with Born’s rule. Adv. Neural. Inf. Process. Syst. 35, 30990–31001 (2022)
Acknowledgements
This work was supported in part by the Shandong Provincial Natural Science Foundation (No. ZR2020MF149) and (No. ZR2021MD115), in part by the Science and Technology Commission of Shanghai Municipality (21511100302).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bao, X., Zhou, Z., Gao, S., Dong, Z., Lin, Y. (2023). Multi-display Graph Attention Network for Text Classification. In: Jin, Z., Jiang, Y., Buchmann, R.A., Bi, Y., Ghiran, AM., Ma, W. (eds) Knowledge Science, Engineering and Management. KSEM 2023. Lecture Notes in Computer Science(), vol 14119. Springer, Cham. https://doi.org/10.1007/978-3-031-40289-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-40289-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40288-3
Online ISBN: 978-3-031-40289-0
eBook Packages: Computer ScienceComputer Science (R0)