Abstract
As a fundamental task in natural language processing, text classification, which is to predict the class label of a given text, has been intensively studied. Consequently, a host of techniques have been developed, among which techniques that are based on graph neural network and its variant \(e.g., \) graph attention network (\(\textsc {GAT}\)) achieved impressive performances, as they show superiority in dealing with complex graph-structured data. Despite effectiveness, most of these techniques suffer from several limitations, \(e.g., \) incapability in well-capturing correlation among words in a text. In light of these, we propose a comprehensive approach \(\textsc {KGAT}\) which incorporates multi-head \(\textsc {GAT}\) with enhanced attention and customized ReadOut operation for text classification. (1) Our approach constructs a text graph \(G_T\) with edge weights from a text such that both semantic and structural information (with correlation degree) can be well captured. (2) On text graph \(G_T\), a novel attention mechanism is incorporated in a multi-head \(\textsc {GAT}\) for representation learning. (3) Our approach customizes ReadOut operation such that the representation of a text is refined by using a set of influential nodes of \(G_T\). Intensive experimental studies on both typical benchmark datasets and a newly created one (\({\textsf{Sensitive}}\)) show that our approach substantially outperforms other baseline methods and yields a promising technique for text classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cheng, J., Dong, L., Lapata, M.: Long short-term memory-networks for machine reading. In: EMNLP, pp. 551–561 (2016)
Ding, K., Wang, J., Li, J., Li, D., Liu, H.: Be more with less: hypergraph attention networks for inductive text classification. In: EMNLP, pp. 4927–4936 (2020)
Forman, G.: BNS feature scaling: an improved representation over TF-IDF for SVM text classification. In: CIKM, pp. 263–270 (2008)
Huang, L., Ma, D., Li, S., Zhang, X., Wang, H.: Text level graph neural network for text classification. In: EMNLP-IJCNLP, pp. 3442–3448 (2019)
Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: EACL, pp. 427–431 (2017)
Kim, S.B., Han, K.S., Rim, H.C., Myaeng, S.H.: Some effective techniques for Naive Bayes text classification. IEEE TKDE 18(11), 1457–1466 (2006)
Kim, Y.: Convolutional neural networks for sentence classification. In: EMNLP, pp. 1746–1751 (2014)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: AAAI, pp. 2267–2273 (2015)
Li, X., Roth, D.: Learning question classifiers. In: COLING (2002)
Lin, Y., Xu, G., Xu, G., Chen, Y., Sun, D.: Sensitive information detection based on convolution neural network and bi-directional LSTM. In: TrustCom, pp. 1614–1621 (2020)
Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: IJCAI, pp. 2873–2879 (2016)
Liu, Y., Guan, R., Giunchiglia, F., Liang, Y., Feng, X.: Deep attention diffusion graph neural networks for text classification. In: EMNLP, pp. 8142–8152 (2021)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: ICLR (2013)
Mikolov, T., Karafiát, M., Burget, L., Cernocký, J., Khudanpur, S.: Recurrent neural network based language model. In: ISCA, pp. 1045–1048 (2010)
Peng, H., et al.: Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In: Proceedings of the 2018 World Wide Web Conference, pp. 1063–1072 (2018)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: EMNLP, pp. 1532–1543 (2014)
Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2009)
Tan, S.: An effective refinement strategy for KNN text classifier. Elsevier ESWA 30(2), 290–298 (2006)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR (2018)
Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: EMNLP, pp. 606–615 (2016)
Was, T., Skibski, O.: An axiomatization of the eigenvector and Katz centralities. In: AAAI, pp. 1258–1265 (2018)
Wu, F., de Souza, A.H., Zhang, T., Fifty, C., Yu, T., Weinberger, K.Q.: Simplifying graph convolutional networks. In: ICML, pp. 6861–6871 (2019)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE TNNLS 32(1), 4–24 (2021)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A.J., Hovy, E.H.: Hierarchical attention networks for document classification. In: NAACL-HLT, pp. 1480–1489 (2016)
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: AAAI, pp. 7370–7377 (2019)
Zhang, Y., Yu, X., Cui, Z., Wu, S., Wen, Z., Wang, L.: Every document owns its structure: inductive text classification via graph neural networks. In: ACL, pp. 334–339 (2020)
Zhang, Z., Cui, P., Zhu, W.: Deep learning on graphs: a survey. IEEE TKDE 34(1), 249–270 (2022)
Zhou, J., et al.: Graph neural networks: a review of methods and applications. AI Open 1, 57–81 (2020)
Zhou, X., Wan, X., Xiao, J.: Attention-based LSTM network for cross-lingual sentiment classification. In: EMNLP, pp. 247–256 (2016)
Acknowledgement
This work is supported by Sichuan Scientific Innovation Fund (No. 2022JDRC0009) and the National Key Research and Development Program of China (No. 2017YFA0700800).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, X. et al. (2022). KGAT: An Enhanced Graph-Based Model for Text Classification. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13551. Springer, Cham. https://doi.org/10.1007/978-3-031-17120-8_51
Download citation
DOI: https://doi.org/10.1007/978-3-031-17120-8_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17119-2
Online ISBN: 978-3-031-17120-8
eBook Packages: Computer ScienceComputer Science (R0)
