Abstract
Text classification, aiming for discovering corresponding relationships between labels and texts, is a pivotal task in Natural Language Processing (NLP). The existing joint text-label models help input texts to establish early global category semantic awareness via label embedding techniques, but they cannot simultaneously capture literal and semantic relationships between texts and labels. It may lead models to ignore obvious clues or semantic relations on different cognitive levels. In this paper, we propose a Bidirectional Multi-channel semantic Interaction model (BMI) to handle both explicit and implicit category semantics in texts for text classification. On the explicit semantic level, BMI designs a word representation similarity match channel for shallow interaction to get rid of semantic mismatch based on assumptions that words have different meanings under the same context. On the implicit semantic level, BMI provides a novel attended attention mechanism over texts and labels for deep interaction to model bidirectional text explanation for labels and label guidance for texts. Furthermore, a gated residual mechanism is employed to obtain core information of labels to improve efficiency. Experiments on benchmark datasets show that BMI achieves competitive results over 15 strong baseline methods, especially in the case of short texts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)
Chen, J., et al.: End-to-end learning of lda by mirror-descent back propagation over a deep architecture. In: NIPS (2015)
Chung, J., Gülçehre Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR (2014)
Gehring, J., et al.: Convolutional sequence to sequence learning. In: ICML (2017)
Gong, C., Shi, K., Niu, Z.: Hierarchical text-label integrated attention network for document classification. In: HPCCT (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jiang, J., Zhang, J., Zhang, K.: Cascaded semantic and positional self-attention network for document classification. In: EMNLP (2020)
Joulin, J., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: EACL (2017)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: ACL (2014)
Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. In: ICLR (2015)
Lin, Z., et al.: A structured self-attentive sentence embedding. In: ICLR (Poster) (2017)
Pang, B., Lee, L.: Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In: ACL (2005)
Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: EMNLP (2014)
Schick, T., Schütze, H.: Exploiting cloze questions for few-shot text classification and natural language inference. In: EACL (2021)
Shen, D., et al.: Baseline needs more love: on simple word-embedding-based models and associated pooling mechanisms. In: ACL (2018)
Shen, T., Zhou, T., Long, G., Jiang, J., Pan, S., Zhang, C.: Disan: directional self-attention network for rnn/cnn-free language understanding. In: AAAI (2018)
Shen, T., Zhou, T., Long, G., Jiang, J., Zhang, C.: Bi-directional block self-attention for fast and memory-efficient sequence modeling. In: ICLR (2018)
Sun, J., Jin, R., Ma, X., Park, J.Y., Sohn, K.A., Chung, T.S.: Gated convolutional neural networks for text classification. Adv. Comput. Sci. Ubiquit. Comput. 32(1), 309–316 (2021)
Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: EMNLP (2015)
Vaswani, A., et al.: Attention is all you need. In: NIPS (2017)
Wang, G., et al.: Joint embedding of words and labels for text classification. In: ACL (2018)
Xiao, L., Huang, X., Chen, B., Jing, L.: Label-specific document representation for multi-label text classification. In: EMNLP (2019)
Zanzotto, F.M., Santilli, A., Ranaldi, L., Onorati, D., Tommasino, P., Fallucchi, F.: Kermit: complementing transformer architectures with encoders of explicit syntactic interpretations. In: EMNLP (2020)
Zhang, H., Xiao, L., Chen, W., Wang, Y., Jin, Y.: Multi-task label embedding for text classification. In: EMNLP (2018)
Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: NIPS (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Y., Zhou, Y., Hu, P., Xu, M., Zhao, T., Chen, Y. (2022). Bidirectional Multi-channel Semantic Interaction Model of Labels and Texts for Text Classification. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13552. Springer, Cham. https://doi.org/10.1007/978-3-031-17189-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-17189-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17188-8
Online ISBN: 978-3-031-17189-5
eBook Packages: Computer ScienceComputer Science (R0)
