Abstract
In text classification tasks, RNNs are usually used to establish global relationships. However, RNNs have the problems that the semantic information coding of key words is not prominent and cannot be calculated in parallel. In addition, hierarchical information of text is usually ignored during feature extraction. Aiming at the above problems, a text classification model based on hierarchical capsule network (HCapsNet) is proposed. In order to capture the hierarchical features, text is divided into granularities and constantly aggregate according to the characteristics of the data. A parallel LSTM network fused with self-attention is utilized to complete the encoding of multiple natural sentences. Then, we construct sentence features into sentence capsules to extract richer semantic information. The spatial relationship between sentence capsule as part and chapter capsule as whole is established by dynamic routing algorithm. Our experiments show that HCapsNet gives better results compared with the state-of-the-art methods on six public data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lu, Z., Gai, K., Duan, Q., Xu, Y.: Machine learning empowered content delivery: status challenges and opportunities. IEEE Netw. 34, 228–234 (2020)
Dai, W., Qiu, L., Wu, A., Qiu, M.: Cloud infrastructure resource allocation for big data applications. IEEE Trans. Big Data 4, 313–324 (2016)
Li, Q., et al.: A survey on text classification: from shallow to deep learning. ACM Comput. 4–41 (2020)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceeding of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751 (2014)
Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Meeting of the Association for Computational Linguistics, pp. 562–570 (2017)
Dai, J., Chen, C.: A backdoor attack against LSTM-based text classification systems. IEEE Access 7, 138872–138878 (2019)
Chowdhury, S., Rahman, M., Ali, S.: A RNN based parallel deep learning framework for detecting sentiment polarity from Twitter derived textual data. In: 11th International Conference on Electrical and Computer Engineering (2020)
Lin, J.C.-W., Shao, Y., Djenouri, Y., Yun, U.: ASRNN: A recurrent neural network with an attention model for sequence labeling. Knowl.-Based Syst. 212, 106548–106556 (2020)
Katarya, R., Arora, Y.: Study on text classification using capsule networks. In: 2019 5th International Conference on Advanced Computing & Communication Systems (2019)
Chen, B., Xu, Z., Wang, X., Long, X., Zhang, W.: Capsule network-based text sentiment classification. IFAC-PapersOnLine 53, 698–703 (2020)
Sabour, S., Frosst, N.: Dynamic routing between capsules. In: Conference and Workshop on Neural Information Processing Systems (NIPS), pp. 3856–3866 (2017)
Bing, L., Pan, W., Lu, J.: Multi-granularity dynamic analysis of complex software networks. In: IEEE International Symposium on Circuits & Systems (2011)
Pavlinek, M., Podgorelec, V.: Text classification method based on self-training and LDA topic models. Expert Syst. Appl. 80, 83–93 (2017)
Ge, J., Lin, S., Fang, Y.: A text classification algorithm based on topic model and convolutional neural network. J. Phys. Conf. Ser. 32–36 (2021)
Zeng, J., Li, J., Song, Y.: Topic memory networks for shorttext classification. In: Proceedings of Empirical Methods in Natural Language Processing, Brussels, Belgium, EMNLP, pp. 3120–3131 (2018)
Yang, Z., Yang, D., Dyer, C.: Hierarchical attention networks for document classification. In: Annual Meeting of the Association for Computational Linguistics, pp. 1480–1489 (2016)
Tong, G., Li, Y., Gao, H., Chen, H., Wang, H., Yang, X.: MA-CRNN: a multi-scale attention CRNN for Chinese text line recognition in natural scenes. Int. J. Document Anal. Recogn. (IJDAR) 23(2), 103–114 (2019). https://doi.org/10.1007/s10032-019-00348-7
Klaren, B., Ek, G., Harmanny, R., Cifola, L.: Multi-target human gait classification using LSTM recurrent neural networks applied to micro-doppler. In: European Radar Conference, pp. 167–170 (2017)
Nie, Y., Bansal, M.: Shortcut-stacked sentence encoders for multi-domain inference. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pp. 41–45 (2017)
Zhou, C., Sun, C., Liu, Z.: A C-LSTM neural network for text classification. Comput. Sci. 1, 39–44 (2015)
Cao, Y., Ma, S., Pan, H.: FDTA: fully convolutional scene text detection with text attention. IEEE Access 8, 155441–155449 (2020)
Zhan, Z., Hou, Z., Yang, Q.: Knowledge attention sandwich neural network for text classification. Neurocomputing 406, 1–11 (2020)
Tang, X., Chen, Y., Dai, Y.: A multi-scale convolutional attention based GRU network for text classification. In: 2019 Chinese Automation Congress (2020)
Min, Y., Wei, Z., Lei, C.: Investigating the transferring capability of capsule networks for text classification . Neural Netw. 118, 247–261 (2019)
Zhao, W., Ye, J., Yang, M.: Investigating capsule networks with dynamic routing for text classification. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pp. 3110–3119 (2018)
Kim, J., Jang, S.: Text classification using capsules. Neurocomputing. 376, 214–221 (2020)
Gong, J., Qiu, X., Wang, S.: Information aggregation via dynamic routing for sequence encoding. In: Annual Meeting of the Association for Computational Linguistics, pp. 2742–2752 (2018)
Deng, X., Yin, S., Deng, H.: A short text classification model based on cross-layer connected gated recurrent unit capsule network. In: Big Data, pp. 1–17 (2020)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Y., Ye, M., Hu, Q. (2021). HCapsNet: A Text Classification Model Based on Hierarchical Capsule Network. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management . KSEM 2021. Lecture Notes in Computer Science(), vol 12816. Springer, Cham. https://doi.org/10.1007/978-3-030-82147-0_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-82147-0_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82146-3
Online ISBN: 978-3-030-82147-0
eBook Packages: Computer ScienceComputer Science (R0)