Abstract
Scientific research literature usually has multi-level labels, and there are often dependencies between multi-level labels. It is crucial for the model to learn and integrate the information between multi-level labels for the hierarchical multi-label text classification (HMTC) of scientific research literature texts. Therefore, for the HMTC task in the scientific research literature, we use the pre-trained language model SciBERT trained on scientific texts. And we introduce a shared TextCNN layer in our multi-task learning architecture to learn the dependency information between labels at each level. Then the hierarchical feature information is fused and propagated from top to bottom according to the task level. We conduct ablation experiments on the dependency information interaction module and the hierarchical information fusion propagation module. Experimental results on the NLPCC2022 SharedTask5 Track1 dataset demonstrate the effectiveness of our model, and we rank 4th place in the task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
\(B C E {\text {Loss}}(x, y)=-(y \log x+(1-y) \log (1-x))\).
References
Beltagy, I., Lo, K., Cohan, A.: SciBERT: a pretrained language model for scientific text. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3615–3620 (2019)
Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)
Chen, Y.: Convolutional neural network for sentence classification. Master’s thesis, University of Waterloo (2015)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Fox, E.A., Akscyn, R.M., Furuta, R.K., Leggett, J.J.: Digital libraries. Commun. ACM 38(4), 22–28 (1995)
Garfield, E.: The evolution of the science citation index. Int. Microbiol. 10(1), 65 (2007)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Lewandowski, D.: Web searching, search engines and information retrieval. Inf. Serv. Use 25(3–4), 137–147 (2005)
Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)
Liu, J., Chang, W.C., Wu, Y., Yang, Y.: Deep learning for extreme multi-label text classification. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 115–124 (2017)
LTD, C.K.: Datasets for NLPCC2022.SharedTask5.Track1. https://doi.org/10.11922/sciencedb.j00104.00100
Peng, H., et al.: Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In: Proceedings of the 2018 World Wide Web Conference, pp. 1063–1072 (2018)
Ruder, S.: An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017)
Silla, C.N., Freitas, A.A.: A survey of hierarchical classification across different application domains. Data Min. Knowl. Disc. 22(1), 31–72 (2011)
Wu, T., Huang, Q., Liu, Z., Wang, Yu., Lin, D.: Distribution-balanced loss for multi-label classification in long-tailed datasets. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 162–178. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_10
Xu, L., et al.: Hierarchical multi-label text classification with horizontal and vertical category correlations. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2459–2468 (2021)
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 7370–7377 (2019)
Acknowledgment
This work was supported by the National Key R &D Program of China (Grant No. 2018YFB1404500 and No. 2018YFB1404503).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, X. et al. (2022). An Interactive Fusion Model for Hierarchical Multi-label Text Classification. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13552. Springer, Cham. https://doi.org/10.1007/978-3-031-17189-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-17189-5_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17188-8
Online ISBN: 978-3-031-17189-5
eBook Packages: Computer ScienceComputer Science (R0)
