Abstract
Scientific paper classification refers to assigning one or more subject categories to papers. This task requires a lot of domain knowledge and heavy manual annotation. With the gradual increase in interdisciplinary research, a paper often has multiple categories. For instance, both Chinese Library Classification (http://www.ztflh.com/) and Engineering Village (EI) have a complete classification system, and there is a hierarchical relationship between the categories. The category of the paper has a hierarchical structure, so the paper classification can be converted into a hierarchical classification problem. However, the existing methods cannot effectively classify papers due to the following two reasons: First, these methods cannot well capture the semantic relationship between papers. Second, they neglect to model the hierarchical structure of labels. In this paper, we propose a hierarchical label attention model based on graph attention network, which utilizes word co-occurrence to model the semantic relationship of papers. We use multiple linear layers to model the category hierarchy and combine every hierarchy of labels through an attention mechanism. The experiments are conducted on CNKI (https://www.cnki.net/) and RCV1 datasets. The experimental results demonstrate that our method is superior to the other methods in the task of scientific paper classification.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhang, W., Yan, J., Wang, X., Zha, H.: Deep extreme multi-label learning. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, ICMR 2018, pp. 100–107. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3206025.3206030
Zhang, J., Lin, Y., Jiang, M., Li, S., Tang, Y., Tan, K.C.: Multi-label feature selection via global relevance and redundancy optimization. In: Bessiere, C. (ed.) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 2512–2518. International Joint Conferences on Artificial Intelligence Organization, July 2020. Main track
Feng, L., An, B., He, S.: Collaboration based multi-label learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 3550–3557 (2019)
Lv, J., Xu, N., Zheng, R., Geng, X.: Weakly supervised multi-label learning via label enhancement. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 3101–3107. International Joint Conferences on Artificial Intelligence Organization, July 2019. https://doi.org/10.24963/ijcai.2019/430
Xing, Y., Yu, G., Domeniconi, C., Wang, J., Zhang, Z.: Multi-label co-training. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 2882–2888. International Joint Conferences on Artificial Intelligence Organization, July 2018. https://doi.org/10.24963/ijcai.2018/400
You, R., Zhang, Z., Wang, Z., Dai, S., Mamitsuka, H., Zhu, S.: AttentionXML: label tree-based attention-aware deep model for high-performance extreme multi-label text classification. arXiv preprint arXiv:1811.01727 (2018)
Xun, G., Jha, K., Sun, J., Zhang, A.: Correlation networks for extreme multi-label text classification. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2020, pp. 1074–1082. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3394486.3403151
Shi, W., Sheng, V.S., Li, X., Gu, B.: Semi-supervised multi-label learning from crowds via deep sequential generative model. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2020, pp. 1141–1149. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3394486.3403167
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 7370–7377 (2019)
Tang, P., Jiang, M., Xia, B.N., Pitera, J.W., Welser, J., Chawla, N.V.: Multi-label patent categorization with non-local attention-based graph convolutional network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 9024–9031 (2020)
Wang, Y., et al.: Multi-label classification with label graph superimposing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 12 265–12 272 (2020)
Chen, B., Huang, X., Xiao, L., Cai, Z., Jing, L.: Hyperbolic interaction model for hierarchical multi-label classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 7496–7503 (2020)
Wehrmann, J., Cerri, R., Barros, R.: Hierarchical multi-label classification networks. In: International Conference on Machine Learning, pp. 5075–5084. PMLR (2018)
Yan, Y.-F., Huang, S.-J.: Cost-effective active learning for hierarchical multi-label classification. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 2962–2968. International Joint Conferences on Artificial Intelligence Organization, July 2018. https://doi.org/10.24963/ijcai.2018/411
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017. Conference Track Proceedings. OpenReview.net (2017). https://openreview.net/forum?id=SJU4ayYgl
Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004). http://jmlr.org/papers/volume5/lewis04a/lewis04a.pdf
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, June 2019. https://www.aclweb.org/anthology/N19-1423
Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Kambhampati, S. (ed.) Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2873–2879. IJCAI/AAAI Press (2016). http://www.ijcai.org/Abstract/16/408
Wang, B.: Disconnected recurrent neural networks for text categorization. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2311–2320. Association for Computational Linguistics, Melbourne, July 2018. https://www.aclweb.org/anthology/P18-1215
Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 562–570. Association for Computational Linguistics, Vancouver, July 2017. https://www.aclweb.org/anthology/P17-1052
Yin, W., Schütze, H.: Attentive convolution: equipping CNNs with RNN-style attention mechanisms. Trans. Assoc. Comput. Linguist. 6, 687–702 (2018)
Acknowledgements
This work was supported by National Key R&D Program of China (2018YFB1402600), and by the National Natural Science Foundation of China (61802028, 61772083, 61877006, 62002027), and sponsored by CCF-Baidu Open Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, C., Xue, Z., Du, J., Kou, F., Liang, M., Xu, M. (2021). A Hierarchical Multi-label Classification Algorithm for Scientific Papers Based on Graph Attention Networks. In: Fang, L., Chen, Y., Zhai, G., Wang, J., Wang, R., Dong, W. (eds) Artificial Intelligence. CICAI 2021. Lecture Notes in Computer Science(), vol 13069. Springer, Cham. https://doi.org/10.1007/978-3-030-93046-2_62
Download citation
DOI: https://doi.org/10.1007/978-3-030-93046-2_62
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93045-5
Online ISBN: 978-3-030-93046-2
eBook Packages: Computer ScienceComputer Science (R0)