Skip to main content

A Hierarchical Multi-label Classification Algorithm for Scientific Papers Based on Graph Attention Networks

  • Conference paper
  • First Online:
Book cover Artificial Intelligence (CICAI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13069))

Included in the following conference series:

  • 2042 Accesses

Abstract

Scientific paper classification refers to assigning one or more subject categories to papers. This task requires a lot of domain knowledge and heavy manual annotation. With the gradual increase in interdisciplinary research, a paper often has multiple categories. For instance, both Chinese Library Classification (http://www.ztflh.com/) and Engineering Village (EI) have a complete classification system, and there is a hierarchical relationship between the categories. The category of the paper has a hierarchical structure, so the paper classification can be converted into a hierarchical classification problem. However, the existing methods cannot effectively classify papers due to the following two reasons: First, these methods cannot well capture the semantic relationship between papers. Second, they neglect to model the hierarchical structure of labels. In this paper, we propose a hierarchical label attention model based on graph attention network, which utilizes word co-occurrence to model the semantic relationship of papers. We use multiple linear layers to model the category hierarchy and combine every hierarchy of labels through an attention mechanism. The experiments are conducted on CNKI (https://www.cnki.net/) and RCV1 datasets. The experimental results demonstrate that our method is superior to the other methods in the task of scientific paper classification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhang, W., Yan, J., Wang, X., Zha, H.: Deep extreme multi-label learning. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, ICMR 2018, pp. 100–107. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3206025.3206030

  2. Zhang, J., Lin, Y., Jiang, M., Li, S., Tang, Y., Tan, K.C.: Multi-label feature selection via global relevance and redundancy optimization. In: Bessiere, C. (ed.) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 2512–2518. International Joint Conferences on Artificial Intelligence Organization, July 2020. Main track

    Google Scholar 

  3. Feng, L., An, B., He, S.: Collaboration based multi-label learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 3550–3557 (2019)

    Google Scholar 

  4. Lv, J., Xu, N., Zheng, R., Geng, X.: Weakly supervised multi-label learning via label enhancement. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 3101–3107. International Joint Conferences on Artificial Intelligence Organization, July 2019. https://doi.org/10.24963/ijcai.2019/430

  5. Xing, Y., Yu, G., Domeniconi, C., Wang, J., Zhang, Z.: Multi-label co-training. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 2882–2888. International Joint Conferences on Artificial Intelligence Organization, July 2018. https://doi.org/10.24963/ijcai.2018/400

  6. You, R., Zhang, Z., Wang, Z., Dai, S., Mamitsuka, H., Zhu, S.: AttentionXML: label tree-based attention-aware deep model for high-performance extreme multi-label text classification. arXiv preprint arXiv:1811.01727 (2018)

  7. Xun, G., Jha, K., Sun, J., Zhang, A.: Correlation networks for extreme multi-label text classification. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2020, pp. 1074–1082. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3394486.3403151

  8. Shi, W., Sheng, V.S., Li, X., Gu, B.: Semi-supervised multi-label learning from crowds via deep sequential generative model. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2020, pp. 1141–1149. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3394486.3403167

  9. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  10. Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 7370–7377 (2019)

    Google Scholar 

  11. Tang, P., Jiang, M., Xia, B.N., Pitera, J.W., Welser, J., Chawla, N.V.: Multi-label patent categorization with non-local attention-based graph convolutional network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 9024–9031 (2020)

    Google Scholar 

  12. Wang, Y., et al.: Multi-label classification with label graph superimposing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 12 265–12 272 (2020)

    Google Scholar 

  13. Chen, B., Huang, X., Xiao, L., Cai, Z., Jing, L.: Hyperbolic interaction model for hierarchical multi-label classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 7496–7503 (2020)

    Google Scholar 

  14. Wehrmann, J., Cerri, R., Barros, R.: Hierarchical multi-label classification networks. In: International Conference on Machine Learning, pp. 5075–5084. PMLR (2018)

    Google Scholar 

  15. Yan, Y.-F., Huang, S.-J.: Cost-effective active learning for hierarchical multi-label classification. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 2962–2968. International Joint Conferences on Artificial Intelligence Organization, July 2018. https://doi.org/10.24963/ijcai.2018/411

  16. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017. Conference Track Proceedings. OpenReview.net (2017). https://openreview.net/forum?id=SJU4ayYgl

  17. Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004). http://jmlr.org/papers/volume5/lewis04a/lewis04a.pdf

  18. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, June 2019. https://www.aclweb.org/anthology/N19-1423

  19. Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Kambhampati, S. (ed.) Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2873–2879. IJCAI/AAAI Press (2016). http://www.ijcai.org/Abstract/16/408

  20. Wang, B.: Disconnected recurrent neural networks for text categorization. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2311–2320. Association for Computational Linguistics, Melbourne, July 2018. https://www.aclweb.org/anthology/P18-1215

  21. Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 562–570. Association for Computational Linguistics, Vancouver, July 2017. https://www.aclweb.org/anthology/P17-1052

  22. Yin, W., Schütze, H.: Attentive convolution: equipping CNNs with RNN-style attention mechanisms. Trans. Assoc. Comput. Linguist. 6, 687–702 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Key R&D Program of China (2018YFB1402600), and by the National Natural Science Foundation of China (61802028, 61772083, 61877006, 62002027), and sponsored by CCF-Baidu Open Fund.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhe Xue .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zheng, C., Xue, Z., Du, J., Kou, F., Liang, M., Xu, M. (2021). A Hierarchical Multi-label Classification Algorithm for Scientific Papers Based on Graph Attention Networks. In: Fang, L., Chen, Y., Zhai, G., Wang, J., Wang, R., Dong, W. (eds) Artificial Intelligence. CICAI 2021. Lecture Notes in Computer Science(), vol 13069. Springer, Cham. https://doi.org/10.1007/978-3-030-93046-2_62

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-93046-2_62

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-93045-5

  • Online ISBN: 978-3-030-93046-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics