Abstract
Automatic academic paper rating (AAPR) remains a difficult but useful task to automatically predict whether to accept or reject a paper. Having found more task-specific structure features of academic papers, we present a modularized hierarchical attention network (MHAN) to predict paper quality. MHAN uses a three-level hierarchical attention network to shorten the sequence for each level. In the network, the modularized parameter distinguishes the semantics of functional chapters. And a label-smoothing mechanism is used as a loss function to avoid inappropriate labeling. Compared with MHCNN and plain HAN on an AAPR dataset, MHAN achieves a state-of-the-art accuracy of 65.33%. Ablation experiments show that the proposed methods are effective.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Models can be found at https://huggingface.co/prajjwal1/bert-medium..
References
Kang, D., et al.: A dataset of peer reviews (PeerRead): collection, insights and NLP applications. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1647–1661 (2018)
Yang, P., Sun, X., Li, W., Ma, S.: Automatic academic paper rating based on modularized hierarchical convolutional neural network. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 496–502 (2018)
Qiao, Feng, Xu, Lizhen, Han, Xiaowei: Modularized and attention-based recurrent convolutional neural network for automatic academic paper aspect scoring. In: Meng, Xiaofeng, Li, Ruixuan, Wang, Kanliang, Niu, Baoning, Wang, Xin, Zhao, Gansen (eds.) WISA 2018. LNCS, vol. 11242, pp. 68–76. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02934-0_7
Leng, Y., Yu, L., Xiong, J.: DeepReviewer: collaborative grammar and innovation neural network for automatic paper review. In: 2019 International Conference on Multimodal Interaction, pp. 395–403 (2019)
Skorikov, M., Momen, S.: Machine learning approach to predicting the acceptance of academic papers. In: 2020 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT). IEEE (2020)
Vincent-Lamarre, P., Larivière, V.: Textual analysis of artificial intelligence manuscripts reveals features associated with peer review outcome. Quant. Sci. Stud. 2(2), 662–677 (2021)
Shen, A., Salehi, B., Baldwin, T., Qi, J.: A joint model for multimodal document quality assessment. In: 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL), pp. 107–110. IEEE (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Kenton, J.D.M.W.C., Toutanova, L.K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)
Beltagy, I., Peters, M.E., Cohan, A.: Longformer: the long-document transformer. arXiv preprint arXiv:2004.05150 (2020)
Zaheer, M., et al.: Big bird: transformers for longer sequences. Adv. Neural Inf. Process. Syst. 33, 17283–17297 (2020)
Langford, J., Guzdial, M.: The arbitrariness of reviews, and advice for school administrators. Commun. ACM 58(4), 12–13 (2015)
Lin, J., Song, J., Zhou, Z., Shi, X.: Automated scholarly paper review: possibility and challenges. arXiv preprint arXiv:2111.07533 (2021)
Shibayama, S., Yin, D., Matsumoto, K.: Measuring novelty in science with word embedding. PLoS ONE 16(7), e0254034 (2021)
Daudaravicius, V.: Automated evaluation of scientific writing: AESW shared task proposal. In: Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 56–63 (2015)
Springstein, M., Müller-Budack, E., Ewerth, R.: QuTI! quantifying text-image consistency in multimodal documents. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2575–2579 (2021)
Hou, Y., Jochim, C., Gleize, M., Bonin, F., Ganguly, D.: TDMSci: a specialized corpus for scientific literature entity tagging of tasks datasets and metrics. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 707–714 (2021)
Gupta, Y., et al.: The effect of pretraining on extractive summarization for scientific documents. In: Proceedings of the Second Workshop on Scholarly Document Processing, pp. 73–82 (2021)
Wang, Q., Zeng, Q., Huang, L., Knight, K., Ji, H., Rajani, N.F.: ReviewRobot: explainable paper review generation based on knowledge synthesis. In: Proceedings of the 13th International Conference on Natural Language Generation, pp. 384–397 (2020)
Yuan, W., Liu, P., Neubig, G.: Can we automate scientific reviewing? arXiv preprint arXiv:2102.00176 (2021)
de Buy Wenniger, G.M., van Dongen, T., Aedmaa, E., Kruitbosch, H.T., Valentijn, E.A., Schomaker, L.: Structure-tags improve text classification for scholarly document quality prediction. In: Proceedings of the First Workshop on Scholarly Document Processing, pp. 158–167 (2020)
Huang, J.B.: Deep paper gestalt. arXiv preprint arXiv:1812.08775 (2018)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751. Association for Computational Linguistics, Doha, Qatar (2014)
Acknowledgments
This work is partly supported by the Beijing Natural Science Foundation (No. 4212026) and the Fundamental Strengthening Program Technology Field Fund (No. 2021-JCJQ-JJ-0059).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kang, K., Zhang, H., Li, Y., Luo, X., Wushour, S. (2022). Automatic Academic Paper Rating Based on Modularized Hierarchical Attention Network. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13551. Springer, Cham. https://doi.org/10.1007/978-3-031-17120-8_52
Download citation
DOI: https://doi.org/10.1007/978-3-031-17120-8_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17119-2
Online ISBN: 978-3-031-17120-8
eBook Packages: Computer ScienceComputer Science (R0)
