Abstract
Peer-review process is fraught with issues like bias, inconsistencies, arbitrariness, non-committal weak rejects, etc. However, it is anticipated that the peer reviews provide constructive feedback to the authors against some aspects of the paper such as Motivation/Impact, Soundness/Correctness, Novelty, Substance, etc. A good review is expected to evaluate a paper under the lens of these aspects. An automated system to extract these implicit aspects from the reviews would help determine the quality/goodness of the peer review. In this work, we propose a deep neural architecture to extract the aspects of the paper on which the reviewer commented in their review. Our automatic aspect-extraction model based on BERT and neural attention mechanism achieves superior performance over the standard baselines. We make our codes, analyses and other matrials available at https://github.com/cruxieu17/aspect-extraction-peer-reviews.
R. Verma and K. Shinde—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Change history
02 December 2021
In the originally published version of chapter 88 the acknowledgement statement was erroneously omitted. The acknowledgement statement has been added to the chapter.
References
Tomkins, A., Zhang, M., Heavlin, W.D.: Reviewer bias in single-versus double-blind peer review. Proc. Natl. Acad. Sci. 114(48), 12708–12713 (2017)
Shah, N.B., Tabibian, B., Muandet, K., Guyon, I., Von Luxburg, U.: Design and analysis of the NIPS 2016 review process. J. Mach. Learn. Res. 19(49), 1–34 (2018)
Langford, J., Guzdial, M.: The arbitrariness of reviews, and advice for school administrators. Commun. ACM 58(4), 12–13 (2015)
Brezis, E.S., Birukou, A.: Arbitrariness in the peer review process. Scientometrics 123(1), 393–411 (2020). https://doi.org/10.1007/s11192-020-03348-1
Sculley, D., Snoek, J., Wiltschko, A.: Avoiding a tragedy of the commons in the peer review process. arXiv preprint arXiv:1901.06246 (2018)
Rogers, A., Augenstein, I.: What can we do to improve peer review in NLP?. arXiv preprint arXiv:2010.03863 (2020)
Tran, D., et al.: An open review of OpenReview: a critical analysis of the machine learning conference review process. arXiv preprint arXiv:2010.05137 (2020)
Yuan, W., Liu, P., Neubig, G.: Can we automate scientific reviewing?. arXiv preprint arXiv:2102.00176 (2021)
Xiong, W., Litman, D.: Automatically predicting peer-review helpfulness. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 502–507, June 2011
Reimers, N., Gurevych, I.: Sentence-HERT: sentence embeddings using Siamese HERT-networks. arXiv preprint arXiv:1908.10084 (2019)
Liu, Y., et al.: Roberta: a robustly optimized HERT pretraining approach. arXiv preprint arXiv:1907.11692, (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45, October2020
Acknowledgement
Tirthankar Ghosal is funded by Cactus Communications, India (Award # CAC-2021-01) to carry out this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Verma, R., Shinde, K., Arora, H., Ghosal, T. (2021). Attend to Your Review: A Deep Neural Network to Extract Aspects from Peer Reviews. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1517. Springer, Cham. https://doi.org/10.1007/978-3-030-92310-5_88
Download citation
DOI: https://doi.org/10.1007/978-3-030-92310-5_88
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92309-9
Online ISBN: 978-3-030-92310-5
eBook Packages: Computer ScienceComputer Science (R0)