Abstract
Assigning effective and accurate reviewers is a crucial step in the peer-reviewing process to ensure high-quality publications. Automatic reviewer assignment problem, which aims to assign experts for rating the submitted research, relies critically on determining the expertise of the reviewers in the topics covered in the manuscript and assigning them so that all research topics covered in the manuscript are rated by proficient reviewers.
In this paper, we consider research expertise as a commixture of interest and proficiency of the reviewers in the topics that span the manuscript. We take reviewers’ recent publications and self-declaration about the areas of research interest as input. Lexical variations in the topics declared by reviewers are leveled using sentence transformers and the research publications. Similarity between the semantic content of the manuscript and the research topics is also matched using the sentence transformer framework. Subsequently, we quantify the reviewers’ interest in each topic along with their proficiency in that topic based on the number of authored publications. Reviewers are scored for expertise, which is a function of the two computed quantities. Top scoring reviewers are assigned the manuscript for review ratings.
We evaluate the effectiveness of the proposed method using topic coverage and review confidence as metrics. We observe that the proposed algorithm for expertise computation and reviewer assignment performs better than the baseline method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For example, ACL publications do not contain keywords.
- 2.
References
Anjum, O., Gong, H., Bhat, S., Hwu, W.M., Xiong, J.: PaRe: a paper-reviewer matching approach using a common topic space. In: Proceedings of the 2019 Conference on EMNLP-IJCNLP, pp. 518–528. ACL, Hong Kong, China (2019)
Charlin, L., Zemel, R.: The toronto paper matching system: an automated paper-reviewer assignment system. In: Proceedings of the 30th International Conference on Machine Learning, Atlanta, Georgia, USA, 2013 (2013)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Duan, Z., Tan, S., Zhao, S., Wang, Q., Chen, J., Zhang, Y.: Reviewer assignment based on sentence pair modeling. Neurocomputing 366, 97–108 (2019)
Jecmen, S., Zhang, H., Liu, R., Shah, N., Conitzer, V., Fang, F.: Mitigating manipulation in peer review via randomized reviewer assignments. Adv. Neural Inf. Process. Syst. 33, 12533–12545 (2020)
Jin, J., Niu, B., Ji, P., Geng, Q.: An integer linear programming model of reviewer assignment with research interest considerations. Ann. Oper. Res. 291(1), 409–433 (2020)
Karimzadehgan, M., Zhai, C.: Constrained multi-aspect expertise matching for committee review assignment. In: Proceedings of the 18th ACM CIKM, pp. 1697–1700 (2009)
Karimzadehgan, M., Zhai, C.: Integer linear programming for constrained multi-aspect committee review assignment. Inf. Process. Manag. 48(4), 725–740 (2012)
Karimzadehgan, M., Zhai, C., Belford, G.: Multi-aspect expertise matching for review assignment. In: Proceedings of the 17th ACM CIKM, pp. 1113–1122 (2008)
Kobren, A., Saha, B., McCallum, A.: Paper matching with local fairness constraints. In: Proceedings of the 25th ACM SIGKDD, pp. 1247–1257 (2019)
Kou, N.M., U, L.H., Mamoulis, N., Gong, Z.: Weighted coverage based reviewer assignment. In: Proceedings of the 2015 ACM SIGMOD, pp. 2031–2046 (2015)
Kou, N.M., U, L.H., Mamoulis, N., Li, Y., Li, Y., Gong, Z.: A topic-based reviewer assignment system. Proc. VLDB Endow. 8(12), 1852–1855 (2015)
Kreutz, C.K., Schenkel, R.: Revaside: assignment of suitable reviewer sets for publications from fixed candidate pools. In: The 23rd International Conference on Information Integration and Web Intelligence, pp. 57–68 (2021)
Li, B., Hou, Y.T.: The new automated IEEE infocom review assignment system. IEEE Netw. 30(5), 18–24 (2016)
Li, X., Watanabe, T.: Automatic paper-to-reviewer assignment, based on the matching degree of the reviewers. Procedia Comput. Sci. 22, 633–642 (2013)
Liu, X., Suel, T., Memon, N.: A robust model for paper reviewer assignment. In: Proceedings of the 8th ACM RecSyS, pp. 25–32 (2014)
Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Mimno, D., McCallum, A.: Expertise modeling for matching papers with reviewers. In: Proceedings of the 13th ACM SIGKDD, pp. 500–509 (2007)
Mirzaei, M., Sander, J., Stroulia, E.: Multi-aspect review-team assignment using latent research areas. Inf. Process. Manag. 56(3), 858–878 (2019)
Nguyen, J., Sánchez-Hernández, G., Agell, N., Rovira, X., Angulo, C.: A decision support tool using order weighted averaging for conference review assignment. Pattern Recogn. Lett. 105, 114–120 (2018)
Ogunleye, O., Ifebanjo, T., Abiodun, T., Adebiyi, A.: Proposed framework for a paper-reviewer assignment system using word2vec. In: Covenant University Conference on E-Governance in Nigeria (2017)
Patil, A.H., Mahalle, P.N.: Multi-label reviewer profile building and ranking based on expertise, recency, authority and h-index: vital module of reviewer paper assignment. Turk. J. Comput. Math. Educ. (TURCOMAT) 12(6), 3026–3035 (2021)
Peng, H., Hu, H., Wang, K., Wang, X.: Time-aware and topic-based reviewer assignment. In: Bao, Z., Trajcevski, G., Chang, L., Hua, W. (eds.) DASFAA 2017. LNCS, vol. 10179, pp. 145–157. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-55705-2_11
Rajapaksha, P., Farahbakhsh, R., Crespi, N.: Bert, xlnet or roberta: the best transfer learning model to detect clickbaits. IEEE Access 9, 154704–154716 (2021)
Reimers, N., Gurevych, I.: Sentence-bert: sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)
Tan, S., Duan, Z., Zhao, S., Chen, J., Zhang, Y.: Improved reviewer assignment based on both word and semantic features. Inf. Retr. J. 24(3), 175–204 (2021). https://doi.org/10.1007/s10791-021-09390-8
Xu, Y., Ma, J., Sun, Y., Hao, G., Xu, W., Zhao, D.: A decision support approach for assigning reviewers to proposals. Expert Syst. Appl. 37(10), 6948–6956 (2010)
Zhao, S., Zhang, D., Duan, Z., Chen, J., Zhang, Y.P., Tang, J.: A novel classification method for paper-reviewer recommendation. Scientometrics 115(3), 1293–1313 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kwatra, D., Bhatnagar, V. (2022). Expertise Computation for Automatic Reviewer Assignment. In: Pardede, E., Delir Haghighi, P., Khalil, I., Kotsis, G. (eds) Information Integration and Web Intelligence. iiWAS 2022. Lecture Notes in Computer Science, vol 13635. Springer, Cham. https://doi.org/10.1007/978-3-031-21047-1_48
Download citation
DOI: https://doi.org/10.1007/978-3-031-21047-1_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21046-4
Online ISBN: 978-3-031-21047-1
eBook Packages: Computer ScienceComputer Science (R0)