Skip to main content

Advertisement

Springer Nature Link
Log in

Code context-based reviewer recommendation

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Code review is a critical process in software development, contributing to the overall quality of the product by identifying errors early. A key aspect of this process is the selection of appropriate reviewers to scrutinize changes made to source code. However, in large-scale open-source projects, selecting the most suitable reviewers for a specific change can be a challenging task. To address this, we introduce the Code Context Based Reviewer Recommendation (CCB-RR), a model that leverages information from changesets to recommend the most suitable reviewers. The model takes into consideration the paths of modified files and the context derived from the changesets, including their titles and descriptions. Additionally, CCB-RR employs KeyBERT to extract the most relevant keywords and compare the semantic similarity across changesets. The model integrates the paths of modified files, keyword information, and the context of code changes to form a comprehensive picture of the changeset. We conducted extensive experiments on four open-source projects, demonstrating the effectiveness of CCB-RR. The model achieved a Top-1 accuracy of 60%, 55%, 51%, and 45% on the Android, OpenStack, QT, and LibreOffice projects respectively. For Mean Reciprocal Rank (MRR), CCB achieved 71%, 62%, 52%, and 68% on the same projects respectively, thereby highlighting its potential for practical application in code reviewer recommendation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Xia X, Lo D, Wang X, Yang X. Who should review this change?: Putting text and file location analyses together for more accurate recommendations. In: Proceedings of the 31st IEEE International Conference on Software Maintenance and Evolution. 2015, 261–270

    Google Scholar 

  2. Shull F, Seaman C. Inspecting the history of inspections: an example of evidence-based technology diffusion. IEEE Software, 2008, 25(1): 88–90

    Article  Google Scholar 

  3. Chueshev A, Lawall J, Bendraou R, Ziadi T. Expanding the number of reviewers in open-source projects by recommending appropriate developers. In: Proceedings of the 36th IEEE International Conference on Software Maintenance and Evolution. 2020, 499–510

    Google Scholar 

  4. Thongtanunam P, Tantithamthavorn C, Kula R G, Yoshida N, Iida H, Matsumoto K I. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In: Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering. 2015, 141–150

    Google Scholar 

  5. Goto I, Tanaka H. Detecting untranslated content for neural machine translation. In: Proceedings of the 1st Workshop on Neural Machine Translation. 2017, 47–55

    Chapter  Google Scholar 

  6. Stahlberg F. Neural machine translation: a review. Journal of Artificial Intelligence Research, 2020, 69: 343–418

    Article  MathSciNet  Google Scholar 

  7. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010

    Google Scholar 

  8. Yu Y, Wang H, Yin G, Wang T. Reviewer recommendation for pull-requests in GitHub: what can we learn from code review and bug assignment? Information and Software Technology, 2016, 74: 204–218

    Article  Google Scholar 

  9. Zanjani M B, Kagdi H, Bird C. Automatically recommending peer reviewers in modern code review. Transactions on Software Engineering, 2016, 42(6): 530–543

    Article  Google Scholar 

  10. Bacchelli A, Bird C. Expectations, outcomes, and challenges of modern code review. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 712–721

    Google Scholar 

  11. Rigby P C, Storey M A. Understanding broadcast based peer review on open source software projects. In: Proceedings of the 33rd International Conference on Software Engineering. 2011, 541–550

    Chapter  Google Scholar 

  12. Yu Y, Wang H, Filkov V, Devanbu P, Vasilescu B. Wait for it: determinants of pull request evaluation latency on GitHub. In: Proceedings of the 12th Working Conference on Mining Software Repositories. 2015, 367–371

    Google Scholar 

  13. Tymchuk Y, Mocci A, Lanza M. Code review: veni, ViDI, vici. In: Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering. 2015, 151–160

    Google Scholar 

  14. Qiu X, Sun T, Xu Y, Shao Y, Dai N, Huang X. Pre-trained models for natural language processing: a survey. Science China Technological Sciences, 2020, 63(10): 1872–1897

    Article  Google Scholar 

  15. Lavrenko V, Croft W B. Relevance-based language models. ACM SIGIR Forum, 2017, 51(2): 260–267

    Article  Google Scholar 

  16. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 3111–3119

    Google Scholar 

  17. Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of the 19th Conference on Empirical Methods in Natural Language Processing. 2014, 1532–1543

    Google Scholar 

  18. Cherney L R, Kaye R C, Lee J B, van Vuuren S. Impact of personal relevance on acquisition and generalization of script training for aphasia: a preliminary analysis. American Journal of Speech-Language Pathology, 2015, 24(4): S913–S922

    Article  Google Scholar 

  19. Lourie N, Le Bras R, Bhagavatula C, Choi Y. UNICORN on RAINBOW: a universal commonsense reasoning model on a new multitask benchmark. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 13480–13488

    Google Scholar 

  20. Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le Q V. XLNet: generalized autoregressive pretraining for language understanding. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 517

    Google Scholar 

  21. Liu Z, Lin W, Shi Y, Zhao J. A robustly optimized BERT pre-training approach with post-training. In: Proceedings of the 20th China National Conference on Chinese Computational Linguistics. 2021, 471–484

    Chapter  Google Scholar 

  22. Chi P H, Chung P H, Wu T H, Hsieh C C, Chen Y H, Li S W, Lee H Y. Audio albert: a lite bert for self-supervised learning of audio representation. In: Proceedings of 2021 IEEE Spoken Language Technology Workshop. 2021, 344–350

    Chapter  Google Scholar 

  23. Brown T B, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler D M, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D. Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 159

    Google Scholar 

  24. Hannebauer C, Patalas M, Stünkel S, Gruhn V. Automatically recommending code reviewers based on their expertise: an empirical comparison. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 2016, 99–110

    Chapter  Google Scholar 

  25. Doğan E, Tüzün E, Tecimer K A, Güvenir H A. Investigating the validity of ground truth in code reviewer recommendation studies. In: Proceedings of the 13th International Symposium on Empirical Software Engineering and Measurement. 2019, 1–6

    Google Scholar 

  26. Ye X, Zheng Y, Aljedaani W, Mkaouer M W. Recommending pull request reviewers based on code changes. Soft Computing, 2021, 25(7): 5619–5632

    Article  Google Scholar 

  27. Fejzer M, Przymus P, Stencel K. Profile based recommendation of code reviewers. Journal of Intelligent Information Systems, 2018, 50(3): 597–619

    Article  Google Scholar 

  28. Ye X. Learning to rank reviewers for pull requests. IEEE Access, 2019, 7: 85382–85391

    Article  Google Scholar 

  29. Firoozeh N, Nazarenko A, Alizon F, Daille B. Keyword extraction: issues and methods. Natural Language Engineering, 2020, 26(3): 259–291

    Article  Google Scholar 

  30. Piskorski J, Stefanovitch N, Jacquet G, Podavini A. Exploring linguistically-lightweight keyword extraction techniques for indexing news articles in a multilingual set-up. In: Proceedings of the 16th EACL Hackashop on News Media Content Analysis and Automated Report Generation. 2021, 35–44

    Google Scholar 

  31. Weninger F, Geiger J, Wöllmer M, Schuller B, Rigoll G. Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments. Computer Speech & Language, 2014, 28(4): 888–902

    Article  Google Scholar 

  32. Jiang N, Lutellier T, Tan L. CURE: code-aware neural machine translation for automatic program repair. In: Proceedings of the 43rd International Conference on Software Engineering. 2021, 1161–1173

    Google Scholar 

  33. Ouni A, Kula R G, Inoue K. Search-based peer reviewers recommendation in modern code review. In: Proceedings of the 32nd International Conference on Software Maintenance and Evolution. 2016, 367–377

    Google Scholar 

  34. Chouchen M, Ouni A, Mkaouer M W, Kula R G, Inoue K. WhoReview: a multi-objective search-based approach for code reviewers recommendation in modern code review. Applied Soft Computing, 2021, 100: 106908

    Article  Google Scholar 

  35. Campos R, Mangaravite V, Pasquali A, Jorge A, Nunes C, Jatowt A. YAKE! Keyword extraction from single documents using multiple local features. Information Sciences, 2020, 509: 257–289

    Article  Google Scholar 

  36. Nasrabadi N M, King R A. Image coding using vector quantization: a review. IEEE Transactions on Communications, 1988, 36(8): 957–971

    Article  Google Scholar 

  37. de Lima Junior M L, Soares D M, Plastino A, Murta L. Automatic assignment of integrators to pull requests: the importance of selecting appropriate attributes. Journal of Systems and Software, 2018, 144: 181–196

    Article  Google Scholar 

  38. Mirsaeedi E, Rigby P C. Mitigating turnover with code review recommendation: balancing expertise, workload, and knowledge distribution. In: Proceedings of the 42nd International Conference on Software Engineering. 2020, 1183–1195

    Google Scholar 

  39. Al-Zubaidi W H A, Thongtanunam P, Dam H K, Tantithamthavorn C, Ghose A. Workload-aware reviewer recommendation using a multi-objective search-based approach. In: Proceedings of the 16th ACM International Conference on Predictive Models and Data Analytics in Software Engineering. 2020, 21–30

    Chapter  Google Scholar 

  40. Balachandran V. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 931–940

    Google Scholar 

  41. Asthana S, Kumar R, Bhagwan R, Bird C, Bansal C, Maddila C, Mehta S, Ashok B. WhoDo: automating reviewer suggestions at scale. In: Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2019, 937–945

    Google Scholar 

  42. Salton G, Buckley C. Term-weighting approaches in automatic text retrieval. Information Processing & Management, 1988, 24(5): 513–523

    Article  Google Scholar 

  43. Mihalcea R, Tarau P. TextRank: bringing order into text. In: Proceedings of the 2nd Conference on Empirical Methods in Natural Language Processing. 2004, 404–411

    Google Scholar 

  44. Erkan G, Radev D R. LexRank: graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, 2004, 22: 457–479

    Article  Google Scholar 

  45. Nguyen T D, Luong M T. WINGNUS: keyphrase extraction utilizing document logical structure. In: Proceedings of the 5th International Workshop on Semantic Evaluation. 2010, 166–169

    Google Scholar 

  46. Rebai S, Amich A, Molaei S, Kessentini M, Kazman R. Multi-objective code reviewer recommendations: balancing expertise, availability and collaborations. Automated Software Engineering, 2020, 27(3–4): 301–328

    Article  Google Scholar 

  47. Çetin H A, Doğan E, Tüzün E. A review of code reviewer recommendation studies: challenges and future directions. Science of Computer Programming, 2021, 208: 102652

    Article  Google Scholar 

  48. Kong D, Chen Q, Bao L, Sun C, Xia X, Li S. Recommending code reviewers for proprietary software projects: a large scale study. In: Proceedings of 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). 2022, 630–640

    Google Scholar 

  49. Li R, Liang P, Avgeriou P. Code reviewer recommendation for architecture violations: an exploratory study. In: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 2023, 42–51

    Chapter  Google Scholar 

Download references

Acknowledgement

This work was supported in part by the Science and Technology Development Fund (FDCT), Macau SAR, China (Nos. 0047/2020/A1 and 0014/2022/A).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Macau University of Science and Technology, Macao, 999078, China

    Dawei Yuan, Xiao Peng, Zijie Chen & Tao Zhang

  2. Faculty of Science, University of Amsterdam, Amsterdam, WB 1018, Netherlands

    Ruijia Lei

Authors
  1. Dawei Yuan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiao Peng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zijie Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Tao Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Ruijia Lei
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Tao Zhang.

Ethics declarations

Competing interests The authors declare that they have no competing interests or financial conflicts to disclose.

Additional information

Dawei Yuan is a PhD student in the School of Computer Science and Engineering, Macau University of Science and Technology (MUST), China. Before joining MUST, he worked as a software engineer at United Imaging Healthcare Corporation (UIH), China. He also got an MSc in Software Engineering from the University of Science and Technology of China and an BSc degree in Network Engineering from Nanjing University of Posts and Telecommunications, China. His research interests mainly focus on using artificial intelligence techniques to solve problems in software engineering, such as bug localization and code analysis.

Xiao Peng received the BSc in information and computing science from the University of Jinan, China in 2020, and MSc in applied mathematics and data science from Macau University of Science and Technology, China in 2022. She is currently working toward the PhD degree in science at Macau University of Science and Technology, China. Her research interests include software engineering and deep learning.

Zijie Chen received the MSc degrees in Applied Mathematics and Data Science from Macau University of Science and Technology, China, and the BSc degree in Software Engineering from Beijing Institute of Technology, China. He is currently pursuing the PhD degree in the School of Computer Science and Engineering, Macau University of Science and Technology, China. His research interests include deep learning, human activity recognition, and Internet of Things.

Tao Zhang received the BS degree in automation, the MEng degree in software engineering from Northeastern University, China, and the PhD degree in computer science from the University of Seoul, Republic of Korea. After that, he spent one year with The Hong Kong Polytechnic University, China as a postdoctoral research fellow. Currently, he is an associate professor with the School of Computer Science and Engineering, Macau University of Science and Technology (MUST), China. Before joining MUST, he was the faculty member of Harbin Engineering University and Nanjing University of Posts and Telecommunications, China. He published more than 90 high-quality papers at renowned software engineering and security journals and concerences. He served as the General Chair of SANER 2023 and the PC members of several top-tier SE concerences such as FSE and ASE. He is a senior member of ACM and IEEE.

Ruijia Lei received the BSc degree in Software Engineering from Macau University of Science and Technology, China in 2021, the MSc degree in Computer Science from University of Amsterdam, the Netherlands in 2023, respectively. His research interests include software technology, machine learning, large-scale data processing, distributed systems, and principles and applications of LLMs.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, D., Peng, X., Chen, Z. et al. Code context-based reviewer recommendation. Front. Comput. Sci. 19, 191202 (2025). https://doi.org/10.1007/s11704-023-3256-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-023-3256-9

Keywords