Skip to main content
Springer Nature Link
Log in

Multi-objective code reviewer recommendations: balancing expertise, availability and collaborations

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Modern Code review is one of the most critical tasks in software maintenance and evolution. A rigorous code review leads to fewer bugs and reduced overall maintenance costs. Most existing studies focus on automatically identifying the most qualified reviewers, based on their expertise, to review pull-up requests. However, the management of code reviews is a complex problem in practice due to a project’s limited resources, including the availability of peer reviewers. Furthermore, the history of collaborations between developers and reviewers could affect the quality of the reviews, in positive or negative ways. In this paper, we formulate the recommendation of code reviewers as a multi-objective search problem to balance the conflicting objectives of expertise, availability, and history of collaborations. Our validation confirms the effectiveness of our multi-objective approach on 9 open source projects by making better recommendations, on average, than the state of the art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://www.gerritcodereview.com/intro-quick.html.

References

  • Almhana, R., Mkaouer, W., Kessentini, M., Ouni, A.: Recommending relevant classes for bug reports using multi-objective search. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, pp. 286–295. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2970276.2970344

  • Amal, B., Kessentini, M., Bechikh, S., Dea, J., Said, L.B.: On the use of machine learning and search-based software engineering for ill-defined fitness function: a case study on software refactoring. In: International Symposium on Search Based Software Engineering, pp. 31–45. Springer, Cham (2014)

  • Bacchelli, A., Bird, C.: Expectations, outcomes, and challenges of modern code review. In: Proceedings of the 2013 International Conference on Software Engineering, pp. 712–721. IEEE Press (2013)

  • Balachandran, V.: Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In: Proceedings of the 2013 International Conference on Software Engineering, pp. 931–940. IEEE Press (2013)

  • Baysal, O., Holmes, R.: A qualitative study of mozilla’s process management practices. David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Canada, Tech. Rep. CS-2012-10 (2012)

  • Baysal, O., Kononenko, O., Holmes, R., Godfrey, M.W.: The influence of non-technical factors on code review. In: 2013 20th Working Conference on Reverse Engineering (WCRE), pp. 122–131. IEEE (2013)

  • Bird, C., Pattison, D., D’Souza, R., Filkov, V., Devanbu, P.: Latent social structure in open source projects. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 24–35. ACM (2008)

  • Bosu, A., Carver, J.C.: Impact of peer code review on peer impression formation: A survey. In: 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 133–142. IEEE (2013)

  • Bosu, A., Carver, J.C.: How do social interaction networks influence peer impressions formation? A case study. In: IFIP International Conference on Open Source Systems, pp. 31–40. Springer (2014)

  • Bosu, A., Carver, J.C.: Impact of developer reputation on code review outcomes in oss projects: an empirical investigation. In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, p. 33. ACM (2014)

  • Bosu, A., Carver, J.C., Bird, C., Orbeck, J., Chockley, C.: Process aspects and social dynamics of contemporary code review: insights from open source development and industrial practice at microsoft. IEEE Trans. Softw. Eng. 43(1), 56–75 (2016)

    Article  Google Scholar 

  • Cohen, J., Brown, E., DuRette, B., Teleki, S.: Best kept secrets of peer code review. Smart Bear Somerville (2006)

  • Committee, S.E.S., et al.: IEEE standard for software reviews. IEEE Std, pp. 1028–1997 (1997)

  • Deb, K., Gupta, S.: Understanding knee points in bicriteria problems and their implications as preferred solution principles. Eng. Optim. 43(11), 1175–1204 (2011)

    Article  MathSciNet  Google Scholar 

  • Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans. Evolut. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  • Emmerich, M.T., Deutz, A.H.: A tutorial on multiobjective optimization: fundamentals and evolutionary methods. Nat. Comput. 17(3), 585–609 (2018)

    Article  MathSciNet  Google Scholar 

  • Fagan, M.: Design and code inspections to reduce errors in program development. In: Software Pioneers, pp. 575–607. Springer (2002)

  • Ghannem, A., El Boussaidi, G., Kessentini, M.: On the use of design defect examples to detect model refactoring opportunities. Softw. Qual. J. 24(4), 947–965 (2016)

    Article  Google Scholar 

  • Ghannem, A., Kessentini, M., El Boussaidi, G.: Detecting model refactoring opportunities using heuristic search. In: Proceedings of the 2011 Conference of the Center for Advanced Studies on Collaborative Research, pp. 175–187 (2011)

  • Harman, M., Mansouri, S.A., Zhang, Y.: Search-based software engineering: trends, techniques and applications. ACM Comput. Surv. (CSUR) 45(1), 11 (2012)

    Article  Google Scholar 

  • Jackson, R.R., Carter, C.M., Tarsitano, M.S.: Trial-and-error solving of a confinement problem by a jumping spider, portia fimbriata. Behaviour 138(10), 1215–1234 (2001)

    Article  Google Scholar 

  • Keller, A.A.: Multi-Objective Optimization in Theory and Practice II: Metaheuristic Algorithms. Bentham Science Publishers, Sharjah (2019)

    Book  Google Scholar 

  • Kononenko, O., Baysal, O., Guerrouj, L., Cao, Y., Godfrey, M.W.: Investigating code review quality: do people and participation matter? In: 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 111–120. IEEE (2015)

  • Ouni, A., Kessentini, M., Ó Cinnéide, M., Sahraoui, H., Deb, K., Inoue, K.: More: a multi-objective refactoring recommendation approach to introducing design patterns and fixing code smells. J. Softw. Evol. Process 29(5), e1843 (2017)

    Article  Google Scholar 

  • Ouni, A., Kula, R.G., Inoue, K.: Search-based peer reviewers recommendation in modern code review. In: 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 367–377. IEEE (2016)

  • Ouni, A., Kula, R.G., Kessentini, M., Ishio, T., German, D.M., Inoue, K.: Search-based software library recommendation using multi-objective optimization. Inf. Softw. Technol. 83, 55–75 (2017)

    Article  Google Scholar 

  • Rachmawati, L., Srinivasan, D.: Multiobjective evolutionary algorithm with controllable focus on the knees of the pareto front. IEEE Trans. Evol. Comput. 13(4), 810–824 (2009)

    Article  Google Scholar 

  • Rigby, P.C., Bird, C.: Convergent contemporary software peer review practices. In: Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, pp. 202–212. ACM (2013)

  • Rigby, P.C., German, D.M., Storey, M.A.: Open source software peer review practices: a case study of the apache server. In: Proceedings of the 30th International Conference on Software Engineering, pp. 541–550. ACM (2008)

  • Rigby, P.C., Storey, M.A.: Understanding broadcast based peer review on open source software projects. In: 2011 33rd International Conference on Software Engineering (ICSE), pp. 541–550. IEEE (2011)

  • Thongtanunam, P., Kula, R.G., Cruz, A.E.C., Yoshida, N., Iida, H.: Improving code review effectiveness through reviewer recommendations. In: Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering, pp. 119–122. ACM (2014)

  • Thongtanunam, P., Tantithamthavorn, C., Kula, R.G., Yoshida, N., Iida, H., Matsumoto, K.i.: Who should review my code? a file location-based code-reviewer recommendation approach for modern code review. In: 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER), pp. 141–150. IEEE (2015)

  • Wang, H., Kessentini, M., Ouni, A.: Bi-level identification of web service defects. In: International Conference on Service-Oriented Computing, pp. 352–368. Springer, Cham (2016)

  • Wilcoxon, F., Katti, S., Wilcox, R.A.: Critical values and probability levels for the wilcoxon rank sum test and the Wilcoxon signed rank test. Sel. Tab. Math. Stat. 1, 171–259 (1970)

    MATH  Google Scholar 

  • Xia, X., Lo, D., Wang, X., Yang, X.: Who should review this change?: Putting text and file location analyses together for more accurate recommendations. In: 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 261–270. IEEE (2015)

  • Xia, X., Lo, D., Wang, X., Zhou, B.: Accurate developer recommendation for bug resolution. In: 2013 20th Working Conference on Reverse Engineering (WCRE), pp. 72–81. IEEE (2013)

  • Yang, X., Kula, R.G., Yoshida, N., Iida, H.: Mining the modern code review repositories: a dataset of people, process and product. In: Proceedings of the 13th International Conference on Mining Software Repositories, pp. 460–463. ACM (2016)

  • Yang, X., Yoshida, N., Kula, R.G., Iida, H.: Peer review social network (person) in open source projects. IEICE Trans. Inf. Syst. 99(3), 661–670 (2016)

    Article  Google Scholar 

  • Yu, Y., Wang, H., Yin, G., Wang, T.: Reviewer recommendation for pull-requests in github: what can we learn from code review and bug assignment? Inf. Softw. Technol. 74, 204–218 (2016)

    Article  Google Scholar 

  • Zanjani, M.B., Kagdi, H., Bird, C.: Automatically recommending peer reviewers in modern code review. IEEE Trans. Softw. Eng. 42(6), 530–543 (2015)

    Article  Google Scholar 

  • Zanjani, M.B., Kagdi, H., Bird, C.: Automatically recommending peer reviewers in modern code review. IEEE Trans. Softw. Eng. 42(6), 530–543 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Michigan, Dearborn, USA

    Soumaya Rebai, Abderrahmen Amich, Somayeh Molaei & Marouane Kessentini

  2. University of Hawaii, Honolulu, USA

    Rick Kazman

Authors
  1. Soumaya Rebai
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Abderrahmen Amich
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Somayeh Molaei
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Marouane Kessentini
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Rick Kazman
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Marouane Kessentini.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rebai, S., Amich, A., Molaei, S. et al. Multi-objective code reviewer recommendations: balancing expertise, availability and collaborations. Autom Softw Eng 27, 301–328 (2020). https://doi.org/10.1007/s10515-020-00275-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-020-00275-6

Keywords