skip to main content
10.1145/2889160.2889243acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

A study of the quality-impacting practices of modern code review at Sony mobile

Published: 14 May 2016 Publication History

Abstract

Nowadays, a flexible, lightweight variant of the code review process (i.e., the practice of having other team members critique software changes) is adopted by open source and proprietary software projects. While this flexibility is a blessing (e.g., enabling code reviews to span the globe), it does not mandate minimum review quality criteria like the formal code inspections of the past. Recent work shows that lax reviewing can impact the quality of open source systems. In this paper, we investigate the impact that code reviewing practices have on the quality of a proprietary system that is developed by Sony Mobile. We begin by replicating open source analyses of the relationship between software quality (as approximated by post-release defect-proneness) and: (1) code review coverage, i.e., the proportion of code changes that have been reviewed and (2) code review participation, i.e., the degree of reviewer involvement in the code review process. We also perform a qualitative analysis, with a survey of 93 stakeholders, semi-structured interviews with 15 stakeholders, and a follow-up survey of 25 senior engineers. Our results indicate that while past measures of review coverage and participation do not share a relationship with defect-proneness at Sony Mobile, reviewing measures that are aware of the Sony Mobile development context are associated with defect-proneness. Our results have lead to improvements of the Sony Mobile code review process.

References

[1]
H. Akaike. Information theory and an extension of the maximum likelihood principle. In Proc. of the Int'l Symp. on Information Theory, pages 267--281, 1973.
[2]
J. Aranda and G. Venolia. The secret life of bugs: Going past the errors and omissions in software repositories. In Proc. of the Int'l Conf. on Software Eng., pages 298--308, 2009.
[3]
A. Bacchelli and C. Bird. Expectations, outcomes, and challenges of modern code review. In Proc. of the Int'l Conf. on Software Eng., pages 712--721, 2013.
[4]
G. Bavota and R. Barbara. Four eyes are better than two: On the impact of code reviews on software quality. In Proc. of the Int'l Conf. on Software Maint. and Evolution, pages 81--90, 2015.
[5]
M. Beller, A. Bacchelli, A. Zaidman, and E. Juergens. Modern code reviews in open-source projects: Which problems do they fix? In Proc. of Working Conf. on Mining Software Repositories, pages 202--211, 2014.
[6]
C. Bird, N. Nagappan, B. Murphy, H. Gall, and P. Devanbu. Don't touch my code!: Examining the effects of ownership on software quality. In Proc. of the ACM SIGSOFT Symp. and the European Conf. on Foundations of Software Eng., pages 4--14, 2011.
[7]
J. David W. Hosmer, S. Lemeshow, and R. X. Sturdivant. Applied Logistic Regression. Wiley, 2013.
[8]
B. Efron and R. J. Tibshirani. An Introduction to the Bootstrap. Chapman & Hall, 1993.
[9]
M. E. Fagan. Design and code inspections to reduce errors in program development. IBM Syst. J., 38(2-3):258--287, 1999.
[10]
F. E. Harrell Jr. Regression modeling strategies. Springer, 2001.
[11]
F. E. Harrell Jr. rms: Regression Modeling Strategies, 2015. R package version 4.3-0.
[12]
A. Hassan. Predicting faults using the complexity of code changes. In Proc. of the Int'l Conf. on Software Eng., pages 78--88, 2009.
[13]
J. D. Herbsleb and R. E. Grinter. Architectures, coordination, and distance: Conway's law and beyond. IEEE Software, 16(5):63--70, 1999.
[14]
C. Kemerer and M. Paulk. The impact of design and code reviews on software quality: An empirical study based on PSP data. IEEE Trans. Software Eng., 35(4):534--550, 2009.
[15]
O. Kononenko, O. Baysal, L. Guerrouj, Y. Cao, and M. Godfrey, W. Investigating code review quality: Do people and participation matter? In Proc. of the Int'l Conf. on Software Maint. and Evolution, pages 111--120, 2015.
[16]
M. Lavallée and P. N. Robillard. Why good developers write bad code: An observational case study of the impacts of organizational factors on software quality. In Proc. of the Int'l Conf. on Software Eng., pages 677--687, 2015.
[17]
S. McIntosh, Y. Kamei, B. Adams, and A. E. Hassan. The impact of code review coverage and code review participation on software quality: A case study of the Qt, VTK, and ITK projects. In Proc. of the Working Conf. on Mining Software Repositories, pages 192--201, 2013.
[18]
S. McIntosh, Y. Kamei, B. Adams, and A. E. Hassan. An empirical study of the impact of modern code review practices on software quality. Empirical Software Eng., page To appear, 2015.
[19]
A. Meneely, A. C. R. Tejeda, B. Spates, S. Trudeau, D. Neuberger, K. Whitlock, C. Ketant, and K. Davis. An empirical investigation of socio-technical code review metrics and security vulnerabilities. In Proc. of the Int'l Workshop on Social Software Eng., pages 37--44, 2014.
[20]
A. Mockus, R. T. Fielding, and J. D. Herbsleb. Two case studies of open source software development: Apache and mozilla. ACM Trans. on Software Eng. and Methodology, 11(3):309--346, 2002.
[21]
R. Morales, S. McIntosh, and F. Khomh. Do code review practices impact design quality? a case study of the Qt, VTK, and ITK projects. In Proc. of the Int'l Conf. on Software Analysis, Evolution and Reengineering, pages 171--180, 2015.
[22]
N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In Proc. of the Int'l Conf. on Software Eng., pages 284--292, 2005.
[23]
N. Nagappan, B. Murphy, and V. Basili. The influence of organizational structure on software quality: An empirical case study. In Proc. of the Int'l Conf. on Software Eng., pages 521--530, 2008.
[24]
D. L. Parnas. On the criteria to be used in decomposing systems into modules. Commun. ACM, 15(12):1053--1058, 1972.
[25]
P. C. Rigby and C. Bird. Convergent contemporary software peer review practices. In Proc. of the ACM SIGSOFT Symp. and the European Conf. on Foundations of Software Eng., pages 202--212, 2013.
[26]
P. C. Rigby and M. Storey. Understanding broadcast based peer review on open source software projects. In Proc. of the Int'l Conf. on Software Eng., pages 541--550, 2011.
[27]
C. B. Seaman. Qualitative methods. Guide to Advanced Empirical Software Engineering, Springer, pages 35--62, 2008.
[28]
T. Tanaka, K. Sakamoto, S. Kusumoto, K. Matsumoto, and T. Kikuno. Improvement of software process by process description and benefit estimation. In Proc. of the Int'l Conf. on Software Eng., pages 123--132, 1995.
[29]
Y. Tao, Y. Dang, T. Xie, D. Zhang, and S. Kim. How do software engineers understand code changes?: An exploratory study in industry. In Proc. of the ACM SIGSOFT Symp. on the Foundations of Software Eng., pages 51:1--51:11, 2012.
[30]
P. Thongtanunam, S. McIntosh, A. E. Hassan, and H. Iida. Investigating code review practices in defective files: An empirical study of the qt system. In Proc. of the Working Conf. on Mining Software Repositories, pages 168--179, 2015.
[31]
L. G. Votta, Jr. Does every inspection need a meeting? In Proc. of the ACM SIGSOFT Symp. on Foundations of Software Eng., pages 107--114, 1993.
[32]
T. Yu, V. Y. Shen, and H. E. Dunsmore. An analysis of several software defect models. IEEE Trans. Software Eng., 14(9):1261--1270, 1988.

Cited By

View all
  • (2024)Embedded-check a Code Quality Tool for Automatic Firmware VerificationProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653577(66-72)Online publication date: 3-Jul-2024
  • (2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
  • (2023)Are Code Review Smells and Metrics Useful in Pull Request-Level Software Defect Prediction?Developments in Information and Knowledge Management Systems for Business Applications10.1007/978-3-031-27506-7_2(27-52)Online publication date: 8-Apr-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICSE '16: Proceedings of the 38th International Conference on Software Engineering Companion
May 2016
946 pages
ISBN:9781450342056
DOI:10.1145/2889160
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 May 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. code review
  2. software quality

Qualifiers

  • Research-article

Conference

ICSE '16
Sponsor:

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)23
  • Downloads (Last 6 weeks)4
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Embedded-check a Code Quality Tool for Automatic Firmware VerificationProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653577(66-72)Online publication date: 3-Jul-2024
  • (2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
  • (2023)Are Code Review Smells and Metrics Useful in Pull Request-Level Software Defect Prediction?Developments in Information and Knowledge Management Systems for Business Applications10.1007/978-3-031-27506-7_2(27-52)Online publication date: 8-Apr-2023
  • (2022)An industrial experience report on retro-inspectionProceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice10.1145/3510457.3513055(43-52)Online publication date: 21-May-2022
  • (2022)Modeling review history for reviewer recommendationProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510213(1381-1392)Online publication date: 21-May-2022
  • (2022)An Industrial Experience Report on Retro-Inspection2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)10.1109/ICSE-SEIP55303.2022.9793980(43-52)Online publication date: May-2022
  • (2022)An empirical study on self-admitted technical debt in modern code reviewInformation and Software Technology10.1016/j.infsof.2022.106855146:COnline publication date: 1-Jun-2022
  • (2022)Towards a taxonomy of code review smellsInformation and Software Technology10.1016/j.infsof.2021.106737142:COnline publication date: 1-Feb-2022
  • (2022)A Neural Architecture for Detecting Identifier Renaming from DiffIntelligent Data Engineering and Automated Learning – IDEAL 202110.1007/978-3-030-91608-4_4(33-44)Online publication date: 1-Jan-2022
  • (2021)ConfigMiner: Identifying the Appropriate Configuration Options for Config-Related User Questions by Mining Online ForumsIEEE Transactions on Software Engineering10.1109/TSE.2020.297399747:12(2907-2918)Online publication date: 1-Dec-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media