Abstract
Software review is a fundamental component of the software quality assurance process, yet significant controversies surround the most efficient and effective review method. A central question surrounds the use of meetings; traditional review practice views them as essential, while more recent findings question their utility. To provide insight into this question, we conducted a controlled experiment to assess several measures of cost and effectiveness for a meeting and non-meeting-based review method. The experiment used CSRS, a computer mediated collaborative software review environment, and 24 three person groups. We found that the meeting-based review method studied was significantly more costly than the non-meeting-based method, but that meeting-based review did not find significantly more defects than the non-meeting-based method. However, the meeting-based review method was significantly better at reducing the level of false positives, and subjects subjectively preferred meeting-based review over non-meeting-based review. This paper presents the motivation for this experiment, its design and implementation, our empirical findings, pointers to Internet repositories for replication or additional analysis of this experiment, conclusions, and future directions.
Similar content being viewed by others
References
Ackerman, A. F., Buchwald, L. S., and Lewski, F. H. 1989. Software inspections: An effective verification process. IEEE Software, 31–36.
Bisant, D. B., and Lyle, J. R. 1989. A two-person inspection method to improve programming productivity. IEEE Transactions on Software Engineering 15(10): 1294–1304.
Diehl, M., and Stroebe, W. 1987. Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology 53(3): 497–509.
Dyer, M. 1992. The Cleanroom Approach to Quality Software Development. John Wiley & Sons, Inc.
Eick, S. G., Loader, C. R., Long, M. D., Vander Wiel, S. A., and Votta, L. G. Jr. 1992. Estimating software fault content before coding. Proceedings of the 14th International Conference on Software Engineering, Melbourne, Australia, 59–65.
Fagan, M. E. 1976. Design and code inspections to reduce errors in program development. IBM Systems Journal 15(3): 182–211.
Fagan, M. E. 1986. Advances in software inspections. IEEE Transactions on Software Engineering SE-12(7): 744–751.
Ferguson, G. A., and Takane, Y. 1989. Statistical Analysis In Psychology And Education, 6th edition. McGraw-Hill Book Company.
Freedman, D. P., and Weinberg, G. M. 1990. Handbook of Walkthroughs, Inspections and Technical Reviews, 4th edition. Little, Brown.
Gilb, T., and Graham, D. 1993. Software Inspection. Addison-Wesley.
Humphrey, W. S. 1990. Managing the Software Process. Addison Wesley Publishing Company Inc.
Johnson, P. M. 1994. An instrumented approach to improving software quality through formal technical review. Proceedings of the 16th International Conference on Software Engineering, Sorrento, Italy, 113–122.
Knight, J. C., and Myers, E. A. 1993. An improved inspection technique. Communications of the ACM 11(11): 51–61.
Martin, J., and Tsai, W.-T. 1990. N-fold inspection: A requirement analysis technique. Communications of the ACM 33(2): 225–232.
Mullen, B., Johnson, C. and Salas, E. 1991. Productivity loss in brainstorming groups. Basic and Applied Social Psychology 12(2): 3–24.
Parnas, D. L., and Weiss, D. M. 1987. Active design reviews: Principles and practices. Journal of Systems and Software 7: 259–265.
Porter, A. A., and Johnson, P. M. 1997. Assessing software review meetings: Results of a comparative analysis of two experimental studies. IEEE Transactions on Software Engineering, 23(3): 129–145.
Porter, A. A., Votta, L. G., Jr., and Basili, V. R. 1995. Comparing detection methods for software requirements inspections: A replicated experiment. IEEE Transactions on Software Engineering 21(6): 563–575.
Russell, G. W. 1991. Experience with inspection in ultralarge-scale developments. IEEE Software, January: 25–31.
Strauss, S. H., and Ebenau, R. G. 1994. Software Inspection Process. McGraw-Hill.
Tjahjono, D. 1996. Exploring the effectiveness of formal technical review factors with CSRS, a collaborative software review system. PhD thesis, Department of Information and Computer Sciences, University of Hawaii.
Votta, L. G., Jr. 1993. Does every inspection need a meeting? Proceedings of the ACM SIGSOFT 1993 Symposium on Foundations of Software Engineering, volume 18(5) of ACM Software Engineering Notes, December: 107–114.
Weller, E. F. 1993. Lessons learned from three years of inspection data. IEEE Software, September: 38–45.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Johnson, P.M., Tjahjono, D. Does Every Inspection Really Need a Meeting?. Empirical Software Engineering 3, 9–35 (1998). https://doi.org/10.1023/A:1009787822215
Issue Date:
DOI: https://doi.org/10.1023/A:1009787822215