Skip to main content
Log in

Does Every Inspection Really Need a Meeting?

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software review is a fundamental component of the software quality assurance process, yet significant controversies surround the most efficient and effective review method. A central question surrounds the use of meetings; traditional review practice views them as essential, while more recent findings question their utility. To provide insight into this question, we conducted a controlled experiment to assess several measures of cost and effectiveness for a meeting and non-meeting-based review method. The experiment used CSRS, a computer mediated collaborative software review environment, and 24 three person groups. We found that the meeting-based review method studied was significantly more costly than the non-meeting-based method, but that meeting-based review did not find significantly more defects than the non-meeting-based method. However, the meeting-based review method was significantly better at reducing the level of false positives, and subjects subjectively preferred meeting-based review over non-meeting-based review. This paper presents the motivation for this experiment, its design and implementation, our empirical findings, pointers to Internet repositories for replication or additional analysis of this experiment, conclusions, and future directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ackerman, A. F., Buchwald, L. S., and Lewski, F. H. 1989. Software inspections: An effective verification process. IEEE Software, 31–36.

  • Bisant, D. B., and Lyle, J. R. 1989. A two-person inspection method to improve programming productivity. IEEE Transactions on Software Engineering 15(10): 1294–1304.

    Google Scholar 

  • Diehl, M., and Stroebe, W. 1987. Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology 53(3): 497–509.

    Google Scholar 

  • Dyer, M. 1992. The Cleanroom Approach to Quality Software Development. John Wiley & Sons, Inc.

    Google Scholar 

  • Eick, S. G., Loader, C. R., Long, M. D., Vander Wiel, S. A., and Votta, L. G. Jr. 1992. Estimating software fault content before coding. Proceedings of the 14th International Conference on Software Engineering, Melbourne, Australia, 59–65.

  • Fagan, M. E. 1976. Design and code inspections to reduce errors in program development. IBM Systems Journal 15(3): 182–211.

    Google Scholar 

  • Fagan, M. E. 1986. Advances in software inspections. IEEE Transactions on Software Engineering SE-12(7): 744–751.

    Google Scholar 

  • Ferguson, G. A., and Takane, Y. 1989. Statistical Analysis In Psychology And Education, 6th edition. McGraw-Hill Book Company.

    Google Scholar 

  • Freedman, D. P., and Weinberg, G. M. 1990. Handbook of Walkthroughs, Inspections and Technical Reviews, 4th edition. Little, Brown.

  • Gilb, T., and Graham, D. 1993. Software Inspection. Addison-Wesley.

  • Humphrey, W. S. 1990. Managing the Software Process. Addison Wesley Publishing Company Inc.

  • Johnson, P. M. 1994. An instrumented approach to improving software quality through formal technical review. Proceedings of the 16th International Conference on Software Engineering, Sorrento, Italy, 113–122.

  • Knight, J. C., and Myers, E. A. 1993. An improved inspection technique. Communications of the ACM 11(11): 51–61.

    Google Scholar 

  • Martin, J., and Tsai, W.-T. 1990. N-fold inspection: A requirement analysis technique. Communications of the ACM 33(2): 225–232.

    Google Scholar 

  • Mullen, B., Johnson, C. and Salas, E. 1991. Productivity loss in brainstorming groups. Basic and Applied Social Psychology 12(2): 3–24.

    Google Scholar 

  • Parnas, D. L., and Weiss, D. M. 1987. Active design reviews: Principles and practices. Journal of Systems and Software 7: 259–265.

    Google Scholar 

  • Porter, A. A., and Johnson, P. M. 1997. Assessing software review meetings: Results of a comparative analysis of two experimental studies. IEEE Transactions on Software Engineering, 23(3): 129–145.

    Google Scholar 

  • Porter, A. A., Votta, L. G., Jr., and Basili, V. R. 1995. Comparing detection methods for software requirements inspections: A replicated experiment. IEEE Transactions on Software Engineering 21(6): 563–575.

    Google Scholar 

  • Russell, G. W. 1991. Experience with inspection in ultralarge-scale developments. IEEE Software, January: 25–31.

  • Strauss, S. H., and Ebenau, R. G. 1994. Software Inspection Process. McGraw-Hill.

  • Tjahjono, D. 1996. Exploring the effectiveness of formal technical review factors with CSRS, a collaborative software review system. PhD thesis, Department of Information and Computer Sciences, University of Hawaii.

  • Votta, L. G., Jr. 1993. Does every inspection need a meeting? Proceedings of the ACM SIGSOFT 1993 Symposium on Foundations of Software Engineering, volume 18(5) of ACM Software Engineering Notes, December: 107–114.

    Google Scholar 

  • Weller, E. F. 1993. Lessons learned from three years of inspection data. IEEE Software, September: 38–45.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Johnson, P.M., Tjahjono, D. Does Every Inspection Really Need a Meeting?. Empirical Software Engineering 3, 9–35 (1998). https://doi.org/10.1023/A:1009787822215

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1009787822215

Navigation