Skip to main content
Log in

Evaluation of Usage-Based Reading—Conclusions after Three Experiments

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software inspections have been introduced in software engineering in order to detect faults before testing is performed. Reading techniques provide reviewers in software inspections with guidelines on how they should check the documents under inspection. Several reading techniques with different purposes have been introduced and empirically evaluated. In this paper, we describe a reading technique with the special aim to detect faults that are severe from a user’s point of view. The reading technique is named usage-based reading (UBR) and it can be used to inspect all software artefacts. In the series of experiments, a high-level design document is used. The main focus of the paper is on the third experiment, which investigates the information needed for UBR in the individual preparation and the meeting of software inspections. Hence, the paper discusses (1) the series of three experiments of UBR, (2) the individual preparation of the third experiment, and (3) the meeting part of the third experiment. For each of these three parts, results are produced. The main results are (1) UBR is an efficient and effective reading technique that can be used for user-focused software inspections, (2) UBR is more efficient and effective if the information used for UBR is developed prior to, instead of during the individual preparation, and (3) the meeting affects the UBR inspection in terms of increased effectiveness and decreased efficiency. In summary, the empirical evidence shows that UBR is an efficient and effective reading technique to be used by software organizations that produce software for which the user perceived quality is important.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aurum, A., Petersson, H., and Wohlin, C. 2002. State-of-the-art: Software inspections after 25 years. Software Testing, Verification and Reliability 12(3): 133–154.

    Google Scholar 

  • Basili, V. R., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sørumgård, S., and Zelkowitz, M. V. 1996. The empirical investigation of perspective-based reading. Empirical Software Engineering: An International Journal 1(2): 133–164.

    Google Scholar 

  • Basili, V. R., Shull, F., and Lanubile, F. 1999. Building knowledge through families of experiments. IEEE Transactions on Software Engineering 25(4): 456–473.

    Google Scholar 

  • Cheng, B., and Jeffery, R. 1996. Comparing inspection strategies for software requirement specifications. Proceedings of the 8th Australian Software Engineering Conference. pp. 203-211.

  • Dunsmore, A., Roper, M., and Wood, M. 2002. Further investigations into the development and evaluation of reading techniques for object-oriented code inspection. Proceedings of the 24th International Conference on Software Engineering. pp. 47-57.

  • Fagan, M. E. 1976. Design and code inspections to reduce errors in program development. IBM System Journal 15(3): 182–211.

    Google Scholar 

  • ITU-T Z.100. 1993. Specification and Description Language. SDL, ITU-T Recommendation Z.100.

  • ITU-T Z.120. 1996. Message Sequence Charts. MSC, ITU-T Recommendation Z.120.

  • Jacobson, I., Christerson, M., Jonsson, P., and Övergaard, G. 1992. Object-Oriented Software Engineering: A Use Case Driven Approach. Addison-Wesley, USA.

    Google Scholar 

  • Knight, J. C., and Myers, A. E. 1993. An improved inspection technique. Communications of ACM 36(11): 50–69.

    Google Scholar 

  • Laitenberger, O., and DeBaud, J.-M. 2000. An encompassing life cycle centric survey of software inspection. Journal of Systems and Software 50(1): 5–31.

    Google Scholar 

  • Lauesen, S. 2002. Software Requirements-Styles and Techniques. Addison-Wesley, UK.

    Google Scholar 

  • Martin, J., and Tsai, W. T. 1990. N-fold inspection: A requirements analysis technique. Communications of ACM 33(2): 225–232.

    Google Scholar 

  • Musa, J. D. 1998. Software Reliability Engineering: More Reliable Software, Faster Development and Testing. McGraw-Hill, USA.

    Google Scholar 

  • Olofsson, M., and Wennberg, M. 1996. Statistical Usage Inspection. Master's Thesis, Dept. of Communication Systems, Lund University, CODEN: LUTEDX (TETS-5244)/1-81/(1996)&local 9.

  • Petersson, H., Thelin, T., Runeson, P., and Wohlin, C. 2004. Capture-recapture in software inspections after 10 years research-theory, evaluation and application. To appear in Journal of Systems and Software.

  • Porter, A., Votta, L., and Basili, V. R. 1995. Comparing detection methods for software requirements inspection: A replicated experiment. IEEE Transactions on Software Engineering 21(6): 563–575.

    Google Scholar 

  • Runeson, P. 2000. A new software engineering program-structure and initial experiences. Proceedings of the 13th International Conference on Software Engineering Education & Training. pp. 223-232.

  • Saaty, T. L., and Vargas, L. G. 2001. Models, methods, concepts & applications of the analytic hierarchy process. Netherlands: Kluwer Academic Publishers.

    Google Scholar 

  • Siegel, S., and Castellan, N. J. 1988. Nonparametric Statistics for the Behavioral Sciences. Singapore: McGraw-Hill.

    Google Scholar 

  • Thelin, T., Runeson, P., and Regnell, B. 2001. Usage-based reading-an experiment to guide reviewers with use cases. Information and Software Technology 43(15): 925–938.

    Google Scholar 

  • Thelin, T., Runeson, P., and Wohlin, C. 2003a. An experimental comparison of usage-based and checklistbased reading. IEEE Transactions on Software Engineering August, 29(8): 687–704.

    Google Scholar 

  • Thelin, T., Runeson, P., and Wohlin, C. 2003b. Prioritized use cases as a vehicle for software inspections. IEEE Software July/August, 20(4): 30–33.

    Google Scholar 

  • Travassos, G., Shull, F., Fredericks, M., and Basili, V. R. 1999. Detecting defects in object-oriented designs: Using reading techniques to increase software quality. Proc. of the International Conference on Object-Oriented Programming Systems, Languages & Applications.

  • Votta, L. G. 1993. Does every inspection need a meeting? Proceedings of the 1st ACM SIGSOFT Symposium on Foundations of Software Engineering, ACM Software Engineering Notes 18(5): 107–114.

    Google Scholar 

  • Wohlin, C. 1997. Meeting the challenge of large-scale software engineering in a university environment. Proceedings of the 10th Conference on Software Engineering Education & Training. pp. 40-52.

  • Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. 2000. Experimentation in Software Engineering: An Introduction. USA: Kluwer Academic Publisher.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Thelin, T., Runeson, P., Wohlin, C. et al. Evaluation of Usage-Based Reading—Conclusions after Three Experiments. Empirical Software Engineering 9, 77–110 (2004). https://doi.org/10.1023/B:EMSE.0000013515.86806.d4

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:EMSE.0000013515.86806.d4

Navigation