Skip to main content
SpringerLink
Log in

On the application of measurement theory in software engineering

  • Viewpoint
  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Elements of measurement theory have recently been introduced into the software engineering discipline. It has been suggested that these elements should serve as the basis for developing, reasoning about, and applying measures. For example, it has been suggested that software complexity measures should be additive, that measures fall into a number of distinct types (i.e., levels of measurement: nominal, ordinal, interval, and ratio), that certain statistical techniques are not appropriate for certain types of measures (e.g., parametric statistics for less-than-interval measures), and that certain transformations are not permissible for certain types of measures (e.g., non-linear transformations for interval measures). In this paper we argue that, inspite of the importance of measurement theory, and in the context of software engineering, many of these prescriptions and proscriptions are either premature or, if strictly applied, would represent a substantial hindrance to the progress of empirical research in software engineering. This argument is based partially on studies that have been conducted by behavioral scientists and by statisticians over the last five decades. We also present a pragmatic approach to the application of measurement theory in software engineering. While following our approach may lead to violations of the strict prescriptions and proscriptions of measurement theory, we demonstrate that in practical terms these violations would have diminished consequences, especially when compared to the advantages afforded to the practicing researcher.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andrews, F., Klem, L., Davidson, T., O'Malley, P. and Rodgers, W. 1981. A Guide for Selecting Statistical Techniques for Analyzing Social Science Data. Institute for Social Research, University of Michigan.

  • Bailey, J. and Basili, V. 1981. A meta-model for software development resource expenditures. Proceedings of the International Conference on Software Engineering, 107–116.

  • Baker, B., Hardyck, C. and Petrinovich, L. 1966. Weak measurements vs. strong statistics: An empirical critique of S. S. Stevens' proscriptions on statistics. Educational and Psychological Measurement 26: 291–309.

    Google Scholar 

  • Basili, V. 1980. Resource models. Tutorial on Models and Metrics for Software Management and Engineering IEEE Computer Society Press, V. Basili (ed.).

  • Baroudi, J. and Orlikowski, W. 1989. The problem of statistical power in MIS research. MIS Quarterly 87–106.

  • Bieman, J. and Ott, L. M. 1994. Measuring functional cohesion. IEEE Trans. Software Eng. 20(8): 644–657, August.

    Google Scholar 

  • Bollman, P. 1984. Two axioms for evaluation measures in information retrieval. Research and Development in Information Retrieval, ACM, British Computer Society Workshop, Series, 233–246.

  • Boneau, C. 1962. A comparison of the power of the U and t tests. Psychological Review 69(3): 246–256.

    Google Scholar 

  • Briand, L., Morasca, S. and Basili, V. 1994. Property based software engineering measurement. Technical Report, CS-TR-119, University of Maryland.

  • Chidamber, S. R. and Kemerer, C. 1994. A metrics suite for object oriented design. IEEE Trans. Software Eng. 20(6): 476–493.

    Google Scholar 

  • Cohen, J. 1965. Some statistical issues in psychological research. Handbook of Clinical Psychology, B. Woleman (ed.), McGraw-Hill.

  • Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates.

  • Cohen, J. and Cohen, P. 1983. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates.

  • Dillon, W. and Goldstein, M. 1984. Multivariate Analysis: Methods and Applications. Wiley & Sons.

  • Fenton, N. 1991. Software Metrics: A Rigorous Approach. Chapman & Hall.

  • Fenton, N. 1994. Software measurement: A necessary scientific basis. IEEE Transactions on Software Engineering 20(3): 199–206.

    Google Scholar 

  • Galletta, D. and Lederer, A. 1989. Some cautions on the measurement of user information satisfaction. Decision Sciences 20: 419–438.

    Google Scholar 

  • Gardner, P. 1975. Scales and statistics. Review of Educational Research 45(1): 43–57.

    Google Scholar 

  • Gibbons, J. 1971. Nonparametric Statistical Inference. McGraw-Hill.

  • Gibbons, J. 1993. Nonparametric Statistics. Sage Publications.

  • Gibbons, J. 1993. Nonparametric Measures of Association. Sage Publications.

  • Ives, B., Olson, M. and Baroudi, J. 1983. The measurement of user information satisfaction. Communications of the ACM 26(10): 785–793.

    Google Scholar 

  • Kim, K. 1989. User information satisfaction: Toward conceptual clarity. Proceedings of the 11th International Conference on Information Systems. 183–191.

  • Kraemer, H. and Thiemann, S. 1987. How Many Subjects? Statistical Power Analysis in Research. Sage Publications.

  • Krantz, D., Luce, R., Suppes, P. and Tversky, A. 1971. Foundations of Measurement. Vol. 1, Academic Press.

  • Labovitz, S. 1967. Some observations on measurement and statistics. Social Forces 46(2): 151–160.

    Google Scholar 

  • Labovitz, S. 1970. The assignment of numbers to rank order categories. American Sociological Review 35: 515–524.

    Google Scholar 

  • Labovitz, S. 1971. In defense of assigning numbers to ranks. American Sociological Review 36: 521–522.

    Google Scholar 

  • Luce, R. and Krumhansl, C. 1988. Measurement, scaling, and psychophysics. Stevens' Handbook of Experimental Psychology. Wiley.

  • Mayer, L. 1971. A note on treating ordinal data as interval data. American Sociological Review 36: 519–520.

    Google Scholar 

  • McIver, J. and Carmines, E. 1981. Unidimensional Scaling. Sage Publications.

  • Michell, J. 1986. Measurement scales and statistics: A clash of paradigms. Psychological Bulletin 100(3): 398–407.

    Google Scholar 

  • Mosteller, F. and Tukey, J. 1977. Data Analysis and Regression. Addison-Wesley. 1977.

  • Nunnally, J. 1978. Psychometric Theory. McGraw-Hill.

  • Oviedo, E. 1980. Control flow, data flow, and program complexity. Proceedings of COMPSAC, 146–152.

  • Roberts, F. 1979. Measurement Theory with Applications to Decisionmaking, Utility, and the Social Sciences. Addison-Wesley.

  • Siegel, S. and Castellan, J. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw Hill.

  • Stevens, S. 1946. On the theory of scales of measurement. Science 103(2684): 677–680.

    Google Scholar 

  • Stevens, S. 1951. Mathematics, measurement, and psychophysics. Handbook of Experimental Psychology S. Stevens (ed.), John Wiley.

  • Stevens, S. 1962. Measurement, psychophysics, and utility. Measurement: definitions and theories, C. Churchman and P. Ratoosh (eds.), John Wiley.

  • Stevens, S. 1968. Measurement, statistics and the schemapiric view. Science 161: 849–856.

    Google Scholar 

  • Suppes, P. and Zinnes, J. 1963. Basic measurement theory. Handbook of Mathematical Psychology. Vol. 1, R. Luce, R. Bush, and E. Galanter (eds.), John Wiley.

  • Townsend, J. and Ashby, F. 1984. Measurement scales and statistics: The misconception misconceived. Psychological Bulletin 96(2): 394–401.

    Google Scholar 

  • Tukey, J. 1986. Data analysis and behavioral science or learning to bear the quantitative man's burden by shunning badmandments. The Collected Works of John W. Tukey. Vol. III, Wadsworth.

  • Tukey, J. 1986. The future of data analysis. The Collected Works of John W. Tukey. Vol. III, Wadsworth.

  • Velleman, P. and Wilkinson, L. 1993. Nominal, ordinal, interval, and ratio typologies are misleading. The American Statistician 47(1): 65–72.

    Google Scholar 

  • Walston, C. and Felix, C. 1977. A method of programming measurement and estimation. IBM Systems Journal 1: 54–73.

    Google Scholar 

  • Weyuker, E. 1988. Evaluating software complexity measures. IEEE Transactions on Software Engineering 14(9): 1357–1365.

    Google Scholar 

  • Zuse, H. 1991. Software Complexity: Measures and Methods. de Gruyter.

  • Zuse, H. 1992. Measuring factors contributing to software maintenance complexity. Proceedings of the 2nd International Conference on Software Quality Triangle Research Park, NC.

  • Zuse, H. 1994. Software complexity metrics/analysis. Encylopedia of Software Engineering. J. Marciniak, (ed.), Vol. I, pp. 31–166, John Wiley & Sons.

  • Zuse, H. and Fetcke, T. 1995. Properties of object-oriented software measures. Proceedings of the Annual Oregon Workshop on Software Metrics.

Download references

Author information

Authors and Affiliations

  1. Software Engineering Group, Centre de Recherche Informatique de Montréal, (CRIM), 1801 McGill College av., PQ, H3A 2N4, Montréal, Canada

    Lionel Briand

  2. Software Engineering Group, Centre de Recherche Informatique de Montréal, (CRIM), 1801 McGill College av., PQ, H3A 2N4, Montréal, Canada

    Khaled El Emam

  3. Dipartimento di Elettronica e Informazione, Politecnico di Milano, Piazza L. Da Vinci 32, I-20133, Milano, Italy

    Sandro Morasca

Authors
  1. Lionel Briand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khaled El Emam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sandro Morasca
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Briand, L., Emam, K.E. & Morasca, S. On the application of measurement theory in software engineering. Empir Software Eng 1, 61–88 (1996). https://doi.org/10.1007/BF00125812

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00125812

Keywords

Search

Navigation