Skip to main content
SpringerLink
Log in

Statistical Methods and Measurement

  • Chapter
Guide to Advanced Empirical Software Engineering

Useful ways of measuring software engineering phenomena have to address two challenges: defining realistic and valid metrics that can feasibly be collected under the constraints and time pressures of real-world software development contexts, and determining valid and accurate ways of analysing the resulting data to guide decisions. Too often, the difficulties of addressing the first challenge mean that the second is given little attention. The purpose of this chapter is to present different techniques for the definition and analysis of metrics such as product quality data. Specifically, statistical issues in the definition and application of metrics are presented with reference to software engineering examples.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Agresti, A, Analysis of Ordinal Categorical Data. New York: Wiley. 1984.

    MATH  Google Scholar 

  • Agresti, A, An Introduction to Categorical Data Analysis. New York: Wiley. 1998.

    Google Scholar 

  • Anscombe, F, Graphs in statistical analysis. American Statistician. 27(1):17–21. 1973.

    Article  Google Scholar 

  • Basili, V, Caldiera, G, and Rombach, D, The goal question metric approach. In: Marciniak, J, ed., Encyclopedia of Software Engineering. New York: Wiley. 1994.

    Google Scholar 

  • Bevington, P, and Robinson, D, Data Reduction and Error Analysis for the Physical Sciences, 2nd ed. New York: McGraw-Hill. 1992.

    Google Scholar 

  • Bowerman, B, and O’Connell, R, Forecasting and Time Series: An Applied Approach, 3rd. ed.Belmont, CA: Wadsworth. 1993.

    MATH  Google Scholar 

  • Box, G and Draper, N, Evolutionary Operation: A Statistical Method for Process Improvement. New York: Wiley. 1969.

    Google Scholar 

  • Box, G and Luceño, A, Statistical Control by Monitoring and Feedback Adjustment. New York: Wiley. 1997.

    MATH  Google Scholar 

  • Briand, L, El Emam, K, and Morasca, S, On the application of measurement theory to software engineering. Empirical Software Engineering. 1(1). 1996.

    Google Scholar 

  • Chayes, F, Ratio Correlation. Chicago: University of Chicago Press. 1971.

    Google Scholar 

  • Cleveland, W, The Elements of Graphing Data. Summit, NJ: Hobart Press. 1994.

    Google Scholar 

  • Cliff, N, What is and isn’t measurement. In: Keren, G and Lewis, C, eds., A Handbook For Data Analysis in the Behavioral Sciences, Vol. 1: Methodological Issues. Hillsdale, NJ: Erlbaum. 1992.

    Google Scholar 

  • Cohen, J, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillside, NJ: Erlbaum. 1988.

    MATH  Google Scholar 

  • Comrey, A and Lee, H, A First Course in Factor Analysis, 2nd ed.Hillsdale, NJ: Erlbaum. 1992.

    Google Scholar 

  • Crowder, M, and Hand, D, Analysis of Repeated Measures. New York: Chapman and Hall. 1990.

    MATH  Google Scholar 

  • Dobson, A, An Introduction to Generalized Linear Models, 2nd ed.New York: Chapman and Hall/CRC. 2001.

    Google Scholar 

  • Draper, N and Smith, H, Applied Regression Analysis, 2nd ed.New York: Wiley. 1998.

    MATH  Google Scholar 

  • Duncan, A, Quality Control and Industrial Statistics, 5th ed.New York: Irwin. 1986.

    Google Scholar 

  • El Emam, K, Benlarbi, S, and Goel, N, Comparing case-based reasoning classifiers for predicting high risk software components. National Research Council Canada technical report NRC 43602/ERB-1058. 1999.

    Google Scholar 

  • Fenton, N and Pfleeger, S, Software Metrics: A Rigorous and Practical Approach, 2nd ed.Boston: PWS Publishing. 1997.

    Google Scholar 

  • Fliess, J, Statistical Methods for Rates and Proportions, 2nd ed.New York: Wiley. 1981.

    Google Scholar 

  • Ghiselli, E, Campbell, J, and Zedeck, S, Measurement Theory for the Behavioral Sciences. San Francisco: Freeman. 1981.

    Google Scholar 

  • Good, P, Permutation Tests. New York: Springer. 1994.

    MATH  Google Scholar 

  • Goodman, L and Kruskal, W, Measures of Association for Cross Classifications. New York: Springer. 1979.

    MATH  Google Scholar 

  • Gottman, J, ed., The Analysis of Change. Hillsdale, NJ: Erlbaum. 1995.

    Google Scholar 

  • Haccou, P, and Meelis, E, Statistical Analysis of Behavioural Data: An Approach Based on Time-Structured Models. Oxford: Oxford University Press. 1994.

    Google Scholar 

  • Hand, D, Construction and Assessment of Classification Rules. New York: Wiley. 1997.

    MATH  Google Scholar 

  • Hand, D, Measurement Theory and Practice: The World through Quantification. Oxford: Oxford University Press. 2004.

    MATH  Google Scholar 

  • Hosmer, D and Lemeshow, S, Applied Logistic Regression. New York: Wiley. 1989.

    Google Scholar 

  • Hosmer, D and Lemeshow, S, Applied Survival Analysis. New York: Wiley. 1999.

    MATH  Google Scholar 

  • Jacobs, R, Smith, P, and Street, A, Measuring Efficiency in Health Care: Analytic Techniques and Health Policy. Cambridge: Cambridge University Press. 2006.

    Google Scholar 

  • Keppel, G, Design and Analysis: A Researcher’s Handbook, 3rd ed.New York: Prentice Hall. 1991.

    Google Scholar 

  • Kleinbaum, D, Logistic Regression. New York: Springer. 1994.

    MATH  Google Scholar 

  • Kleinbaum, D, Survival Analysis. New York: Springer. 1996.

    MATH  Google Scholar 

  • Krantz, D, Luce, R, Suppes, P, and Tversky, A, Foundations of Measurement. New York: Academic. 1971.

    MATH  Google Scholar 

  • Long, J, Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks, CA: Sage. 1997.

    MATH  Google Scholar 

  • Maddala, G, Limited-Dependent and Qualitative Variables in Econometrics. Cambridge: Cambridge University Press. 1986.

    Google Scholar 

  • Makridakis, S, Wheelwright, S, and Hyndman, R, Forecasting: Methods and Applications, 3rd ed.New York: Wiley. 1998.

    Google Scholar 

  • Montgomery, D, Introduction to Statistical Quality Control, 3rd ed.New York: Wiley. 1996.

    Google Scholar 

  • Montgomery, D and Myers, R, Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 2nd ed.New York: Wiley. 2002.

    MATH  Google Scholar 

  • Phelps, C, and Huston, A, Estimating diagnostic accuracy using a “fuzzy gold standard”. Medical Decision Making 15:44–57. 1995.

    Article  Google Scholar 

  • Rawlings, J, Pantula, S, and Dickey, D, Applied Regression Analysis, 2nd ed.New York: Springer. 1998.

    MATH  Google Scholar 

  • Rosenbaum, P, Observational Studies, 2nd ed.New York: Springer. 2002.

    MATH  Google Scholar 

  • Rosenberg, J, A methodology for evaluating predictive metrics. In: Zelkowitz, M., ed., Advances in Computers, Vol. 23. New York: Academic. 2000.

    Google Scholar 

  • Shepperd, M and Ince, D, Derivation and Validation of Software Metrics. Oxford: Clarendon Press. 1993.

    MATH  Google Scholar 

  • Singer, J and Willett, J, Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford: Oxford University Press. 2003.

    Google Scholar 

  • Sprent, P, Applied Non-Parametric Statistical Methods, 2nd ed.New York: Chapman and Hall. 1993.

    Google Scholar 

  • Swets, J, Signal Detection Theory and ROC Analysis in Psychology and Diagnostics. Hillsdale, NJ: Erlbaum. 1996.

    MATH  Google Scholar 

  • Taylor, J, An Introduction to Error Analysis, 2nd ed.Sausalito, CA: University Science Books. 1997.

    Google Scholar 

  • Valenstein, P, Evaluating diagnostic tests with imperfect standards. American Journal of Clinical Pathology 93:252–258. 1990.

    Google Scholar 

  • Velleman, P, Nominal, ordinal, interval, and ratio typologies are misleading. American Statistician. 47:65–72. 1993.

    Article  Google Scholar 

  • Wellek, S, Testing Statistical Hypotheses of Equivalence. New York: Chapman and Hall/CRC Press. 2002.

    Google Scholar 

  • Wickens, T, Multiway Contingency Tables Analysis for the Social Sciences. Hillsdale, NJ: Erlbaum. 1989.

    MATH  Google Scholar 

  • Zhou, X, Obuchowski, N, and McClish, D, Statistical Methods in Diagnostic Medicine. New York: Wiley. 2002.

    Book  MATH  Google Scholar 

  • Zuse, H, Software Complexity: Measures and Methods. New York: Walter de Gruyter. 1990.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sun Microsystems, 94303, Palo Alto, CA, USA

    Jarrett Rosenberg

Authors
  1. Jarrett Rosenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Fraunhofer Center Maryland, 20742, College Park, MD, USA

    Forrest Shull

  2. National Research Council Canada, K1A 0R6, Ottawa, Ontario, Canada

    Janice Singer

  3. Simula Research Laboratory, NO-1325, Lysaker, Norway

    Dag I. K. Sjøberg

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag London Limited

About this chapter

Cite this chapter

Rosenberg, J. (2008). Statistical Methods and Measurement. In: Shull, F., Singer, J., Sjøberg, D.I.K. (eds) Guide to Advanced Empirical Software Engineering. Springer, London. https://doi.org/10.1007/978-1-84800-044-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-1-84800-044-5_6

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-84800-043-8

  • Online ISBN: 978-1-84800-044-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation