Skip to main content
SpringerLink
Log in

Software Quality Evaluation Based on Expert Judgement

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

A method using expert judgement for the evaluation of software quality is presented. The underlying principle of the approach is the encoding of experts' tacit knowledge into probabilistic measures associated with the achievement level of software quality attributes. An aggregated quality measure is obtained based on preference statements related to the quality attributes. The technical objectives of the paper are

• to develop of a generic and operationally feasible measurement technique to transform the tacit knowledge of a software expert to a probability distribution depicting his/her uncertainty of the level of achievement related to a quality attribute;

• to develop rules for the construction of a consensus probability measure based on expert-specific probability measures;

• to derive a framework for specifying software quality strategy and for evaluating the acceptance of a software produced in a software development process;

The above technical developments are used to support group decision-making regarding

• the launch or implementation decision of a software version;

• the allocation of resources during the software development process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Basili, V.R. and Rombach, H.D. 1988. The TAME project: Towards improvement-oriented software environments, IEEE Transactions on Software Engineering 14(6): 758–773.

    Google Scholar 

  • Baumert, J.H. and McWhinney, M.S. 1992. Software measures and the capability maturity model, Technical Report CMU/SEI-92-TR-25, Software Engineering Institute, Carnegie Mellow University, Pittsburgh.

    Google Scholar 

  • Butler, R.W. and Finelli, G.B. 1993. The infeasibility of quantifying the reliability of life-critical real-time software, IEEE Trans. Software Reliability 19: 3–12.

    Google Scholar 

  • Cooke, R. 1991. Experts in Uncertainty, Oxford University Press, Oxford.

    Google Scholar 

  • Fenton, N.E. and Neil, M. 1999. Software metrics: successes, failures and new directions, The Journal of Systems and Software 47: 149–157.

    Google Scholar 

  • Fenton, N.E. and Pfleeger, S.L. 1997. Software Metrics-A Rigorous and Practical Approach, 2nd ed., PWS Publishing Company, Boston.

    Google Scholar 

  • Fowler, M. and Scott, K. 1997. UML Distilled-Applying the Standard Object Modelling Language, Addison-Wesley Object Technology Series.

  • Garvin, D. 1984. What does “product quality” really mean?, Sloan Management Review 26(1): 25–45.

    Google Scholar 

  • ISO 9126. 1991. ISO 9126 Information Technology-Software Product Evaluation-Quality Characteristics and Guidelines for their Use, Geneva.

  • Järvinen, J. 2000. Measurement Based Continuous Assessment of Software Engineering Processes, VTT Publications 426 (Dissertation), VTT Technical Research Centre of Finland.

  • Keiller, P.A. and Miller, D.R. 1991. On the use and the performance of software reliability growth models, Reliability Engineering and Systems Safety 13: 95–117.

    Google Scholar 

  • Kelly, G.A. 1955. The Psychology of Personal Constructs-A Theory of Personality, Norton, New York.

    Google Scholar 

  • Kilpi, T. 2001. Implementing a Software Metrics Program at Nokia, IEEE Software 18(6): 72–77.

    Google Scholar 

  • Littlewood, B. and Strigini, L. 1993. Validation of ultra-high dependability for software based systems, Communications of ACM 36: 69–88.

    Google Scholar 

  • Mason, D. and Woit, D. 1998. Software system reliability form component reliability, In Proc. of 1998 Workshop on Software Reliability Engineering (SRE'98), Ottawa, Ontario.

  • McCall, J.A. 1994. Quality factors, In J.J. Marciniak (ed.), Encyclopedia of Software Engineering, John Wiley & Sons, New York.

    Google Scholar 

  • Musa, J.D. 1993. Operational profiles in software-reliability engineering, IEEE Software 10(2): 14–32.

    Google Scholar 

  • NRC. 1989. NUREG-1150: Reactor risk reference document, Report NUREG-1150, U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research, Maryland.

    Google Scholar 

  • Palisade Corporation. 1996. @Risk-Advanced Risk Analysis for Spreadsheets, Palisade Corporation, New York.

    Google Scholar 

  • Pulkkinen, U. 1993. Methods for combination of expert judgements, Reliability Engineering and Systems Safety 40: 111–118.

    Google Scholar 

  • Rumbaugh, J., Booch, G., and Jacobson, I. 1998. Unified Modelling Language, Addison-Wesley Object Technology Series.

  • Saaty, T.L. 1994. How to make decisions: The analytic hierarchy process, Interfaces 24(6): 19–43.

    Google Scholar 

  • Tervonen, I. 1996. Support for quality-based design and inspection, IEEE Software 13(1): 44–54.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. VTT Industrial Systems, 02044 VTT, Espoo, Finland

    Tony Rosqvist, Mika Koskela & Hannu Harju

Authors
  1. Tony Rosqvist
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mika Koskela
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hannu Harju
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rosqvist, T., Koskela, M. & Harju, H. Software Quality Evaluation Based on Expert Judgement. Software Quality Journal 11, 39–55 (2003). https://doi.org/10.1023/A:1023741528816

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1023741528816

Search

Navigation