Abstract
A method using expert judgement for the evaluation of software quality is presented. The underlying principle of the approach is the encoding of experts' tacit knowledge into probabilistic measures associated with the achievement level of software quality attributes. An aggregated quality measure is obtained based on preference statements related to the quality attributes. The technical objectives of the paper are
• to develop of a generic and operationally feasible measurement technique to transform the tacit knowledge of a software expert to a probability distribution depicting his/her uncertainty of the level of achievement related to a quality attribute;
• to develop rules for the construction of a consensus probability measure based on expert-specific probability measures;
• to derive a framework for specifying software quality strategy and for evaluating the acceptance of a software produced in a software development process;
The above technical developments are used to support group decision-making regarding
• the launch or implementation decision of a software version;
• the allocation of resources during the software development process.
Similar content being viewed by others
References
Basili, V.R. and Rombach, H.D. 1988. The TAME project: Towards improvement-oriented software environments, IEEE Transactions on Software Engineering 14(6): 758–773.
Baumert, J.H. and McWhinney, M.S. 1992. Software measures and the capability maturity model, Technical Report CMU/SEI-92-TR-25, Software Engineering Institute, Carnegie Mellow University, Pittsburgh.
Butler, R.W. and Finelli, G.B. 1993. The infeasibility of quantifying the reliability of life-critical real-time software, IEEE Trans. Software Reliability 19: 3–12.
Cooke, R. 1991. Experts in Uncertainty, Oxford University Press, Oxford.
Fenton, N.E. and Neil, M. 1999. Software metrics: successes, failures and new directions, The Journal of Systems and Software 47: 149–157.
Fenton, N.E. and Pfleeger, S.L. 1997. Software Metrics-A Rigorous and Practical Approach, 2nd ed., PWS Publishing Company, Boston.
Fowler, M. and Scott, K. 1997. UML Distilled-Applying the Standard Object Modelling Language, Addison-Wesley Object Technology Series.
Garvin, D. 1984. What does “product quality” really mean?, Sloan Management Review 26(1): 25–45.
ISO 9126. 1991. ISO 9126 Information Technology-Software Product Evaluation-Quality Characteristics and Guidelines for their Use, Geneva.
Järvinen, J. 2000. Measurement Based Continuous Assessment of Software Engineering Processes, VTT Publications 426 (Dissertation), VTT Technical Research Centre of Finland.
Keiller, P.A. and Miller, D.R. 1991. On the use and the performance of software reliability growth models, Reliability Engineering and Systems Safety 13: 95–117.
Kelly, G.A. 1955. The Psychology of Personal Constructs-A Theory of Personality, Norton, New York.
Kilpi, T. 2001. Implementing a Software Metrics Program at Nokia, IEEE Software 18(6): 72–77.
Littlewood, B. and Strigini, L. 1993. Validation of ultra-high dependability for software based systems, Communications of ACM 36: 69–88.
Mason, D. and Woit, D. 1998. Software system reliability form component reliability, In Proc. of 1998 Workshop on Software Reliability Engineering (SRE'98), Ottawa, Ontario.
McCall, J.A. 1994. Quality factors, In J.J. Marciniak (ed.), Encyclopedia of Software Engineering, John Wiley & Sons, New York.
Musa, J.D. 1993. Operational profiles in software-reliability engineering, IEEE Software 10(2): 14–32.
NRC. 1989. NUREG-1150: Reactor risk reference document, Report NUREG-1150, U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research, Maryland.
Palisade Corporation. 1996. @Risk-Advanced Risk Analysis for Spreadsheets, Palisade Corporation, New York.
Pulkkinen, U. 1993. Methods for combination of expert judgements, Reliability Engineering and Systems Safety 40: 111–118.
Rumbaugh, J., Booch, G., and Jacobson, I. 1998. Unified Modelling Language, Addison-Wesley Object Technology Series.
Saaty, T.L. 1994. How to make decisions: The analytic hierarchy process, Interfaces 24(6): 19–43.
Tervonen, I. 1996. Support for quality-based design and inspection, IEEE Software 13(1): 44–54.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Rosqvist, T., Koskela, M. & Harju, H. Software Quality Evaluation Based on Expert Judgement. Software Quality Journal 11, 39–55 (2003). https://doi.org/10.1023/A:1023741528816
Issue Date:
DOI: https://doi.org/10.1023/A:1023741528816