Jean Scholtz
ABSTRACT
This paper introduces an evaluation method that provides the capability of comparing results of like-structured evaluations that occur over time and with changing toolsets or environmental conditions. This makes use of the framework ideal for comparison of collaboration tools. The framework helps to structure evaluations by mapping system goals to evaluation objectives, metrics, and measures. The upper-most levels of the framework are conceptual in nature, while the bottom level is implementation-specific, i.e., evaluation-specific. Careful attention during construction of the conceptual elements for an evaluation template allows for its reuse in a series of like-structured evaluations and comparison of those results.
- Basili, V. R. and Rombach, H. Tailoring the software process to project goals and environments. In Proceedings of the 9th International Conference on Software Engineering, 1987, 345--357. Google Scholar
Digital Library
- Basili, V. R and Weiss, D. M. A Methodology for Collecting Valid Software Engineering Data, IEEE Transactions on Software Engineering, Vol. SE-10, No. 3, 1984, 728--738.Google ScholarDigital Library
- Boehm, B. W., Brown, J. R., and Lipow, M. Quantitative Evaluation of Software Quality. In Proceedings of the 2nd International Conference on Software Engineering, 1976, 592--605. Google ScholarDigital Library
- Emam, K. E., Moukheiber, N., and Madhavji, N. H. An Empirical Evaluation of the G/Q/M Method. In Proceedings of the 1993 Conference of the Centre for Advanced Studies on Collaborative Research: software engineering, Vol. 1, Toronto, Ontario, 1993, 265--289. Google ScholarDigital Library
- Grudin, J. Why CSCW Applications Fail: Problems in the Design and Evaluation of Organizational Interfaces. In Proceedings of CSCW, 1988, 85--93. Google ScholarDigital Library
- IEEE Standard for a Software Quality Metrics Methodology, IEEE Std 1061--1998, 1998, 2-3.Google Scholar
- Kogure, M. and Akao, Y. Quality Function Deployment and CWQC in Japan. Quality Progress, October 1983, 25--29.Google Scholar
- Mack, G., Longeran, K., Scholtz, J., Steves, M. P., and Hale, C. A Framework for Metrics in Large, Complex Systems. In Proceedings of the IEEE Aerospace Conference, Big Sky, Montana, 2004, T10.0409.Google Scholar
- McCall, J. A., Richards, P. K., and Walters, G. F. Factors in Software Quality. RADC TR-77-369, 1977.Google Scholar
- MEL/ITL Task Group on Metrology for Information Technology (IT), "Metrology for Information Technology", NISTIR 6025, National Institute of Standards and Technology, Gaithersburg, MD, 1997, 8.Google Scholar
- Murine, G. E. Applying Software Quality Metrics in the Requirements Analysis Phase of a Distributive System. In Proceedings of the Minnowbrook Workshop, Blue Mountain Lake, New York, 1980.Google Scholar
- Rombach, H. D. Practical Benefits of Goal-Oriented Measurement. In Software Reliability and Metrics, eds. N. Fenton and B. Littlewood, Elsevier Science Publishing Co, London, 1991, 217--235.Google Scholar
- Steves, M. P. and Scholtz, J. Metrics Mapping Concepts, NISTIR 7061, National Institute of Standards and Technology, Gaithersburg, MD, 2003.Google Scholar
Index Terms
- A framework for real-world software system evaluations
Recommendations
Evaluation theory and practice as applied to security education
Security education and critical infrastructuresThis paper will overview general evaluation purposes, elements, and steps for designing an evaluation in order to provide foundational information that can be used to conduct an evaluation of any security awareness, training, or education programs. An ...
A framework for evaluating a software bidding model
This paper discusses the issues involved in evaluating a software bidding model. We found it difficult to assess the appropriateness of any model evaluation activities without a baseline or standard against which to assess them. This paper describes our ...
Towards a General Framework for Evaluating Software Development Methodologies
COMPSAC '10: Proceedings of the 2010 IEEE 34th Annual Computer Software and Applications ConferenceIt has become essential to scrutinize and evaluate software development methodologies, mainly because of their increasing number and variety. Evaluation is required to gain a better understanding of the features, strengths, and weaknesses of the ...
Comments