Abstract
Decision support system developers and users agree on the need for rigorous evaluations of performance and impact. Evaluating simple reminder systems is relatively easy because there is usually a gold standard of decision quality. However, when a system generates complex output (such as a critique or graphical report), it is much less obvious how to evaluate it. We discuss some generic problems and how one might resolve them, using as a case study Design- a-Trial, a DSS to help clinicians write a lengthy trial protocol.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Wyatt, J., Altman, D., Heathfield, H., Pantin, C.: Development of Design-a-Trial, a Knowledge-based Critiquing System for Authors of Clinical Trial Protocols. Comp. Meth. Prog. Biomed. 43 (1994) 283–291
Modgil, S., Hammond, P., Wyatt, J.C., Potts, H.: The Design-a-Trial Project: Developing a Knowledge-based Tool for Authoring Clinical Trial Protocols. In: 1st European Workshop on Computer-based Support for Clinical Guidelines and Protocols (EWGLP-2000). IOS Press, Amsterdam (in press)
Fox, J., Johns, N., Rahmanzadeh, A., Thompson, R.: ProForma: A General Technology for Clinical Decision Support Systems. Comp. Meth. Prog. Biomed. 54 (1997) 59–67
Friedman, C., Wyatt, J.: Evaluation Methods in Medical Informatics. Springer Verlag, New York (1997)
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., Walsh, S.: Assessing the Quality of Randomized Controlled Trials. Control. Clin. Trials 16 (1995) 62–73
Bose, D. K., Heathfield, H. A., Andrew, M.: Collective Decision Problems in Medicine — A Basic Approach Looking for Cross-Fertilization in Clinical Surgery. Theoretical Surg. 7 (1992) 186–193
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Potts, H.W.W., Wyatt, J.C., Altman, D.G. (2001). Challenges in Evaluating Complex Decision Support Systems: Lessons from Design-a-Trial. In: Quaglini, S., Barahona, P., Andreassen, S. (eds) Artificial Intelligence in Medicine. AIME 2001. Lecture Notes in Computer Science(), vol 2101. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48229-6_61
Download citation
DOI: https://doi.org/10.1007/3-540-48229-6_61
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42294-5
Online ISBN: 978-3-540-48229-1
eBook Packages: Springer Book Archive