Abstract
In classifier ensembles predictions of different classifiers regarding a query are combined into one final decision. It was previously shown that using ensemble techniques can significantly improve classification performance. In this study we build upon this result and propose to use variability in the predictions of classifiers contributing to the final decision as an indicator of its reliability. The study hypothesis is tested with respect to previously proposed information-theoretic computer-aided decision (IT-CAD) system for detection of masses in mammograms. A database of 1820 regions of interest (ROIs) extracted from digital database of screening mammography (DDSM) is used. Experimental results show that the proposed reliability assessment successfully identifies decisions that can not be trusted. Further, a low correlation between reliability and the classifier output is noted. This opens a possibility of combining reliability and ensemble output into one improved decision.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kittler, J., Hatef, M., Duin, R.P., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 226–239 (1998)
Kuncheva, L.I.: Combining Pattern Classifiers. Willey-Interscience (2004)
Ranawana, R., Palade, V.: Multi-classifier systems: Review and a roadmap for developers. International Journal of Hybrid Intelligent Systems 3, 35–61 (2006)
Bloch, I.: Some aspects of dempster-shafer evidence theory for classification of multi-modality medical images taking partial volume effect into account. Pattern Recognition Letters 17, 905–919 (1996)
Kuncheva, L.I.: Switching between selection and fusion in combining classifiers: An experiment. IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics 32, 146–156 (2002)
Zhou, Z.-H., Jiang, Y.: Medical diagnosis with c4.5 rule preceded by artificial neural network ensemble. IEEE Transaction on Information Technology in Biomedicine 7, 37–42 (2003)
Greene, D., Tsymbal, A., Bolshakova, N., Cunningham, P.: Ensemble clustering in medical diagnostics. In: Proceedings of 17th IEEE Symposium on Computer-Based Medical Systems (CBMS 2004), pp. 575–581 (2004)
West, D., Mangiameli, P., Rampal, R., West, V.: Ensemble strategies for a medical diagnostic decision support system: A breast cancer diagnosis application. European Journal of Operational Research 162, 532–551 (2005)
Raza, M., Gondal, I., David Green, R.L.C.: Classifier fusion using Dempster-Shafer theory of evidence to predict breast cancer tumors. In: 2006 IEEE Region 10 Conference (TENCON 2006), pp. 1–4 (2006)
Mazurowski, M.A., Zurada, J.M., Tourassi, G.D.: Database decomposition of a knowledge base cad system in mammography; an ensemble approach to improve detection performance. In: Proc. SPIE Medical Imaging 2008 (in press) (2008)
Habas, P.A., Zurada, J.M., Elmaghraby, A.S., Tourassi, G.D.: Reliability analysis framework for computer-assisted medical decision systems. Med. Phys. 34, 763–772 (2007)
Habas, P.A., Zurada, J.M., Elmaghraby, A.S., Tourassi, G.D.: Probabilistic framework for reliability analysis of information-theoretic cad systems in mammography. In: Proceedings of the 28th IEEE EMBS Annual International Conference, New York City, USA, August 30-September 3, 2006, pp. 6113–6116 (2006)
Tourassi, G.D., Vargas-Voracek, R., Catarious Jr., D.M., Floyd Jr., C.E.: Computer-assisted detection of mammographic masses: A template matching scheme based on mutual information. Medical Physics 30, 2123–2130 (2003)
Tourassi, G.D., Haarawood, B., Singh, S., Lo, J.Y., Floyd, C.E.: Evaluation of information-theoretic similarity measures for content-based retrieval and detection of masses in mammograms. Medical Physics 34, 140–150 (2007)
Jiang, Y.: Uncertainty in the output of artificial neural networks. IEEE Trans. Med. Imaging 22, 913–921 (2003)
Heath, M., et al.: Current status of the digital database for screening mammography, ch. In: Digital Mammography. Kluwer Academic, Dordrecht (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mazurowski, M.A., Zurada, J.M., Tourassi, G.D. (2008). Reliability Assessment of Ensemble Classifiers: Application in Mammography. In: Krupinski, E.A. (eds) Digital Mammography. IWDM 2008. Lecture Notes in Computer Science, vol 5116. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70538-3_51
Download citation
DOI: https://doi.org/10.1007/978-3-540-70538-3_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70537-6
Online ISBN: 978-3-540-70538-3
eBook Packages: Computer ScienceComputer Science (R0)