Abstract
Receiver Operating Characteristics (ROC) Analysis originated from signal detection theory, as a model of how well a receiver is able to detect a signal in the presence of noise [1,9]. Its key feature is the distinction between hit rate (or true positive rate) and false alarm rate (or false positive rate) as two separate performance measures. ROC analysis has also widely been used in medical data analysis to study the effect of varying the threshold on the numerical outcome of a diagnostic test. It has been introduced to machine learning relatively recently, in response to classification tasks with skewed class distributions or misclassification costs [11,12,5].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Egan, J.P.: Signal detection theory and ROC analysis. Academic Press, New York (1975)
Ferri, C., Flach, P.A., Hernández-Orallo, J.: Learning decision trees using the area under the ROC curve. In: Proceedings of the Nineteenth International Conference on Machine Learning (ICML 2002), pp. 139–146 (2002)
Ferri, C., Flach, P.A., Hernández-Orallo, J., Senad, A.: Modifying ROC curves to incorporate predicted probabilities. In: Proceedings of the second workshop on ROC analysis in machine learning, ICML 2005 workshop proceedings (2005)
Flach, P.A.: The geometry of ROC space: Understanding machine learning metrics through ROC isometrics. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML 2003), pp. 194–201 (2003)
Flach, P.A.: The many faces of ROC analysis in machine learning, ICML 2004 tutorial notes (2004), http://www.cs.bris.ac.uk/~flach/ICML04tutorial/
Flach, P.A., Wu, S.: Repairing concavities in ROC curves. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 702–707 (2005)
Fürnkranz, J., Flach, P.A.: An analysis of rule evaluation metrics. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML 2003), pp. 202–209 (2003)
Fürnkranz, J., Flach, P.A.: ROC ’n’ rule learning-towards a better understanding of covering algorithms. Machine Learning 58(1), 39–77 (2005)
Green, D.M., Swets, J.A.: Signal detection theory and psychophysics. Wiley, New York (1966)
Prati, R.C., Flach, P.A.: ROCCER: An algorithm for rule learning based on ROC analysis. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 823–828 (2005)
Provost, F.J., Fawcett, T.: Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In: Proceedings of the Third International Conference on Knowledge Discovery and Data Mining (KDD 1997), pp. 43–48 (1997)
Provost, F.J., Fawcett, T.: Robust classification for imprecise environments. Machine Learning 42(3), 203–231 (2001)
Wu, S., Flach, P.A.: A scored AUC metric for model evaluation and selection. In: Proceedings of the second workshop on ROC analysis in machine learning. In: ICML 2005 workshop proceedings (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Flach, P.A. (2006). Reinventing Machine Learning with ROC Analysis. In: Sichman, J.S., Coelho, H., Rezende, S.O. (eds) Advances in Artificial Intelligence - IBERAMIA-SBIA 2006. IBERAMIA SBIA 2006 2006. Lecture Notes in Computer Science(), vol 4140. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11874850_3
Download citation
DOI: https://doi.org/10.1007/11874850_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45462-5
Online ISBN: 978-3-540-45464-9
eBook Packages: Computer ScienceComputer Science (R0)