Definition
ROC analysis investigates and employs the relationship between sensitivity and specificity of a binary classifier. Sensitivity or true positive rate measures the proportion of positives correctly classified; specificity or true negative rate measures the proportion of negatives correctly classified. Conventionally, the true positive rate (tpr) is plotted against the false positive rate (fpr), which is one minus true negative rate. If a classifier outputs a score proportional to its belief that an instance belongs to the positive class, decreasing the decision threshold – above which an instance is deemed to belong to the positive class – will increase both true and false positive rates. Varying the decision threshold from its maximal to its minimal value results in a piecewise linear curve from (0, 0) to (1, 1), such that each segment has a nonnegative slope (Fig. 1). This ROC curve is the main tool used in ROC analysis....
This is a preview of subscription content, log in via an institution.
Recommended Reading
Bourke, C., Deng, K., Scott, S., Schapire, R., & Vinodchandran, N. V. (2008). On reoptimizing multi-class classifiers. Machine Learning, 71(2–3), 219–242.
Brier, G. (1950). Verification of forecasts expressed in terms of probabilities. Monthly Weather Review, 78, 1–3.
Drummond, C., & Holte, R. (2006). Cost curves: An improved method for visualizing classifier performance. Machine Learning, 65(1), 95–130.
Egan, J. (1975). Signal detection theory and ROC analysis. Series in cognitition and perception. New York: Academic Press.
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.
Fawcett, T., & Niculescu-Mizil, A. (2007). PAV and the ROC convex hull. Machine Learning, 68(1), 97–106.
Ferri, C., Hernández-Orallo, J., & Salido, M. (2003). Volume under the ROC surface for multi-class problems. In Proceedings of the fourteenth (ECML 2003) (pp. 108–120). Lecture Notes in Computer Science 2837. Berlin: Springer.
Flach, P. (2003). The geometry of ROC space: Understanding machine learning metrics through ROC isometrics. In Proceedings of the twentieth international conference on machine learning (ICML 2003) (pp. 194–201). Washington, DC: AAAI Press.
Flach, P. (2004). The many faces of ROC analysis in machine learning. ICML-04 Tutorial. http://www.cs.bris.ac.uk/flach/ICML04tutorial/. Accessed on 16 December 2009.
Fuernkranz, J., & Flach, P. (2005). ROC ‘n’ Rule learning – towards a better understanding of covering algorithms. Machine Learning, 58(1), 39–77.
Hand, D., & Till, R. (2001). A simple generalization of the area under the ROC curve to multiple class classification problems. Machine Learning, 45(2), 171–186.
Lachiche, N., & Flach, P. (2003). Improving accuracy and cost of two-class and multi-class probabilistic classifiers using ROC curves. In Proceedings of the twentieth international conference on machine learning (ICML’03) (pp. 416–423). Washington, DC: AAAI Press.
Mossman, D. (1999). Three-way ROCs. Medical Decision Making, 19, 78–89.
Provost, F., & Fawcett, T. (2001). Robust classification for imprecise environments. Machine Learning, 42(3), 203–231.
Srinivasan, A. (1999). Note on the location of optimal classifiers in n-dimensional ROC space. Technical report PRG-TR-2-99. Oxford University Computing Laboratory, Oxford.
Zadrozny, B., & Elkan, C. (2002). Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 694–699). New York: ACM.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this entry
Cite this entry
Flach, P.A. (2011). ROC Analysis. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-30164-8_733
Download citation
DOI: https://doi.org/10.1007/978-0-387-30164-8_733
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-30768-8
Online ISBN: 978-0-387-30164-8
eBook Packages: Computer ScienceReference Module Computer Science and Engineering