Performance Measures for Binary Classification

doi:10.1016/B978-0-12-809633-8.20351-8

Reference Module in Life Sciences

Encyclopedia of Bioinformatics and Computational Biology

Volume 1, 2019, Pages 546-560

https://doi.org/10.1016/B978-0-12-809633-8.20351-8 Get rights and content

Abstract

This article is an introduction to some of the most commonly used performance measures for the evaluation of binary classifiers. These measures are categorized into three broad families: measures based on a single classification threshold, measures based on a probabilistic interpretation of error, and ranking measures. Graphical methods, such as ROC curves, precision-recall curves, TPR-FPR plots, gain charts, and lift charts, are also discussed. Using a simple example, we illustrate how to calculate the various performance measures and show how they are related. The article also explains how to assess the statistical significance of an obtained performance value, how to calculate approximate and exact parametric confidence intervals, and how to derive percentile bootstrap confidence intervals for a performance measure.

References (0)

Cited by (49)

Quantifying and comparing the effects of key chemical descriptors on metal–organic frameworks water stability with CatBoost and SHAP
2024, Microchemical Journal
Metal-organic frameworks (MOFs) were considered suitable candidates for a range of industrial applications, including adsorption, separation, sensing and catalysis, due to their advantages of diverse structures and adjustable functions. One of the criteria for determining the commercial viability of MOFs is their stability in water vapor. Here, we established a novel Categorical Boosting (CatBoost) machine learning approach to model more than 200 datasets of empirical measurements of MOF water stability, and used a comprehensive set of chemical descriptors to represent MOF composition including metal ions, organic ligands, and metal–ligand molar ratios. CatBoost algorithm was significantly superior to other gradient algorithms in accuracy, precision and F1-Score. Also, the CatBoost output was interpreted using the Shapley additive interpretation (SHAP) method. Besides providing guidelines for future experimental screening of stable candidates for MOFs, the interpretable Catboost model can also be used for MOFs screening of other design criteria.
Assessment of the relationship between the postpartum diseases susceptibility and the bovine monocyte subsets via Bayesian logistic regression, under various prior distributions
2022, Research in Veterinary Science
Postpartum diseases (PD) in dairy cows cause serious concerns about economic losses worldwide. This study intended to investigate the relationship between PD susceptibility and counts of monocyte subgroup cells (MCC), in the blood samples taken from 27 German Holstein cows 42 and 14 days before the expected calving by adopting the Bayesian approach. The paper also aimed to discuss the prior selection problem in the Bayesian approach and to reveal the parameter estimation difference based on the available data. The parameters were estimated according to the models established at two different time points with eight different prior distributions. As a result of the study, all the models revealed strong evidence that cows with PD, compared to healthy cows, had a higher increase in MCC counts on Day 14. There was no difference between the models according to their WAIC and LOO values. In terms of the parameter estimates, the models produced identical results; however, the models with noninformative priors presented strong evidence for the absence of effects by Bayes factor but, provided evidence for the existence of the effect according to the credible interval. The models with weakly informative and shrinkage priors provided strong evidence for the presence of the effect. The findings suggest that MCC can be considered to serve as a prospective indicator for early detection of PD.
A Survey of Machine Learning and Deep Learning Methods for Estimating Automatic Engagement and Attention in Offline Classroom Environment
2024, Lecture Notes in Networks and Systems
Computational Approaches for the Automatic Quantification of Cells from Brain Images
2024, Lecture Notes in Networks and Systems
Sepsis mimics among presumed sepsis patients at intensive care admission: a retrospective observational study
2024, Infection
ArrayNet: A Combined Seismic Phase Classification and Back-Azimuth Regression Neural Network for Array Processing Pipelines
2023, Bulletin of the Seismological Society of America

View all citing articles on Scopus

View full text