Model Evaluation as Approach to Predict a Diagnosis

Abstract

The paper presents an approach to a relevant issue of the supervised learning: classification. Creating the models that are able to generalize to new data, tuning the model so that the performance can be increased and models evaluation are relevant task of this sub-field of machine learning. In this paper, we have chosen to treat the evaluation of a model. The paper consists in two experiments, each of them with a particular purpose. First experiment outlines different model evaluation methods using specific performance metrics. This paper analyzes two models, logistic regression and the k-nearest neighbor with three methods of evaluation: train-test approach, test-set approach and the k-cross validation approach. On our analyzed data set, we achieved reasonable results using logistic regression as model with k-cross validation approach as evaluating method of the model. The second experiment determines the rank of the observations by predicting the probabilities depending on the response vector. Based on the predicted probability we try to improve the metric performance. For our particular task, one metric that allows us to extract relevant information from data and can be improved is the sensitivity metric.