Prediction Performance Analysis for ML Models Based on Impacts of Data Imbalance and Bias

An imbalanced dataset is characterized by a substantial disparity in the distribution of examples among its classes, with one class containing significantly more instances than others. Most of the credit fraud datasets are imbalanced. Addressing the challenges posed by imbalanced datasets in classification problems is a complex task, as many classification algorithms struggle to provide satisfactory performance under such conditions. In this article, we have conducted a comparative analysis of various classifiers to assess their performance in handling imbalanced data related to credit card fraud. Then, we employed the Synthetic Minority Oversampling Technique (SMOTE) to synthesize imbalanced data into a relatively balanced dataset. Subsequently, we reevaluated the classification results using different classifiers. Ultimately, our findings revealed that the Naive Bayes classifier was less sensitive to the dataset imbalance, which the AUC score increase rate is 40.19%, the KNN classifier is the most sensitive one, the AUC score increase rate is 61.27%. In all, AdaBoost and Random Forest perform much higher AUC score, which are both higher than 95% after the SMOTE.