Regularization in skewed binary classification

Summary

Skewed binary classification concerns the assignment of a new unknown object to one of two populations, 0 or 1, on the basis of a q-dimensional vector x = (x₁, …x_q), where one of the populations, for example population 0, is the prevalent class. Assignment rules are developed from learning samples of known objects, that is, objects known to come from each of the two populations. Since population 1 is the rare class, overfitting and generalization problems arise easily for many classification models. We propose an effective solution by assigning more weights to class 1. The idea is to produce noisy replicates of the rare cases while keeping the dominant class 0 cases unchanged. The classification models considered are: nearest neighbor method, neural networks, classification trees, and quadratic discriminant. Noisy replication of the rare cases was applied to three real world and simulated data sets. Encouraging results were obtained for all the classification models considered.