Handling Small Disjuncts and Class Skew Using Sequential Ellipsoidal Partitioning

Abstract

Data irregularities, such as small disjuncts, class skew and imbalance, and outliers significantly affect the performance of classifiers. In this paper, we focus on identifying small disjuncts, which hitherto, has been addressed mainly by rule-based or inductive algorithms. Small disjuncts have been identified as distribution-based irregularities which provide significant learning, although they cover a subset of examples in the training set, which may be considered as being rare. Such samples are more error-prone than large disjuncts. Eliminating small disjuncts by removal or pruning is seen to affect the learning of the classifier adversely. Widely used non-rule-based learning algorithms like SVM, kNN, Logistic Regression, and Neural networks perform poorly in the presence of small disjuncts in the dataset. In this paper, a novel Sequential Ellipsoidal Partitioning method is proposed to identify small disjuncts in the dataset. This method is a supervised classifier that iteratively partitions the dataset into Minimum Volume Ellipsoids that contain points of the same label; this is performed based on the idea of Reduced Convex Hulls. By allowing an ellipsoid that contains points of one label to contain a few points of the other, such small disjuncts may be identified. As we discuss, the proposed technique is agnostic of underlying data distributions and is applicable as a supervised classifier when the datasets are highly skewed and imbalanced even. We demonstrate the performance of the approach using a few publicly available datasets.