Adapting the CBA algorithm by means of intensity of implication

Abstract

In recent years, extensive research has been carried out by using association rules to build more accurate classifiers. The idea behind these integrated approaches is to focus on a limited subset of association rules. This paper aims to contribute to this integrated framework by adapting the Classification Based on Associations (CBA) algorithm. CBA was adapted by coupling it with another measurement of the quality of association rules: i.e. intensity of implication. The new algorithm has been implemented and empirically tested on an authentic financial dataset for purposes of bankruptcy prediction. We validated our results with an association ruleset, with C4.5, with original CBA and with CART by statistically comparing its performance via the area under the ROC-curve. The adapted CBA algorithm presented in this paper proved to generate significantly better results than the other classifiers at the 5% level of significance.

Introduction

Classification and association-rule discovery are two of the most important tasks addressed in the data mining literature. Association rules have received significant attention for extracting knowledge from large databases. Their study is focused on using exhaustive search to find all rules in data that satisfy the user-specified minimum support and minimum confidence criteria. The Apriori algorithm is the best known algorithm in this field [2]. Probably, an even more popular technique is classification rule mining. It aims to discover a small set of rules to form an accurate classifier. Given a set of cases with class labels as a training set, the aim of classification is to build a model (called classifier) to predict future data objects for which the class label is unknown. Many systems have been proposed for classification rule mining but Quinlan’s C4.5 classifier [17] is known as the state-of-the-art method.

In recent years, extensive research has been carried out to integrate both approaches. By focusing on a limited subset of association rules, i.e. those rules where the consequent of the rule is restricted to the classification class attribute, it is possible to build more accurate classifiers. Several publications [5], [7], [12], [14], [20] have shown that association-based classification in general generates at least equal accuracy than state-of-the-art classification algorithms such as C4.5. The reasons for the good performance are obvious. Association rules will search globally for all rules that satisfy minimum support and minimum confidence norms. They will therefore contain the full set of rules, which may incorporate important information. The richness of the rules gives this technique the potential of reflecting the true classification structure in the data [20]. Associative classification is therefore gaining increasing popularity. However, the comprehensiveness and complexity of dealing with the often large number of association rules have lead to difficulties and (accuracy versus generality) trade-off questions which are part of a lot of research which is currently going on. Contributions to tackle a number of these difficulties can be found in [7], in [13] and in [20]. Liu et al., proposed an improvement of their original Classification Based on Associations (CBA)-system [14] in [15] to cope with the weaknesses in the system. In spite of the fact that the presented adaptations of CBA are valuable, some important issues still remain unsolved. Our goal is to address them in this paper.

The potential weakness which we were able to determine is situated in the way CBA sorts its (class) association rules. As will be explained in Section 2, the sorting in CBA is quite important because the rules for the final classifier will be selected by following the sorted sequence. CBA sorts its rules by using the conditional probability (confidence). This is a good measure when classes are equally distributed. However, as we will show, when class distributions differ significantly, and especially for classes whose frequency is low, this is not the most adequate approach to follow. For this reason, we propose intensity of implication [8] as a better measure to sort the class association rules. Apart from this, the CBA algorithm which we have implemented also traces the evolution of the number of false positives (FP) and false negatives (FN) and not only of the total number of errors, as the original CBA algorithm does. Section 3 elaborates on both issues. The results of our empirical evaluation are given in Section 4. Finally, conclusions and recommendations for further research are presented in Section 5.