Decision rule mining using classification consistency rate

Abstract

Decision rule mining is an important technique in many applications. In this paper, we propose a new rough set approach for rule induction based on a significance measure, called classification consistency rate. The approach implements the rule induction from the viewpoint of attribute rather than descriptor. The proposed algorithm is tested and compared with LEM2 algorithm on several real-life data sets added with different levels of inconsistent data. The results show that the proposed algorithm is effective in rule induction for inconsistent data.

Introduction

Rule induction is one of the most important techniques of machine learning, expert system, knowledge discovery and data mining. To handle this issue, many inductive learning methods, such as induction of decision trees [1], [2], rule induction methods [3], [4], [5], [6], [7] and rough set theory [8], [9], [10], [11], [12] are introduced and applied to extract knowledge from databases.

Rough set theory, introduced by Pawlak, is a useful mathematic approach for dealing with vague and uncertain information. It has attracted the attention of many researchers who have studied its theories and its applications during the last decades [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33].

A number of approaches of rule induction based on rough set theory have been proposed. These approaches can be broadly divided into two categories. One is exhaustive category. Skowron [34], [35] proposed famous rule induction approaches based on discernibility matrix and boolean reasoning. For particular clinic application, Tsumoto [27], [28] introduced a rule induction approach called PRIMEROSE, which extracts not only classification rules but also other medical knowledge needed for diagnosis. The induction algorithm consists of two procedures, one is an exhaustive search procedure to induce the exclusive rule through all the attribute–value pairs, and the other is a postprocessing procedure to induce inclusive rules through the combination of all the attribute–value pairs. The other category is heuristic. Grzymala-Busse proposed famous LEM2 algorithm in LERS [9], [36], [37], [38], [39] which is a representative approach for rule induction by rough set theory. LEM2 explores the search space of the whole attribute–value pairs (called descriptors), and a pair who covers the most number of objects is selected to induce decision rule each time. In other words, it implements the rule induction procedure from the viewpoint of descriptors.

However, in some cases, rule induction from the viewpoint of attributes is more reasonable. For example, users may be interested in the rules about some appointed attribute or attribute sets. Based on this observation, we suggest the viewpoint of attribute space rather than viewpoint of descriptor space for the issue of rule induction in this paper. This viewpoint supplies us an approach to study the data at the understandable semantic level of attributes. From the viewpoint of attribute space, we propose a new strategy for decision rules induction from decision systems based on the concept of classification consistency rate which is defined in this paper to search the attribute space. The proposed rule mining strategy is denoted as DRICA (Decision Rule mIning using Consistency rAte).

The remainder of the paper is organized as follows. Some preliminaries about rough set theory are reviewed in Section 2. In Section 3, the algorithm of rule induction based on classification consistency is introduced. An example is used to illustrate the process of generating rules in Section 4. Experiments on several data sets are conducted to test the proposed approach, and the results are compared in Section 5. Section 6 concludes the paper.