Finding fuzzy classification rules using data mining techniques

https://doi.org/10.1016/S0167-8655(02)00273-8Get rights and content

Abstract

Data mining techniques can be used to discover useful patterns by exploring and analyzing data, so, it is feasible to incorporate data mining techniques into the classification process to discover useful patterns or classification rules from training samples. This paper thus proposes a data mining technique to discover fuzzy classification rules based on the well-known Apriori algorithm. Significantly, since it is difficult for users to specify the minimum fuzzy support used to determine the frequent fuzzy grids or the minimum fuzzy confidence used to determine the effective classification rules derived from frequent fuzzy grids, therefore the genetic algorithms are incorporated into the proposed method to determine those two thresholds with binary chromosomes. For classification generalization ability, the simulation results from the iris data and the appendicitis data demonstrate that the proposed method performs well in comparison with other classification methods.

Introduction

Data mining is the exploration and analysis of data in order to discover meaningful patterns (Berry and Linoff, 1997). In addition, data mining problems involving classification can be viewed within a common framework of rule discovery (Agrawal et al., 1993). The goal of this paper is just to propose an effective method that can find a compact set of fuzzy rules for classification problems by data mining techniques.

Recently, the discovery of association rules from databases has become an important research topic, and association rules have been applied for analysis to help managers determine which items are frequently purchased together by customers (Berry and Linoff, 1997; Han and Kamber, 2001). The Apriori algorithm proposed by Agrawal et al. (1996) is an influential algorithm that can be used to find association rules. In this algorithm, a candidate k-itemset (k⩾1) containing k items is frequent (i.e., frequent k-itemset) if its support is larger than or equal to a user-specified minimum support. Significantly, the well-known Apriori property (Han and Kamber, 2001) for mining association rules shows that any subset of a frequent itemset must also be frequent. Subsequently, we use frequent itemsets to generate association rules.

A fuzzy classification rule is a fuzzy if–then rule whose consequent part is a class label. Since the comprehensibility of fuzzy rules by human users is a criterion in designing a fuzzy rule-based system (Ishibuchi et al., 1999), fuzzy classification rules with linguistic interpretations must be taken into account. To cope with this problem, we consider both quantitative and categorical attributes, which are used to describe each sample data, as linguistic variables. Then, each linguistic variable can be partitioned by its linguistic values represented by fuzzy numbers with triangular membership functions. Simple fuzzy grids or grid partitions (Ishibuchi et al., 1999; Jang and Sun, 1995) in feature space resulting from the fuzzy partition are thus obtained.

In this paper, we propose a two-phase data mining technique to discover fuzzy rules for classification problems based on the Apriori algorithm. The first phase finds frequent fuzzy grids by dividing each quantitative attribute with a pre-specified number of various linguistic values. The second phase generates effective fuzzy classification rules from those frequent fuzzy grids. The fuzzy support and the fuzzy confidence, which have been defined previously (e.g., Ishibuchi et al., 2001a; Ishibuchi et al., 2001b; Hu et al., 2002), are employed to determine which fuzzy grids are frequent and which rules are effective by comparison with the minimum fuzzy support (min FS) and the minimum fuzzy confidence (min FC), respectively.

However, both min FS and min FC are not easily user-specified for each classification problem. To solve this problem, the genetic algorithm (GA) (Goldberg, 1989) is thus incorporated into the proposed algorithm to automatically determine those two parameters. A binary chromosome with sufficiently large length used in this paper is composed of two substrings: one for the min FS, and the other for the min FC. Each generation of the GA can obtain the fitness value of each chromosome, which maximizes the classification accuracy rate and minimizes the number of fuzzy rules. When reaching the termination condition, a chromosome with the maximum fitness value is used to test the performance of the proposed method.

For classification generalization ability, the simulation results from the iris data and the appendicitis data demonstrate that proposed learning algorithm performs well in comparison with other fuzzy or non-fuzzy classification methods. Thus, the goal of acquiring an effectively compact set of fuzzy rules for classification problems can be achieved.

This paper is organized as follows. Notations used in this paper are described in Section 2. The fuzzy partition methods are detailed introduced in Section 3. In Section 4, the proposed learning algorithm incorporated with the GA is presented. In Section 5, the performance of the proposed method is examined by computer simulation on Anderson’s iris data (Anderson, 1935) and the appendicitis data. Discussions and conclusions are presented in Section 6 and Section 7, respectively.

Section snippets

Notations

Notations used in this paper are as follows:

    C

    number of class labels

    d

    number of attributes used to describe each sample data, where 1⩽d

    k

    dimension of one fuzzy grid, where 1⩽kd

    K

    number of various linguistic values defined in each quantitative attribute, where K⩾2

    AK,ikxk

    ikth linguistic value of K linguistic values in the linguistic variable xk, where 1⩽ikK and 1⩽md

    μK,ikxk

    membership function of AK,ikxk

    tp

    pth training sample, where tp=(tp1,t2,…,tpd), and tpi is the value with respect to the ith

Fuzzy partition methods

The concepts of linguistic variables were proposed by Zadeh, 1975a, Zadeh, 1975b, Zadeh, 1976 and it is reasonable that we view each attribute as a linguistic variable. Formally, a linguistic variable is characterized by a quintuple (Pedrycz and Gomide, 1998; Zimmermann, 1991) denoted by (x,T(x),U,G,M), in which x is the name of the variable; T(x) denotes the set of names of linguistic values or terms, which are linguistic words or sentences in a natural language (Chen and Jong, 1997), of x; U

Finding fuzzy classification rules

As we have mentioned above, the generation of frequent fuzzy grids and fuzzy classification rules are two significant phases of the proposed learning algorithm. In this section, we thus describe the individual phase of the proposed method in 4.1 Determining frequent fuzzy grids, 4.2 Determining effective fuzzy rules. The proposed learning algorithm incorporated with the GA is presented in detail in Section 4.3.

Experiments

To examine the performance of the proposed learning algorithm for testing samples, we perform the leave-one-out technique, which is an almost unbiased estimator of the true error rate of a classifier (Weiss and Kulikowski, 1991). Based on the leave-one-out technique, we try to make a comparison between the proposed learning algorithm and other fuzzy or non-fuzzy classification methods.

In 4.1 Determining frequent fuzzy grids, 4.2 Determining effective fuzzy rules, we employ the proposed learning

Discussions

The performance of the proposed method is examined by the iris data and appendicitis data. From Table 2, we may conclude that K is not an influential factor when it is larger than 2.

We also find that the fuzzy classification methods proposed by Ishibuchi et al., 2001a, Ishibuchi et al., 2001b employed each (d−1)-fuzzy grid, which does not contain the dimension of the class label, as an antecedence of one fuzzy rule, whose consequence can be determined by computing the fuzzy confidence for each

Conclusions

In this paper, we propose a learning algorithm that can find fuzzy rules for classification problems based on the processing of the Apriori algorithm. Significantly, our method tries to find a compact set of fuzzy rules by using the GA to automatically find the appropriately min FS and min FC.

Simulation results on the iris data and the appendicitis data demonstrate that the classification accuracy rates of the proposed method are comparable to the other fuzzy or non-fuzzy methods. Thus, the

References (27)

  • Grabisch, M., Dispot, F., 1992. A comparison of some methods of fuzzy classification on real data. In: Proceedings of...
  • J.W. Han et al.

    Data Mining: Concepts and Techniques

    (2001)
  • T.P. Hong et al.

    Trade-off between computation time and number of rules for fuzzy mining from quantitative data

    International Journal of Uncertainty, Fuzziness and Knowledge Based Systems

    (2001)
  • Cited by (116)

    • Semi-overlap functions and novel fuzzy reasoning algorithms with applications

      2022, Information Sciences
      Citation Excerpt :

      Fuzzy Rule-Based Classification Systems (FRBCSs) [19] are one of the most popular methods in pattern recognition and machine learning, which is also used in practical applications, for example image processing [27] and anomaly intrusion detection [32]. To improve the interpretability of classification rules and avoid unnatural boundaries in attribute division, researchers have made different studies on classification systems based on fuzzy association rules [7,16–18,28,37]. In particular, Alcalá-Fdez [1] propose a fuzzy association rule-based classification method for high-dimensional problems (FARC–HD) to obtain an accurate and compact fuzzy rule-based classifier with a low computational cost.

    • Steganalysis based on steganography pattern discovery

      2016, Journal of Information Security and Applications
      Citation Excerpt :

      The process of generating the signature of stego images is done by an Evolutionary Algorithm (EA). EAs have been used as rule generation and optimization tools in the design of fuzzy rule-based systems (Cordon et al., 2004; Hu et al., 2003). To obtain accurate fuzzy rules, we employ an evolutionary rule generation algorithm based on Iterative Rule Learning (IRL) approach (Sajedi and Jamzad, 2009b, 2010e).

    View all citing articles on Scopus
    View full text