Finding fuzzy classification rules using data mining techniques
Introduction
Data mining is the exploration and analysis of data in order to discover meaningful patterns (Berry and Linoff, 1997). In addition, data mining problems involving classification can be viewed within a common framework of rule discovery (Agrawal et al., 1993). The goal of this paper is just to propose an effective method that can find a compact set of fuzzy rules for classification problems by data mining techniques.
Recently, the discovery of association rules from databases has become an important research topic, and association rules have been applied for analysis to help managers determine which items are frequently purchased together by customers (Berry and Linoff, 1997; Han and Kamber, 2001). The Apriori algorithm proposed by Agrawal et al. (1996) is an influential algorithm that can be used to find association rules. In this algorithm, a candidate k-itemset (k⩾1) containing k items is frequent (i.e., frequent k-itemset) if its support is larger than or equal to a user-specified minimum support. Significantly, the well-known Apriori property (Han and Kamber, 2001) for mining association rules shows that any subset of a frequent itemset must also be frequent. Subsequently, we use frequent itemsets to generate association rules.
A fuzzy classification rule is a fuzzy if–then rule whose consequent part is a class label. Since the comprehensibility of fuzzy rules by human users is a criterion in designing a fuzzy rule-based system (Ishibuchi et al., 1999), fuzzy classification rules with linguistic interpretations must be taken into account. To cope with this problem, we consider both quantitative and categorical attributes, which are used to describe each sample data, as linguistic variables. Then, each linguistic variable can be partitioned by its linguistic values represented by fuzzy numbers with triangular membership functions. Simple fuzzy grids or grid partitions (Ishibuchi et al., 1999; Jang and Sun, 1995) in feature space resulting from the fuzzy partition are thus obtained.
In this paper, we propose a two-phase data mining technique to discover fuzzy rules for classification problems based on the Apriori algorithm. The first phase finds frequent fuzzy grids by dividing each quantitative attribute with a pre-specified number of various linguistic values. The second phase generates effective fuzzy classification rules from those frequent fuzzy grids. The fuzzy support and the fuzzy confidence, which have been defined previously (e.g., Ishibuchi et al., 2001a; Ishibuchi et al., 2001b; Hu et al., 2002), are employed to determine which fuzzy grids are frequent and which rules are effective by comparison with the minimum fuzzy support (min FS) and the minimum fuzzy confidence (min FC), respectively.
However, both min FS and min FC are not easily user-specified for each classification problem. To solve this problem, the genetic algorithm (GA) (Goldberg, 1989) is thus incorporated into the proposed algorithm to automatically determine those two parameters. A binary chromosome with sufficiently large length used in this paper is composed of two substrings: one for the min FS, and the other for the min FC. Each generation of the GA can obtain the fitness value of each chromosome, which maximizes the classification accuracy rate and minimizes the number of fuzzy rules. When reaching the termination condition, a chromosome with the maximum fitness value is used to test the performance of the proposed method.
For classification generalization ability, the simulation results from the iris data and the appendicitis data demonstrate that proposed learning algorithm performs well in comparison with other fuzzy or non-fuzzy classification methods. Thus, the goal of acquiring an effectively compact set of fuzzy rules for classification problems can be achieved.
This paper is organized as follows. Notations used in this paper are described in Section 2. The fuzzy partition methods are detailed introduced in Section 3. In Section 4, the proposed learning algorithm incorporated with the GA is presented. In Section 5, the performance of the proposed method is examined by computer simulation on Anderson’s iris data (Anderson, 1935) and the appendicitis data. Discussions and conclusions are presented in Section 6 and Section 7, respectively.
Section snippets
Notations
Notations used in this paper are as follows:
- C
number of class labels
- d
number of attributes used to describe each sample data, where 1⩽d
- k
dimension of one fuzzy grid, where 1⩽k⩽d
- K
number of various linguistic values defined in each quantitative attribute, where K⩾2
- AK,ikxk
ikth linguistic value of K linguistic values in the linguistic variable xk, where 1⩽ik⩽K and 1⩽m⩽d
- μK,ikxk
membership function of AK,ikxk
- tp
pth training sample, where tp=(tp1,t2,…,tpd), and tpi is the value with respect to the ith
Fuzzy partition methods
The concepts of linguistic variables were proposed by Zadeh, 1975a, Zadeh, 1975b, Zadeh, 1976 and it is reasonable that we view each attribute as a linguistic variable. Formally, a linguistic variable is characterized by a quintuple (Pedrycz and Gomide, 1998; Zimmermann, 1991) denoted by (x,T(x),U,G,M), in which x is the name of the variable; T(x) denotes the set of names of linguistic values or terms, which are linguistic words or sentences in a natural language (Chen and Jong, 1997), of x; U
Finding fuzzy classification rules
As we have mentioned above, the generation of frequent fuzzy grids and fuzzy classification rules are two significant phases of the proposed learning algorithm. In this section, we thus describe the individual phase of the proposed method in 4.1 Determining frequent fuzzy grids, 4.2 Determining effective fuzzy rules. The proposed learning algorithm incorporated with the GA is presented in detail in Section 4.3.
Experiments
To examine the performance of the proposed learning algorithm for testing samples, we perform the leave-one-out technique, which is an almost unbiased estimator of the true error rate of a classifier (Weiss and Kulikowski, 1991). Based on the leave-one-out technique, we try to make a comparison between the proposed learning algorithm and other fuzzy or non-fuzzy classification methods.
In 4.1 Determining frequent fuzzy grids, 4.2 Determining effective fuzzy rules, we employ the proposed learning
Discussions
The performance of the proposed method is examined by the iris data and appendicitis data. From Table 2, we may conclude that K is not an influential factor when it is larger than 2.
We also find that the fuzzy classification methods proposed by Ishibuchi et al., 2001a, Ishibuchi et al., 2001b employed each (d−1)-fuzzy grid, which does not contain the dimension of the class label, as an antecedence of one fuzzy rule, whose consequence can be determined by computing the fuzzy confidence for each
Conclusions
In this paper, we propose a learning algorithm that can find fuzzy rules for classification problems based on the processing of the Apriori algorithm. Significantly, our method tries to find a compact set of fuzzy rules by using the GA to automatically find the appropriately min FS and min FC.
Simulation results on the iris data and the appendicitis data demonstrate that the classification accuracy rates of the proposed method are comparable to the other fuzzy or non-fuzzy methods. Thus, the
References (27)
- et al.
Mining fuzzy association rules for classification problems
Computers and Industrial Engineering
(2002) - et al.
Distributed representation of fuzzy rules and its application to pattern classification
Fuzzy Sets and Systems
(1992) The concept of a linguistic variable and its application to approximate reasoning
Information Science (part 1)
(1975)The concept of a linguistic variable and its application to approximate reasoning
Information Science
(1975)- et al.
Database mining: a performance perspective
IEEE Transactions on Knowledge and Data Engineering
(1993) - et al.
Fast discovery of association rules
The irises of the gaspe peninsula
Bull. Amer. Iris Soc.
(1935)- et al.
Data Mining Techniques: for Marketing, Sales, and Customer Support
(1997) - et al.
Fuzzy query translation for relational database systems
IEEE Transactions on Systems, Man, and Cybernetics
(1997) Genetic Algorithms in Search, Optimization, and Machine Learning
(1989)
Data Mining: Concepts and Techniques
Trade-off between computation time and number of rules for fuzzy mining from quantitative data
International Journal of Uncertainty, Fuzziness and Knowledge Based Systems
Cited by (116)
A hybrid approach based on mathematical modelling and improved online learning algorithm for data classification
2023, Expert Systems with ApplicationsSemi-overlap functions and novel fuzzy reasoning algorithms with applications
2022, Information SciencesCitation Excerpt :Fuzzy Rule-Based Classification Systems (FRBCSs) [19] are one of the most popular methods in pattern recognition and machine learning, which is also used in practical applications, for example image processing [27] and anomaly intrusion detection [32]. To improve the interpretability of classification rules and avoid unnatural boundaries in attribute division, researchers have made different studies on classification systems based on fuzzy association rules [7,16–18,28,37]. In particular, Alcalá-Fdez [1] propose a fuzzy association rule-based classification method for high-dimensional problems (FARC–HD) to obtain an accurate and compact fuzzy rule-based classifier with a low computational cost.
ARC-SL: Association rule-based classification with soft labels
2021, Knowledge-Based SystemsEARC: Evidential association rule-based classification
2021, Information SciencesComplexity-based parallel rule induction for multiclass classification
2017, Information SciencesSteganalysis based on steganography pattern discovery
2016, Journal of Information Security and ApplicationsCitation Excerpt :The process of generating the signature of stego images is done by an Evolutionary Algorithm (EA). EAs have been used as rule generation and optimization tools in the design of fuzzy rule-based systems (Cordon et al., 2004; Hu et al., 2003). To obtain accurate fuzzy rules, we employ an evolutionary rule generation algorithm based on Iterative Rule Learning (IRL) approach (Sajedi and Jamzad, 2009b, 2010e).