Fuzzy DIFACONN-miner: A novel approach for fuzzy rule extraction from neural networks

Abstract

Artificial neural networks (ANNs) are mathematical models inspired from the biological nervous system. They have the ability of predicting, learning from experiences and generalizing from previous examples. An important drawback of ANNs is their very limited explanation capability, mainly due to the fact that knowledge embedded within ANNs is distributed over the activations and the connection weights. Therefore, one of the main challenges in the recent decades is to extract classification rules from ANNs. This paper presents a novel approach to extract fuzzy classification rules (FCR) from ANNs because of the fact that fuzzy rules are more interpretable and cope better with pervasive uncertainty and vagueness with respect to crisp rules. A soft computing based algorithm is developed to generate fuzzy rules based on a data mining tool (DIFACONN-miner), which was recently developed by the authors. Fuzzy DIFACONN-miner algorithm can extract fuzzy classification rules from datasets containing both categorical and continuous attributes. Experimental research on the benchmark datasets and comparisons with other fuzzy rule based classification (FRBC) algorithms has shown that the proposed algorithm yields high classification accuracies and comprehensible rule sets.

Introduction

Artificial neural network is a mathematical or computational model based on biological neural networks. In most cases, an ANN is an adaptive system that changes its structure based on internal and external information that flows through the network during the learning process. Even in the case of an element of the neural network failing, it has the ability to continue learning without any problem through their parallel structure. In addition, ANNs can tolerate incomplete or noisy inputs. These characteristics make ANNs a convenient tool for classification and prediction.

However, classification and function approximation concepts of ANNs are usually incomprehensible to the end user. This is because typical ANN solutions consist of a large number of interacting non-linear elements, characterized by large sets of real-valued parameters that are difficult to interpret. Distributed internal representations make it even more difficult to understand what exactly an ANN has learned and where it will fail to generate the correct answer (Kuttuyil, 2004). Because of this fact, many researchers tend to develop humanly understandable representations of ANNs. This can be achieved by extracting production rules from trained ANNs (Huang & Xing, 2002).

Algorithms for rule extraction from ANNs are grouped into three categories. The first category is the decompositional approach that involves rule extraction at the level of hidden and output units, which are mapped in a binary form (Hruschka & Ebecken, 2006). The second is the pedagogical approach which attempts to map inputs directly into outputs, and views ANNs as a black-box. Finally, the eclectic approach incorporates elements of both pedagogical and decomposition techniques.

Besides these approaches, researchers also extended the scope of rule extraction algorithms from ANNs so as to incorporate the fuzziness in rules (Huang and Xing, 2002, Muslimi et al., 2006, Quteishat and Lim, 2008). The motivation behind this theory is to integrate the advantages of ANNs such as learning, adaptation, fault-tolerance, parallelism and generalization, with fuzzy inference mechanisms under cognitive uncertainty. Extracting fuzzy rules from a trained ANN offers the advantage of being able to build a fuzzy system which is transparent to the user, even when domain expert knowledge is unavailable (Muslimi et al., 2006).

In this paper, a new approach, named the Fuzzy DIFACONN-miner, is proposed for extracting conjunctive normal form (CNF) fuzzy rules from feed-forward ANNs. Conjunction is used between the attributes of rule, and disjunction is used between the sub-attributes. The continuous valued attributes of the datasets are fuzzified using a triangular membership function. A fixed length binary encoding scheme is mapped to represent rules in the proposed approach. The performance of the algorithm is tested on six classification benchmark problems, and compared with a number of other FRBC algorithms available in the literature. Non-parametric statistical tests have been used to compare and analyze the accuracy of the experimental results. They show the superiority of the proposed algorithm over compared algorithms with regard to the accuracy of the results.

The paper is organized in the following way: In Section 2, the rule extraction problem is briefly described. Fuzzy DIFACONN-miner algorithm is comprehensibly explained in Section 3. The experimental study is presented in Section 4. Experimental results and their analysis are given in Section 5. Finally, in Section 6, concluding remarks are reported.