Elsevier

Expert Systems with Applications

Volume 79, 15 August 2017, Pages 140-152
Expert Systems with Applications

Interval type-2 neuro-fuzzy system with implication-based inference mechanism

https://doi.org/10.1016/j.eswa.2017.02.046Get rights and content

Highlights

  • The system uses interval type-2 fuzzy sets in premises and consequences of rules.

  • The system uses several interval type-2 fuzzy implications.

  • The system applies logical interpretation to fuzzy rules.

  • The paper is accompanied by numerical examples.

  • The system can elaborate models with lower errors than type-1 and type-2 systems.

Abstract

Neuro-fuzzy systems have been proved to be an efficient tool for modelling real life systems. They are precise and have ability to generalise knowledge from presented data. Neuro-fuzzy systems use fuzzy sets – most commonly type-1 fuzzy sets. Type-2 fuzzy sets model uncertainties better than type-1 fuzzy sets because of their fuzzy membership function. Unfortunately computational complexity of type reduction in general type-2 systems is high enough to hinder their practical application. This burden can be alleviated by application of interval type-2 fuzzy sets. The paper presents an interval type-2 neuro-fuzzy system with interval type-2 fuzzy sets both in premises (Gaussian interval type-2 fuzzy sets with uncertain fuzziness) and consequences (trapezoid interval type-2 fuzzy set). The inference mechanism is based on the interval type-2 fuzzy Łukasiewicz, Reichenbach, Kleene-Dienes, or Brouwer–Gödel implications. The paper is accompanied by numerical examples. The system can elaborate models with lower error rate than type-1 neuro-fuzzy system with implication-based inference mechanism. The system outperforms some known type-2 neuro-fuzzy systems.

Introduction

Neuro-fuzzy systems have been proved to be an efficient tool for modelling real life systems. They are precise and have ability to generalise knowledge from presented data. Their important advantage is interpretability of created models. Some kinds of neuro-fuzzy systems have been proved to be universal approximators that satisfy the Stone–Weierstrass theorem (Kosko, 1994). Neuro-fuzzy systems use fuzzy sets to handle vagueness. Mendel and John (2002) list noise and uncertainty of data among main sources of uncertainties in fuzzy systems with type-1 fuzzy sets. The membership function of type-1 fuzzy sets has strictly crisp values. That deteriorates the ability to represent noisy or uncertain data. Type-2 fuzzy sets (Zadeh, 1975) model better such uncertainties because of their fuzzy membership function.

Type-1 and type-2 fuzzy systems share the same architecture, they have four main components: fuzzifier, inference engine with a rule base, and output processor. In type-2 fuzzy systems the output processor has two tasks: type-reduction and defuzzification. Type reduction is the main cause of high computation burden of the type-2 fuzzy systems. It is their main disadvantage and can be mitigated by application of interval type-2 (IT2) fuzzy sets. An interval type-2 fuzzy set is a kind of type-2 fuzzy set. A secondary membership function of an interval type-2 (IT2) fuzzy sets can have only two values: 0 or 1. A system is called an interval type-2 fuzzy system when it has interval type-2 sets in some part (mostly in rule premises).

Interval type-2 neuro-fuzzy systems have been profoundly theoretically analysed by Liang and Mendel (2000), Karnik, Mendel, and Liang (1999), and Mendel and Rajati (2014). These papers present the theory and design patterns. Mendel (2004) provides mathematical formulae for steepest-descent parameter tuning of type-2 fuzzy systems with a Karnik–Mendel type reducer. The Karnik–Mendel algorithm (Karnik & Mendel, 2001) is consistent with an extension principle. Unfortunately this complicates the theoretical analysis and implementation of systems. To avoid this Wu and Mendel (2002), Nie and Tan (2008), Du and Ying (2010) formulate type-reducers in a closed-form formulae. These solutions do not satisfy the requirements of the fuzzy theory, but are easier to analyse than the Karnik–Mendel algorithm. Neuro-fuzzy systems can be ordered starting with type-1 (T1) through interval type-2 (IT2) to general type-2 systems. This sequence presents both increase in ability to handle uncertainties and computational burden. Computational complexity of type reduction in general type-2 systems is high enough to hinder their practical application. Interval type-2 systems have lower computational overhead than general type-2 fuzzy systems. They can better handle uncertainties and can model more complex surfaces than type-1 systems with similar number of rules. An interval type-2 fuzzy system with the KM type reducer cannot be implemented by a type-1 fuzzy system using the same rule base (Wu, 2012).

There have been proposed many interval type-2 neuro-fuzzy systems. Most systems have interval type-2 fuzzy sets only in premises. The DIT2LIFR-IP system (Juang & Chen, 2013) uses interval type-2 sets in premises and zero-order TSK interval type-2 consequences (interval constants). The initial rule base is generated by a self-splitting clustering algorithm in the input–output space. Aliasghary, Eksin, Guzelkaya, and Kumbasar (2015) describe an interval type-2 fuzzy system with diamond-shaped type-2 fuzzy sets in premises and singletons in consequences and Nie–Tan type reduction. Melin, Mendoza, and Castillo (2010) present an interval type-2 fuzzy inference system with interval fuzzy constants in consequences. Starczewski and Rutkowski (2003) propose a type-2 neuro-fuzzy system with interval fuzzy membership grades in rule antecedents and intervals in consequents. Chen and Chang (2011a) apply interval type-2 Gaussian fuzzy sets with fuzzy cores in premises for sparse fuzzy rule-based system. Juang and Tsao (2008a) propose a self-evolving interval type-2 fuzzy neural network with online structure and parameter learning. The premises of the rules are built with interval type-2 fuzzy sets with uncertain means. The consequences follow a TSK pattern. The rules are generated with an online clustering method.

Non interval type-2 fuzzy systems are not as popular as interval type-2 systems. One of the examples may be a system with type-2 fuzzy set in premises (with triangular secondary membership function) and Mamdani type consequences (Starczewski, Scherer, Korytkowski, & Nowicki, 2008).

Tuning of (interval) type-2 neuro-fuzzy system is a challenging task in spite of wide research (Starczewski et al., 2008). Many techniques are applied, eg: gradient descent and the rule-ordered recursive least squares algorithm (Juang & Chen, 2013), hybrid backpropagation (Castro, Castillo, Melin, & Rodríguez-Díaz, 2009), genetic algorithm (Park & Lee, 2013), backpropagation and AdaBoost algorithms (Starczewski et al., 2008).

A fuzzy IF-THEN rule is a kind of fuzzy implication. Fuzzy implications have been deeply analysed from mathematical point of view. In the proposed system is an implication-based (Dubois & Prade, 1996) fuzzy inference system where fuzzy rules are evaluated as fuzzy implication. This approach is also called an implicative or deductive approach (Štěpnička & Baets, 2013).

The novelty of this paper is a neuro-fuzzy system with two features: (1) it is an interval type-2 neuro-fuzzy system (with IT2 sets both in premises and consequences of rules) with (2) implication-based inference mechanism. To the best of our knowledge this is the first attempt at creation of an interval type-2 (IT2) fuzzy system with an implication-based inference mechanism and IT2 fuzzy sets both in premises and consequences of the rules.

Our system is an extension of the ANNBFIS (Artificial Neural Network Based Fuzzy Inference System) system (Czogała & Łęski, 2000). This type-1 system has been proved to be precise and fast in calculations and has many applications and modifications as ε-insensitive learning (Leski, 2003), rough-fuzzy paradigm (Siminski, 2015), deterministic annealing optimisation (Czabański, 2006), subspace approach (Siminski, 2017), hybrid system with SVM (Siminski, 2014a, Siminski, 2014b), incomplete data mining (Siminski, 2016a), and inversion of neuro-fuzzy system (Siminski, 2016b).

The paper is organized as follows: Section 2 describes the interval type-2 fuzzy implications. Section 3 describes the proposed interval type-2 neuro-fuzzy system with implication-based inference mechanism. Section 4 shortly presents the creation of rules from train data. Section 5 describes datasets and experiments. Finally Section 6 summarizes the paper. The calculation of derivatives is moved to Appendix to keep the text clearer.

Section snippets

Interval type-2 implication

The paper describes an interval type-2 neuro-fuzzy system with implication-based inference mechanism (IT2NFSIB). Each rule in rule base is a fuzzy interval type-2 implication. Alcalde, Burusco, and Fuentes-González (2005) discuss the construction and features of interval type-2 fuzzy implications based on known type-1 fuzzy implications. The authors propose methods for construction of internal type-2 fuzzy implications from continuous T-norms for interval type-2 fuzzy numbers (denoted as [l, u

IT2 neuro-fuzzy system with implication-based inference mechanism

Interval type-2 neuro-fuzzy system with implication-based inference mechanism is a multiple input single output (MISO) system. Fuzzy rule base is a crucial part of the system. Each fuzzy rule is an interval type-2 fuzzy logical implication. The value of the rule is a value of fuzzy implication of premise and consequence.

Rule base L contains fuzzy rules l (fuzzy implications) l:xisayisb,where x=[x1,x2,,xD]T and y are linguistic variables, a and b are fuzzy linguistic terms. The squiggle arrow

Creation of fuzzy model

The rule base of the system is created with the train data. In the system we use scatter partition (clustering) of the input domain to avoid curse of dimensionality occurring in a grid partition. In the first step the parameters of premises are elaborated with clustering and consequences with least square error method. Then they are tuned with gradient method. The needed parameters are: number of rules in the rule base, number of clustering iterations, and number of tuning iterations.

Experiments

This section describes the experiments. In all experiments the number of rules is set to L=6. Number of rules in neuro-fuzzy systems is an open question. A practical approach is to keep the number of rules reasonably low, commonly the rule bases hold several rules. Small number of rules facilitates the interpretability of the fuzzy model, what is an important feature of neuro-fuzzy systems. In our experiments we aimed at comparison of precision of the models and we have set the number of rules

Conclusions

The paper presents an interval type-2 fuzzy system with implication-based inference mechanism (IT2NFSIB). This is the first attempt to create such a kind of an IT2 neuro-fuzzy system. The system has interval type-2 fuzzy sets both in premises (Gaussian membership function with uncertain fuzziness) and consequences (trapezoid membership function). The system is tuned with the gradient method.

The experiments show that the proposed system can elaborate more generalised model than T1 systems with

Acknowledgements

The author is grateful to the anonymous Referees for their constructive comments that have helped to improve the paper. This study was funded by Politechnika Slaska (Silesian University of Technology), grant number: BK-219/RAU2/2016.

References (54)

  • M. Štěpnička et al.

    Implication-based models of monotone fuzzy rule bases

    Fuzzy Sets and Systems

    (2013)
  • L.A. Zadeh

    The concept of a linguistic variable and its application to approximate reasoning-I

    Information Science

    (1975)
  • J. Alcalá-Fdez et al.

    KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework

    Journal of Multiple-Valued Logic and Soft Computing

    (2011)
  • M. Aliasghary et al.

    General derivation and analysis for input-output relations in interval type-2 fuzzy logic systems

    Soft Computing

    (2015)
  • S.-M. Chen et al.

    Fuzzy rule interpolation based on interval type-2 Gaussian fuzzy sets and genetic algorithms

    Fuzzy systems (FUZZ), 2011  IEEE international conference on

    (2011)
  • S.-M. Chen et al.

    Weighted fuzzy rule interpolation based on ga-based weight-learning techniques

    IEEE Transactions on Fuzzy Systems

    (2011)
  • S.-M. Chen et al.

    Fuzzy interpolative reasoning for sparse fuzzy rule-based systems based on α-cuts and transformations techniques

    Fuzzy Systems, IEEE Transactions

    (2008)
  • R. Czabański

    Extraction of fuzzy rules using deterministic annealing integrated with ϵ-insensitive learning

    International Journal of Applied Mathematics and Computer Science

    (2006)
  • E. Czogała et al.

    Fuzzy and neuro-fuzzy intelligent systems

    (2000)
  • X. Du et al.

    Derivation and analysis of the analytical structures of the interval type-2 fuzzy-PI and PD controllers

    IEEE Transactions on Fuzzy Systems

    (2010)
  • Z. Huang et al.

    Fuzzy interpolation and extrapolation: A practical approach

    IEEE Transactions on Fuzzy Systems

    (2008)
  • C. Hwang et al.

    Uncertain fuzzy clustering: Interval type-2 fuzzy approach to c-means

    IEEE Transactions On Fuzzy Systems

    (2007)
  • J. de Jesus Rubio

    SOFMLS: Online self-organizing fuzzy modified least-squares network

    IEEE Transactions on Fuzzy Systems

    (2009)
  • C.-F. Juang et al.

    Data-driven interval type-2 neural fuzzy system with high learning accuracy and improved model interpretability

    IEEE Transactions on Cybernetics

    (2013)
  • C.F. Juang et al.

    An on-line self-constructing neural fuzzy inference network and its applications

    IEEE Transactions on Fuzzy Systems

    (1998)
  • C.-F. Juang et al.

    A self-evolving interval type-2 fuzzy neural network with online structure and parameter learning

    IEEE Transactions on Fuzzy Systems

    (2008)
  • C.F. Juang et al.

    A type-2 self-organizing neural fuzzy system and its FPGA implementation

    IEEE Transactions on Systems, Man, Cybernetics B

    (2008)
  • Cited by (26)

    • Prognosis of forest production using machine learning techniques

      2023, Information Processing in Agriculture
      Citation Excerpt :

      Therefore, it is important to check whether the use of these techniques can provide gains in terms of accuracy. ANFIS is a combination of artificial neural network (ANN) and fuzzy logic systems, in which an ANN generates rules or functions of relevance to fuzzy systems [19], with good interpretability and precision, to generalize the acquired knowledge [20]. RF is a machine learning method that uses a bootstrap approach to build several decision trees.

    • An embedded deep fuzzy association model for learning and explanation

      2022, Applied Soft Computing
      Citation Excerpt :

      The conventional fuzzy inference mechanisms are inappropriate under such scenarios, which require a dynamic fuzzy rule base [18] or intermediate fuzzy rules to be generated by fuzzy interpolation and extrapolation [19]. In addition, it is infeasible to specify a universal fuzzy implication operator to be used in diverse tasks [20]. Another possible approach is to embed the convolutional architecture into a neuro-fuzzy system with the aim of handling the nonlinear behaviors under the conditions of data shifts.

    • Prototype based granular neuro-fuzzy system for regression task

      2022, Fuzzy Sets and Systems
      Citation Excerpt :

      In general, type-2 systems are adversely affected by high computational complexity (type reduction from type 2 to type 1 is a bottleneck in these systems) [37], and thus interval type-2 fuzzy sets are used because they have the advantages of type-2 sets but lower computational complexity [53,68]. Only a few systems use an interval type-2 fuzzy set in the whole information flow path from premises to consequences [61]. Granular computing is a paradigm that mimics the human approach to information processing.

    • Functional characterization of unknown protein sequences using Neuro-Fuzzy based machine learning approach and sequence augmented feature

      2022, Expert Systems with Applications
      Citation Excerpt :

      Dehzangi et al., 2015) proposed bacterial protein function prediction model using support vector machine. As per the recent periodicals very less amount of work has been carried out for the function prediction of UPS using Neuro-Fuzzy-based machine learning approach, while rule based membership function is reliable for the prediction of unknown samples as compared to the probability and likelihood based methods (Siminski, 2017). Protein secondary structure classification was performed by (Hering et al., 2003) using an artificial neural network with fuzzy approach.

    View all citing articles on Scopus
    View full text