Interval type-2 neuro-fuzzy system with implication-based inference mechanism
Introduction
Neuro-fuzzy systems have been proved to be an efficient tool for modelling real life systems. They are precise and have ability to generalise knowledge from presented data. Their important advantage is interpretability of created models. Some kinds of neuro-fuzzy systems have been proved to be universal approximators that satisfy the Stone–Weierstrass theorem (Kosko, 1994). Neuro-fuzzy systems use fuzzy sets to handle vagueness. Mendel and John (2002) list noise and uncertainty of data among main sources of uncertainties in fuzzy systems with type-1 fuzzy sets. The membership function of type-1 fuzzy sets has strictly crisp values. That deteriorates the ability to represent noisy or uncertain data. Type-2 fuzzy sets (Zadeh, 1975) model better such uncertainties because of their fuzzy membership function.
Type-1 and type-2 fuzzy systems share the same architecture, they have four main components: fuzzifier, inference engine with a rule base, and output processor. In type-2 fuzzy systems the output processor has two tasks: type-reduction and defuzzification. Type reduction is the main cause of high computation burden of the type-2 fuzzy systems. It is their main disadvantage and can be mitigated by application of interval type-2 (IT2) fuzzy sets. An interval type-2 fuzzy set is a kind of type-2 fuzzy set. A secondary membership function of an interval type-2 (IT2) fuzzy sets can have only two values: 0 or 1. A system is called an interval type-2 fuzzy system when it has interval type-2 sets in some part (mostly in rule premises).
Interval type-2 neuro-fuzzy systems have been profoundly theoretically analysed by Liang and Mendel (2000), Karnik, Mendel, and Liang (1999), and Mendel and Rajati (2014). These papers present the theory and design patterns. Mendel (2004) provides mathematical formulae for steepest-descent parameter tuning of type-2 fuzzy systems with a Karnik–Mendel type reducer. The Karnik–Mendel algorithm (Karnik & Mendel, 2001) is consistent with an extension principle. Unfortunately this complicates the theoretical analysis and implementation of systems. To avoid this Wu and Mendel (2002), Nie and Tan (2008), Du and Ying (2010) formulate type-reducers in a closed-form formulae. These solutions do not satisfy the requirements of the fuzzy theory, but are easier to analyse than the Karnik–Mendel algorithm. Neuro-fuzzy systems can be ordered starting with type-1 (T1) through interval type-2 (IT2) to general type-2 systems. This sequence presents both increase in ability to handle uncertainties and computational burden. Computational complexity of type reduction in general type-2 systems is high enough to hinder their practical application. Interval type-2 systems have lower computational overhead than general type-2 fuzzy systems. They can better handle uncertainties and can model more complex surfaces than type-1 systems with similar number of rules. An interval type-2 fuzzy system with the KM type reducer cannot be implemented by a type-1 fuzzy system using the same rule base (Wu, 2012).
There have been proposed many interval type-2 neuro-fuzzy systems. Most systems have interval type-2 fuzzy sets only in premises. The DIT2LIFR-IP system (Juang & Chen, 2013) uses interval type-2 sets in premises and zero-order TSK interval type-2 consequences (interval constants). The initial rule base is generated by a self-splitting clustering algorithm in the input–output space. Aliasghary, Eksin, Guzelkaya, and Kumbasar (2015) describe an interval type-2 fuzzy system with diamond-shaped type-2 fuzzy sets in premises and singletons in consequences and Nie–Tan type reduction. Melin, Mendoza, and Castillo (2010) present an interval type-2 fuzzy inference system with interval fuzzy constants in consequences. Starczewski and Rutkowski (2003) propose a type-2 neuro-fuzzy system with interval fuzzy membership grades in rule antecedents and intervals in consequents. Chen and Chang (2011a) apply interval type-2 Gaussian fuzzy sets with fuzzy cores in premises for sparse fuzzy rule-based system. Juang and Tsao (2008a) propose a self-evolving interval type-2 fuzzy neural network with online structure and parameter learning. The premises of the rules are built with interval type-2 fuzzy sets with uncertain means. The consequences follow a TSK pattern. The rules are generated with an online clustering method.
Non interval type-2 fuzzy systems are not as popular as interval type-2 systems. One of the examples may be a system with type-2 fuzzy set in premises (with triangular secondary membership function) and Mamdani type consequences (Starczewski, Scherer, Korytkowski, & Nowicki, 2008).
Tuning of (interval) type-2 neuro-fuzzy system is a challenging task in spite of wide research (Starczewski et al., 2008). Many techniques are applied, eg: gradient descent and the rule-ordered recursive least squares algorithm (Juang & Chen, 2013), hybrid backpropagation (Castro, Castillo, Melin, & Rodríguez-Díaz, 2009), genetic algorithm (Park & Lee, 2013), backpropagation and AdaBoost algorithms (Starczewski et al., 2008).
A fuzzy IF-THEN rule is a kind of fuzzy implication. Fuzzy implications have been deeply analysed from mathematical point of view. In the proposed system is an implication-based (Dubois & Prade, 1996) fuzzy inference system where fuzzy rules are evaluated as fuzzy implication. This approach is also called an implicative or deductive approach (Štěpnička & Baets, 2013).
The novelty of this paper is a neuro-fuzzy system with two features: (1) it is an interval type-2 neuro-fuzzy system (with IT2 sets both in premises and consequences of rules) with (2) implication-based inference mechanism. To the best of our knowledge this is the first attempt at creation of an interval type-2 (IT2) fuzzy system with an implication-based inference mechanism and IT2 fuzzy sets both in premises and consequences of the rules.
Our system is an extension of the ANNBFIS (Artificial Neural Network Based Fuzzy Inference System) system (Czogała & Łęski, 2000). This type-1 system has been proved to be precise and fast in calculations and has many applications and modifications as ε-insensitive learning (Leski, 2003), rough-fuzzy paradigm (Siminski, 2015), deterministic annealing optimisation (Czabański, 2006), subspace approach (Siminski, 2017), hybrid system with SVM (Siminski, 2014a, Siminski, 2014b), incomplete data mining (Siminski, 2016a), and inversion of neuro-fuzzy system (Siminski, 2016b).
The paper is organized as follows: Section 2 describes the interval type-2 fuzzy implications. Section 3 describes the proposed interval type-2 neuro-fuzzy system with implication-based inference mechanism. Section 4 shortly presents the creation of rules from train data. Section 5 describes datasets and experiments. Finally Section 6 summarizes the paper. The calculation of derivatives is moved to Appendix to keep the text clearer.
Section snippets
Interval type-2 implication
The paper describes an interval type-2 neuro-fuzzy system with implication-based inference mechanism (IT2NFSIB). Each rule in rule base is a fuzzy interval type-2 implication. Alcalde, Burusco, and Fuentes-González (2005) discuss the construction and features of interval type-2 fuzzy implications based on known type-1 fuzzy implications. The authors propose methods for construction of internal type-2 fuzzy implications from continuous T-norms for interval type-2 fuzzy numbers (denoted as [l, u
IT2 neuro-fuzzy system with implication-based inference mechanism
Interval type-2 neuro-fuzzy system with implication-based inference mechanism is a multiple input single output (MISO) system. Fuzzy rule base is a crucial part of the system. Each fuzzy rule is an interval type-2 fuzzy logical implication. The value of the rule is a value of fuzzy implication of premise and consequence.
Rule base contains fuzzy rules l (fuzzy implications) where and y are linguistic variables, and are fuzzy linguistic terms. The squiggle arrow
Creation of fuzzy model
The rule base of the system is created with the train data. In the system we use scatter partition (clustering) of the input domain to avoid curse of dimensionality occurring in a grid partition. In the first step the parameters of premises are elaborated with clustering and consequences with least square error method. Then they are tuned with gradient method. The needed parameters are: number of rules in the rule base, number of clustering iterations, and number of tuning iterations.
Experiments
This section describes the experiments. In all experiments the number of rules is set to . Number of rules in neuro-fuzzy systems is an open question. A practical approach is to keep the number of rules reasonably low, commonly the rule bases hold several rules. Small number of rules facilitates the interpretability of the fuzzy model, what is an important feature of neuro-fuzzy systems. In our experiments we aimed at comparison of precision of the models and we have set the number of rules
Conclusions
The paper presents an interval type-2 fuzzy system with implication-based inference mechanism (IT2NFSIB). This is the first attempt to create such a kind of an IT2 neuro-fuzzy system. The system has interval type-2 fuzzy sets both in premises (Gaussian membership function with uncertain fuzziness) and consequences (trapezoid membership function). The system is tuned with the gradient method.
The experiments show that the proposed system can elaborate more generalised model than T1 systems with
Acknowledgements
The author is grateful to the anonymous Referees for their constructive comments that have helped to improve the paper. This study was funded by Politechnika Slaska (Silesian University of Technology), grant number: BK-219/RAU2/2016.
References (54)
- et al.
A constructive method for the definition of interval-valued fuzzy implication operators
Fuzzy Sets and Systems
(2005) - et al.
A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks
Information Sciences
(2009) - et al.
Interval type-2 fuzzy membership function generation methods for pattern recognition
Information Sciences
(2009) - et al.
What are fuzzy rules and how to use them
Fuzzy Sets and Systems
(1996) - et al.
Centroid of a type-2 fuzzy set
Information Sciences
(2001) - et al.
An enhanced fuzzy linear regression model with more flexible spreads
Fuzzy Sets and Systems
(2009) - et al.
An improved method for edge detection based on interval type-2 fuzzy logic
Expert Systems with Applications
(2010) - et al.
Neuro-fuzzy systems for function approximation
Fuzzy Sets and Systems
(1999) Rough subspace neuro-fuzzy system
Fuzzy Sets and Systems
(2015)Ridders algorithm in approximate inversion of fuzzy model with parameterized consequences
Expert Systems with Applications
(2016)
Implication-based models of monotone fuzzy rule bases
Fuzzy Sets and Systems
The concept of a linguistic variable and its application to approximate reasoning-I
Information Science
KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework
Journal of Multiple-Valued Logic and Soft Computing
General derivation and analysis for input-output relations in interval type-2 fuzzy logic systems
Soft Computing
Fuzzy rule interpolation based on interval type-2 Gaussian fuzzy sets and genetic algorithms
Fuzzy systems (FUZZ), 2011 IEEE international conference on
Weighted fuzzy rule interpolation based on ga-based weight-learning techniques
IEEE Transactions on Fuzzy Systems
Fuzzy interpolative reasoning for sparse fuzzy rule-based systems based on α-cuts and transformations techniques
Fuzzy Systems, IEEE Transactions
Extraction of fuzzy rules using deterministic annealing integrated with ϵ-insensitive learning
International Journal of Applied Mathematics and Computer Science
Fuzzy and neuro-fuzzy intelligent systems
Derivation and analysis of the analytical structures of the interval type-2 fuzzy-PI and PD controllers
IEEE Transactions on Fuzzy Systems
Fuzzy interpolation and extrapolation: A practical approach
IEEE Transactions on Fuzzy Systems
Uncertain fuzzy clustering: Interval type-2 fuzzy approach to c-means
IEEE Transactions On Fuzzy Systems
SOFMLS: Online self-organizing fuzzy modified least-squares network
IEEE Transactions on Fuzzy Systems
Data-driven interval type-2 neural fuzzy system with high learning accuracy and improved model interpretability
IEEE Transactions on Cybernetics
An on-line self-constructing neural fuzzy inference network and its applications
IEEE Transactions on Fuzzy Systems
A self-evolving interval type-2 fuzzy neural network with online structure and parameter learning
IEEE Transactions on Fuzzy Systems
A type-2 self-organizing neural fuzzy system and its FPGA implementation
IEEE Transactions on Systems, Man, Cybernetics B
Cited by (26)
3WDNFS – Three-way decision neuro-fuzzy system for classification
2023, Fuzzy Sets and SystemsPrognosis of forest production using machine learning techniques
2023, Information Processing in AgricultureCitation Excerpt :Therefore, it is important to check whether the use of these techniques can provide gains in terms of accuracy. ANFIS is a combination of artificial neural network (ANN) and fuzzy logic systems, in which an ANN generates rules or functions of relevance to fuzzy systems [19], with good interpretability and precision, to generalize the acquired knowledge [20]. RF is a machine learning method that uses a bootstrap approach to build several decision trees.
An embedded deep fuzzy association model for learning and explanation
2022, Applied Soft ComputingCitation Excerpt :The conventional fuzzy inference mechanisms are inappropriate under such scenarios, which require a dynamic fuzzy rule base [18] or intermediate fuzzy rules to be generated by fuzzy interpolation and extrapolation [19]. In addition, it is infeasible to specify a universal fuzzy implication operator to be used in diverse tasks [20]. Another possible approach is to embed the convolutional architecture into a neuro-fuzzy system with the aim of handling the nonlinear behaviors under the conditions of data shifts.
Prototype based granular neuro-fuzzy system for regression task
2022, Fuzzy Sets and SystemsCitation Excerpt :In general, type-2 systems are adversely affected by high computational complexity (type reduction from type 2 to type 1 is a bottleneck in these systems) [37], and thus interval type-2 fuzzy sets are used because they have the advantages of type-2 sets but lower computational complexity [53,68]. Only a few systems use an interval type-2 fuzzy set in the whole information flow path from premises to consequences [61]. Granular computing is a paradigm that mimics the human approach to information processing.
Functional characterization of unknown protein sequences using Neuro-Fuzzy based machine learning approach and sequence augmented feature
2022, Expert Systems with ApplicationsCitation Excerpt :Dehzangi et al., 2015) proposed bacterial protein function prediction model using support vector machine. As per the recent periodicals very less amount of work has been carried out for the function prediction of UPS using Neuro-Fuzzy-based machine learning approach, while rule based membership function is reliable for the prediction of unknown samples as compared to the probability and likelihood based methods (Siminski, 2017). Protein secondary structure classification was performed by (Hering et al., 2003) using an artificial neural network with fuzzy approach.