A structure evolving learning method for fuzzy systems

Abstract

This paper proposes a structure evolving learning method for fuzzy systems. The mechanism of the algorithm is that it is an error-reducing driven learning method. The proposed algorithm starts with a simple fuzzy system and evolves the system structure by adding more fuzzy terms and rules to reduce the model errors in a ‘greedy’ way. The main features of the proposed algorithm are summarized as three points. Firstly, it can automatically determine and control the number and location of fuzzy terms by following the error-reducing driven evolving process to achieve the desired accuracy. Secondly, it adds new fuzzy terms and rules by evenly distributing error to each sub-region aiming at an efficient set of fuzzy rules. Thirdly, it uses triangular membership functions with regular partitions and leads to fuzzy system models with good transparency and interpretability. Three benchmark examples are given to illustrate the advantages of the proposed algorithm.

Appendix

Proof of Theorem 1

We assume that the sub-region sub-region _m is selected for splitting, and the splitting point is ${\mathbf{x}}^{splitting} = ( {x_{1}^{splitting}}, x_{2}^{splitting}, \ldots, x_{n}^{splitting} ),$ and the selected attribute for splitting is x _s . Now we prove that for any given input vector ${\mathbf{x}} = (x_{1},x_{2}, \ldots,x_{n} ),$ the output o ^old(x) for the old fuzzy system F ^old(x) (before sub-region splitting), is the same as the output o ^new(x) for the new fuzzy system $\hat{F}^{new} ({\mathbf{x}})$ after sub-region splitting.

1.1 Part A. Proof for Mamdani fuzzy system

1.1.1 Part A.I

If ${\mathbf{x}} = (x_{1},x_{2}, \ldots,x_{n} )$ is located in sub-region _c , and c ≠ m, then the parameters $y_{c}^{{k_{1} k_{2}, \ldots, k_{n} }}$ and membership functions $\mu_{c,i}^{{k_{i} }} (x_{i} )$ for sub-region _c (c ≠ m) do not change after sub-region splitting.

$$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}) = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i = 1}^{n} \mu_{c,i}^{{k_{i} }} (x_{i} )} \right) \times y_{c}^{{k_{1} k_{2}, \ldots, k_{n} }} } $$

(18)

1.1.2 Part A.II

If ${\mathbf{x}} = (x_{1},x_{2}, \ldots,x_{n} )$ is located in sub-region _m , then before sub-region splitting, the model output of F ^old(x) is,

$$ \begin{gathered} o^{old} ({\mathbf{x}}) = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i = 1}^{n} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y_{m}^{{k_{1} k_{2}, \ldots, k_{n} }} } \hfill \\ = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y_{m}^{{k_{1} k_{2}, \ldots, k_{n} }} |_{{k_{s} = 0}} } } \right) \hfill \\ + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y_{m}^{{k_{1} k_{2}, \ldots, k_{n} }} |_{{k_{s} = 1}} } } \right) \hfill \\ \end{gathered} $$

(19)

We define,

$$ A_{0} = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y_{m}^{{k_{1} k_{2}, \ldots, k_{n} }} |_{{k_{s} = 0}} } $$

(20)

$$ A_{1} = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y_{m}^{{k_{1} k_{2}, \ldots, k_{n} }} |_{{k_{s} = 1}} } $$

(21)

Then,

$$ o^{old} ({\mathbf{x}}) = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{0} + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{1} . $$

(22)

• (a) If $x_{s} \in [ {b_{m,s},x_{s}^{splitting} } ),$ then after sub-region splitting, the model output of $\hat{F}^{new} ({\mathbf{x}})$ is,

  $$ \begin{gathered} o^{new} ({\mathbf{x}}) \hfill \\ = \left( {{\frac{{x_{s}^{splitting} - x_{s} }}{{x_{s}^{splitting} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y_{m}^{{k_{1} k_{2}, \ldots, k_{n} }} |_{{k_{s} = 0}} } } \right) \hfill \\ + \left( {{\frac{{x_{s} - b_{m,s} }}{{x_{s}^{splitting} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y^{{k_{1} k_{2}, \ldots, k_{s - 1} k_{s + 1}, \ldots, k_{n} }} } } \right) \hfill \\ \end{gathered} $$

  (23)

  Applying Eqs. 20–22, we obtain,

  $$ o^{new} ({\mathbf{x}}) = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{0} + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{1} $$

  (24)

  where $x_{s}^{splitting}$ is the edge of the new sub-region created after splitting. However, b _m,s and c _m,s are the minimum and maximum value of the old sub-region_m before splitting.Therefore

  $$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}). $$

  (25)

• (b) If $x_{s} \in [ {x_{s}^{splitting},c_{m,s} } ),$ then after sub-region splitting, the model output of $\hat{F}^{new} ({\mathbf{x}})$ is,

  $$ \begin{gathered} o^{new} ({\mathbf{x}}) \hfill \\ = \left( {{\frac{{x_{s} - x_{s}^{splitting} }}{{c_{m,s} - x_{s}^{splitting} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y_{m}^{{k_{1} k_{2}, \ldots, k_{n} }} |_{{k_{s} = 1}} } } \right) \hfill \\ + \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - x_{s}^{splitting} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times y^{{k_{1} k_{2}, \ldots, k_{s - 1} k_{s + 1}, \ldots, k_{n} }} } } \right) \hfill \\ \end{gathered} $$

  (26)

  Applying Eqs. 20–22, we obtain,

  $$ o^{new} ({\mathbf{x}}) = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{0} + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{1} $$

  (27)

  therefore

  $$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}) $$

  (28)

  By combining Eqs. 18 25, and 28,

  $$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}). $$

  (29)

1.2 Part B. Proof for TS fuzzy system

1.2.1 Part B.I

If ${\mathbf{x}} = (x_{1},x_{2}, \ldots,x_{n} )$ is located in sub-region _c , and c ≠ m, then the parameters $a_{m,0}^{{k_{1} k_{2}, \ldots, k_{n} }},a_{m,1}^{{k_{1} k_{2}, \ldots, k_{n} }}, \ldots,a_{m,n}^{{k_{1} k_{2}, \ldots, k_{n} }}$ and membership functions $\mu_{c,i}^{{k_{i} }} (x_{i} )$ for sub-region _c (c ≠ m) do not change after sub-region splitting.

$$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}) = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i = 1}^{n} \mu_{c,i}^{{k_{i} }} (x_{i} )} \right) \times y_{c}^{{k_{1} k_{2}, \ldots, k_{n} }} } $$

(30)

1.2.2 Part B.II

If ${\mathbf{x}} = (x_{1},x_{2}, \ldots,x_{n} )$ is located in sub-region _m , then before sub-region splitting, the model output of F ^old(x) is,

$$ \begin{gathered} o^{old} ({\mathbf{x}}) = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i = 1}^{n} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times \left( {a_{m,0}^{{k_{1} k_{2}, \ldots, k{}_{n}}} + \sum\limits_{i = 1}^{n} {a_{m,i}^{{k_{1} k_{2}, \ldots, k{}_{n}}} x_{i} } } \right)} \hfill \\ = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times \left( {a_{m,0}^{{k_{1} k_{2}, \ldots, k{}_{n}}} + \sum\limits_{i = 1}^{n} {a_{m,i}^{{k_{1} k_{2}, \ldots, k{}_{n}}} x_{i} } } \right)|_{{k_{s} = 0}} } } \right) \hfill \\ + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times \left( {a_{m,0}^{{k_{1} k_{2}, \ldots, k{}_{n}}} + \sum\limits_{i = 1}^{n} {a_{m,i}^{{k_{1} k_{2}, \ldots k{}_{n}}} x_{i} } } \right)|_{{k_{s} = 1}} } } \right) \hfill \\ \end{gathered} $$

(31)

We define,

$$ A_{0} = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times \left( {a_{m,0}^{{k_{1} k_{2}, \ldots, k{}_{n}}} + \sum\limits_{i = 1}^{n} {a_{m,i}^{{k_{1} k_{2}, \ldots, k{}_{n}}} x_{i} } } \right)|_{{k_{s} = 0}} } . $$

(32)

$$ A_{1} = \sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times \left( {a_{m,0}^{{k_{1} k_{2}, \ldots, k{}_{n}}} + \sum\limits_{i = 1}^{n} {a_{m,i}^{{k_{1} k_{2}, \ldots, k{}_{n}}} x_{i} } } \right)|_{{k_{s} = 1}} } . $$

(33)

Then,

$$ o^{old} ({\mathbf{x}}) = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{0} + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{1} $$

(34)

• (a) If $x_{s} \in [ {b_{m,s},x_{s}^{splitting} } ),$ then after sub-region splitting, the model output of $\hat{F}^{new} ({\mathbf{x}})$ is,

  $$ \begin{gathered} o^{new} ({\mathbf{x}}) \hfill \\ = \left( {{\frac{{x_{s}^{splitting} - x_{s} }}{{x_{s}^{splitting} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times \left( {a_{m,0}^{{k_{1} k_{2}, \ldots, k{}_{n}}} + \sum\limits_{i = 1}^{n} {a_{m,i}^{{k_{1} k_{2}, \ldots, k_{n} }} x_{i} } } \right)|_{{k_{s} = 0}} } } \right) \hfill \\ + \left( {{\frac{{x_{s} - b_{m,s} }}{{x_{s}^{splitting} - b_{m,s} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times a_{new,0}^{{k_{1} k_{2}, \ldots, k_{s - 1} k_{s + 1}, \ldots, k_{n} }} } } \right) \hfill \\ \end{gathered} $$

  (35)

  Applying Eqs. 32, 34, 35, we obtain,

  $$ o^{new} ({\mathbf{x}}) = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{0} + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{1} $$

  (36)

  therefore

  $$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}). $$

  (37)

• (b) If $x_{s} \in [ {x_{s}^{splitting},c_{m,s} } ),$ then after sub-region splitting, the model output of $\hat{F}^{new} ({\mathbf{x}})$ is,

  $$ \begin{gathered} o^{new} ({\mathbf{x}}) \hfill \\ = \left( {{\frac{{x_{s} - x_{s}^{splitting} }}{{c_{m,s} - x_{s}^{splitting} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times \left( {a_{m,0}^{{k_{1} k_{2}, \ldots, k{}_{n}}} + \sum\limits_{i = 1}^{n} {a_{m,i}^{{k_{1} k_{2}, \ldots, k{}_{n}}} x_{i} } } \right)|_{{k_{s} = 1}} } } \right) \hfill \\ + \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - x_{s}^{splitting} }}}} \right) \times \left( {\sum\limits_{{k_{1} k_{2}, \ldots, k_{n} }} {\left( {\mathop \Uppi \limits_{i \ne s}^{{}} \mu_{m,i}^{{k_{i} }} (x_{i} )} \right) \times a_{new,0}^{{k_{1} k_{2}, \ldots, k_{s - 1} k_{s + 1}, \ldots, k_{n} }} } } \right) \hfill \\ \end{gathered} $$

  (38)

  Applying Eqs. 32, 34, 35, we obtain,

  $$ o^{new} ({\mathbf{x}}) = \left( {{\frac{{c_{m,s} - x_{s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{0} + \left( {{\frac{{x_{s} - b_{m,s} }}{{c_{m,s} - b_{m,s} }}}} \right) \times A_{1} $$

  (39)

  therefore

  $$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}) $$

  (40)

  By combining Eqs. 30, 37, and 40, we have

  $$ o^{new} ({\mathbf{x}}) = o^{old} ({\mathbf{x}}). $$

  (41)

So we can get conclusion that by setting the initial value of consequent parameters by Eqs. 14 and 17, we obtain a special case of a new fuzzy system which is equivalent to its old fuzzy system before splitting in accuracy.

Proof of Theorem 2

We have proved in Theorem 1, that we can always find a new fuzzy system $\hat{F}^{new} ({\mathbf{x}})$ with its consequent parameters being determined by Eq. 12 or by Eqs. 14 and 17 is equivalent to its old fuzzy system F ^old(x). That is $\hat{F}^{new} ({\mathbf{x}}) = F^{old} ({\mathbf{x}})$ and $E( {\hat{F}^{new} ({\mathbf{x}})} ) = E( {F^{old} ({\mathbf{x}})} ),$ $\hat{F}^{new} ({\mathbf{x}}) \in \{ {F^{new} ({\mathbf{x}})} \}.$ Let $F_{{^{Best} }}^{new} ({\mathbf{x}}) \in \{ {F^{new} ({\mathbf{x}})} \}$ be a new fuzzy system which minimizes the approximation errors, that is, $E[ {F_{{^{Best} }}^{new} (X)} ] = \min_{{\{ {F^{new} ({\mathbf{x}})} \}}} E[ {F^{new} ({\mathbf{x}})} ].$ Then we have

$$ E\left[ {F_{{^{Best} }}^{new} (X)} \right] = \min_{{\left\{ {F^{new} ({\mathbf{x}})} \right\}}} E\left[ {F^{new} ({\mathbf{x}})} \right] \le E\left[ {\hat{F}^{new} ({\mathbf{x}})} \right] = E\left[ {F^{old} ({\mathbf{x}})} \right] $$

where the inequality is obtained based on the fact that $\hat{F}^{new} ({\mathbf{x}}) \in \{ {F^{new} ({\mathbf{x}})} \}.$