Introducing evolving Takagi–Sugeno method based on local least squares support vector machine models

Abstract

In this study, an efficient local online identification method based on the evolving Takagi–Sugeno least square support vector machine (eTS-LS-SVM) for nonlinear time series prediction is introduced. As an innovation, this paper has applied the nonlinear models, i.e. local LS-SVM models, as the consequence parts of the fuzzy rules, instead of the linear models used in the conventional evolving TS fuzzy models. In each step, the proposed learning approach includes two phases. The fuzzy rules (rule premise) are first created and updated adaptively based on a sequential clustering technique to obtain the structure of TS model. Then, the parameters of each local LS-SVM model (rule consequence) are recursively updated by deriving a new recursive algorithm (a local decremental and incremental procedure) to minimize the local modelling error and trace the process’s dynamics. Besides, a new learning algorithm based on the recursive gradient-based method is used to adaptively update the meta-parameters of the LS-SVM models. Comparison of the suggested method with some of the previous approaches based on the online prediction of the nonlinear time series has shown that the introduced identification algorithm has a proper performance in terms of learning and generalization abilities while having a lower redundancy.

Appendix

Proof

First consider $Y_{K }^i = [ 0\quad {y_k^i(1)} \ldots{y_k^i({N - 1})}]^T$ where (N − 1) denotes the size of moving-windows for the ith local model after the pruning stage, y _k+1 is new added output then we can derive (36) as following:

$$\begin{aligned} \Uptheta_{k+1}^{i}&=P_{k+1}^{i}\left[\begin{array}{l} Y_{k }^i\\ y_{k+1} \end{array}\right]\\ &= \left[{\begin{array}{ll} {P_{pr}^{i}-\eta_{k+1}^{i}Z_{k+1}^{i} (Z_{k+1}^{i})^{T}} & {\eta_{k+1}^{i}Z_{k+1}^{i}}\\ {\zeta_{k+1}^{i}\psi_{k+1}^{i}\left[{\eta_{k+1}^{i} Z_{k+1}^{i} (Z_{k+1}^{i})^{T}-P_{N}}\right]}&{-\eta_{k+1}^{i}} \end{array}}\right]\left[{\begin{array}{l} {Y_{k}^{i}}\\ {y_{k+1}}\\ \end{array}}\right]\\ &=\left[{\begin{array}{l} P_{N} Y_{k}^{i}-\eta_{k+1}^{i} Z_{k+1}^{i} (Z_{k+1}^{i})^{T} Y_{k}^{i}+\eta_{k+1}^{i} Z_{k+1}^{i} y_{k+1}\\ \zeta_{k+1}^{i} \psi_{k+1}^{i}\eta_{k+1}^{i} Z_{k+1}^{i} (Z_{k+1} ^{i})^{T} Y_{k}^{i}-\zeta_{k+1}^{i} (Z_{k+1}^{i})^{T} Y_{k}^{i}- \eta_{k+1}^{i} y_{k+1} \end{array}} \right]\\ &=\left[{\begin{array}{l} \Uptheta_{pr}^{i}+\eta_{k+1}^{i} Z_{k+1}^{i} (y_{k+1}-(Z_{k+1}^{i})^{T} Y_{k}^{i})\\ (\zeta_{k+1}^{i}\eta_{k+1}^{i}({\eta_{k+1}^{i}})^{-1}-\zeta _{k+1}^{i})(Z_{k+1} ^{i})^{T} Y_{k}^{i}\\ \qquad \qquad\qquad-\eta_{k+1}^{i} (y_{k+1}-(Z_{k+1}^{i})^{T} Y_{k}^{i}) \end{array}}\right]\\ &=\left[{\begin{array}{l}\Uptheta_{pr}^{i}+\eta_{k+1}^{i} Z_{k+1}^{i} e_{k+1}^{i}\\ (\zeta_{k+1}-\zeta_{k+1})(Z_{k+1}^{i})^{T} Y_{k}^{i}-\eta_{k+1}e_{k+1}^{i} \end{array}}\right]\\ &=\left[{\begin{array}{l} \Uptheta_{pr}^{i}+\eta_{k+1}^{i}Z_{k+1}^{i} e_{k+1}^{i}\\ {-\eta_{k+1}^{i} e_{k+1}^{i}} \end{array}} \right] \end{aligned} $$

(49)

where e ⁱ _k+1 is the prediction error computed by the difference between the desired signal and the output of the ith local model after the pruning stage (Dovžan and Škrjanc 2011).