A generalized TSK model with a novel rule antecedent structure: Structure identification and parameter estimation

Abstract

TSK fuzzy models are convenient tools for describing complex nonlinear behavior. However, the existing combinatorial antecedent structure in TSK models makes them substantially suffer from the curse of dimensionality. In this work, a novel rule antecedent structure is proposed to design an efficient generalized TSK (GTSK) model by using fewer rules. The new rule antecedent only uses nonlinear variables. Additionally, one more degree of freedom is introduced to design antecedents to cover an antecedent space more efficiently, which further reduces the number of rules. The resultant GTSK model is identified in two stages. A novel recursive estimation based on spatially rearranged data is used to determine the consequent and antecedent variables. Model parameter values are obtained from partitioned antecedent space, which is the result of solving a series of splitting and regression problems.

Introduction

TSK (Takagi–Sugeno–Kang) fuzzy models gained popularity in fields including modeling (Chen, 2009, Cococcioni et al., 2007, Jacquin and Shamseldin, 2006), control (Guelton et al., 2009, Khiar et al., 2007), forecasting (Chang & Liu, 2008), and classification (Zhang, Zhou, Liu, & Harrington, 2006). The popularity is partially due to the interpretability rendered by the linguistic terms such as ‘High’, ‘Fast’, etc., used to construct rules. Also, the cause-and-effect format in an IF … THEN structure is friendly for human understanding and interpretation. Interpretability of a TSK model is enhanced by the divide-and-conquer concept inherited in the model, where each rule describes a part of model behavior, and rules work together for a complete description. One then understands a complex model by understanding each part of it. More importantly, the popularity is rooted in a fundamental reason: a TSK fuzzy model is a universal approximator (Kosko, 1994), which guaranties its ability to describe almost any nonlinear behavior given a sufficiently flexible structure.

The following presentation will be based upon a single-input–single-output (SISO) model. The extension to multiple-input–multiple-output (MIMO) models will be addressed when necessary. Eq. (1) is a general expression of SISO dynamic systems with dynamic orders ny, nu, pure time delay d, and an additive disturbance e(t)

$$y(t)=f\left(\begin{array}{l}y(t-1),\dots ,y(t-ny),\\ u(t-d),\dots ,u(t-nu-d)\end{array}\right)+e(t)$$

where y is the system response and u is the system input.

A corresponding rule (the rth rule in a TSK model) to Eq. (1) is described by

$$\begin{array}{l}\text{IF}(y(t-1)\text{is}{A}_1^r\text{AND}\cdots \text{AND}u(t-nu-d)\text{is}{A}_{ny+nu+1}^r)\\ \text{THEN}{\text{A}}^r(z^{-1})y(t)=k^r+{\text{B}}^r(z^{-1})u(t-d)\\ {\text{A}}^r(z^{-1})=1+a_1^r{z}^{-1}+\cdots +a_{ny}^r{z}^{-ny}\\ {\text{B}}^r(z^{-1})=b_0^r+b_1^r{z}^{-1}+\cdots +b_{nu}^r{z}^{-nu}\end{array}$$

where z is the backshift operator, and the expression $y(t-1)\text{is}{A}_1^r\text{AND}\cdots \text{AND}u(t-nu-d)\text{is}{A}_{ny+nu+1}^r$ is the antecedent of the rule. The variables y(t − 1), …, y(t − ny), u(t − d), …, u(t − nu − d) are antecedent variables and $A_1^r$ is the fuzzy subset for y(t − 1) in the rule. The consequent of the rule is a local linear model A^r(z⁻¹)y(t) = k^r + B^r(z⁻¹)u(t − d) with dynamic orders ny and nu, pure time delay d and a constant k^r.

A TSK model is defined by a collection of rules in Eq. (2). The identification of a TSK model needs to determine antecedent and consequent structures, and to estimate parameter values.

An important structure configuration in a TSK model is the number of rules, which should be set to balance model complexity with accuracy. Trials are often needed in practice to obtain the right number. Heuristics based on clustering are used to recognize the prototype rules (Dickerson and Kosko, 1996, Vernieuwe et al., 2006, Wang and Yang, 2009), which automatically results the number of rules. In Nelles (2001) the number of rules progressively grows in each step when an equal division in a dimension is conducted. The number of rules might also be determined in a backward fashion (Yen & Wang, 1999), where the TSK model initially has a large number of rules. In the end, a more compact model results by eliminating redundant rules. The computational burden of these techniques increases geometrically with dimensions (ny and nu). Additionally, heuristic-based stochastic procedures exist to gain both model structures and parameter values simultaneously (Du and Zhang, 2008, Guenounou et al., 2009, Lin, 2008, Lin and Xu, 2006), which however require even more computation resources.

The GTSK models in this work are for dynamic systems. Therefore, dynamic orders such as ny, nu and pure time delay, d in Eq. (1) are also important structure configurations. Dynamic order determination is well developed for linear systems where a preliminary analysis using autocorrelation and partial autocorrelation is able to estimate dynamic orders (Brockwell & Davis, 1998). For static linear systems, subset selection methods (Miller, 1990) are able to find influential regressors. Analysis of variance can also be used for regressor analysis (Lind & Liung, 2008). For nonlinear dynamic systems with unknown nonlinearities, there is no general method, and order analysis falls into two categories. One approach accepts either known or assumed nonlinear structures. There are various choices of nonlinear structures such as bilinear, Wiener, Hammerstein structures or their combinations. With known nonlinear structures, analysis might be conducted rigorously. Another approach does not depend on a predefined nonlinear structure. The geometric method (Molina, Sampson, Fitzgerald, & Niranjan, 1996), False Nearest Neighbor (Rhodes & Morari, 1995), and Lipschitz Quotient (He & Asada, 1993) all belong to the second category. These methods can be roughly argued upon the first-order Taylor series expansion. However, these methods are sensitive to noise (Nelles, 2001).

The number of rules is coupled with dimensions, ny and nu and more rules are required for a higher dimension. Reducing the number of rules in a TSK model is possible by allowing a dimension difference between antecedents and consequents. The difference is implicitly expressed in (Kawamoto, 1992, Takagi and Sugeno, 1985, Tanaka and Wang, 2001), where a TSK model is used to approximate a known nonlinear state-space model and the antecedent variables are defined as functions of system states. This results in a TSK model with different dimensions in its antecedent and consequent. Antecedent variable selection is explicitly mentioned in (Leith and Leithead, 1999, Shorten et al., 1999), where examples are given to select antecedent variables as nonlinear states from a known nonlinear state-space model. However, neither is applicable to the situation where only input–output data are available and the system model is unknown. In Pomares, Rojas, González, and Prieto (2002), variable selection was proposed in constructing a model for function approximation. However, selected variables are included in both antecedent and consequent, which creates difficulty with a high dimension problem.

In addition to the antecedent dimension affecting the number of rules, the geometric structure in the antecedent also has a significant impact. The antecedent in Eq. (2) assumes a combinatorial structure. Its permutations lead to many rule possibilities.

Once the model structure is determined, the next task is to determine model parameter values. Parameters for a dynamic TSK model include both antecedent and consequent parameters. Parameter estimation could be pursued by solving optimization problems. Steepest decent (Dickerson & Kosko, 1996) and Levenberg–Marquardt (Moreno-Velo, Baturone, Barriga, & Sánchez-Solano, 2007) are popular choices. They all provide local optimal solutions, and trials have to be made from different initial guesses in order to increase the probability of obtaining a global optimal solution (Iyer & Rhinehart, 1999). Genetic algorithms (Cordon et al., 2004) are also used for parameter estimation for a better local solution.

This work proposes several novelties in the successive stages of developing models.

In this work, the dynamic orders, ny, nu, and delay d are determined by selecting influential variables based on proposed recursive estimation, which rearranges the raw data spatially to organize nonlinearity (Section 3). The antecedent variables are then selected as a subset of the determined influential variables, as those which have nonlinear influence on the model output, and are identified by the proposed recursive estimation, again (Section 3E).

Also, in this work, the proposed rule structure has different dimensions in antecedents and consequents. The antecedent dimension is generally lower than the consequent dimension because it only includes variables which have a nonlinear impact on the output, argued to be necessary and sufficient. In addition, the antecedent is designed with one more degree of freedom to include variable interactions to efficiently occupy an antecedent space. The novel rule and antecedent structure reduce the number of rules required.

The number of rules, locations and shapes of local regions in the antecedents are determined by recursively partitioning the antecedent space (Section 4). Once the antecedent space is fully partitioned, the antecedent and consequent parameter values are estimated. The partition is conducted by solving a series of splitting and regression problems.

Section 2 presents the method to reduce the number of rules by using a more general antecedent structure. It concludes with a GTSK model consisting of rules with the new antecedent structure. The variable selection for a nonlinear dynamic process is presented in Section 3. It selects the important regressors defining the overall model dimension and the variables to be used in antecedents. Section 4 presents the details on estimating parameter values based on antecedent space partition. The steps for model development are summarized in Section 5. Several testing and applications are presented in Section 6.