On the Hermite Series-Based Generalized Regression Neural Networks for Stream Data Mining

Abstract

In the paper, we develop the mathematically justified stream data mining algorithm for solving regression problems. The algorithm is based on the Hermite expansions of drifting regression functions. The global convergence, in the $L_2$ space, is proved both in probability and with probability one. The examples of several concept drifts to be handled by our algorithm, and the illustrative simulations are presented.

Appendix – Proof of Theorems 1 and 2

Let us denote:

$$\begin{aligned} K_n(x,u) = \sum _{j_{1}=0}^{q(n)}...\sum _{j_{p}=0}^{q(n)}g_{j_{1}}(x^{(1)}) g_{j_{1}}(u^{(1)})\dots \text { }g_{j_{p}}(x^{(p)})g_{j_{p}}(u^{(p)}). \end{aligned}$$

(42)

Obviously,

$$\begin{aligned} \int {Var[Y_n K_n(x,X_n)]dx} \le \int {\mathbb {E}[Y^2_n K^2_n(x,X_n)]dx}. \end{aligned}$$

(43)

Since $X_n$ and $Z_n$ are independent random variables, using (7) and (8), one gets:

$$\begin{aligned} \int {\mathbb {E}[Y^2_n K^2_n(x,X_n)]dx} \end{aligned}$$

$$\begin{aligned} = \int {\mathbb {E}[{\phi }^2_n(X_n) K^2_n(x,X_n)]dx} + \int {\mathbb {E}[Z^2_n K^2_n(x,X_n)]dx} \end{aligned}$$

$$\begin{aligned} = \int \int {\phi }^2_n(u)K^2_n(x,u)f(u) dx du + \sigma _z^2 \int \int K^2_n(x,u)f(u) dx du. \end{aligned}$$

(44)

By the Schwarz inequality, one has:

$$\begin{aligned} K^2_n(x,u) \le \sum _{j_{1}=0}^{q(n)}g^2_{j_{1}}(x^{(1)}) \sum _{j_{1}=0}^{q(n)}g^2_{j_{1}}(u^{(1)}) \dots \sum _{j_{p}=0}^{q(n)}g^2_{j_{p}}(x^{(p)}) \sum _{j_{p}=0}^{q(n)}g^2_{j_{p}}(u^{(p)}) \end{aligned}$$

(45)

Combining (44) with (45), using (3), (18), and the orthonormality of system (1), one obtains the following bound:

$$\begin{aligned} \int {Var[Y_n K_n(x,X_n)]dx} \le q^p(n) \left( \sum _{j=0}^{q(n)}G^2_j \right) ^p d_n. \end{aligned}$$

(46)

In view of (25) and (46), a proper application of Theorems 9.3 and 9.4 in [18] concludes the proof of Theorems 1 and 2.