Discrete time-variant nonlinear optimization and system solving via integral-type error function and twice ZND formula with noises suppressed

Abstract

In this paper, by using integral-type error function and twice zeroing neural-dynamics (or termed, Zhang neural-dynamics, ZND) formula, continuous-time advanced zeroing neural-dynamics (CT-AZND) model is proposed for solving the continuous time-variant nonlinear optimization problem. Furthermore, a discrete-time advanced zeroing neural-dynamics (DT-AZND) model is first proposed, analyzed, and investigated for solving the discrete time-variant nonlinear optimization (DTVNO) problem. Theoretical analyses show that the proposed DT-AZND model is convergent, and its steady-state residual error has an $O(g^3)$ pattern with g denoting the sampling gap. In addition, in the presence of various kinds of noises, the proposed DT-AZND model possesses advantaged performance. In detail, the proposed DT-AZND model converges toward the time-variant theoretical solution of the DTVNO problem with $O(g^3)$ residual error in the presence of an arbitrary constant noise and has excellent ability to suppress linear-form time-variant noise and bounded random noise. Illustrative numerical experiments further substantiate the efficacy and advantage of the proposed DT-AZND model for solving the DTVNO problem.

Appendices

Appendix A

In terms of the constant noise $\mathbf{n }(t)=\mathbf c \in \mathbb R^m$, let us use the Laplace transformation to the jth subsystem of the noise-disturbed CT-AZND model (9). Then, we have

$$\begin{aligned} {z_j}(s) = \frac{{s({z_j}(0) + {c}_j/s)}}{{{s^2} + 2s\mu + \mu ^2 }}, \end{aligned}$$

where $\mu > 0$, which can be concluded that the subsystem is stable. Using the final value theorem of Laplace transformation, we have the following equation:

$$\begin{aligned} \mathop {\lim }\limits _{t \rightarrow \infty } {z_j}(t) = \mathop {\mathrm{{lim}}}\limits _{s \rightarrow 0} s{z_j}(s) = \mathop {\mathrm{{lim}}}\limits _{s \rightarrow 0} \frac{{{s^2}({z_j}(0) + {c_j}/s)}}{{{s^2} + 2s\mu + \mu ^2}} = 0. \end{aligned}$$

Thus, it can be concluded that ${\lim _{t \rightarrow \infty }}{\left\| {\mathbf{{z}}(t)} \right\| _\mathrm{{2}}} = 0$. That is, the noise-disturbed CT-AZND model (9) converges toward theoretical solution of CTVNO (2) with steady-state residual error ${\lim _{t \rightarrow \infty }}{\left\| {\mathbf{{z}}(t)} \right\| _\mathrm{{2}}} = 0$. The proof is thus completed.

Appendix B

In terms of the linear-form time-variant noise $\mathbf{n }(t) = \alpha t + \beta \in \mathbb R^m$, let us use Laplace transformation to the jth subsystem of noise-disturbed CT-AZND model (9), and we have

$$\begin{aligned} {z_j}(s) = \frac{{s{z_j}(0) + {\alpha _j}/s + {\beta _j}}}{{{s^2} + 2s{\mu } + \mu ^2}}, \end{aligned}$$

where $\mu > 0$. Therefore, we know that the subsystem is stable. Using the final value theorem of Laplace transformation, we have the following equation:

$$\begin{aligned} \mathop {\lim }\limits _{t \rightarrow \infty } {z_j}(t) = \mathop {\mathrm{{lim}}}\limits _{s \rightarrow 0} s{z_j}(s) = \mathop {\mathrm{{lim}}}\limits _{s \rightarrow 0} \frac{{{s^2}{z_j}(0) + {\alpha _j} + s{\beta _j}}}{{{s^2} + 2s{\mu } + \mu ^2}} = \frac{{{\alpha _j}}}{{\mu ^2}}. \end{aligned}$$

Thus, it is concluded that ${\lim _{t \rightarrow \infty }}{\left\| {\mathbf{{z}}(t)} \right\| _\mathrm{{2}}}= \left\| \alpha \right\| _\mathrm{{2}}/\mu ^2$. Therefore, it is concluded that the noise-disturbed CT-AZND model (9) converges toward theoretical solution of CTVNO (2) with the upper bound of steady-state residual error ${\lim _{t \rightarrow \infty }}{\left\| {\mathbf{{z}}(t)} \right\| _\mathrm{{2}}}$ being $\left\| \alpha \right\| _\mathrm{{2}}/\mu ^2$. The proof is thus completed.

Appendix C

In terms of the bounded random noise $\mathbf{n }(t) = {\varpi (t)} \in \mathbb R^m$, the jth subsystem can be reformulated as

$$\begin{aligned} {\dot{z}_j}(t) = - 2\mu {z_j}(t) - \mu ^2 \int _0^t {{z_j}(\sigma )} \mathrm{{d}}\sigma + {\varpi _j}(t), \nonumber \end{aligned}$$

we can obtain the solution of the above equation as

$$\begin{aligned} {z_j}(t)= & {} -{z_j}(0)t{\mu }\exp ({-\mu }t) + {z_j}(0)\exp ({-\mu }t)\\ \nonumber&+ \int _0^t {\left( {{-\mu }(t - \sigma )\exp ({-\mu }(t - \sigma ))} \right) \varpi _j (\sigma )} \mathrm{{d}}\sigma \\ \nonumber&+ \int _0^t {\exp ({-\mu }(t - \sigma ))\varpi _j (\sigma )} \mathrm{{d}}\sigma . \end{aligned}$$

Based on Theorem 1 in Zhang and Zhang (2013), we know that there exist $\kappa >0$ and $\gamma >0$ such that

$$\begin{aligned} \left| {{\mu }} \right| t\exp ({-\mu }t) \le \kappa \exp ( - \gamma t). \end{aligned}$$

Thus, we obtain

$$\begin{aligned} \left| {{z_j}(t)} \right|\le & {} \left| {-{z_j}(0)t{\mu }\exp ({-\mu }t) + {z_j}(0)\exp ({-\mu }t)} \right| \\ \nonumber&+ \int _0^t {\left| {\kappa \exp (-\gamma (t - \sigma ))} \right| \left| \varpi _j (\sigma )\right| } \mathrm{{d}}\sigma \\ \nonumber&+ \int _0^t {\left| {\exp ({-\mu }(t - \sigma ))} \right| \left| {\varpi _j (\sigma )} \right| } \mathrm{{d}}\sigma . \end{aligned}$$

We further have

$$\begin{aligned} \left| {{z_j}(t)} \right|\le & {} \left| {-{z_j}(0)t{\mu }\exp ({-\mu }t) + {z_j}(0)\exp ({-\mu }t)} \right| \\ \nonumber&+ \left( {\frac{\kappa }{\gamma } + \frac{1}{{{\mu }}}} \right) \mathop {\max }\limits _{0 \le \sigma \le t} \left| {{\varpi _j}(\sigma )} \right| . \end{aligned}$$

Finally, we have

$$\begin{aligned} \mathop {\lim }\limits _{t \rightarrow \infty } {\left\| {{\mathbf {z}}(t)} \right\| _{\text {2}}} \le \left( {\frac{\kappa }{\gamma } + \frac{1}{\mu }} \right) \sqrt{m} \mathop {\max }\limits _{0 \le \sigma \le t } \mathop {\max }\limits _{1 \le j \le m} \left| {{\varpi _j}(\sigma )} \right| . \end{aligned}$$

From the above analysis, it is concluded that the steady-state residual error ${\lim _{t \rightarrow \infty }}{\left\| {\mathbf{{z}}(t)} \right\| _\mathrm{{2}}}$ of noise-disturbed CT-AZND model (9) is bounded in the presence of bounded random noise $\varpi (t)$. In addition, the steady-state residual error ${\lim _{t \rightarrow \infty }}{\left\| {\mathbf{{z}}(t)} \right\| _\mathrm{{2}}}$ of noise-disturbed CT-AZND model (9) is bounded by

$$\begin{aligned} \left( {\frac{\kappa }{\gamma } + \frac{1}{\mu }} \right) \sqrt{m} \mathop {\max }\limits _{0 \le \sigma \le t } \mathop {\max }\limits _{1 \le j \le m} \left| {{\varpi _j}(\sigma )} \right| . \end{aligned}$$

The proof is thus completed.

Appendix D

First of all, let $x_{k+i}$ denote $x\left( {(k + i)g} \right)$, and the following equation can be derived based on the Taylor expansion:

$$\begin{aligned} x_{k+1}= & {} x\left( {(k + 1)g} \right) \\ \nonumber= & {} x(kg) + g\dot{x}(kg) + \frac{{{g^2}}}{{2!}}\ddot{x}(kg) + \frac{{{g^3}}}{{3!}}{x^{(3)}}({c_1}), \end{aligned}$$

(24)

where ${c_1}$ lies between kg and $(k + 1)g$. Similarly, the following two equations can be obtained:

$$\begin{aligned} x_{k-1}= & {} x\left( {(k - 1)g} \right) \\ \nonumber= & {} x(kg) - g\dot{x}(kg) + \frac{{{g^2}}}{{2!}}\ddot{x}(kg) - \frac{{{g^3}}}{{3!}}{x^{(3)}}({c_2}) \end{aligned}$$

(25)

and

$$\begin{aligned} x_{k-2}= & {} x\left( {(k - 2)g} \right) = x(kg) \\ \nonumber&-\, 2g\dot{x}(kg) + \frac{{{{(2g)}^2}}}{{2!}}\ddot{x}(kg) - \frac{{{{(2g)}^3}}}{{3!}}{x^{(3)}}({c_3}), \end{aligned}$$

(26)

with ${c_2}$ and ${c_3}$ lying in the gap $((k - 1)g, kg)$ and $((k - 2)g, kg)$, respectively. Let (24) multiply 3, let (4) multiply $-1$, and let (26) multiply $-1/2$. Then, add together these results. The following equation is thus obtained:

$$\begin{aligned} {{\dot{x}}_k}= & {} \frac{3}{{5g}}{x_{k + 1}} - \frac{3}{{10g}}{x_k} - \frac{1}{{5g}}{x_{k - 1}} - \frac{1}{{10g}}{x_{k - 2}} \\ \nonumber&-\, \frac{1}{{10}}{g^2}{x^{(3)}}({c_1}) + \frac{1}{{30}}{g^2}{x^{(3)}}({c_2}) - \frac{2}{{15}}{g^2}{x^{(3)}}({c_3}), \end{aligned}$$

which can be rewritten as

$$\begin{aligned} {{\dot{x}}_k} = \frac{3}{{5g}}{x_{k + 1}} - \frac{3}{{10g}}{x_k} - \frac{1}{{5g}}{x_{k - 1}} - \frac{1}{{10g}}{x_{k - 2}} + O({g^2}) \nonumber \end{aligned}$$

with $O(g^2)$ absorbing the three terms $(1/10){g^2}{x^{(3)}}({c_1})$,

$(1/30){g^2}{x^{(3)}}({c_2})$ and $(2/15){g^2}{x^{(3)}}({c_3})$. Computationally, the 4-point 1-step-ahead finite difference formula is proposed as

$$\begin{aligned} {{\dot{x}}_k} \approx \frac{3}{{5g}}{x_{k + 1}} - \frac{3}{{10g}}{x_k} - \frac{1}{{5g}}{x_{k - 1}} - \frac{1}{{10g}}{x_{k - 2}}, \end{aligned}$$

which has a truncation error of $O( {{g^2}} )$.