Reinforcement learning-based online adaptive controller design for a class of unknown nonlinear discrete-time systems with time delays

Abstract

This paper is concerned with online adaptive control strategy for a class of unknown nonlinear discrete-time systems with time delays. The main objective is to establish an online adaptive control strategy based on reinforcement learning (RL) algorithm, so that the nonquadratic performance index can be minimized and the closed-loop system with time delays is stable. In order to simplify the control systems with time delays, a time delay function is designed to eliminate the term of control delays. Then, the online adaptive control algorithm via RL is presented to approach the reasonable control law and optimizes the long-term performance function. On the basis of the Lyapunov theory, it is proved that the design of online adaptive controller is effective and all the signals of control system are ultimately uniformly bounded. The simulation results indicate the validity and feasibility of the proposed adaptive control strategy.

Appendix

1.1 Proof of Theorem 1

Consider the Lyapunov candidate function defined as

$$\begin{aligned} \begin{aligned} L(k)&= \sum \limits _{i = 1}^5 {{L_i}(k)} \\&=\displaystyle \frac{{{\gamma _1}}}{3}x_{}^T(k)x(k) + \frac{{{\gamma _2}}}{{{\eta _a}}}{\tilde{w}}_a^T(k){\tilde{w}}_a^{}(k) + \frac{{{\gamma _3}}}{{{\eta _c}}}{\tilde{w}}_c^T(k){\tilde{w}}_c^{}(k) \\&\quad +\displaystyle \frac{{{\gamma _4}}}{{{\eta _M}}}{\tilde{w}}_M^T(k){\tilde{w}}_M^{}(k) + {\gamma _5}{\left\| {{\zeta _c}(k - 1)} \right\| ^2} \end{aligned} \end{aligned}$$

(40)

where ${\gamma _i} \in {R^ + },{{i = 1,2,3,4,5,}}$ are the design parameters.

Let ${L_1}(k) = \displaystyle \frac{{{\gamma _1}}}{3}x_{}^T(k)x(k)$ and the first-order derivation of ${L_1}(k)$ is

$$\begin{aligned} \begin{aligned} \Delta {L_1}(k)&= \displaystyle \frac{{{\gamma _1}}}{3}(x_{}^T(k + 1)x(k + 1) - x_{}^T(k)x(k)) \\&\le {\gamma _1}{\bar{g}}_M^2\zeta _a^2 + {\gamma _1}{\bar{g}}_M^2\delta _a^2 + {\gamma _1}\lambda _{\max }^2({\alpha _1}){\left\| {x(k)} \right\| ^2} - \displaystyle \frac{{{\gamma _1}}}{3}{\left\| {x(k)} \right\| ^2} \end{aligned} \end{aligned}$$

(41)

where ${\lambda _{\, \, \max }}({\alpha _1})$ is the maximum eigenvalue of matrix ${\alpha _1}$.

Let ${L_2}(k) = \displaystyle \frac{{{\gamma _2}}}{{{\eta _a}}}\tilde{w}_a^T(k){\tilde{w}}_a^{}(k)$ and the first-order derivation of ${L_2}(k)$ is expressed by

$$\begin{aligned} \begin{aligned} \Delta {L_2}(k) = \frac{{{\gamma _2}}}{{{\eta _a}}}{\left\| {{{{\tilde{w}}}_a}(k + 1)} \right\| ^2} - \frac{{{\gamma _2}}}{{{\eta _a}}}{\left\| {{{{\tilde{w}}}_a}(k)} \right\| ^2}. \end{aligned} \end{aligned}$$

(42)

Subtracting the term ${w_a}$ on both sides of (34), one gets

$$\begin{aligned} {{\tilde{w}}_a}(k + 1) = {{\tilde{w}}_a}(k) - {\eta _a}{\psi _a}({s_a}(k))(x(k + 1) - {\alpha _1}x(k) + {\hat{J}}(k)). \end{aligned}$$

(43)

Substituting (43) into (42), it yields

$$\begin{aligned} \begin{aligned} \Delta {L_2}(k)&\le \displaystyle \frac{{{\gamma _2}}}{{{\eta _a}}}[\eta _a^2{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{({\alpha _1}x(k) - x(k + 1) - {\hat{J}}(k))^2}] \\&\quad -\displaystyle \frac{{2{\gamma _2}}}{{{\eta _a}}}{{{\tilde{w}}}_a}(k)[{\eta _a}{\psi _a}({s_a}(k))({\hat{J}}(k) + x(k + 1) - {\alpha _1}x(k))] \\&= {\eta _a}{\gamma _2}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{({\alpha _1}x(k) - x(k + 1) - {\hat{J}}(k))^2} - 2{\gamma _2}{\zeta _a}(k) \\&\quad \times ({\hat{J}}(k) + x(k + 1) - {\alpha _1}x(k))\\&\le {\eta _a}{\gamma _2}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{({\alpha _1}x(k) - x(k + 1) - {\hat{J}}(k))^2} - 2{\gamma _2}{\zeta _a}(k)\\&\quad \times ({g_M}(x(k),x(k - \sigma )){\zeta _a}(k) \\&\quad - {g_M}(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k))\, \, \\&\le {\gamma _2}\left\{ { - ({{{\underline{g}} }_M} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{\underline{g}} _M^2)} \right. \zeta _a^2(k) - 2\zeta _a^{}(k) \\&\quad \times [1 - {\eta _a}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{g_M}(x(k),x(k - \sigma ))]\\&\quad \times [\, - {g_M}(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k)] - {{{\underline{g}} }_M} \zeta _a^2(k)\\&\quad \left. { + {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{[\, - {g_M}(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k)]}^2}} \right\} . \end{aligned} \end{aligned}$$

(44)

Subtracting and adding the term ${{({{(1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M})}^2})} \big / {({{{\underline{g}} }_M} - {\eta _a}}}$ $\times {\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{\bar{g}}_M^2) \times {[ - {g_M}(x(k),x(k - \sigma )){\delta _a} + {\hat{J}}(k)]^2}$ into the above equation, it obtains

$$\begin{aligned} \begin{aligned} \Delta {L_2}(k)&\le {\gamma _2}\left\{ { - {{{\underline{g}} }_M}\zeta _a^2 + \frac{{{{(1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M})}^2}}}{{{{{\underline{g}} }_M} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{\bar{g}}_M^2}}} \right. \\&\quad \times {[ - {g_M}(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k)]^2} \\&\quad - ({{\underline{g}} _M} - {\eta _a}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{\bar{g}}_M^2)\\&\quad \times \left[ {{\zeta _a} + \frac{{1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M}}}{{{{{\underline{g}} }_M} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{\bar{g}}_M^2}}} \right. \\&\quad {\left. { \times ( - {g_M}(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k))} \right] ^2}\\&\quad + \left. {{\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{[ - {g_M}(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k)]}^2}} \right\} .\\ \end{aligned} \end{aligned}$$

(45)

Define ${\gamma _2} = {\gamma _{21}}{\gamma _{22}}$, where ${\gamma _{22}}[{{({{(1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M})}^2}} \big / {({{{\underline{g}} }_M} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}}}$ $\times \bar{g}_M^2)) + {\eta _a}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}] \le {1 / 2}$. Thus, it becomes

$$\begin{aligned} \begin{aligned} \Delta {L_2}(k)&\le -{\gamma _2}({{\underline{g}} _M} - {\eta _a}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{\bar{g}}_M^2)\left[ {{\zeta _a} + \frac{{1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M}}}{{{{{\underline{g}} }_M} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{\bar{g}}_M^2}}} \right. \\&\quad {\left. { \times ( - g_M(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k))} \right] ^2}\\&\quad - {\gamma _2}{{\underline{g}} _M}\zeta _a^2 + \frac{{{\gamma _{21}}}}{2}{[ - g_M(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k)]^2}\\&\le - {\gamma _2}{{\underline{g}} _M}\zeta _a^2 - {\gamma _2}({{\underline{g}} _M} - {\eta _a}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{\bar{g}}_M^2)\\&\quad \times \, \left[ {{\zeta _a} + \frac{{1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M}}}{{{{{\underline{g}} }_M} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{\bar{g}}_M^2}}} \right. \\&\quad {\left. { \times ( - g_M(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k))} \right] ^2}\\&\quad + {\gamma _{21}}\zeta _c^2 + {\gamma _{21}}{[ - g_M(x(k),x(k - \sigma )){\delta _a}(k) + J(k)]^2}. \end{aligned} \end{aligned}$$

(46)

Defining ${{\tilde{w}}_c}(k) = {{\hat{w}}_c}(k) - {w_c}$ and subtracting ${w_c}$ on both sides of (28), one gets

$$\begin{aligned} {{\tilde{w}}_c}(k + 1)&= {{\tilde{w}}_c}(k) - {\eta _c}{\psi _c}({s_c}(k))\times [{\beta ^{N - k + 1}}r(k) \nonumber \\&\quad + {\hat{w}}_c^T(k){\psi _c}(s(k)) - {\hat{w}}_c^T(k - 1){\psi _c}(s(k - 1))]. \end{aligned}$$

(47)

Thus, the first-order derivative of ${L_3}(k)$ can be obtained

$$\begin{aligned} \begin{aligned} \Delta {L_3}(k)&=\displaystyle \frac{{{\gamma _3}}}{{{\eta _c}}}{\left\| {{{{\tilde{w}}}_c}(k + 1)} \right\| ^2} - \frac{{{\gamma _c}}}{{{\eta _c}}}{\left\| {{{{\tilde{w}}}_c}(k)} \right\| ^2}\\&= \frac{{{\gamma _3}}}{{{\eta _c}}}\left\{ { - 2{\eta _c}{\tilde{w}}_c^T{\psi _c}({s_c}(k))({\beta ^{N - k + 1}}p(k) + {\hat{w}}_c^T(k){\psi _c}({s_c}(k))} \right. \\&\quad - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1))) + \eta _c^2{\left\| {{\psi _c}({s_c}(k))} \right\| ^2}[{\beta ^{N - k + 1}}p(k)\\&\quad \left. { + {\hat{w}}_c^T(k){\psi _c}({s_c}(k)) - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1)){]^2}} \right\} \\&= -\, 2{\gamma _3}{\zeta _c}(k)({\beta ^{N - k + 1}}p(k) + {\hat{w}}_c^T(k){\psi _c}({s_c}(k))\, \, \, \\&\quad - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1))) + {\eta _c}{\gamma _3}{\left\| {{\psi _c}({s_c}(k))} \right\| ^2} \\&\quad \times [{\beta ^{N - k + 1}}p(k) + {\hat{w}}_c^T(k){\psi _c}({s_c}(k))\, \, \, \, \\&\quad - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1))){]^2}. \end{aligned} \end{aligned}$$

(48)

Subtracting and adding the term ${\gamma _3}{[{\beta ^{N - k + 1}}r(k) + {\hat{w}}_c^T(k){\psi _c}({s_c}(k)) - {\hat{w}}_c^T(k - 1)}$ $\times {{\psi _c}({s_c}(k - 1))]^2}$ into (48), $\Delta L(3)$ becomes

$$\begin{aligned} \begin{aligned} \Delta {L_3}(k)&= -\, {\gamma _3}[1 - {\eta _c}{\left\| {{\psi _c}({s_c}(k))} \right\| ^2}][{\beta ^{N - k + 1}}p(k) \\&\quad + {\hat{w}}_c^T(k){\psi _c}({s_c}(k)) - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1)){]^2}\\&\quad + {\gamma _3}[{\beta ^{N - k + 1}}p(k) + {\hat{w}}_c^T(k){\psi _c}({s_c}(k)) \\&\quad - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1))][ - 2{\zeta _c}(k) + {\beta ^{N - k + 1}}p(k) \\&\quad + {\hat{w}}_c^T(k){\psi _c}({s_c}(k)) - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1))] \\&= -\, {\gamma _3}[1 - {\eta _c}{\left\| {{\psi _c}({s_c}(k))} \right\| ^2}][{\beta ^{N - k + 1}}p(k) \\&\quad + {\hat{w}}_c^T(k){\psi _c}({s_c}(k)) - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1)){]^2} \\&\quad + {\gamma _3}[{\beta ^{N - k + 1}}p(k) + w_c^T(k){\psi _c}({s_c}(k)) \\&\quad - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1)){]^2} - {\gamma _3}\zeta _c^2(k). \end{aligned} \end{aligned}$$

(49)

According to the equality ${\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1)) = w_c^T(k - 1){\psi _c}({s_c}(k - 1)) + {\zeta _c}(k - 1)$ and the fact ${(m \pm n)^2} \le 2{m^2} + 2{n^2}$, one can deduce

$$\begin{aligned} \begin{aligned} \Delta {L_3}(k)&\le - {\gamma _3}[1 - {\eta _c}{\left\| {{\psi _c}({s_c}(k))} \right\| ^2}][{\beta ^{N - k + 1}}r(k) \\&\quad + {\hat{w}}_c^T(k){\psi _c}({s_c}(k)) - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1)){]^2} \\&\quad + 2{\gamma _3}[{\beta ^{N - k + 1}}p(k) + w_c^T(k){\psi _c}({s_c}(k)) \\&\quad - w_c^T(k - 1){\psi _c}({s_c}(k - 1)){]^2} - {\gamma _3}\zeta _c^2(k) + 2{\gamma _3}\zeta _c^2(k - 1). \end{aligned} \end{aligned}$$

(50)

Let ${L_4}(k) = \displaystyle \frac{{{\gamma _4}}}{{{\eta _M}}}\tilde{w}_M^T(k){\tilde{w}}_M^{}(k)$ and the first-order derivation of ${L_4}(k)$ is expressed by

$$\begin{aligned} \Delta {L_4}(k) = \displaystyle \frac{{{\gamma _4}}}{{{\eta _M}}}{\left\| {{{{\tilde{w}}}_M}(k + 1)} \right\| ^2} - \frac{{{\gamma _4}}}{{{\eta _M}}}{\left\| {{{{\tilde{w}}}_M}(k)} \right\| ^2}. \end{aligned}$$

(51)

Subtracting term ${w_M}$ on both sides of (15), one gets

$$\begin{aligned} {{\tilde{w}}_M}(k + 1) = {{\tilde{w}}_M}(k) - {\eta _M}{\psi _M}(u(k))({\hat{u}}(k - \tau ) - u(k - \tau )). \end{aligned}$$

(52)

Substituting (52) into (51) yields

$$\begin{aligned} \begin{aligned} \Delta {L_4}(k)&=\displaystyle \frac{{{\gamma _4}}}{{{\eta _M}}}\left( { - 2{\eta _M}{{{\tilde{w}}}_M}(k){\psi _M}(u(k))({\hat{u}}(k - \tau ) - u(k - \tau ))} \right. \,\\&\quad + \left. {\eta _M^2\psi _M^2(u(k))({\hat{u}}(k - \tau ) - u(k - \tau ))\, {\, ^2}} \right) . \end{aligned} \end{aligned}$$

(53)

Subtracting and adding the term ${\gamma _4}{\left[ {{\hat{u}}(k - \tau ) - u(k - \tau )} \right] ^2}$, the $\Delta {L_4}(k)$ becomes

$$\begin{aligned} \begin{aligned} \Delta {L_4}(k)\, \, \,&= -\, {\gamma _4}[1 - {\eta _M}\psi _M^2(u(k))]{[{\hat{u}}(k - \tau ) - u(k - \tau )]^2} \\&\quad + {\gamma _4}[ - 2{{{\tilde{w}}}_M}(k){\psi _M}(u(k)) + ({\hat{u}}(k - \tau ) - u(k - \tau ))\, ] \\&\quad \times [{\hat{u}}(k - \tau ) - u(k - \tau )] \\&= -\, {\gamma _4}[1 - {\eta _M}\psi _M^2(u(k))]{[{\hat{u}}(k - \tau ) - u(k - \tau )]^2} \\&\quad +{\gamma _4} [{w_M}{\psi _M}(u(k)) - u(k - \tau ) - {\zeta _M}(k)] \\&\quad \times [{w_M}{\psi _M}(u(k)) - u(k - \tau ) + {\zeta _M}(k)] \\&\le - {\gamma _4}[1 - {\eta _M}\psi _M^2(u(k))]{[{\hat{u}}(k - \tau ) - u(k - \tau )]^2} \\&\quad + {\gamma _4}{[{w_M}{\psi _M}(u(k)) - u(k - \tau )]^2} - {\gamma _4}\zeta _M^2(k). \end{aligned} \end{aligned}$$

(54)

By invoking (7) and (8), one obtains

$$\begin{aligned} \begin{aligned} \Delta {L_4}(k)&\le - {\gamma _4}[1 - {\eta _M}\psi _M^2(u(k))]{e_M^2} \\&\quad + {\gamma _4}\delta _M^2(k) - {\gamma _4}\zeta _M^2(k) \end{aligned} \end{aligned}$$

(55)

where ${\zeta _M}(k) = {{\tilde{w}}_M}{\psi _M}(u(k))$.

In addition, let ${L_5} = {\gamma _5}{\left\| {{\zeta _c}(k - 1)} \right\| ^2}$, it is obviously that

$$\begin{aligned} \Delta {L_5}(k) = {\gamma _5}\zeta _c^2(k) - {\gamma _5}\zeta _c^2(k - 1). \end{aligned}$$

(56)

Combining (41), (46), (50), (55) with (56), the first derivative of L(k) becomes

$$\begin{aligned} \begin{aligned} \Delta L(k)&= \sum \limits _{i = 1}^5 {\Delta {L_i}(k)} \\&\le ({\gamma _1}{\bar{g}}_M^2 - {\gamma _2}{{\underline{g}} _M})\zeta _a^2 + ({\gamma _1}\lambda _{\max }^2({\alpha _1}) - \displaystyle \frac{{{\gamma _1}}}{3}){\left\| {{x_{}}(k)} \right\| ^2} \\&\quad - {\gamma _2}({{\underline{g}} _M} - {\eta _a}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{\bar{g}}_M^2)\, \\&\quad \times {\left[ {{\zeta _a} + \displaystyle \frac{{1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M}}}{{{\underline{g}} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{\bar{g}}_M^2}}{{[ - g(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k)]}^{}}} \right] ^2} \\&\quad + ({\gamma _{21}} - {\gamma _3} + {\gamma _5})\zeta _c^2(k)\, - {\gamma _3}[1 - {\eta _c}{\left\| {{\psi _c}({s_c}(k))} \right\| ^2}] \\&\quad \times {[{\beta ^{N - k + 1}}p(k) + {\hat{w}}_c^T(k){\psi _c}({S_c}(k)) - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1))]^2} \\&\quad - ({\gamma _5} - 2{\gamma _3})\zeta _c^2(k - 1) - {\gamma _4}[1 - {\eta _M}\psi _M^2(u(k))]e_M^2\\&\quad - {\gamma _4}\zeta _M^2(k) - {\gamma _4}[1 - {\eta _M}\psi _M^2(u(k))]e_M^2 + {R_1} \\&= -\, ({\gamma _2}{{\underline{g}} _M} - {\gamma _1}{\bar{g}}_M^2)\zeta _a^2 - {\gamma _1}(\frac{1}{3} - \lambda _{\max }^2({\alpha _1})){\left\| {{x_{}}(k)} \right\| ^2} \\&\quad - ({\gamma _3} - {\gamma _{21}} - {\gamma _5})\zeta _c^2(k)\, \, \, - {\gamma _2}({{\underline{g}} _M} - {\eta _a}{\left\| {{\psi _a}({s_a}(k))} \right\| ^2}{\bar{g}}_M^2) \\&\quad \times {\left[ {{\zeta _a} + \displaystyle \displaystyle \frac{{1 - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{{{\underline{g}} }_M}}}{{{\underline{g}} - {\eta _a}{{\left\| {{\psi _a}({s_a}(k))} \right\| }^2}{\bar{g}}_M^2}}{{[ - g(x(k),x(k - \sigma )){\delta _a}(k) + {\hat{J}}(k)]}^{}}} \right] ^2} \\&\quad - {\gamma _3}[1 - {\eta _c}{\left\| {{\psi _c}({s_c}(k))} \right\| ^2}] \\&\quad \times {[{\beta ^{N - k + 1}}p(k) + {\hat{w}}_c^T(k){\psi _c}({S_c}(k)) - {\hat{w}}_c^T(k - 1){\psi _c}({s_c}(k - 1))]^2}\\&\quad - ({\gamma _5} - 2{\gamma _3})\zeta _c^2(k - 1) - {\gamma _4}\zeta _M^2(k) - {\gamma _4}[1 - {\eta _M}\psi _M^2(u(k))]e_M^2 + {R_1} \end{aligned} \end{aligned}$$

(57)

with

$$\begin{aligned} {R_1}&= {\gamma _1}g_M^2\delta _a^2 + {\gamma _{21}}{[ - g(x(k),x(k - \sigma )){\delta _a}(k) + J(k)]^2} \\&\quad + 2{\gamma _3}[{\beta ^{N - k + 1}}p(k) + \omega _c^T(k){\psi _c}({s_c}(k)) \\&\quad - w_c^T(k - 1){\psi _c}({s_c}(k - 1)){]^2} + {\gamma _4}\delta _M^2(k) \\ &\le {\gamma _1}{\bar{g}}_M^2\delta _{am}^2 + 2{\gamma _{21}}{\bar{g}}_M^2\delta _{am}^2 + 2{\gamma _{21}}J_{m}^2 \\&\quad + 12{\gamma _3}w_{cm}^2\psi _{cm}^2 + 6{\gamma _3} + {\gamma _4}{\bar{\delta }} _M^2, \end{aligned}$$

where ${{\bar{g}}_M}$ and ${{\underline{g}} _M}$ stand for the maximum and minimum eigenvalues of the matrix ${g_M}$, respectively.

Choose the parameters as

$$\begin{aligned} \begin{array}{l} {\gamma _1} \le \displaystyle \frac{{{{\underline{\mathrm{g}} }_M}{\gamma _2}}}{{{\bar{g}}_M^2}},0 \le {\lambda _{{\mathrm{max}}}}({\alpha _1}) \le \displaystyle \frac{{\sqrt{3} }}{3},{\gamma _3}> {\gamma _{21}} + {\gamma _5},{\gamma _5} > 2{\gamma _3},0<{\eta _a}<\displaystyle \frac{{{{{\underline{g}} }_M}}}{{{\psi _{am}^2}{\bar{g}}_M^2}}, \\ 0<{\eta _c}< \displaystyle \frac{1}{{\psi _{cm}^2}},0< {\eta _M} <\displaystyle \frac{1}{{{\bar{\psi }} _M^2}} \\ \end{array} \end{aligned}$$

Hence, $\Delta L(k)$ becomes

$$\begin{aligned} \Delta L(k)\le & {} - ({\gamma _2}{{\underline{g}} _M} - {\gamma _1}{\bar{g}}_M^2)\zeta _a^2 - {\gamma _1}\left( \displaystyle \frac{1}{3} - \lambda _{\max }^2({\alpha _1})\right) {\left\| {x(k)} \right\| ^2} \\&\quad - ({\gamma _3} - {\gamma _{21}} - {\gamma _5})\zeta _c^2(k) - {\gamma _4}\zeta _M^2(k) + {{{\bar{R}}}_1} \end{aligned}$$

where

$$\begin{aligned} {{{\bar{R}}}_1}= & {} {\gamma _1}{\bar{g}}_M^2\delta _{am}^2 + 2{\gamma _{21}}{\bar{g}}_M^2\delta _{am}^2 + 2{\gamma _{21}}J_{m}^2 \\&\quad + 12{\gamma _3}w_{cm}^2\psi _{cm}^2 + 6{\gamma _3} + {\gamma _4}{\bar{\delta }} _M^2. \end{aligned}$$

Thus, if one of the following inequalities holds, $\Delta L(k) < 0.$

$$\begin{aligned} \left\| {x(k)} \right\| \ge \sqrt{\frac{{3{{{\bar{R}}}_1}}}{{1 - 3\lambda _{\max }^2({\alpha _1})}}} \end{aligned}$$

or

$$\begin{aligned} \left\| {{\zeta _a}(k)} \right\| \ge \sqrt{\frac{{{{{\bar{R}}}_1}}}{{{\gamma _2}{{{\underline{g}} }_M} - {\gamma _2}{\bar{g}}_M^2}}} \end{aligned}$$

or

$$\begin{aligned} \left\| {{\zeta _c}(k)} \right\| \ge \sqrt{\frac{{{{{\bar{R}}}_1}}}{{{\gamma _3} - {\gamma _{21}} - {\gamma _5}}}} \end{aligned}$$

or

$$\begin{aligned} \left\| {{\zeta _M}(k)} \right\| \ge \sqrt{\frac{{{{{\bar{R}}}_1}}}{{{\gamma _4}}}}. \end{aligned}$$