PI Controller Design for Suppressing Exogenous Disturbances

Abstract

A novel approach is proposed to suppress bounded exogenous disturbances in linear control systems using a PI controller. The approach is based on reducing the original problem to a nonconvex matrix optimization problem. A gradient method for finding the controller’s parameters is derived and its justification is provided. The corresponding recurrence procedure is rather effective and yields quite satisfactory controllers in terms of engineering performance criteria. This paper continues a series of the author’s research works devoted to the design of feedback control laws from an optimization point of view.

Appendices

APPENDIX A

The lemmas below contain well-known results necessary for the further presentation.

Lemma A.1 [1]. Let X and Y be the solutions of the dual Lyapunov equations with a Hurwitz matrix A:

$${{A}^{{\text{T}}}}X + XA + W = 0\quad and\quad AY + Y{{A}^{{\text{T}}}} + V = 0.$$

Then

$$\operatorname{tr} (XV) = \operatorname{tr} (YW).$$

Lemma A.2 [10].

(1) Matrices A and B of compatible dimensions satisfy the relations

$${{\left\| {AB} \right\|}_{F}}\;\leqslant \;{{\left\| A \right\|}_{F}}\left\| B \right\|,$$

$$\left| {\operatorname{tr} AB} \right|\;\leqslant \;{{\left\| A \right\|}_{F}}{{\left\| B \right\|}_{F}},$$

$$\left\| A \right\|\;\leqslant \;{{\left\| A \right\|}_{F}},$$

$$AB + {{B}^{{\text{T}}}}{{A}^{{\text{T}}}}\;\leqslant \;\varepsilon A{{A}^{{\text{T}}}} + \frac{1}{\varepsilon }{{B}^{{\text{T}}}}B\quad for\;any\;\varepsilon > 0.$$

(2) Nonnegative definite matrices A and B satisfy the relations

$$0\;\leqslant \;{{\lambda }_{{\min }}}(A){{\lambda }_{{\max }}}(B)\;\leqslant \;{{\lambda }_{{\min }}}(A)\operatorname{tr} B\;\leqslant \;\operatorname{tr} AB\;\leqslant \;{{\lambda }_{{\max }}}(A)\operatorname{tr} B\;\leqslant \;\operatorname{tr} A\operatorname{tr} B.$$

Lemma A.3 [1]. The solution P of the Lyapunov equation

$$AP + P{{A}^{{\text{T}}}} + Q = 0$$

with a Hurwitz matrix A and Q $\succ$ 0 obeys the bounds

$${{\lambda }_{{\max }}}(P)\; \geqslant \;\frac{{{{\lambda }_{{\min }}}(Q)}}{{2\sigma }},\quad {{\lambda }_{{\min }}}(P)\; \geqslant \;\frac{{{{\lambda }_{{\min }}}(Q)}}{{2\left\| A \right\|}},$$

where σ = $- \mathop {\max }\limits_i \operatorname{Re} {{\lambda }_{i}}(A)$.

If Q = DD^T and the pair (A, D) is controllable, then

$${{\lambda }_{{\max }}}(P)\; \geqslant \;\frac{{{{{\left\| {u{\kern 1pt} *{\kern 1pt} D} \right\|}}^{2}}}}{{2\sigma }} > 0,$$

where

$$u{\kern 1pt} *{\kern 1pt} A = \lambda u{\kern 1pt} *{\kern 1pt} ,\quad \operatorname{Re} \lambda = - \sigma ,\quad \left\| u \right\| = 1,$$

i.e., u is the left eigenvector of the matrix A corresponding to the eigenvalue λ of the matrix A with the greatest real part. The vector u and the number λ can be complex-valued; here, u* denotes the Hermitian conjugate.

APPENDIX B

Proof of Lemma 1. Indeed, if the matrix ${{\mathcal{A}}_{0}}$ + {$\mathcal{A}$, k} is Hurwitz, then σ(${{\mathcal{A}}_{0}}$ + {$\mathcal{A}$, k}) > 0 and there exists the solution P $\succcurlyeq$ 0 of the Lyapunov equation (10) for 0 < α < 2σ(${{\mathcal{A}}_{0}}$ + {$\mathcal{A}$, k}). Thus, the function f(k, α) > 0 is well-defined and f(k) > 0 by Theorem 1. The proof of Lemma 1 is complete.

Proof of Lemma 2. The optimization problem has the form

$$\min f(k,\alpha ),\quad f(k,\alpha ) = \operatorname{tr} P{{\left( {C\;0} \right)}^{{\text{T}}}}\left( {C\;0} \right) + \rho {{\left| k \right|}^{2}}$$

subject to the constraint described by the Lyapunov equation

$$\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k\} + \frac{\alpha }{2}I} \right)P + P{{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}} + \frac{1}{\alpha }\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}^{{\text{T}}}} = 0.$$

To differentiate with respect to k, we add the increment Δk and denote the corresponding increment of P by ΔP:

$$\begin{gathered} \left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k + \Delta k\} + \frac{\alpha }{2}I} \right)(P + \Delta P) \\ + \;(P + \Delta P){{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k + \Delta k\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}} + \frac{1}{\alpha }\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}^{{\text{T}}}} = 0. \\ \end{gathered} $$

Let us apply linearization and subtract this and the previous equations to obtain

$$\begin{gathered} \left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k\} + \frac{\alpha }{2}I} \right)\Delta P + \Delta P{{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}} \\ + \;\{ \mathcal{A},\Delta k\} P + P{{\{ \mathcal{A},\Delta k\} }^{{\text{T}}}} = 0. \\ \end{gathered} $$

(B.1)

The increment of f(k) is calculated by linearizing the corresponding terms:

$$\begin{gathered} \Delta f(k) = \operatorname{tr} (P + \Delta P){{\left( {C\;0} \right)}^{{\text{T}}}}\left( {C\;0} \right) + \rho {{\left| {k + \Delta k} \right|}^{2}} \\ - \;\left( {\operatorname{tr} P{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + \rho {{{\left| k \right|}}^{2}}} \right) = \operatorname{tr} \Delta P{{\left( {C\;0} \right)}^{{\text{T}}}}\left( {C\;0} \right) + 2\rho {{k}^{{\text{T}}}}\Delta k. \\ \end{gathered} $$

Consider Eq. (14), dual to (B.1). Due to Lemma A.1, from Eqs. (B.1) and (14) it follows that

$$\Delta f(k) = 2\operatorname{tr} Y\{ \mathcal{A},\Delta k\} P + 2\rho {{k}^{{\text{T}}}}\Delta k.$$

Thus,

$$df(k) = 2\operatorname{tr} PY\sum\limits_{i = 1}^2 {{{\mathcal{A}}_{i}}d{{k}_{i}}} + 2\rho \sum\limits_{i = 1}^2 {{{k}_{i}}d{{k}_{i}},} $$

which leads to (12).

The validity of (13) is demonstrated by analogy with ([1], Lemma 1). The proof of Lemma 2 is complete.

Proof of Lemma 3. The value $\left( {\nabla _{{kk}}^{2}f(k)e,e} \right)$ is calculated by differentiating ∇_k f(k) in the direction e ∈ ${{\mathbb{R}}^{2}}$. For this purpose, linearizing the corresponding terms and using the convenient notation

$$[\operatorname{tr} PY\mathcal{A}] = \left( \begin{gathered} \operatorname{tr} PY{{\mathcal{A}}_{1}} \\ \operatorname{tr} PY{{\mathcal{A}}_{2}} \\ \end{gathered} \right),$$

we calculate the increment of ∇_k f(k) in the direction e:

$$\frac{1}{2}\Delta {{\nabla }_{k}}f(k)e = \rho (k + \delta e) + [\operatorname{tr} (P + \Delta P)(Y - \Delta Y)\mathcal{A}] - (\rho k + [\operatorname{tr} PY\mathcal{A}])$$

$$ = \rho (k + \delta e) + [\operatorname{tr} (P + \delta P{\kern 1pt} '(k)e)(Y + \delta Y{\kern 1pt} '(k)e)\mathcal{A}] - (\rho k + [\operatorname{tr} PY\mathcal{A}])$$

$$ = \delta (\rho e + [\operatorname{tr} (PY{\kern 1pt} '(k)e + P{\kern 1pt} '(k)eY)\mathcal{A}]),$$

where

$$\Delta P = P(k + \delta e) - P(k) = \delta P{\kern 1pt} '(k)e,$$

$$\Delta Y = Y(k + \delta e) - Y(k) = \delta Y{\kern 1pt} '(k)e.$$

Thus, with P ' = P '(k)e and Y ' = Y '(k)e, we have

$$\frac{1}{2}\left( {\nabla _{{kk}}^{2}f(k)e,\,\,e} \right) = (\rho e + [\operatorname{tr} (PY{\kern 1pt} '\; + P{\kern 1pt} '{\kern 1pt} Y)\mathcal{A}],\,\,e).$$

Furthermore, P = P(k) is the solution of Eq. (17). We write it in increments in the direction e:

$$\begin{gathered} \left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k + \delta e\} + \frac{\alpha }{2}I} \right)(P + \delta P{\kern 1pt} ') \\ + \;(P + \delta P{\kern 1pt} '){{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k + \delta e\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}} + \frac{1}{\alpha }\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}^{{\text{T}}}} = 0 \\ \end{gathered} $$

or

$$\begin{gathered} \left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,k\} + \frac{\alpha }{2}I} \right)(P + \delta P{\kern 1pt} ') + (P + \delta P{\kern 1pt} '){{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,k\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}} \\ + \;\delta \left( {\{ \mathcal{A},\,\,e\} P + P{{{\{ \mathcal{A},\,\,e\} }}^{{\text{T}}}}} \right) + \frac{1}{\alpha }\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}^{{\text{T}}}} = 0. \\ \end{gathered} $$

Subtracting Eq. (17) from this expression gives Eq. (16).

Similarly, Y = Y(k) is the solution of the Lyapunov equation (14). We write it in increments in the direction e:

$$\begin{gathered} {{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k + \delta e\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}}(Y + \delta Y{\kern 1pt} ') \\ + \;(Y + \delta Y{\kern 1pt} ')\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k + \delta e\} + \frac{\alpha }{2}I} \right) + {{\left( {C\;0} \right)}^{{\text{T}}}}\left( {C\;0} \right) = 0 \\ \end{gathered} $$

or

$$\begin{gathered} {{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,k\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}}(Y + \delta Y{\kern 1pt} ') + (Y + \delta Y{\kern 1pt} ')\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,k + \delta e\} + \frac{\alpha }{2}I} \right) \\ + \;\delta \left( {{{{\{ \mathcal{A},\,\,e\} }}^{{\text{T}}}}Y + Y\{ \mathcal{A},e\} } \right) + {{\left( {C\;0} \right)}^{{\text{T}}}}\left( {C\;0} \right) = 0. \\ \end{gathered} $$

Subtracting Eq. (14) from this expression yields

$${{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,k\} + \frac{\alpha }{2}I} \right)}^{{\text{T}}}}Y + Y{\kern 1pt} '\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,k\} + \frac{\alpha }{2}I} \right) + {{\{ \mathcal{A},\,\,e\} }^{{\text{T}}}}Y + Y\{ \mathcal{A},\,\,e\} = 0.$$

(B.2)

From (16) and (B.2) it follows that

$$\operatorname{tr} P{\kern 1pt} '{\kern 1pt} Y\{ \mathcal{A},\,\,e\} = \operatorname{tr} PY{\kern 1pt} '\{ \mathcal{A},\,\,e\} ,$$

so

$$\frac{1}{2}\left( {\nabla _{{kk}}^{2}f(k)e,\,\,e} \right) = \rho (e,\,\,e) + ([\operatorname{tr} (PY{\kern 1pt} '\; + P{\kern 1pt} '{\kern 1pt} Y)\mathcal{A}],\,\,e) = \rho (e,\,\,e) + 2\operatorname{tr} P{\kern 1pt} '{\kern 1pt} Y\{ \mathcal{A},\,\,e\} .$$

The proof of Lemma 3 is complete.

Proof of Lemma 4. Consider a sequence of stabilizing controllers {k_j} ∈ $\mathcal{S}$ such that k_j → k ∈ $\partial \mathcal{S}$, i.e., σ(${{\mathcal{A}}_{0}}$ + {$\mathcal{A}$, k}) = 0. In other words, for any ε > 0 there exists a number N = N(ε) such that

$$\left| {\sigma ({{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} ) - \sigma ({{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,k\} )} \right| = \sigma ({{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} ) < \epsilon $$

for all j $\geqslant$ N($\epsilon$).

Let P_j be the solution of the Lyapunov equation (10) associated with the controller k_j:

$$\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} + \frac{{{{\alpha }_{j}}}}{2}I} \right){{P}_{j}} + {{P}_{j}}{{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} + \frac{{{{\alpha }_{j}}}}{2}I} \right)}^{{\text{T}}}} + \frac{1}{{{{\alpha }_{j}}}}\left[ {\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}}^{{\text{T}}}} + {{\varepsilon }_{2}}I} \right] = 0.$$

Also, let Y_j be the solution of the dual Lyapunov equation

$${{\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} + \frac{{{{\alpha }_{j}}}}{2}I} \right)}^{{\text{T}}}}{{Y}_{j}} + {{Y}_{j}}\left( {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} + \frac{{{{\alpha }_{j}}}}{2}I} \right) + {{\left( {C\;0} \right)}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I = 0.$$

Using Lemma A.3, we have

$$f({{k}_{j}}) = \operatorname{tr} {{P}_{j}}\left( {{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I} \right) + \rho {{\left| {{{k}_{j}}} \right|}^{2}}\; \geqslant \;\operatorname{tr} {{P}_{j}}\left( {{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I} \right)$$

$$ = \operatorname{tr} {{Y}_{j}}\frac{1}{{{{\alpha }_{j}}}}\left[ {\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}}^{{\text{T}}}} + {{\varepsilon }_{2}}I} \right]\; \geqslant \;\frac{1}{{{{\alpha }_{j}}}}{{\lambda }_{{\min }}}({{Y}_{j}})\operatorname{tr} \left[ {\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}}^{{\text{T}}}} + {{\varepsilon }_{2}}I} \right]$$

$$ \geqslant \;\frac{1}{{{{\alpha }_{j}}}}{{\lambda }_{{\min }}}({{Y}_{j}})\left\| {{\kern 1pt} \left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){\kern 1pt} } \right\|_{F}^{2}\; \geqslant \;\frac{1}{{{{\alpha }_{j}}}}\frac{{{{\lambda }_{{\min }}}\left( {{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I} \right)}}{{2\left\| {{{\mathcal{A}}_{0}} + \{ \mathcal{A},{{k}_{j}}\} + \frac{{{{\alpha }_{j}}}}{2}I} \right\|}}\left\| D \right\|_{F}^{2}$$

$$ \geqslant \;\frac{{{{\varepsilon }_{1}}}}{{4\sigma ({{\mathcal{A}}_{0}} + \{ \mathcal{A},{{k}_{j}}\} )\left\| {{{\mathcal{A}}_{0}} + \{ \mathcal{A},{{k}_{j}}\} + \frac{{{{\alpha }_{j}}}}{2}I} \right\|}}\left\| D \right\|_{F}^{2}$$

$$ \geqslant \;\frac{{{{\varepsilon }_{1}}}}{{4\epsilon \left( {\left\| {{{\mathcal{A}}_{0}} + \{ \mathcal{A},{{k}_{j}}\} } \right\| + \epsilon } \right)}}\left\| D \right\|_{F}^{2}\xrightarrow[{\epsilon \to 0}]{} + \infty $$

since

$$0 < {{\alpha }_{j}} < 2\sigma ({{\mathcal{A}}_{0}} + \{ \mathcal{A},{{k}_{j}}\} )$$

and

$$\left\| {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} + \frac{{{{\alpha }_{j}}}}{2}I} \right\|\;\leqslant \;\left\| {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} {\kern 1pt} } \right\| + \frac{{{{\alpha }_{j}}}}{2} < \left\| {{{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} {\kern 1pt} } \right\| + \sigma ({{\mathcal{A}}_{0}} + \{ \mathcal{A},\,\,{{k}_{j}}\} ).$$

On the other hand,

$$f({{k}_{j}}) = \operatorname{tr} {{P}_{j}}\left( {{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I} \right) + \rho {{\left| {{{k}_{j}}} \right|}^{2}}\; \geqslant \;\rho {{\left| {{{k}_{j}}} \right|}^{2}}\xrightarrow[{\left| {{{k}_{j}}} \right| \to + \infty }]{} + \infty .$$

The proof of Lemma 4 is complete.

Proof of Corollary 2. The function f(k) has a minimum point on the set ${{\mathcal{S}}_{0}}$ (as a continuous function on a compact set), but the set ${{\mathcal{S}}_{0}}$ shares no points with the boundary $\mathcal{S}$ due to (18). Finally, the function f(k) is differentiable on ${{\mathcal{S}}_{0}}$ by Lemma 2, which concludes the proof of Corollary 2.

Proof of Lemma 5. Applying Lemma A.2 to (15) gives

$$\frac{1}{2}\left\| {\nabla _{{kk}}^{2}f(k)} \right\| = \frac{1}{2}\mathop {\sup }\limits_{\left| e \right| = 1} \left| {\left( {\nabla _{{kk}}^{2}f(k)e,e} \right)} \right|\;\leqslant \;\mathop {\sup }\limits_{\left| e \right| = 1} \rho (e,e) + 2\mathop {\sup }\limits_{\left| e \right| = 1} \left| {\operatorname{tr} P{\kern 1pt} '{\kern 1pt} Y\{ \mathcal{A},e\} } \right|$$

$$ = \rho + 2\mathop {\sup }\limits_{\left| e \right| = 1} {{\left\| {P{\kern 1pt} '{\kern 1pt} } \right\|}_{F}}{{\left\| {Y\{ \mathcal{A},e\} } \right\|}_{F}}\;\leqslant \;\rho + 2{{\left\| {P{\kern 1pt} '{\kern 1pt} } \right\|}_{F}}\mathop {\sup }\limits_{\left| e \right| = 1} \left\| Y \right\|{{\left\| {\{ \mathcal{A},e\} } \right\|}_{F}}$$

$$\leqslant \;\rho + 2\sqrt 2 {{\left\| {P{\kern 1pt} '{\kern 1pt} } \right\|}_{F}}\left\| Y \right\|\mathop {\max }\limits_i {{\left\| {{{\mathcal{A}}_{i}}} \right\|}_{F}}$$

since

$${{\left\| {\{ \mathcal{A},\,\,e\} } \right\|}_{F}} = {{\left\| {\sum\limits_i {{{\mathcal{A}}_{i}}{{e}_{i}}} } \right\|}_{F}}\;\leqslant \;\sum\limits_i {{{{\left\| {{{\mathcal{A}}_{i}}} \right\|}}_{F}}\left| {{{e}_{i}}} \right|} \;\leqslant \;\mathop {\max }\limits_i {{\left\| {{{\mathcal{A}}_{i}}} \right\|}_{F}}{{\left| e \right|}_{1}}\;\leqslant \;\sqrt 2 \mathop {\max }\limits_i {{\left\| {{{\mathcal{A}}_{i}}} \right\|}_{F}}\left| e \right|.$$

Thus, it is necessary to estimate from above the value

$$\rho + 2\sqrt 2 \mathop {\max }\limits_i {{\left\| {{{\mathcal{A}}_{i}}} \right\|}_{F}}{{\left\| {P{\kern 1pt} '{\kern 1pt} } \right\|}_{F}}\left\| Y \right\|.$$

For ||Y || we have the upper bound

$$\begin{gathered} \frac{{{{\varepsilon }_{2}}}}{\alpha }\left\| Y \right\|\;\leqslant \;\frac{1}{\alpha }{{\lambda }_{{\min }}}\left[ {\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}}^{{\text{T}}}} + {{\varepsilon }_{2}}I} \right]\operatorname{tr} Y\;\leqslant \;\operatorname{tr} Y\frac{1}{\alpha }\left[ {\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}}^{{\text{T}}}} + {{\varepsilon }_{2}}I} \right] \\ = \operatorname{tr} P\left( {{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I} \right) = f(k) - \rho {{\left| k \right|}^{2}}\;\leqslant \;f(k)\;\leqslant \;f({{k}_{0}}) \\ \end{gathered} $$

and consequently,

$$\left\| Y \right\|\;\leqslant \;\frac{\alpha }{{{{\varepsilon }_{2}}}}f({{k}_{0}}).$$

(B.3)

An upper bound for α is established as follows:

$$\alpha < 2\sigma ({{\mathcal{A}}_{0}} + \{ \mathcal{A},k\} )\;\leqslant \;2\left\| {{{\mathcal{A}}_{0}} + \{ \mathcal{A},k\} } \right\|$$

$$\leqslant \;2\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \sum\limits_i {\left\| {{{\mathcal{A}}_{i}}} \right\|\left| {{{k}_{i}}} \right|} } \right)\;\leqslant \;2\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \mathop {\max }\limits_i \left\| {{{\mathcal{A}}_{i}}} \right\|{{{\left| k \right|}}_{1}}} \right)$$

$$\leqslant \;2\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \mathop {\max }\limits_i \left\| {{{\mathcal{A}}_{i}}} \right\|\sqrt 2 \left| k \right|} \right)\;\leqslant \;2\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \mathop {\max }\limits_i \left\| {{{\mathcal{A}}_{i}}} \right\|\sqrt {\frac{2}{\rho }f(k)} } \right)$$

$$\leqslant \;2\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \mathop {\max }\limits_i \left\| {{{\mathcal{A}}_{i}}} \right\|\sqrt {\frac{2}{\rho }f({{k}_{0}})} } \right),$$

so

$$\left\| Y \right\|\;\leqslant \;\frac{2}{{{{\varepsilon }_{2}}}}\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \mathop {\max }\limits_i \left\| {{{\mathcal{A}}_{i}}} \right\|\sqrt {\frac{2}{\rho }f({{k}_{0}})} } \right)f({{k}_{0}}).$$

Now, let us estimate ||P || from above:

$$\begin{gathered} {{\varepsilon }_{1}}\left\| P \right\|\;\leqslant \;{{\lambda }_{{\min }}}\left( {{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I} \right)\left\| P \right\| \hfill \\ \leqslant \;\operatorname{tr} P\left( {{{{\left( {C\;0} \right)}}^{{\text{T}}}}\left( {C\;0} \right) + {{\varepsilon }_{1}}I} \right) = f(k) - \rho {{\left| k \right|}^{2}}\;\leqslant \;f(k)\;\leqslant \;f({{k}_{0}}), \hfill \\ \end{gathered} $$

which yields

$$\left\| P \right\|\;\leqslant \;\frac{{f({{k}_{0}})}}{{{{\varepsilon }_{1}}}}.$$

It remains to estimate from above the value ||P ' ||_F. In view of Lemma A.2,

$${{\lambda }_{{\max }}}\left( {\{ \mathcal{A},\,\,e\} P + P{{{\{ \mathcal{A},\,\,e\} }}^{{\text{T}}}}} \right) = \left\| {\{ \mathcal{A},\,\,e\} P + P{{{\{ \mathcal{A},\,\,e\} }}^{{\text{T}}}}} \right\|\;\leqslant \;\left\| {{{P}^{2}} + \{ \mathcal{A},\,\,e\} {{{\{ \mathcal{A},\,\,e\} }}^{{\text{T}}}}} \right\|$$

$$\leqslant \;{{\left\| P \right\|}^{2}} + {{\left\| {\{ \mathcal{A},\,\,e\} } \right\|}^{2}}\;\leqslant \;\frac{{{{f}^{2}}({{k}_{0}})}}{{\varepsilon _{1}^{2}}} + 2\mathop {\max }\limits_i {{\left\| {{{\mathcal{A}}_{i}}} \right\|}^{2}}\;\leqslant \;\xi \frac{{{{\varepsilon }_{2}}}}{\alpha }\;\leqslant \;\xi \frac{1}{\alpha }{{\lambda }_{{\min }}}\left[ {\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right){{{\left( \begin{gathered} D \\ 0 \\ \end{gathered} \right)}}^{{\text{T}}}} + {{\varepsilon }_{2}}I} \right]$$

for

$$\xi = \frac{\alpha }{{{{\varepsilon }_{2}}}}\left( {\frac{{{{f}^{2}}({{k}_{0}})}}{{\varepsilon _{1}^{2}}} + 2\mathop {\max }\limits_i {{{\left\| {{{\mathcal{A}}_{i}}} \right\|}}^{2}}} \right).$$

Therefore, the solution P ' of the Lyapunov equation (16) satisfies the inequality

$$\begin{gathered} P' \preccurlyeq \xi P \preccurlyeq \frac{\alpha }{{{{\varepsilon }_{2}}}}\left( {\frac{{{{f}^{2}}({{k}_{0}})}}{{\varepsilon _{1}^{2}}} + 2\mathop {\max }\limits_i {{{\left\| {{{\mathcal{A}}_{i}}} \right\|}}^{2}}} \right)\frac{{f({{k}_{0}})}}{{{{\varepsilon }_{1}}}}I \\ \preccurlyeq \frac{{2f({{k}_{0}})}}{{{{\varepsilon }_{1}}{{\varepsilon }_{2}}}}\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \mathop {\max }\limits_i \left\| {{{\mathcal{A}}_{i}}} \right\|\sqrt {\frac{2}{\rho }f({{k}_{0}})} } \right)\left( {\frac{{{{f}^{2}}({{k}_{0}})}}{{\varepsilon _{1}^{2}}} + 2\mathop {\max }\limits_i {{{\left\| {{{\mathcal{A}}_{i}}} \right\|}}^{2}}} \right)I. \\ \end{gathered} $$

Hence, it follows that

$${{\left\| {P{\kern 1pt} '{\kern 1pt} } \right\|}_{F}}\;\leqslant \;\frac{{2\sqrt n f({{k}_{0}})}}{{{{\varepsilon }_{1}}{{\varepsilon }_{2}}}}\left( {\left\| {{{\mathcal{A}}_{0}}} \right\| + \mathop {\max }\limits_i \left\| {{{\mathcal{A}}_{i}}} \right\|\sqrt {\frac{2}{\rho }f({{k}_{0}})} } \right)\left( {\frac{{{{f}^{2}}({{k}_{0}})}}{{\varepsilon _{1}^{2}}} + 2\mathop {\max }\limits_i {{{\left\| {{{\mathcal{A}}_{i}}} \right\|}}^{2}}} \right).$$

(B.4)

Considering the bounds (B.3) and (B.4), we arrive at the relation (19). The proof of Lemma 5 is complete.