Data-Driven Stabilization of Nonlinear Systems via Taylor’s Expansion

Abstract

Lyapunov’s indirect method is one of the oldest and most popular approaches to model-based controller design for nonlinear systems. When the explicit model of the nonlinear system is unavailable for designing such a linear controller, finite-length off-line data is used to obtain a data-based representation of the closed-loop system, and a data-driven linear control law is designed to render the considered equilibrium locally asymptotically stable. This work presents a systematic approach for data-driven linear stabilizer design for continuous-time and discrete-time general nonlinear systems. Moreover, under mild conditions on the nonlinear dynamics, we show that the region of attraction of the resulting locally asymptotically stable closed-loop system can be estimated using data.

Appendix

1.1 Petersen’s Lemma

In the section for data-driven controller design, Petersen’s lemma is essential for deriving the sufficient condition characterizing the controller. Due to the space limit, the proof of the lemma is omitted and one may refer to works such as [4, 25, 26] for more details.

Lemma 12.8.1

(Petersen’ s lemma [25]) Consider matrices $\mathcal G=\mathcal G^{\top }\in \mathbb {R}^{n\times n}$, $\mathcal M\in \mathbb {R}^{n\times m}$, $\mathcal M\ne 0$, $\mathcal N \in \mathbb {R}^{p\times n}$, $\mathcal N \ne 0$, and a set F defined as

$$\begin{aligned} F =\{\mathcal F\in \mathbb {R}^{m\times p}: \mathcal F^{\top } \mathcal F\preceq \overline{\mathcal F} \}, \end{aligned}$$

where $\overline{\mathcal F}=\overline{\mathcal F}^{\top }\succeq 0$. Then, for all $\mathcal F\in F$,

$$\begin{aligned} \mathcal G + \mathcal M \mathcal F \mathcal N + \mathcal N^{\top } \mathcal F^{\top } \mathcal M^{\top } \preceq 0 \end{aligned}$$

if and only if there exists $\mu >0$ such that

$$\begin{aligned} \mathcal G + \mu \mathcal M \mathcal M^{\top } +\mu ^{-1}\mathcal N^{\top }\overline{\mathcal F}\mathcal N \preceq 0. \end{aligned}$$

1.2 Proof of Lemma 12.5.3

Let $z=\begin{bmatrix} x^{\top } & u^{\top } \end{bmatrix}^{\top }$. Each element $f_i(z)$, $i=1,\dots ,n$ in f(z) can be written as

$$\begin{aligned} f_i(z)= \sum ^{n+m}_{j=1}\frac{\partial f_i}{\partial z_j}(0)z_j+R_i(z). \end{aligned}$$

On the other hand, as shown in [12], the function $f_i(z)$ can be expressed as

$$\begin{aligned} f_i(z)=\sum ^{n+m}_{j=1}z_j\int _{0}^{1} \frac{\partial f_i}{\partial z_j}(tz)dt. \end{aligned}$$

As a consequence, one can write $R_i(z)$ as

$$\begin{aligned} R_i(z) & =& \sum _{j=1}^{n+m}z_j \int ^1_0 \frac{\partial f_i}{\partial z_j}(tz) dt - \sum ^{n+m}_{j=1}\frac{\partial f_i}{\partial z_j}(0)z_j\\ & =& \sum _{j=1}^{n+m}z_j \int ^1_0 \left( \frac{\partial f_i}{\partial z_j}(tz) - \frac{\partial f_i}{\partial z_j}(0) \right) dt. \end{aligned}$$

Under Assumption 12.5.2, one has

$$\begin{aligned} \left| \frac{\partial f_i}{\partial z_j}(tz) - \frac{\partial f_i}{\partial z_j}(0) \right| \le L_i\Vert tz\Vert ,~~t\in (0,1). \end{aligned}$$

Then, it holds that

$$\begin{aligned} \left| R_i(z) \right| &\le & \sum _{j=1}^{n+m}|z_j|\int ^1_0 L_i\Vert z\Vert \cdot |t|dt \\ & =& L_i\Vert z\Vert \int ^1_0 |t|dt \cdot \sum _{j=1}^{n+m}|z_j| . \end{aligned}$$

By the fact that $\int ^1_0 |t|dt=\frac{1}{2}$ and $|z_1+\cdots z_{n+m}|\le \sqrt{n+m}\Vert z\Vert$, it holds that

$$\begin{aligned} \left| R_i(z) \right| \le \frac{\sqrt{n+m} L_i}{2}\Vert z\Vert ^2. \end{aligned}$$

The proof is complete.

1.3 Proof of Lemma 12.5.6

For the closed-loop system with the controller $u=Kx$ designed via Theorem 12.4.1, the derivative of the Lyapunov function $V(x)=x^{\top }P^{-1}x$ satisfies

$$\begin{aligned} \dot{V}(x) = & {} x^{\top }P^{-1}(A+BK)x + x^{\top }(A+BK)^{\top }P^{-1}x+2x^{\top }P^{-1}R(x,Kx) \\ &\le & -wx^{\top }P^{-1}x +2x^{\top }P^{-1}R(x,Kx). \end{aligned}$$

Under Assumption 12.5.2, for all $x\in \mathbb D$ and $i=1,\dots ,n$, the bounds on the approximation error can be found as

$$\begin{aligned} |R_i(x,Kx)| \le \frac{\sqrt{m+n} L_i}{2} \Vert (x,Kx)\Vert ^{2}. \end{aligned}$$

Hence, for all $x\in \mathbb D$, there exists a continuous $\rho _i(x)$ for each $i=1,\dots ,n$ such that for each $x\in \mathbb D$

$$\begin{aligned} R_i(x,Kx) = & {} \rho _i(x) \Vert (x,Kx)\Vert ^{2}, \\ \rho _i(x)&\in & \left[ -\frac{\sqrt{m+n} L_i}{2}, \frac{\sqrt{m+n} L_i}{2}\right] . \end{aligned}$$

Define $\rho (x) = [\rho _1(x)~\dots ~\rho _n(x)]^{\top }$. By the definition of polytope [5, Definition 3.21], the vector $\rho (x)$ belongs to the polytope

$$\begin{aligned} \mathcal H= \{ \varrho : -\bar{h} \preceq \varrho \preceq \bar{h} \}, \end{aligned}$$

where

$$\begin{aligned} \bar{h}=[\bar{h}_1~ \cdots ~\bar{h}_n]^{\top } = \begin{bmatrix} \frac{\sqrt{m+n} L_1}{2} &\cdots & \frac{\sqrt{m+n} L_n}{2} \end{bmatrix}^{\top }. \end{aligned}$$

Denote $Q_i$ as the ith column of $P^{-1}$. It holds that

$$\begin{aligned} {} & {} 2x^{\top }P^{-1}R(x,Kx) \\ = & {} 2\begin{bmatrix} x^{\top }Q_1 & \cdots & x^{\top }Q_n \end{bmatrix} \begin{bmatrix} \rho _1(x)\Vert (x,Kx)\Vert ^{2} \\ \vdots \\ \rho _n(x)\Vert (x,Kx)\Vert ^{2} \end{bmatrix}\\ = & {} 2\sum ^n_{i=1} x^{\top }Q_i\rho _i(x)\Vert (x,Kx)\Vert ^{2}\\ = & {} 2\sum ^n_{i=1} x^{\top }Q_i\Vert (x,Kx)\Vert ^{2}\cdot \rho _i(x)\\ & =& 2\begin{bmatrix} x^{\top }Q_1\Vert (x,Kx)\Vert ^{2} & \cdots & x^{\top }Q_n\Vert (x,Kx)\Vert ^{2} \end{bmatrix}\rho (x). \end{aligned}$$

Denote

$$\begin{aligned} \kappa (x)=\begin{bmatrix} x^{\top }Q_1\Vert (x,Kx)\Vert ^{2} & \cdots & x^{\top }Q_n\Vert (x,Kx)\Vert ^{2} \end{bmatrix}. \end{aligned}$$

Then, the derivative of the Lyapunov function satisfies for all $x\in \mathbb D$

$$\begin{aligned} \dot{V}(x) \le -wx^{\top }P^{-1}x + 2\kappa (x)\rho (x), \end{aligned}$$

where $\rho (x)\in \mathcal H$.

1.4 Sum of Squares Relaxation

As solving positive conditions of multivariable polynomials is in general NP-hard, the SOS relaxations are often used to obtain sufficient conditions that are tractable. The SOS polynomial matrices are defined as follows.

Definition 12.8.2

(SOS polynomial matrix [8]) $M:\mathbb {R}^n\rightarrow \mathbb {R}^{\sigma \times \sigma }$ is an SOS polynomial matrix if there exist $M_{1},\dots ,M_{k}:\mathbb {R}^n\rightarrow \mathbb {R}^{\sigma \times \sigma }$ such that

$$\begin{aligned} M(x)=\sum ^{k}_{i=1}M_{i}(x)^{\top }M_{i}(x)~~\forall x\in \mathbb {R}^n. \end{aligned}$$

(12.28)

Note that when $\sigma =1$, M(x) becomes a scalar SOS polynomial.

It is straightforward to see that if a matrix M(x) is an SOS polynomial matrix, then it is positive semi-definite, i.e., $M(x)\succeq 0~\forall x\in \mathbb {R}^n$. Relaxing the positive polynomial conditions into SOS polynomial conditions makes the conditions tractable and easily solvable by common software.

1.5 Positivstellensatz

In the RoA analysis, we need to characterize polynomials that are positive on a semialgebraic set, and the Positivstellensatz plays an important role in this characterization.

Let $p_1,\dots ,p_k$ be polynomials. The multiplicative monoid, denoted by $\mathcal S_M(p_1,\dots ,p_k)$, is the set generated by taking finite products of the polynomials $p_1,\dots ,p_k$. The cone $\mathcal S_C(p_1,\dots ,p_k)$ generated by the polynomials is defined as

$$\begin{aligned} {} & {} \mathcal S_C(p_1,\dots ,p_k)\\ & =&\{ s_0 +\sum _{i=1}^j s_iq_i:s_0,\dots ,s_j \text { are SOS polynomials}, q_1,.\dots ,q_j\in \mathcal S_M(p_1,\dots ,p_k) \}. \end{aligned}$$

The ideal $\mathcal S_I(p_1,\dots ,p_k)$ generated by the polynomials is defined as

$$\begin{aligned} \mathcal S_I(p_1,\dots ,p_k)= \left\{ \sum _{i=1}^k r_ip_i: r_1,\dots ,r_k \text { are polynomials} \right\} . \end{aligned}$$

Stengle’s Positivstellensatz [28] is presented as follows in [8].

Theorem 12.8.3

(Positivstellensatz) Let $f_1,\dots ,f_k$, $g_1,\dots ,g_l$, and $h_1,\dots ,h_m$ be polynomials. Define the set

$$\begin{aligned} \mathcal X = \{x\in \mathbb {R}^n: {} & {} f_1(x)\ge 0, \dots , f_k(x)\ge 0, \\ {} & {} g_1(x)= 0, \dots , g_l(x)= 0,\\ {} & {} \text {and } h_1(x)\ne 0, \dots , h_m(x)\ne 0 \}. \end{aligned}$$

Then, $\mathcal X = \emptyset$ if and only if

$$\begin{aligned} \exists f \in \mathcal S_C(f_1,\dots ,f_k),~~g \in \mathcal S_I(g_1,\dots ,g_l),~~h \in \mathcal S_M(h_1,\dots ,h_m) \end{aligned}$$

such that

$$\begin{aligned} f(x)+g(x)+h(x)^2=0. \end{aligned}$$

For the subsequent RoA analysis, we will use the following result derived from the Positivstellensatz.

Lemma 12.8.4

Let $\varphi _1$ and $\varphi _2$ be polynomials in x. If there exist SOS polynomials $s_1$ and $s_2$ in x such that

$$\begin{aligned} -(s_1\varphi _1(x)+s_2\varphi _2(x) +x^{{{\tiny \mathsf T}}}x) \text { is SOS }~\forall x\in \mathbb {R}^n \end{aligned}$$

(12.29)

then the set inclusion condition

$$\begin{aligned} \{x\in \mathbb {R}^n: \varphi _1(x)\ge 0, x\ne 0\} \subseteq \{ x\in \mathbb {R}^n: \varphi _2(x)<0 \} \end{aligned}$$

(12.30)

holds.

Proof

The set inclusion condition (12.30) can be equivalently written as

$$\begin{aligned} \{ x\in \mathbb {R}^n: \varphi _1(x)\ge 0, \varphi _2(x)\ge 0, x\ne 0\} = \emptyset . \end{aligned}$$

By Theorem 12.8.3, we know that this is true if and only if there exist $\varphi (x)\in \mathcal S_C(\varphi _1,\varphi _2)$ and $\zeta (x)\in \mathcal S_M(x)$, such that

$$\begin{aligned} \varphi (x) + \zeta (x)^2=0. \end{aligned}$$

(12.31)

Let

$$\begin{aligned} \varphi =s_0+s_1\varphi _1+s_2\varphi _2, \end{aligned}$$

where $s_j$, $j=0,1,2$ are SOS polynomials. By the definition of the cone $\mathcal S_C$, one has that $\varphi \in \mathcal S_C(\varphi _1,\varphi _2)$. Choosing $\zeta (x)^2=x^{{{\tiny \mathsf T}}}x$, we write the condition (12.31) as

$$\begin{aligned} s_0+s_1\varphi _1+s_2\varphi _2 +x^{{{\tiny \mathsf T}}}x=0. \end{aligned}$$

(12.32)

As $s_0=-(s_1\varphi _1+s_2\varphi _2 +x^{{{\tiny \mathsf T}}}x)$ from (12.32), if there exist SOS polynomials $s_1$ and $s_2$ such that the SOS condition (12.29) holds, then there exist SOS polynomials $s_j$, $j=0,1,2$ such that (12.32) is true, and hence the set inclusion condition (12.30) holds.    $\square$

1.6 Proof of Lemma 12.5.8

For the closed-loop system with the controller $u=Kx$ designed via Theorem 12.4.2, the difference between the Lyapunov functions $V(x^{+})=(x^{+})^{\top }P^{-1}x^{+}$ and $V(x)=x^{\top }P^{-1}x$ is

$$\begin{aligned} {} & {} V(x^{+})-V(x) \\ = & {} \left[ (A+BK)x+R(x,Kx) \right] ^{\top } P^{-1} [ (A+BK)x +R(x,Kx) ]-x^{\top } P^{-1} x \\ = & {} x^{\top } \left[ (A+BK)^{\top }P^{-1} (A+BK)-P^{-1} \right] x +2R(x,Kx)^{\top }P^{-1}(A+BK)x \\ {} & {} +R(x,Kx)^{\top } P^{-1}R(x,Kx). \end{aligned}$$

Observe that

$$\begin{aligned} A+BK = \left( -\overline{{\textbf {A}}}^{-1}\overline{{\textbf {B}}} + \overline{{\textbf {A}}}^{-1/2}\Delta \right) ^{\top } \begin{bmatrix} K\\ I \end{bmatrix} \end{aligned}$$

with $\Delta \Delta ^{\top }\preceq \delta I$. Then, it holds that

$$\begin{aligned} \left\| P^{-1}(A+BK) \right\| \le \left\| - P^{-1}\overline{{\textbf {B}}}^{\top }\overline{{\textbf {A}}}^{-1} \begin{bmatrix} K\\ I \end{bmatrix} \right\| + \sqrt{\delta } \left\| P^{-1} \right\| \left\| \overline{{\textbf {A}}}^{-1/2} \begin{bmatrix} K\\ I \end{bmatrix} \right\| =r_1 . \end{aligned}$$

For any $\varepsilon >0$, it holds that

$$\begin{aligned} {} & {} 2R(x,Kx)^{\top }P^{-1}(A+BK)x\\ &\le & \varepsilon R(x,Kx)^{\top }R(x,Kx) +\varepsilon ^{-1} \left\| P^{-1}(A+BK) \right\| ^2 x^{\top }x\\ &\le &\varepsilon \Vert R(x,Kx)\Vert ^2 + \varepsilon ^{-1}r_1^2x^{\top }x. \end{aligned}$$

Recall that, by Theorem 12.4.2, $(A+BK)^{\top }P^{-1} (A+BK)-P^{-1}\preceq -w P^{-1}$. Hence, one has that

$$\begin{aligned} V(x^{+})-V(x) = -x^{\top }\left( w P^{-1} - \varepsilon ^{-1} r_1^2I \right) x + \left( \varepsilon + \Vert P^{-1}\Vert \right) \Vert R(x,Kx)\Vert ^2. \end{aligned}$$

Under Assumption 12.5.2,

$$\begin{aligned} \Vert R(x,Kx)\Vert ^2= \sum ^n_{i=1} R_i(x,Kx)^2 \le \sum ^n_{i=1}\frac{(m+n)L_i^2}{4} \Vert (x,Kx)\Vert ^4. \end{aligned}$$

If we write $\Vert R(x,Kx)\Vert ^2$ as

$$\begin{aligned} \Vert R(x,Kx)\Vert ^2 = \begin{bmatrix} \Vert (x,Kx)\Vert ^{4} & \cdots & \Vert (x,Kx)\Vert ^{4} \end{bmatrix} \begin{bmatrix} \widehat{\rho }_1(x) \\ \vdots \\ \widehat{\rho }_n(x) \end{bmatrix}, \end{aligned}$$

then the scalars $\widehat{\rho }_i(x)$ are such that $\widehat{\rho }_i(x)\in \left[ 0,\frac{(m+n)L_i^2}{4} \right]$, $i=1,\dots ,n$ for all $x\in \mathbb D$. Defining

$$\begin{aligned} \widehat{\kappa }(x) = \begin{bmatrix} \Vert (x,Kx)\Vert ^{4} & \cdots & \Vert (x,Kx)\Vert ^{4} \end{bmatrix} ~~ \text { and }~~ \widehat{\rho }(x) = \begin{bmatrix} \widehat{\rho }_1(x) & \cdots & \widehat{\rho }_n(x) \end{bmatrix}^{\top } \end{aligned}$$

gives $\Vert R(x,Kx)\Vert ^2 =\widehat{\kappa }(x)\widehat{\rho }(x)$, and for any $x\in \mathbb D$ the vector $\widehat{\rho }(x)$ is contained in the polytope $\widehat{\mathcal H}$ defined in (12.26).

1.7 Dynamics Used for Data Generation in the Example

The dynamics used for data generation in Sect. 12.6 is the inverted pendulum written as

$$\begin{aligned} \dot{x}_1 &= x_2, \nonumber \\ \dot{x}_2 &= \frac{mgl}{J}\textrm{sin}(x_1) -\frac{r}{J}x_2 +\frac{l}{J}\textrm{cos}(x_1) u, \end{aligned}$$

(12.33)

where $m=0.1$, $g=9.8$, $r=l=J=1$. For the discrete-time case, consider the Euler discretization of the inverted pendulum, i.e.,

$$\begin{aligned} x_1^{+} = & {} x_1 + T_sx_2,\\ x_2^{+} = & {} \frac{T_s g}{l} \sin (x_1) +\left( 1- \frac{T_s r}{ml^2} \right) x_2 + \frac{T_s}{ml^2} \cos (x_1) u, \end{aligned}$$

where $m=0.1$, $g=9.8$, $T_s=0.1$, $l=1$, and $r=0.01$.