A limited-memory BFGS-based differential evolution algorithm for optimal control of nonlinear systems with mixed control variables and probability constraints

Abstract

In this paper, we consider an optimal control problem of nonlinear systems with mixed control variables and probability constraints. To obtain a numerical solution of this optimal control problem, our target is to formulate this problem as a constrained nonlinear parameter optimization problem (CNPOP), which can be solved by using any gradient-based numerical computation method. Firstly, some binary functions are introduced for each value of the discrete-valued control variable (DCV). Following that, we relax these binary functions and impose a penalty term on the relaxation such that the solution of the resulting relaxed problem (RP) can converge to the solution of the original problem as the penalty parameter increases. Secondly, we introduce a simple initial transformation for the probability constraints. Following that, an adaptive sample approximation method (ASAM) and a novel smooth approximation technique (NSAT) are adopted to formulate the probability constraints as some deterministic constraints. Thirdly, a control parameterization approach (CPA) is used to transform the deterministic problem (i.e., an infinite dimensional problem) into a finite dimensional CNPOP. Fourthly, in order to combine the advantages of limited memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithms and differential evolution (DE) algorithms, a L-BFGS-based DE (L-BFGS-DE) algorithm is proposed for solving the resulting approximation problem based on an improvied L-BFGS (IL-BFGS) method and an improved DE (IDE) algorithm. Following that, we establish the convergence result of the L-BFGS-DE algorithm. The L-BFGS-DE algorithm consists of two stages. The objectives of the first and second stages are to obtain a probable position of the global solution and to accelerate the convergence rate, respectively. In the IL-BFGS method, we propose a novel updating rule (NUR), which uses not only the gradient information of the objective function but also the value of the objective function. This will improved the performance of the IL-BFGS method. In the IDE algorithm, a novel adaptive parameter adjustment (NAPA) method, a novel population size decrease (NPSD) strategy, and an improved mutation (IM) scheme are proposed to improve its performance. Finally, an anti-cancer drug therapy problem (ADTP) is further extended to illustrate the effectiveness of the L-BFGS-DE algorithm by taking into account some probability constraints. Numerical results show that the L-BFGS-DE algorithm has good performance and can obtain a stable and robust performance when considering the small noise perturbations in initial state.

Appendix A: Theorems for Section 3.5

For simplicity of notation, let $\tilde u\left (t \right ) = \left [ {{\left (u\left (t \right )\right ]^{\mathrm {T}}}, {\left (z\left (t \right )\right )^{\mathrm {T}}}} \right ]^{\mathrm {T}}$. Then, Problem (3.16) can be written equivalently as follows:

$$\begin{array}{@{}rcl@{}} && \min\limits_{\tilde u\left(t \right)} \quad J_{\text{relax}} = \hat \varphi_{0} \left({x\left({t_{f} } \right)} \right) + {\int}_{t_{0} }^{t_{f} } {\hat L_{0} \left({x\left(t \right),\tilde u\left(t \right)} \right)dt} \end{array}$$

(A.1a)

$$\begin{array}{@{}rcl@{}} && s.t.\quad \dot x\left(t \right) = f\left({x\left(t \right),\tilde u\left(t \right)} \right) \end{array}$$

(A.1b)

$$\begin{array}{@{}rcl@{}} && \quad \quad x\left({t_{0} } \right) = x_{0} \end{array}$$

(A.1c)

$$\begin{array}{@{}rcl@{}} && \quad \quad \bar H_{j} \left({\tilde u\left(t \right)} \right) = \hat \varphi_{j} \left({x\left({t_{f} } \right)} \right) + {\int}_{t_{0} }^{t_{f} } {\hat L_{j} \left({x\left(t \right),\tilde u\left(t \right)} \right)dt} = 0,\quad j = 1, {\cdots} ,M_{2} + 1 \end{array}$$

(A.1d)

where $\tilde u\left (t \right ) \in {\mathscr{A}}$; ${\mathscr{A}} = U \times \left [ {0,1} \right ]^{r}$; $\hat \varphi _{0} \left ({x\left ({t_{f} } \right )} \right ) = \varphi \left ({x\left ({t_{f} } \right )} \right )$; $\hat L_{0} \left ({x\left (t \right ),\tilde u\left (t \right )} \right ) = \bar L\left ({x\left (t \right ),u\left (t \right ), v\left (t \right )} \right )$; $\hat \varphi _{j} \left ({x\left ({t_{f} } \right )} \right ) = 0$, j = 1,⋯,M₂ + 1; $\hat L_{j} \left ({x\left (t \right ),\tilde u\left (t \right )} \right ) = \left ({{\min \limits } \left \{ {\hat p_{j} \left ({u\left (t \right )} \right ) - 1 + \epsilon _{j},0} \right \}} \right )^{2}$, j = 1,⋯,M₂; and $\hat L_{M_{2} + 1} \left ({x\left (t \right ),\tilde u\left (t \right )} \right ) = \left ({\sum \limits _{q = 1}^{r} {z_{q} \left (t \right )} - 1} \right )^{2}$.

Note that the approximate inequality (3.17) and $z_{q} \left (t \right ) = \sum \limits _{i = 1}^{M_{3} } {\theta _{qi} \chi _{i} \left (t \right )}$, q ∈ I₃ can be combined as follows:

$$\tilde u \left(t\right) \approx \tilde u^{M_{3} } \left(t \right) = \sum\limits_{i = 1}^{M_{3} } {\tilde \theta_{i} \chi_{i} \left(t \right)},$$

(A.2)

where $\tilde \theta _{i} = \left [ {\left (b_{i}\right )^{\mathrm {T}} , \left (\theta _{i}\right )^{\mathrm {T}} } \right ]^{\mathrm {T}}$, i = 1,⋯,M₃. Let $\tilde \theta ^{M_{3} } = \left [ {\left (\tilde \theta _{1}\right )^{\mathrm {T}} , {\cdots } , \left (\tilde \theta _{M_{3} }\right )^{\mathrm {T}} } \right ]^{\mathrm {T}}$. Then, Problem (3.18) can be written equivalently as follows:

$$\begin{array}{@{}rcl@{}} && \min\limits_{\tilde \theta^{M_{3} }\left(t\right) } \quad \tilde J = \hat \varphi_{0} \left({x\left({t_{f} } \right)} \right) + {\int}_{t_{0} }^{t_{f} } {\mathscr{L}_{0} \left({x\left(t \right),\tilde \theta^{M_{3} } } \right)dt} \end{array}$$

(A.3a)

$$\begin{array}{@{}rcl@{}} && s.t.\quad \dot x\left(t \right) = \tilde f\left({x\left(t \right),\tilde \theta^{M_{3} } } \right) \end{array}$$

(A.3b)

$$\begin{array}{@{}rcl@{}} && \quad \quad x\left({t_{0} } \right) = x_{0} \end{array}$$

(A.3c)

$$\begin{array}{@{}rcl@{}} && \quad \quad \tilde H_{j} \left({\tilde \theta^{M_{3} } } \right) = \hat \varphi_{j} \left({x\left({t_{f} } \right)} \right) + {\int}_{t_{0} }^{t_{f} } {\mathscr{L}_{j} \left({x\left(t \right),\bar u\left(t \right)} \right)dt} = 0,\quad j = 1, {\cdots} ,M_{2} + 1 \end{array}$$

(A.3d)

where $\tilde \theta ^{M_{3} } \in \hat U$, $\hat U = \underbrace {\tilde U \times {\cdots } \times \tilde U}_{M_{3} } \hfill$, $\tilde U = U \times \left [ {0,1} \right ]^{r}$; $\tilde f\left ({x\left (t \right ),\tilde \theta ^{M_{3} } } \right ) = f\left ({x\left (t \right ),\sum \limits _{i = 1}^{M_{3} } {\tilde \theta _{i} \chi _{i} \left (t \right )} } \right )$; and ${\mathscr{L}}_{j} \left ({x\left (t \right ),\tilde \theta ^{M_{3} } } \right ) = \hat L_{j} \left ({x\left (t \right ),\sum \limits _{i = 1}^{M_{3} } {\tilde \theta _{i} \chi _{i} \left (t \right )} } \right )$, j = 0, 1,⋯ ,M₂ + 1.

Note that Problem (A.1) is a canonical optimal problem described by the Chapter 6 of the work [39]. Then, we can establish four Lemmas as follows:

Lemma A.1

For any $\tilde u\left (t\right ) \in {\mathscr{B}}$, let $\tilde u^{M_{3} } \left (t \right )$ is defined by

$$\tilde u^{M_{3} } \left(t \right) = \sum\limits_{i = 1}^{M_{3} } {\tilde \theta_{i} \chi_{i} \left(t \right)}$$

(A.4)

where

$$\tilde \theta_{i} = \frac{1}{{\rvert {\tau_{i} - \tau_{i - 1} } \rvert}}{\int}_{\tau_{i - 1} }^{\tau_{i} } {\tilde u\left(t \right)dt}.$$

Then, $\tilde u^{M_{3} } \left (t \right )$ converges to $\tilde u\left (t \right )$ a.e. on $\left [t_{0}, t_{f}\right ]$ and $\lim \limits _{M_{3} \to + \infty } {\int \limits }_{t_{0} }^{t_{f} } {\left \| {\tilde u^{M_{3} } \left (t \right ) - \tilde u\left (t \right )} \right \|dt} = 0$, where ${\mathscr{B}}$ is the set of all $\tilde u\left (t\right ) \in {\mathscr{A}}$.

Proof

The proof is similar to that given for Lemma 6.4.1 in [39].

Lemma A.2

Suppose that $\left \{ {\tilde u^{M_{3} } \left (t \right )} \right \}_{M_{3} = 1}^{+ \infty }$ is a bounded function sequence in $L_{\infty }$. Then, the corresponding solution sequence $\left \{ {x\left ({ \cdot \rvert \tilde u^{M_{3} } \left (t \right )} \right )} \right \}_{M_{3} = 1}^{+ \infty }$ of the ODE (A.3b) with the initial condition (A.3c) is also bounded in $L_{\infty }$.

Proof

The proof is similar to that given for Lemma 6.4.2 in [39].

Lemma A.3

Suppose that $\left \{ {\tilde u^{M_{3} } \left (t \right )} \right \}_{M_{3} = 1}^{+ \infty }$ is a bounded function sequence in $L_{\infty }$ and $\tilde u^{M_{3} } \left (t \right )$ converges to $\tilde u\left (t\right )$ a.e. on $\left [t_{0}, t_{f}\right ]$. Then, for any $t \in \left [t_{0}, t_{f}\right ]$, we have

$$\lim\limits_{M_{3} \to + \infty } \left\| {x\left({ t \rvert\tilde u^{M_{3} } \left(t \right)} \right) - x\left({ t \rvert\tilde u\left(t \right)} \right)} \right\| = 0,$$

where ${x\left ({t \rvert \tilde u^{M_{3} } \left (t \right )} \right )}$ is the solution of the ODE (A.3b) with the initial condition (A.3c) and ${x\left ({t \rvert \tilde u\left (t \right )} \right )}$ is the solution of the ODE (A.1b) with the initial condition (A.1c).

Proof

The proof is similar to that given for Lemma 6.4.3 in [39].

Lemma A.4

Suppose that $\left \{ {\tilde u^{M_{3} } \left (t \right )} \right \}_{M_{3} = 1}^{+ \infty }$ is a bounded function sequence in $L_{\infty }$ and $\tilde u^{M_{3} } \left (t \right )$ converges to $\tilde u\left (t\right )$ a.e. on $\left [t_{0}, t_{f}\right ]$. Then, we have

$$\lim\limits_{M_{3} \to + \infty } J_{\text{relax}} \left({\tilde u^{M_{3} } \left(t \right)} \right) = J_{\text{relax}} \left({\tilde u\left(t \right)} \right).$$

Proof

The proof is similar to that given for Lemma 6.4.4 in [39].

Suppose that the following conditions are satisfied:

Assumption A.1

The functions f and $\hat L_{j}$, j = 0, 1,⋯ ,M₂ + 1, and their partial derivatives with respect to each components of $x\left (t\right )$ and $\tilde u\left (t\right )$ are piecewise continuous on $\left [t_{0}, t_{f}\right ]$ for each $\left (x\left (t\right ), \tilde u\left (t\right )\right ) \in \mathbb {R}^{n} \times {\mathscr{R}}^{r}$ and continuous on $\mathbb {R}^{n} \times {\mathscr{R}}^{r}$ for each $t \in \left [t_{0}, t_{f}\right ]$, where ${\mathscr{R}}^{r} = \mathbb {R}^{r} \times \mathbb {R}^{r}$.

Assumption A.2

The functions $\hat \varphi _{j}$, j = 0, 1,⋯ ,M₂ + 1 are continuously differentiable with respect to $x\left (t\right )$.

Let ${\mathscr{F}}^{M_{3}}$ be the set of all $\tilde \theta ^{M_{3} } \in \hat U$. Then, a definition can be introduced as follows:

Definition A.1

$\tilde \theta ^{M_{3} } \in {\mathscr{F}}^{M_{3}}$ is said to be $\tilde \varepsilon$-tolerated feasible, if satisfies the following conditions:

$$- \tilde \varepsilon \leqslant \tilde H_{j} \left({\tilde \theta^{M_{3} } } \right) \leqslant \tilde \varepsilon ,\quad j = 1, {\cdots} ,M_{2} + 1.$$

(A.5)

Let ${\mathscr{B}}^{M_{3}}$ be the subset of ${\mathscr{F}}^{M_{3}}$ such that the equalities described by (A.3d) are satisfied. Furthermore, let ${\mathscr{B}}^{M_{3}, \tilde \varepsilon }$ be the subset of ${\mathscr{F}}^{M_{3}}$ such that the inequalities described by (A.5) are satisfied. Clearly, ${\mathscr{B}}^{M_{3}} \subset {\mathscr{B}}^{M_{3}, \tilde \varepsilon }$.

Then, the $\tilde \varepsilon$-tolerated version of Problem (A.3) can be state as follows:

Choose a $\tilde \theta ^{M_{3} } \in {\mathscr{B}}^{M_{3}, \tilde \varepsilon }$ such that the objective function (A.3a) is minimized subject to the ODE (A.3b) with the initial condition (A.3c).

For convenience, this problem is referred to as Problem ($P_{\tilde \varepsilon }$). Furthermore, the following assumption can be introduced:

Assumption A.3

There exists an integer ${\mathscr{M}}_{0}$ such that

$$\lim\limits_{M_{3} \to + \infty } \tilde J \left({\tilde u^{M_{3} ,\tilde \varepsilon , * } } \right) = \tilde J \left({\tilde u^{M_{3} , * } } \right),$$

uniformly with respect to $M_{3} \ge {\mathscr{M}}_{0}$, where ${\tilde u^{M_{3} ,\tilde \varepsilon , * } }$ and ${\tilde u^{M_{3} , * } }$ are the optimal solutions of Problems ($P_{\tilde \varepsilon }$) and (A.3), respectively.

Now, the following two theorems will be provided to illustrate the relationship between Problems (A.1) and (A.3):

Theorem A.1

Let ${\tilde u^{M_{3} , * } }$ and $\tilde u^{*}$ be the optimal solutions of Problems (A.3) and (A.1), respectively. Then, we have

$$\lim\limits_{M_{3} \to + \infty } J_{\text{relax}} \left({\tilde u^{M_{3} , * } } \right) = J_{\text{relax}} \left({\tilde u^ * } \right).$$

Proof

Let ${\tilde u^{M_{3} ,\tilde \varepsilon , * } }$ be the optimal solution of Problem ($P_{\tilde \varepsilon }$). Then, by using Assumption (A.3), it follows that for any $\mathcal {N} > 0$, there exists a ε₀ > 0 such that

$$\tilde J_{\text{relax}} \left({\tilde u^{M_{3} ,\tilde \varepsilon , * } } \right) > \tilde J_{\text{relax}} \left({\tilde u^{M_{3} , * } } \right) - \mathcal{N}$$

(A.6)

for any $\tilde \varepsilon \in \left (0, \varepsilon _{0}\right )$ and $M_{3} \in \left ({\mathscr{M}}, +\infty \right )$.

Let ${\tilde u^{*,M_{3}} }$ is defined from $\tilde u^{*}$ by using Equality (A.4). Then, for any $\tilde \varepsilon \in \left (0, \varepsilon _{0}\right )$, by using Lemmas A.1, A.3, Assumption A.1, and Assumption A.2, it follows that there exists an integer ${\mathscr{M}}_{1} > {\mathscr{M}}_{0}$ such that ${\tilde u^{*,M_{3}} } \in {\mathscr{B}}^{M_{3},\tilde \varepsilon }$ for all $M_{3} > {\mathscr{M}}_{1}$. This implies that

$$\tilde J_{\text{relax}} \left({\tilde u^{M_{3} ,\tilde \varepsilon , * } } \right) \le \tilde J_{\text{relax}} \left({\tilde u^{*,M_{3} } } \right),$$

(A.7)

for all $M_{3} > {\mathscr{M}}_{1}$.

By using Inequalities (A.6) and (A.7), we have

$$\tilde J_{\text{relax}} \left({\tilde u^{*,M_{3} } } \right) > \tilde J_{\text{relax}} \left({\tilde u^{M_{3} , * } } \right) - \mathcal{N},$$

(A.8)

for all $M_{3} > {\mathscr{M}}_{1}$.

In addition, by using Lemmas A.1 and A.4, it is obtained that

$$\lim\limits_{M_{3} \to + \infty } \tilde J_{\text{relax}} \left({\tilde u^{*,M_{3} } } \right) = \tilde J_{\text{relax}} \left({\tilde u^ * } \right).$$

(A.9)

Then, from Inequality (A.8) and Equality (A.9), we have

$$\mathcal{N} + \tilde J_{\text{relax}} \left({\tilde u^{*} } \right) \ge \lim\limits_{M_{3} \to + \infty } \tilde J_{\text{relax}} \left({\tilde u^{M_{3} , * } } \right).$$

(A.10)

Thus, from Inequality (A.10), it follows that

$$\lim\limits_{M_{3} \to + \infty } \tilde J_{\text{relax}} \left({\tilde u^{M_{3} , * } } \right) = \tilde J_{\text{relax}} \left({\tilde u^{*} } \right),$$

because $\mathcal {N} > 0$ is arbitrary and $\tilde u^{*}$ is the optimal solutions of Problems (A.1).

Theorem A.2

Let ${\tilde u^{M_{3} , * } }$ and $\tilde u^{*}$ be the optimal solutions of Problems (A.3) and (A.1), respectively. Suppose that ${\tilde u^{M_{3} , * } }$ converges to $\hat u$ a.e. on $\left [t_{0}, t_{f}\right ]$. Then, $\hat u$ is also the optimal solution of Problem (A.1).

Proof

Note that ${\tilde u^{M_{3} , * } }$ converges to $\hat u$ a.e. on $\left [t_{0}, t_{f}\right ]$. Then, by using Lemma A.4, it is obtained that

$$\lim\limits_{M_{3} \to + \infty } \tilde J_{\text{relax}} \left({\tilde u^{M_{3} , * } } \right) = \tilde J_{\text{relax}} \left({\hat u} \right).$$

(A.11)

Furthermore, by using Lemma A.3, Assumptions A.1, and A.2, it follows that $\hat u$ is feasible for Problem (A.1). In addition, from Theorem A.1, we can obtain that

$$\lim\limits_{M_{3} \to + \infty } \tilde J_{\text{relax}} \left({\tilde u^{M_{3} , * } } \right) = \tilde J_{\text{relax}} \left({u^ * } \right).$$

(A.12)

Thus, by using Equalities (A.11) and (A.12) together with the feasibility of $\hat u$, it is obtained that $\hat u$ is also the optimal solution of Problem (A.1).