On feedback strengthening of the maximum principle for measure differential equations

Abstract

For a class of nonlinear nonconvex impulsive control problems with states of bounded variation driven by Borel measures, we derive a new type non-local necessary optimality condition, named impulsive feedback maximum principle. This optimality condition is expressed completely within the objects of the impulsive maximum principle (IMP), while employs certain “feedback variations” of impulsive control. The obtained optimality condition is shown to, potentially, discard non-optimal IMP-extrema, and can be viewed as a deterministic non-local iterative algorithm for optimal impulsive control.

Appendices

Appendix A: Feedback control and control synthesis of ordinary dynamical systems

The goal of this appendix is to provide a motivation for the chosen concept of impulsive feedback control. To clarify the idea, we appeal to ordinary control systems.

On a finite time interval ${\mathcal {T}}$, consider a control dynamical system

$$\begin{aligned} \dot{x}(t) = f\big (t,x(t),u(t)\big ), \quad x(0)=x^0, \quad u(t)\in U, \end{aligned}$$

(41)

with given $x^0\in {\mathbb {R}}^n$ and a compact $U\subset {\mathbb {R}}^m$. Control inputs are $u=u(\cdot )\in {\mathcal {U}}\doteq L_\infty ({\mathcal {T}}, U)$, and, under standard regularity assumptions (Lipschitz continuity and sub-linear growth of f w.r.t. x), a trajectory is the unique Carathéodory solution $x=x[u](\cdot )\in {\mathcal {X}}\doteq AC({\mathcal {T}}, {\mathbb {R}}^n)$ of system (41) corresponding to a control $u\in {\mathcal {U}}$.

Given an arbitrary function $w: \ {\mathcal {T}}\times {\mathbb {R}}^n\mapsto U,$ which is said to be feedback control (or closed-loop control, or, simply, feedback) of system (41), we focus on consistent definitions of solution to the closed-looped system

$$\begin{aligned} \dot{x}(t) = f\big (t,x(t),w(t, x(t))\big ), \quad x(0)=x^0. \end{aligned}$$

The Carathéodory solution concept $({\mathcal {C}})$ is still a cornerstone here. This type solution is uniquely defined for feedbacks w which produce vector fields $F(t, x)\doteq f\big (t, x, w(t, x)\big )$ enjoying the well-known Carathéodory properties, and can exist for discontinuous w.r.t. x functions w(t, x). However, one can loose to guarantee the existence of a Carathéodory feedback solution under weaker regularity assumptions. On the other hand, there is a series of alternative relevant solution concepts [1, 7, 10, 22, 23, 33, 37], that make sense in much more general cases, even for an arbitrary feedback control.

In this list, we mark out three concepts, which are of a “constructive” (iterative) feature:

• Euler feedback solution (${\mathcal {E}}$),

• Krasovskii-Subbotin (generalized sampling) feedback solution ($\mathcal {KS}$), and

• Model-predictive feedback solution ($\mathcal {MP}$).

The mentioned solutions are defined as outcomes of certain step-by-step procedures—as uniform limits of sequences of Carathéodory solutions produced by control inputs of a relatively simple structure. The first two types of feedback solution are rather conventional, and we focus on $\mathcal {MP}$ solutions, which is a novel concept.

Definition 3

A control synthesis of system (41) is an arbitrary everywhere defined single-valued mapping $\omega : \, {\mathbb {R}}^n \mapsto {\mathcal {U}}.$ In other words, it is a family of control functions $u_x(\cdot )\in {\mathcal {U}}$, parameterized by state $x\in {\mathbb {R}}^n$.

A measurable in t feedback control w trivially produces a control synthesis by setting $u_x(t)\doteq w(t,x)$ with a further factorization in $L_\infty$.

Given a partition $\pi$ of ${\mathcal {T}}$, and a synthesis $\omega : \, x\mapsto u_x(\cdot )$, a respective $\pi$-polygonal arc is the function $\varsigma _\pi [ \omega ]^S : {\mathcal {T}}\mapsto {\mathbb {R}}^n$ obtained through the following iterations with $x_0\doteq x^0$ and $x_i \doteq \varsigma (t_i)$:

$$\begin{aligned} \varsigma (t)=\varsigma (t_i)+\int _{t_i}^{t}f\big (s, \varsigma (s), u_{x_i}(s)\big )ds, \quad t\in [t_i, t_{i+1}], \quad i={1,\ldots , N-1}. \end{aligned}$$

Definition 4

A function $\varsigma : \ {\mathcal {T}}\mapsto {\mathbb {R}}^n$, being a partial uniform limit of a sequence of $\pi$-polygonal arcs $\varsigma _\pi [ \omega ]^S$, as $\mathrm{diam}(\pi ) \rightarrow 0$, is said to be a model predictive feedback solution associated with synthesis $\omega$.

Proposition 6

Assume that f satisfies standard assumptions about the Lipschitz continuity and sublinear growth. Then, for any synthesis $\omega : \, x \mapsto u_x$, there exists at least one $\mathcal {MP}$ solution of closed-looped system (41), and this solution is absolutely continuous.

Proof

Let $\pi _k=\{t_i^k\}$, $k =1, 2, \ldots$, be partitions of the interval ${\mathcal {T}}$ such that $\mathrm{diam}(\pi _k)\le \varepsilon _k \rightarrow 0$ as $k\rightarrow \infty$, $\varsigma _k=\varsigma _{\pi _k} [ \omega ]^S$—the respective $\pi _k$-polygonal arcs, and $u_k$—the concatenation of functions $u_{\varsigma _{k}(t_i^k)}$, defined though the above step-by-step procedure (note that $u_k$ is an admissible open loop control and $\varsigma _k=\varsigma [u_k]$).

Standard arguments based on the Gronwall’s inequality ensure that the tube of Carathéodory solutions to system (41) is bounded, which implies that the family $\{\varsigma _k\}$ is equi-bounded. Furthermore, for any $\varsigma _k$, the compositions $f\big (t, \varsigma _k(t), u_k(t)\big )$ are also uniformly bounded, which implies that solutions $\varsigma _k$ are uniformly Lipschitzian on ${\mathcal {T}}$, and hence, the family $\{\varsigma _k\}$ is equicontinuous. By the Arzelá-Ascoli selection principle, there is a uniformly converging subsequence of $\{\varsigma _k\}$, whose limit $\varsigma$ is an $\mathcal {MP}$ solution, by definition. The absolute continuity of $\varsigma$ follows from [4, Theorem 4]. $\square$

It is important to note that the raised solution concepts are pairwise different, i.e., the respective sets of solutions are not, in general, proper subsets one of another: to ensure the difference between ${\mathcal {C}}$, ${\mathcal {E}}$, and $\mathcal {KS}$, we refer to [10] and the bibliography therein, where relevant counter-examples are collected, and perform a comparison of the concepts $\mathcal {KS}$ and $\mathcal {MP}$. The following two simplest examples show that, in fact, $\mathcal {KS} \nsubseteq \mathcal {MP}$, and $\mathcal {MP} \nsubseteq \mathcal {KS}$.

Example A.1

Consider the dynamic system:

$$\begin{aligned} \dot{x} =u, \quad x(0)=0, \quad u\in [0,1], \quad t\in [0,1]. \end{aligned}$$

The feedback control, defined formally as $w(t, x)=v(t)$, $t\in [0,1]$, $x\in {\mathbb {R}}$, with

$$\begin{aligned} v(t)=\left\{ \!\begin{array}{ll} 0, &{}\quad \text{ if } t \text{ is } \text{ irrational },\\ 1, &{}\quad \text{ otherwise, }\end{array}\right. \end{aligned}$$

produces a unique natural Carathéodory solution $x\equiv 0$, and a continuum of $\mathcal {KS}$ solutions, in particular, $x=t$ (corresponding to partitions by rational numbers). On the other hand, the respective control synthesis $x\mapsto u_x$ gives a single $\mathcal {MP}$-solution $x\equiv 0$, which coincides with the Carathéodory one. Thus, $\mathcal {KS} \nsubseteq \mathcal {MP}$.

Note that the addressed “feedback” is, obviously, a representative of the measurable control $u\equiv 0$. At the same time, sampling of the respective solution $x\equiv 0$ by the $\mathcal {KS}$ or ${\mathcal {E}}$ algorithms leads to very different solutions, while the $\mathcal {MP}$ scheme naturally weeds out such pathological cases.

Example A.2

Consider the system

$$\begin{aligned} \dot{x} =1+u, \quad x(0)=0, \quad u\in [0,1], \quad t\in [0,1]. \end{aligned}$$

The feedback control

$$\begin{aligned} w(t,x)=\left\{ \!\begin{array}{ll} 1, &{}\quad \text{ if } t \text{ is } \text{ irrational } \text{ and } x \text{ is } \text{ rational },\\ 0, &{}\quad \text{ otherwise, }\end{array}\right. \end{aligned}$$

generates a single $\mathcal {KS}$-solution $x=t$, while the respective control synthesis $\omega : \ u_x(t)=w(t,x)$ produces a continuum of $\mathcal {MP}$ solutions, in particular, $x=2t$. Thus, $\mathcal {MP} \nsubseteq \mathcal {KS}$.

Appendix B: Feedback necessary optimality conditions: how feedbacks serve to discard non-optimal extrema

This appendix is assumed to give the reader some intuition about the paradigm of feedback necessary optimality conditions, in particular, to answer the question: “Why one can expect that feedback controls, used in the feedback maximum principle, serve to “filter out” non-optimal processes, in particular, non-optimal extrema?”.

The idea is, in fact, quite simple; it comes back to the technique of the modified Lagrangian with a linear (in state variable) weakly monotone function [16,17,18,19], and refers to the principle of extremal aiming [12, 37]: Given a reference (examined) process ${\bar{\gamma }}=({\bar{y}},{\bar{v}})$, ${\bar{y}}\doteq (\bar{{\mathbf {t}}},\bar{{\mathbf {x}}})$, we try to construct another process ${\hat{\gamma }}$ of a lower cost (this would mean that ${\bar{\gamma }}$ is not optimal, and, hence, is discarded by ${\hat{\gamma }}$). Fixed $\xi$ and ${\bar{\phi }}$, introduce the function

$$\begin{aligned} \eta (s, y)\doteq \eta (s, {\mathbf {t}}, {\mathbf {x}})= \xi \big (\mathbf t-\bar{{\mathbf {t}}}(s)\big )+\langle {\bar{\phi }}(s), {\mathbf {x}}- \bar{{{\mathbf {x}}}}(s) \rangle \end{aligned}$$

and define

$$\begin{aligned} {\bar{\eta }} (s,y)=\eta (s,y)+\int \limits _s^S \inf \limits _{y\in O_r} \left[ \max \limits _{|v|\le 1} \frac{d\eta }{d\tau }(\tau ,y,v) \right] d\tau , \end{aligned}$$

where

$$\begin{aligned} \displaystyle \frac{d\eta }{ds}(s,y, v)\doteq \eta _s+H\big ({\mathbf {x}}, \eta _{{\mathbf {x}}}(s),\eta _{{\mathbf {t}}},v\big ) \end{aligned}$$

(42)

is a total derivative of function $\eta$ w.r.t. system (15), and $O_r$ is an outer estimate of the trajectory tube (the existence of such an outer estimate follows from the uniform boundedness, thanks to the sublinear growth assumption in (H)).

One can discover the following properties of function ${\bar{\eta }}$:

• 1) ${\bar{\eta }}$ is weakly increasing w.r.t. dynamical system [12], i.e., for any initial data $(s_*,y_*)$, there is at least one trajectory y of (15), (14) with $y(s_*)=y_*$ such that the composition ${\bar{\eta }}\big (s,y(s)\big )$ does not decrease on $[s_*, S]$. This property is equivalent to the following Hamilton-Jacobi inequality:

  $$\begin{aligned} \max \limits _{|v|\le 1} \frac{d{\bar{\eta }}}{ds}(s,y,v)\ge 0,\quad \forall \, \big (s,y\big )\in {\mathcal {S}}\times {\mathcal {T}}\times {\mathbb {R}}^n. \end{aligned}$$

• 2) Given an admissible process $\gamma =(y, v)$, $y=(\mathbf t,{\mathbf {x}})$, one has:

  $$\begin{aligned} \begin{aligned} {\mathcal {I}}(\gamma )-{\mathcal {I}}({\bar{\gamma }})&=\langle c, {{\mathbf {x}}}(S)\rangle - \langle c,\bar{{\mathbf {x}}}(S)\rangle \\&= -{\bar{\eta }} \big (S,y(S)\big )= -\displaystyle \int \limits _0^S \frac{d{\bar{\eta }}}{ds}\big (s,y(s),v(s)\big )ds-{\bar{\eta }}(0,y_0). \end{aligned} \end{aligned}$$

  (43)

From the exact increment formula (43) it is clear that, in order to minimize the cost ${\mathcal {I}}$, one should search for a state trajectory $s \mapsto y(s)$, which ensures the “fastest increasing” of ${\bar{\eta }} (s, y)$. In view of (42), this leads to the following finite-dimensional optimization problem:

$$\begin{aligned} H\big ({\mathbf {x}}, {\bar{\phi }}(s),\xi ,v\big )\rightarrow \max \limits _{|v|\le 1},\quad \big (s,y=({\mathbf {t}}, {\mathbf {x}})\big ) \in {{\mathcal {S}}} \times {\mathcal {T}}\times {\mathbb {R}}^n, \end{aligned}$$

whose formal solutions are feedback controls of the structure

$$\begin{aligned} w=w_\xi (s,{\mathbf {x}})\in W_\xi \big ({\mathbf {x}}, {\bar{\phi }}(s)\big ), \end{aligned}$$

arising in Sect. 4. The associated feedback polygonal arcs and piecewise open-loop (piecewise constant) controls are, thus, the desired candidates on the role of discarding process ${\hat{\gamma }}$.