Constrained decision-making for low-count radiation detection by mobile sensors

Abstract

This paper approaches from an optimal control perspective the problem of fixed-time detection of mobile radioactive sources in transit by means of a collection of mobile sensors. Under simplifying assumptions on the motion and geometry of the source, the sensors, and the surrounding environment, the optimal control problem admits an intuitive, analytic closed-form solution. This solution is obtained thanks to analytic expressions for bounds on the probabilities of detection and false alarm for a Neyman–Pearson detection test. The intuition derived from this analytic solution supports the development of a motion control law that steers (suboptimally) the sensors to a given neighborhood of the suspected source, while navigating among stationary obstacles in their environment. This motion controller closes the loop at the acceleration level of a heterogeneous collection of sensor platforms. Experimental studies with these platforms corroborate the theoretical convergence results.

Appendix

To simplify notation, we drop the subscript $x_i$ from the expressions of the gradient and hessian of the navigation, with the understanding that all these differentiations are with respect to $x_i$. Similarly, instead of distinguishing the obstacle and navigation function of agent i by writing $\beta _i$ and $\varphi _i$, we simply refer to it generically as $\beta$ and $\varphi$. We will use the index i to range over obstacles in the environment.

With a slight abuse of notation, we will now think of the free workspace $\mathcal {P}$ as a subset of $\mathbb {R}^n$ (instead of just $\mathbb {R}^3$); the results established in this section hold irrespectively of the particular value of $n \in \mathbb {N}_+$. Let $\partial \mathcal {S}_T = \{x \in \mathbb {R}^n: \Vert x-x_t\Vert =r_t\}$, for a small $\epsilon >0$ $\mathcal {B}_i(\epsilon ) \triangleq \{x \in \mathbb {R}^n: 0<\beta _i(x)<\epsilon )$, and (re)define the decomposition of $\overline{\mathcal {P}}$ into sets $\partial \mathcal {F}$, $\mathcal {F}_0(\epsilon )$, $\mathcal {F}_1(\epsilon )$, $\mathcal {F}_2(\epsilon )$ and $\mathcal {W}(\epsilon )$ as follows.

• 1. the free space boundary

  $$\begin{aligned} \partial \mathcal {F} = \partial \mathcal {P} = \beta ^{-1}(0); \end{aligned}$$

• 2. the set “near the obstacles”

  $$\begin{aligned} \mathcal {F}_0(\epsilon ) \triangleq \bigcup _{i=1}^M\mathcal {B}_i(\epsilon ) \, {\setminus } \, \partial \mathcal {S}_T ; \end{aligned}$$

• 3. the set “near the workspace boundary”

  $$\begin{aligned} \mathcal {F}_1(\epsilon ) \triangleq \mathcal {B}_0(\epsilon ) \, {\setminus } \, \left( \partial \mathcal {S}_T \cup \mathcal {F}_0(\epsilon )\right) ; \end{aligned}$$

• 4. the set “away from the obstacles”

  $$\begin{aligned} \mathcal {F}_2(\epsilon ) \triangleq \mathcal {P} \, {\setminus } \, \left( \partial \mathcal {S}_T \cup \partial \mathcal {F} \cup \mathcal {F}_0(\epsilon ) \cup \mathcal {F}_1(\epsilon )\right) . \end{aligned}$$

• 5. the set “away from the obstacles and target”

  $$\begin{aligned} \mathcal {W}(\epsilon ) = \mathcal {F}_2(\epsilon )\, {\setminus }\, \mathcal {B}_{x_t}(\delta _t) \end{aligned}$$

Recall that a workspace is called valid, if the obstacles do not overlap with each other and the destination (set).

Proposition 3

If the workspace is valid, any $x_d \in \partial \mathcal {S}_T$ is a degenerate local minimum of $\varphi$. A vector v satisfying ${v}^\intercal \;\nabla ^2\varphi (x_d) \; {v} = 0$ has to be tangent to $\partial \mathcal {S}_T$.

Proof

Evaluate

and note that since and , it is $\nabla \varphi ({x_d}) = 0$. Now

Consider arbitrary vector $v\in \mathbb {R}^n$ and evaluate the quadratic form

$$\begin{aligned} v^\intercal \, \nabla ^2\varphi (x_d) \,v= & {} 8\,\beta ^{-1/k}\, v^\intercal \,(x_d-x_t)(x_d-x_t)^\intercal \,v \\= & {} 8\,\beta ^{-1/k}\,\Vert v^\intercal (x_d-x_t) \Vert ^2 . \end{aligned}$$

This means that $v^\intercal \, \nabla ^2\varphi ( x_d) \,v\ge 0$ with equality if and only if v is normal to $(x_d-x_t)$, that is, when v is tangent to $\partial \mathcal {S}_T$. $\square$

Proposition 4

If the workspace is valid, all the critical points of $\varphi$ are in the interior of the free space.

Proof

Let $x_0$ be a point in $\partial \mathcal {F}$. Then by definition, $\beta _i(x_0)=0$ for some $i \in \{0,\ldots ,M\}$. From the workspace being valid, it follows that $\beta _j>0$ for all $j \in \{0,\ldots ,M\}$, $j \ne i$. Then,

which completes the proof. $\square$

Proposition 5

For every $\epsilon > 0$ there exists a positive integer $N(\epsilon )$ such that if $k \ge N(\epsilon )$ then there are no critical points of $\frac{J^k}{\beta }$ in $\mathcal {W}(\epsilon )$.

Proof

A sufficient condition for $\frac{J^k}{\beta }$ not having critical points in $\mathcal {W}$ is (Koditschek and Rimon 1990, Proposition 3.4)

$$\begin{aligned} k > \frac{J \, \Vert \nabla \beta \Vert }{\beta \, \Vert \nabla J\Vert } . \end{aligned}$$

For this, it is sufficient to have

$$\begin{aligned} k \ge \sup _\mathcal {W} \frac{J}{\Vert \nabla J \Vert } \,\sup _\mathcal {W} \frac{\Vert \nabla \beta \Vert }{\beta } > \frac{J \, \Vert \nabla \beta \Vert }{\beta \, \Vert \nabla J\Vert } . \end{aligned}$$

The existence of a finite bound of $\sup _\mathcal {W} \frac{J}{\Vert \nabla J \Vert } \,\sup _\mathcal {W} \frac{\Vert \nabla \beta \Vert }{\beta }$ can be established analytically as follows.

$$\begin{aligned} \sup _\mathcal {W} \frac{J}{\Vert \nabla J\Vert }= & {} \sup _\mathcal {W} \frac{\big (\Vert x-x_t\Vert ^2-r_t^2\big )^2}{4\Vert x-x_t\Vert ^3-4r_t^2\Vert x-x_t\Vert } \\= & {} \sup _\mathcal {W}\frac{\Vert x-x_t\Vert ^2-r_t^2}{4\Vert x-x_t\Vert } . \end{aligned}$$

Since $\Vert x-x_t\Vert$ is bounded from below and above in $\mathcal {W}$, and $\inf _\mathcal {W} \Vert x-x_t\Vert = \delta _t$, it is ensured that $\sup _\mathcal {W} \frac{J}{\Vert \nabla J\Vert }$ is finite. For the other bound, we have

$$\begin{aligned} \frac{\Vert \nabla \beta \Vert }{\beta }< & {} \sup _\mathcal {W} \frac{\Vert \nabla \beta \Vert }{\beta } \nonumber \\\le & {} \sup _\mathcal {W} \sum _{i=0}^M \frac{\Vert \nabla \beta _i\Vert }{\beta _i} \nonumber \\\le & {} \frac{2}{\epsilon }\left[ \rho _0 + \sum _{i=1}^M \sup _\mathcal {W} \Vert x-o_i \Vert \right] \end{aligned}$$

(17)

The strict inequality is due to the fact that $\Vert x\Vert < \rho _0$ for any point in $\mathcal {W}$. Equation (17) implies that $\sup _\mathcal {W} \frac{\Vert \nabla \beta \Vert }{\beta }$ exists and is bounded. Thus, a choice of a sufficiently large $k \ge N(\epsilon )$ would be

$$\begin{aligned} N(\epsilon ) := \frac{1}{\epsilon } \sup _\mathcal {W} \left\{ \frac{\Vert x-x_t\Vert ^2-r_t^2}{2\Vert x-x_t\Vert } \right\} \left( \rho _0+ \sum _{i=1}^M \sup _\mathcal {W} \Vert x-o_i \Vert \right) . \nonumber \\ \end{aligned}$$

(18)

The proof is thus completed. $\square$

Proposition 6

For any valid workspace, there exists an $\epsilon _0>0$ such that $\frac{J^k}{\beta }$ has no local minima in $\mathcal {F}_0(\epsilon )$, as long as $\epsilon < \epsilon _0$.

Proof

The analysis focuses on $\mathcal {F}_0(\epsilon )$, and that implies that for any critical point $x_c \in \mathcal {F}_0(\epsilon )$, for some i we will have $x_c \in \mathcal {B}_i(\epsilon )$; therefore, $0<\beta _i(x_c)<\epsilon$. The validity of the workspace guarantees that $\Vert o_i-x_t\Vert >r_t+\rho _i+\epsilon _t$. Because of this, that particular ball $\mathcal {B}_i(\epsilon )$ is bounded away from $\partial \mathcal {S}_T$: for any $x \in \mathcal {B}_i(\epsilon )$, it is $\Vert x-x_t\Vert >r_t$ as long as $0<\Vert x-o_i\Vert -\rho _i< \sqrt{\epsilon +\rho _i^2}-\rho _i< \epsilon _t <\Vert o_i-x_t\Vert -r_t-\rho _i$. Since $x_c$ is a critical point, $k\beta \, \nabla J = J \;\nabla \beta$ at $x_c$. Note that everywhere in $\mathcal {F}_0(\epsilon )$, $J \ne 0$ and $\beta \ne 0$. Therefore, $\nabla J$ is aligned with $\nabla \beta$. Using the concept of the omitted product (Koditschek and Rimon 1990)

$$\begin{aligned} \bar{\beta }_i \triangleq \prod _{j=0,\,j\ne i}^M \beta _j \end{aligned}$$

vector $\nabla \beta$ expands to

$$\begin{aligned} \nabla \beta= & {} \sum _{j=1}^M 2(x_c-o_j)\bar{\beta }_j - 2 \bar{\beta }_0 x_c \\= & {} 2(x_c-o_i)\bar{\beta }_i+2 \beta _i \sum _{j=1,j\ne i}^M (x_c-o_j) \;\frac{\bar{\beta }_j}{\beta _i} - 2\bar{\beta }_0\,x_c \end{aligned}$$

and by defining

$$\begin{aligned} \alpha _i\triangleq 2\!\!\sum _{j=1,j\ne i}^M (x_c-o_j)\,\frac{\bar{\beta }_j}{\beta _i} -2\frac{\bar{\beta }_0}{\beta _i}\,x_c \end{aligned}$$

which is a vector independent from $\epsilon$, and bounded in $\mathcal {F}_0(\epsilon )$, one has

$$\begin{aligned} \nabla \beta = 2(x_c-o_i)\bar{\beta }_i+\beta _i \,\alpha _i . \end{aligned}$$

From $k\beta \, \nabla J = J \;\nabla \beta$ at $x_c$ it now follows that

$$\begin{aligned} \nabla J= & {} \frac{J}{k\beta }\nabla \beta \\\iff & {} 4\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) (x_c-x_t) \\= & {} \frac{J}{k\beta }\left[ 2(x_c-o_i)\bar{\beta }_i+\beta _i \,\alpha _i\right] \end{aligned}$$

which leads to

$$\begin{aligned} x_c-x_t=\frac{\Vert x_c-x_t\Vert ^2-r_t^2}{4}\left( 2\frac{x_c-o_i}{k\beta _i}+\frac{\alpha _i}{k\bar{\beta }_i} \right) . \end{aligned}$$

(19)

If one now sets

$$\begin{aligned} C_k\triangleq \sup _\mathcal {W}\left\{ \frac{\Vert x_c-x_t\Vert ^2-r_t^2}{2\Vert x_c-x_t\Vert }\right\} \Big (\rho _0+\sum _{i=1}^M\sup _\mathcal {W}\Vert x_c-o_i\Vert \Big ) \end{aligned}$$

then according to (18) in the proof of Proposition 5, a suitable choice of k would be

$$\begin{aligned} k := \frac{C_k}{\epsilon } \end{aligned}$$

in which case (19) becomes

$$\begin{aligned} x_c-x_t=\frac{\epsilon (\Vert x_c-x_t\Vert ^2-r_t^2)}{4C_k}\left( 2\frac{x_c-o_i}{\beta _i}+\frac{\alpha _i}{\bar{\beta }_i} \right) . \end{aligned}$$

(20)

Taking the inner product of both sides of $k\beta \nabla J = J \nabla \beta$ with $\nabla J$ yields

$$\begin{aligned}&k\beta \nabla J^\intercal \, \nabla J = J\; \nabla \beta ^\intercal \, \nabla J \nonumber \\&\quad \implies k \beta = \frac{\bar{\beta }_i \,\nabla \beta _i^\intercal \, \nabla J + \beta _i \,\nabla \bar{\beta }_i^\intercal \, \nabla J}{16 \Vert x_c-x_t\Vert ^2} \end{aligned}$$

(21)

From this point, one can then prove that the critical point of $\frac{J^k}{\beta }$ is not a local minimum by showing that $\nabla ^2\frac{J^k}{\beta }$ has at least one negative eigenvalue at that point. (The procedure follows the exact same steps as Koditschek and Rimon 1990, Proposition 3.6.) Essentially, it amounts to using any vector $\hat{v}$ orthogonal to $\frac{\nabla \beta _i}{\Vert \nabla \beta _i \Vert }$ as a test vector, and showing that at $x_c$ and for small enough $\epsilon$, $\hat{v}^\intercal \; \nabla ^2\frac{J^k}{\beta }\; \hat{v} < 0$. The process in detail is as follows:

(22)

To determine the sign of the far right side of (22), perform the expansion of $\hat{v}^\intercal \; \nabla ^2J \; \hat{v}$ into

$$\begin{aligned}&\hat{v}^\intercal \; \Big [4\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) \mathbf{I}+8(x_c-x_t)(x_c-x_t)^\intercal \Big ] \; \hat{v} \nonumber \\&\quad =4\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) + 8 \,\hat{v}^\intercal \; (x_c-x_t)(x_c-x_t)^\intercal \; \hat{v} \nonumber \\&\quad =4\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) +8 \left( \hat{v}^\intercal (x_c-x_t)\right) ^2, \end{aligned}$$

(23)

where $\mathbf I$ denotes the identity matrix, and plug (20) into (23), to express $\hat{v}^\intercal \; \nabla ^2J \; \hat{v}$ in the form

$$\begin{aligned}&4\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) \nonumber \\&\qquad +\, 8\left| \hat{v}^\intercal \, \frac{\epsilon \left( \Vert x_c-x_t\Vert ^2-r_t^2\right) }{4C_k}\Big (2\frac{x_c-o_i}{\beta _i}+\frac{\alpha _i}{\bar{\beta }_i} \Big )\right| ^2 \nonumber \\&\quad =4\sqrt{J}+\frac{2\epsilon \sqrt{J}\; |\hat{v}^\intercal \alpha _i|^2}{C_k\bar{\beta }_i}. \end{aligned}$$

(24)

where $\hat{v}^\intercal (x_c-o_i)=0$ and $\sqrt{J}$ is substituted for $\Vert x_c-x_t\Vert ^2-r_t^2$ (for brevity), since in $\mathcal {F}_0(\epsilon )$ it holds $\Vert x_c-x_t\Vert >r_t$.

Given now that the second term in (22) can be made arbitrarily small by choosing $\epsilon > \beta _i$, one can establish the negative definiteness of (22) by ensuring that the first term is strictly below zero. The second factor in the first term in (22) can be expanded

$$\begin{aligned}&\frac{\hat{v}^\intercal \; \nabla ^2J \; \hat{v}}{16\Vert x_c-x_t\Vert ^2}\; \nabla \beta _i^\intercal \,\nabla J-2J \\&\quad \mathop {=}\limits ^{(24)} \frac{2\sqrt{J}+\frac{\epsilon \sqrt{J}}{C_k\bar{\beta }_i}|\hat{v}^T\alpha _i|^2}{8\Vert x_c-x_t\Vert ^2}\; 2(x_c-o_i) \, 4(x_c-x_t)\sqrt{J}-2J \\&\quad = 2J\left[ \frac{(x_c-o_i)^\intercal (x_c-x_t)}{ \Vert x_c-x_t \Vert ^2}-1\right] \\&\qquad + \,\frac{\epsilon J|\hat{v}^\intercal \alpha _i|^2}{C_k\bar{\beta }_i\Vert x_c-x_t\Vert ^2}\;(x_c-o_i)^\intercal (x_c-x_t)\\&\quad =\frac{ 2J (x_t-o_i)^\intercal (x_c-x_t)}{\Vert x_c-x_t\Vert ^2}+ \frac{\epsilon J |\hat{v}^\intercal \alpha _i|^2\; (x_c-o_i)^\intercal (x_c-x_t)}{C_k\bar{\beta }_i\Vert x_c-x_t\Vert ^2} \end{aligned}$$

and by applying known relations (Koditschek and Rimon 1990, Lemma 3.5) one arrives at

$$\begin{aligned}&\frac{\hat{v}^\intercal \; \nabla ^2J \; \hat{v}}{16\Vert x_c-x_t\Vert ^2}\; \nabla \beta _i^\intercal \,\nabla J-2J \nonumber \\&\quad \le \frac{2J \Vert x_t-o_i\Vert \, \left( \sqrt{\epsilon +\rho _i^2}-\Vert x_t-o_i\Vert \right) }{\Vert x_c-x_t\Vert ^2}\nonumber \\&\qquad +\, \epsilon \sup _{\mathcal {F}_0(\epsilon )} \frac{J|\hat{v}^T\alpha _i|^2\;(x_c-o_i)^\intercal (x_c-x_t)}{C_k\bar{\beta }_i\Vert x_c-x_t\Vert ^2} . \end{aligned}$$

(25)

At this point, (25) is used in (22) to upper bound the left hand side of (22)

Now choosing $\epsilon$ appropriately small, the second term can be made sufficiently small so that the sign of the first term dominates. The sign of the latter is determined by the expression $\sqrt{\epsilon +\rho _i^2}-\Vert x_t-o_i\Vert$, which for small $\epsilon$ approaches $\rho _i-\Vert x_t-o_i\Vert$, which is guaranteed negative by the validity of the workspace. (The target $x_t$ is $(r_t+\rho _i)$ away from the center of obstacle i.) $\square$

Proposition 7

If $k \ge \frac{C_k}{\epsilon }$, then there exists an $\epsilon _1 >0$ such that $\hat{\varphi }=\frac{J^k}{\beta }$ has no critical points on $\mathcal {F}_1(\epsilon )$, as long as $\epsilon < \epsilon _1$.

Proof

The set $\mathcal {F}_1(\epsilon )$ expresses the neighborhood of the workspace (outer) boundary. Select $\epsilon$ small enough so that the $\mathcal {B}_0(\epsilon )$ neighborhood of the outer boundary is disjoint from the $r_t$-neighborhood of the target: $\beta _0 < \epsilon <\rho _0^2-(\Vert x_t\Vert +r_t)^2$. Then any critical point $x_c \in \mathcal {F}_1(\epsilon )$ will satisfy $\beta _0(x_c)= \rho _0 ^2-\Vert x_c\Vert ^2<\epsilon$, implying $\Vert x_c\Vert >\Vert x_t\Vert +r_t$. Then in $\mathcal {B}_0(\epsilon )$

$$\begin{aligned} \nabla J^\intercal \, \nabla \beta _0 =&\,4\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) (x_c-x_t)^\intercal \, (-2x_c) \nonumber \\ =&\,8\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) \left( x_t^\intercal \,x_c - \Vert x_c\Vert ^2\right) \nonumber \\ \le&\,8\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) \left( \Vert x_c\Vert \Vert x_t\Vert - \Vert x_c\Vert ^2 \right) \nonumber \\ =&\, 8\left( \Vert x_c-x_t\Vert ^2-r_t^2\right) \,\Vert x_c\Vert \, ( \Vert x_t\Vert - \Vert x_c\Vert ) \nonumber \\ <&\,0. \end{aligned}$$

By choosing $\epsilon$ small enough, we can ensure that $\nabla \hat{\varphi }$ does not vanish in $\mathcal {F}_1(\epsilon )$. Here is why:

$$\begin{aligned} \nabla \hat{\varphi }^\intercal \,\nabla J= & {} \Big [ \frac{kJ^{k-1}}{\beta } \nabla J - \frac{J^k}{\beta ^2} \nabla \beta \Big ]^\intercal \,\nabla J \\= & {} \frac{J^k (16k \beta \, \Vert x_c-x_t\Vert ^2 - \nabla \beta ^\intercal \,\nabla J) }{\beta ^2} \\= & {} \frac{J^k [16k \beta \Vert x_c-x_t\Vert ^2 - (\beta _0 \; \nabla \bar{\beta }_0^\intercal \nabla J+ \bar{\beta }_0 \; \nabla \beta _0^\intercal \nabla J)] }{\beta ^2}\\&\mathop { >}\limits ^{(\nabla J^\intercal \nabla \beta _0 < 0)} \frac{J^k\beta _0(16k\bar{\beta }_0\Vert x_c-x_t\Vert ^2 - \nabla \bar{\beta }_0^\intercal \nabla J ) }{\beta ^2} \end{aligned}$$

and thus any $\epsilon$ small enough to make $k> \frac{\nabla \bar{\beta }_0^\intercal \, \nabla J }{ 16 \bar{\beta }_0\Vert x_c-x_t\Vert ^2}$, will also make $\nabla \hat{\varphi }^\intercal \,\nabla J$ positive. In fact, the choice utilized earlier, i.e., $k = \frac{C_k}{\epsilon }$ suffices. To see this,

$$\begin{aligned} \frac{\nabla \bar{\beta }_0^\intercal \nabla J }{ 16 \bar{\beta }_0\Vert x_c-x_t\Vert ^2}\le & {} \frac{\Vert \nabla \bar{\beta }_0\Vert \Vert \nabla J \Vert }{ 16 \bar{\beta }_0\Vert x_c-x_t\Vert ^2} = \frac{\frac{\sqrt{J}}{\beta } \sum _{i=1}^M \bar{\beta }_i \Vert \nabla \beta _i\Vert }{4\Vert x_c-x_t\Vert } \\< & {} \frac{1}{\epsilon }\sup _\mathcal {W}\left\{ \frac{\sqrt{J}}{2\Vert x_c-x_t\Vert }\right\} \sum _{i=1}^M\sup _\mathcal {W}\Vert x_c-o_i\Vert , \end{aligned}$$

and compare to

$$\begin{aligned} k := \frac{C_k}{\epsilon }= & {} \frac{1}{\epsilon }\sup _\mathcal {W}\left\{ \frac{\sqrt{J}}{2\Vert x_c-x_t\Vert }\right\} \left( \rho _0\!+\! \sum _{i=1}^M\sup _\mathcal {W}\Vert x_c-o_i\Vert \right) \\> & {} \frac{1}{\epsilon }\sup _\mathcal {W}\left\{ \frac{\sqrt{J}}{2\Vert x_c-x_t\Vert }\right\} \sum _{i=1}^M\sup _\mathcal {W}\Vert x_c-o_i\Vert . \end{aligned}$$

It thus suffices to pick $\epsilon < \epsilon _1=(\rho _0)^2-(\Vert x_t\Vert +r_t)^2$ to ensure that no critical points are in $\mathcal {F}_1(\epsilon )$.$\square$

Proposition 8

Critical points in the interior of $\mathcal {F}_0(\epsilon )$ are non-degenerate.

Proof

One way to establish such a claim [Koditschek and Rimon (1990, Proposition 3.9)] is to partition the tangent space of $\hat{\varphi }$ into a subspace that yields positive values for the quadratic form constructed with $\nabla ^2 \hat{\varphi }$, and a subset that yields negative values. The negative case is established by Proposition 6. The positive case, again along the lines of [Koditschek and Rimon (1990, Proposition 3.9)], is established here by taking a test direction ${{\mathrm{\widehat{\nabla {\beta }}}}}_i =\frac{\nabla \beta _i}{\Vert \nabla \beta _i \Vert }$, and picking $\epsilon$ small enough to obtain ${{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; \nabla ^2 \hat{\varphi }\; {{\mathrm{\widehat{\nabla {\beta }}}}}_i>0$. Note that for a given i, ${{\mathrm{\widehat{\nabla {\beta }}}}}_i$ defines one subspace, and all the vectors $\hat{v}$ form its orthogonal complement. To verify the sign of ${{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; \nabla ^2 \hat{\varphi }\; {{\mathrm{\widehat{\nabla {\beta }}}}}_i$, expand the expression

$$\begin{aligned}&\frac{\beta ^2}{J^{k-1}} \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; \nabla ^2 \hat{\varphi } \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i\nonumber \\&\quad = {{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; k\beta \nabla ^2J \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i \nonumber \\&\qquad + \frac{J(1-\frac{1}{k}) }{\beta } \; (\nabla \beta ^\intercal \, {{\mathrm{\widehat{\nabla {\beta }}}}}_i)^2 - J \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; \nabla ^2 \beta \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i . \end{aligned}$$

(26)

We know [Koditschek and Rimon (1990, Proposition 3.9)] that for small enough $\epsilon$,

$$\begin{aligned} \frac{J \Vert \nabla \beta \Vert ^2}{2k\beta } \!+\! \frac{J(1-\frac{1}{k})}{\beta } (\nabla \beta ^\intercal \, {{\mathrm{\widehat{\nabla {\beta }}}}}_i)^2 \!-\! J\; {{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; \nabla ^2 \beta \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i>0 \end{aligned}$$

And although different J function is used here, the same derivation in [Koditschek and Rimon (1990, Proposition 3.9)] holds here. So to set the sign of (26), it suffices to make

$$\begin{aligned} {{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; k\beta \, \nabla ^2J \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i \ge \frac{J\Vert \nabla \beta \Vert ^2}{2k\beta } \end{aligned}$$

(27)

Recalling (23), and that $\Vert x_c-x_t\Vert ^2-r_t^2 = \sqrt{J}$ since $x_c$ is a critical point, the left hand side of (27) is

$$\begin{aligned} {{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \; k\beta \, \nabla ^2J \; {{\mathrm{\widehat{\nabla {\beta }}}}}_i = 4k\beta \sqrt{J}+8k\beta |{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal (x_c-x_t) |^2 \end{aligned}$$

(28)

and because $x_c$ is a critical point, taking squared norms of both sides of $k\beta \,\nabla J = J \,\nabla \beta$ yields

$$\begin{aligned} \big (4k\beta \sqrt{J}\, \Vert x_c-x_t\Vert \big )^2=J^2\,\Vert \nabla \beta \Vert ^2 \end{aligned}$$

from which one extracts that

$$\begin{aligned} 4k\beta =\frac{J\Vert \nabla \beta \Vert ^2}{4k\beta \Vert x_c-x_t\Vert ^2} . \end{aligned}$$

(29)

Plugging now (29) back into (28) yields

$$\begin{aligned}&{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal k\beta \nabla ^2J {{\mathrm{\widehat{\nabla {\beta }}}}}_i \\&\quad =\frac{J^{3/2}\,\Vert \nabla \beta \Vert ^2}{4k\beta \Vert x_c-x_t\Vert ^2} +\frac{J\Vert \nabla \beta \Vert ^2\; \big |{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \,(x_c-x_t)\big |^2}{2k\beta \Vert x_c-x_t\Vert ^2}. \end{aligned}$$

Now (27) takes the form

$$\begin{aligned}&\frac{J^{3/2}\,\Vert \nabla \beta \Vert ^2}{4k\beta \Vert x_c-x_t\Vert ^2} +\frac{J\Vert \nabla \beta \Vert ^2\; \big |{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal \,(x_c-x_t)\big |^2}{2k\beta \Vert x_c-x_t\Vert ^2} \ge \frac{J\Vert \nabla \beta \Vert ^2}{2k\beta } \nonumber \\&\iff \frac{J^{1/2}}{2\Vert x_c-x_t\Vert ^2}+\frac{\big |{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal (x_c-x_t)\big |^2}{\Vert x_c-x_t\Vert ^2} \ge 1 \nonumber \\&\iff \frac{ \Vert x_c-x_t\Vert ^2-r_t^2 +2\,\big |{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal (x_c-x_t)\big |^2}{2\Vert x_c-x_t\Vert ^2} \ge 1 \nonumber \\&\iff 2\;\big |{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal (x_c-x_t)\big |^2 \ge \Vert x_c-x_t\Vert ^2+r_t^2 \end{aligned}$$

(30)

For $x_c \in \mathcal {B}_i(\epsilon )$ (guaranteed by Proposition 6), $\Vert x_c-x_t\Vert > r_t$. Now let $r_t$ assume the form $r_t=\zeta \inf _{\mathcal {B}_i(\epsilon )} \Vert x_c-x_t\Vert$ for an appropriate $\zeta <1$, and recall that ${{\mathrm{\widehat{\nabla {\beta }}}}}_i =\frac{\nabla \beta _i}{\Vert \nabla \beta _i \Vert }$, where $\nabla \beta _i = 2 (x-o_i)$. With this in mind, one satisfies (30) by ensuring that

$$\begin{aligned} \frac{1+\zeta ^2}{2}&\le \left( \frac{(x_c-o_i)^\intercal (x_c-x_t)}{\Vert x_c-o_i\Vert \, \Vert x_c-x_t\Vert } \right) ^2 \\&\implies 2\; \big |{{\mathrm{\widehat{\nabla {\beta }}}}}_i^\intercal (x_c-x_t)\big |^2 \ge \Vert x_c-x_t\Vert ^2+r_t^2 . \nonumber \end{aligned}$$

(31)

An appropriately small choice of $\epsilon$ can establish (31), as the following derivation shows:

$$\begin{aligned}&\frac{(x_c-o_i)^\intercal (x_c-x_t)}{\Vert x_c-o_i\Vert \, \Vert x_c-x_t\Vert }\\&\quad \ge \frac{ \frac{\sqrt{J}}{4k}\left[ \left( 2\Vert x_c-o_i\Vert ^2\right) /\beta _i+\left[ \alpha _i^\intercal (x_c-o_i)\right] /\bar{\beta }_i \right] }{\frac{\sqrt{J}}{4k}\left[ (2\Vert x_c-o_i\Vert )/\beta _i+\Vert \alpha _i\Vert /\bar{\beta }_i \right] \Vert x_c-o_i\Vert } \\&\quad \ge \frac{ \left( 2\Vert x_c-o_i\Vert ^2\right) /\beta _i-(\Vert \alpha _i\Vert \Vert x_c-o_i\Vert )/\bar{\beta }_i}{\left( 2\Vert x_c-o_i\Vert ^2\right) /\beta _i+(\Vert \alpha _i\Vert \Vert x_c-o_i\Vert )/\bar{\beta }_i} \\&\quad = \frac{1-(\beta _i \Vert \alpha _i\Vert )/(2\bar{\beta }_i \Vert x_c-o_i\Vert )}{1+(\beta _i\Vert \alpha _i\Vert )/(2\bar{\beta }_i \Vert x_c-o_i\Vert )} \\&\quad = 1-\frac{(\beta _i \Vert \alpha _i\Vert )/(\bar{\beta }_i \Vert x_c-o_i\Vert )}{1+(\beta _i\Vert \alpha _i\Vert )/(2\bar{\beta }_i \Vert x_c-o_i\Vert )} \\&\quad \ge 1-\frac{\beta _i \Vert \alpha _i\Vert }{\bar{\beta }_i \Vert x_c-o_i\Vert } \ge 1-\frac{\epsilon \,\Vert \alpha _i\Vert }{\bar{\beta }_i\Vert x_c-o_i\Vert } \end{aligned}$$

and thus to satisfy (27), it sufficies to pick

$$\begin{aligned} \epsilon&< \left( 1-\sqrt{\frac{1+\zeta ^2}{2}}\right) \frac{\inf _i \rho _i^m}{\sup _{ {\mathcal {F}_0(\epsilon )}} \Vert \alpha _i\Vert } \\&\implies \epsilon < \left( 1-\sqrt{\frac{1+\zeta ^2}{2}}\right) \frac{\bar{\beta }_i\Vert x_c-o_i\Vert }{\Vert \alpha _i\Vert } \end{aligned}$$

$\square$

Proposition 9

There exists $k_0>0$ such that for any $k>k_0$, any critical point $x_c \in \mathcal {B}_{x_t}(\delta _t)$ is a local maximum of $\frac{J^k}{\beta }$.

Proof

To study the critical points in $\mathcal {B}_{x_t}(\delta _t)$, we work on the two cases:

Case I: $\nabla \beta |_{x_t} = 0$, $x_c = x_t$,

$\nabla J|_{x_t} = 0 \Rightarrow k\beta (x_t)\nabla J|_{x_t} = J(x_t)\nabla \beta |_{x_t}=0$, we shall have $x_t$ as one of the critical points in $\mathcal {B}_{x_t}(\delta _t)$. In this case, for any unit vector $q\in \mathbb {R}^n$:

$$\begin{aligned}&\frac{\beta ^2}{J^{k-2}}q^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_t}q\\&\quad =q^\intercal \left( k \beta J \nabla ^2 J + k(k-1)\beta \nabla J \nabla J^\intercal - J^2 \nabla ^2\beta \right) q\\&\quad =q^\intercal \big (-4 J^{1.5}k \beta \mathbf {I} - J^2 \nabla ^2 \beta \big )q \\&\quad =-\,4r_t^6k\beta - r_t^8q^\intercal \nabla ^2\beta q \end{aligned}$$

Recall that to make the workspace valid, $\mathcal {B}_{x_t}(r_t)$ should not intersect $\partial \mathcal {F}$, then at $x_t$, $\beta _i>r_t^2$ for $i \in \{0\ldots m\} \Rightarrow \beta >r_t^{2m}$. So as long as

$$\begin{aligned} k > k_1 = \frac{1}{4}r_t^{(2-2m)}\sup _q(|q^\intercal \nabla ^2\beta q|) \end{aligned}$$

$q^\intercal \nabla ^2(\frac{J^k}{\beta })|_{x_t}q$ is guaranteed to be negative for any unit vector $q\in \mathbb {R}^n$ and the critical point $x_t$ is a local maximum of $\frac{J^k}{\beta }$.

Case II: For any $x_c \ne x_t, x_c\in \mathcal {B}_{x_t}(\delta _t)$:

For any unit vector $q \in \mathbb {R}^n$, q can be presented as scaled sum of $v_1 = \frac{\nabla J}{\Vert \nabla J\Vert }$ and $v_2$, a unit vector perpendicular to $v_1$, i.e. $q = q_1 v_1+q_2 v_2, q_1^2+q_2^2=1, q_1 q_2>0$. In order to ensure that $\nabla ^2(\frac{J^k}{\beta })$ is negative definite at $x_c$, a critical point of $\frac{J^k}{\beta }$ in $\mathcal {B}_{x_t}(\delta _t)$, we study the sign of $\frac{\beta ^2}{J^{k-2}}q^\intercal \nabla ^2(\frac{J^k}{\beta })|_{x_c}q$:

$$\begin{aligned}&\frac{\beta ^2}{J^{k-2}}q^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}q \nonumber \\&\quad = \frac{\beta ^2}{J^{k-2}}\left( q_1^2 v_1^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_1 +q_2^2 v_2^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_2\right. \nonumber \\&\qquad \left. +\, q_1 q_2 v_1^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_2 + q_1 q_2 v_2^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_1\right) \nonumber \\ \end{aligned}$$

(32)

Recall that at critical point $x_c$, $k\beta \nabla J = J \nabla \beta$. Take the norm of both side:

$$\begin{aligned} k\beta (4J^{0.5}\Vert x_c-x_t\Vert ) =&J\Vert \nabla \beta \Vert \nonumber \\ k\Vert x_c-x_t\Vert =&J^{0.5}\frac{\Vert \nabla \beta \Vert }{4\beta } \end{aligned}$$

(33)

The first term in (32) can be expanded as:

$$\begin{aligned}&q_1^2\frac{\beta ^2}{J^{k-2}}v_1^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_1\\&\quad =q_1^2v_1^\intercal (k \beta J \nabla ^2 J + k(k-1)\beta \nabla J \nabla J^\intercal - J^2 \nabla ^2\beta ) v_1\\&\quad =q_1^2v_1^\intercal \big (k \beta J [8(x_c-x_t)(x_c-x_t)^\intercal -4 J^{0.5} \mathbf {I}] + \cdots \\&\qquad + k(k-1)\beta \nabla J \nabla J^\intercal - J^2 \nabla ^2 \beta \big )v_1 \\&\quad =q_1^2(8k(2k-1)\beta J\Vert x_c-x_t\Vert ^2 \!-\! 4k\beta J^{1.5}-J^2 v_1^\intercal \nabla ^2\beta v_1 )\\&\quad \mathop {=}\limits ^{(33)} q_1^2\left( \frac{J^2\Vert \nabla \beta \Vert ^2}{\beta } -2J^{1.5}\Vert \nabla \beta \Vert \Vert x_c-x_t\Vert \right. \\&\qquad \qquad \qquad \left. -J^2 v_1^\intercal \nabla ^2\beta v_1 -..- \,4k\beta J^{1.5}\right) \\&\quad \le q_1^2\left( \frac{J^2\Vert \nabla \beta \Vert ^2}{\beta } -J^2 v_1^\intercal \nabla ^2\beta v_1- 4k\beta J^{1.5}\right) \end{aligned}$$

Since $v_2$ is an arbitrary unit vector that is perpendicular to $\nabla J$, it is also perpendicular to $(x_c-x_t)$. The second term in (32) can be expanded as:

$$\begin{aligned}&q_2^2\frac{\beta ^2}{J^{k-2}}v_2^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_2\\&\quad =q_2^2v_2^\intercal (k \beta J \nabla ^2 J + k(k-1)\beta \nabla J \nabla J^\intercal - J^2 \nabla ^2\beta ) v_2\\&\quad =q_2^2v_2^\intercal \big (k \beta J [8(x_c-x_t)(x_c-x_t)^\intercal -4 J^{0.5} \mathbf {I}] + ..\\&\qquad +\, k(k-1)\beta \nabla J \nabla J^\intercal - J^2 \nabla ^2 \beta \big )v_2 \\&\quad =q_2^2(-4k\beta J^{1.5}-J^2 v_2^\intercal \nabla ^2\beta v_2) \end{aligned}$$

Similarly we shall have:

$$\begin{aligned}&q_1q_2\frac{\beta ^2}{J^{k-2}}v_1^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_2 =q_1q_2(-J^2 v_1^\intercal \nabla ^2\beta v_2) \\&q_1q_2\frac{\beta ^2}{J^{k-2}}v_2^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}v_1 =q_1q_2(-J^2 v_2^\intercal \nabla ^2\beta v_1) \\ \end{aligned}$$

And since $\nabla ^2\beta$ is symmetric, $v_1^\intercal \nabla ^2\beta v_2 = v_2^\intercal \nabla ^2\beta v_1$. Now (32) can be upper bounded by:

$$\begin{aligned}&\frac{\beta ^2}{J^{k-2}}q^\intercal \nabla ^2\left( \frac{J^k}{\beta }\right) \Big |_{x_c}q \\&\quad \le q_1^2\left( \frac{J^2\Vert \nabla \beta \Vert ^2}{\beta } -J^2 v_1^\intercal \nabla ^2\beta v_1- 4k\beta J^{1.5}\right) \\&\qquad +\, q_2^2\left( -4k\beta J^{1.5}-J^2 v_2^\intercal \nabla ^2\beta v_2\right) \\&\qquad +\, q_1q_2\left( -J^2 v_1^\intercal \nabla ^2\beta v_2\right) + q_1q_2\left( -J^2 v_2^\intercal \nabla ^2\beta v_1\right) \\&\quad =-\,4k\beta J^{1.5} + q_1^2\left( \frac{J^2\Vert \nabla \beta \Vert ^2}{\beta } -J^2 v_1^\intercal \nabla ^2\beta v_1\right) \\&\qquad + q_2^2\left( -J^2 v_2^\intercal \nabla ^2\beta v_2\right) + 2q_1q_2\left( -J^2 v_1^\intercal \nabla ^2\beta v_2\right) \\&\quad \le -\,4k\beta J^{1.5} \\&\qquad + J^2\left( \frac{\Vert \nabla \beta \Vert ^2}{\beta }+ |v_1^\intercal \nabla ^2\beta v_1| + |v_2^\intercal \nabla ^2\beta v_2| + |v_1^\intercal \nabla ^2\beta v_2|\right) \end{aligned}$$

Recall that to make the workspace valid, $\mathcal {B}_{x_t}(r_t)$ should not intersect $\partial \mathcal {F}$, then in $\mathcal {B}_{x_t}(\delta _t)$, $\beta _i>(r_t-\delta _t)^2$ for $i \in \{0\ldots m\} \Rightarrow \beta >(r_t-\delta _t)^{2m}$, and $J \le r_t^4$. So as long as

$$\begin{aligned}&k > k_2 \\&\quad = \frac{r_t^2}{4(r_t-\delta _t)^{2m}}\bigg [\sup _{x_c\in \mathcal {B}_{x_t}(\delta _t)}\Big ( \frac{\Vert \nabla \beta \Vert ^2}{\beta } \\&\qquad + |v_1^\intercal \nabla ^2\beta v_1| + |v_2^\intercal \nabla ^2\beta v_2| +\, |v_1^\intercal \nabla ^2\beta v_2|\Big )\bigg ] \end{aligned}$$

we shall satisfy $q^\intercal \nabla ^2(\frac{J^k}{\beta })|_{x_c}q<0$ for any $q\in \mathbb {R}^n$ and any critical point $x_c$ in $\mathcal {B}_{x_t}(\delta _t)$ other than $x_t$ is guaranteed to be local maximum of $\frac{J^k}{\beta }$.

To sum up two cases, $k>k_0=\max \{k_1,k_2\}$ will guarantee any critical point in $\mathcal {B}_{x_t}(\delta _t)$ to be a local maximum of $\frac{J^k}{\beta }$. $\square$