An upper bound on the Hausdorff distance between a Pareto set and its discretization in bi-objective convex quadratic optimization

Abstract

We provide upper bounds on the Hausdorff distances between the efficient set and its discretization in the decision space, and between the Pareto set (also called the Pareto front) and its discretization in the objective space, in the context of bi-objective convex quadratic optimization on a compact feasible set. Our results imply that if $t$ is the dispersion of the sampled points in the discretized feasible set, then the Hausdorff distances in both the decision space and the objective space are $O(\sqrt{t})$ as $t$ decreases to zero.

A Supporting results: gradients, tangents, and inscribed q-Balls

In this section, we provide several supporting results regarding the geometry of Problem (Q) under Assumption 1. These results involve viewing the efficient set as a differentiable curve (Sect. A.1); determining bounds on the angles between tangent vectors, gradients, and efficient points (Sect. A.2); and whether a q-ball of certain radius fits inside an ellipsoid (Sect. A.3).

1.1 A.1 The efficient set as a differentiable curve

To begin, from (2), the efficient set can be traced as the curve (see also [32])

$$\begin{aligned} { {z}}(\beta ){:}{=}(\beta H_1+(1-\beta ) H_2)^{-1} (\beta H_1{ {x}}_1^*+(1-\beta )H_2{ {x}}_2^*), \quad \beta \in [0,1], \end{aligned}$$

(34)

where ${ {z}}(0)={ {x}}_2^*={{0}}_q$, ${ {z}}(1)={ {x}}_1^*$, and for simplicity, define the shorthand notation ${ {z}}_n{:}{=}{ {z}}(\beta _n)$ for any subscript n.

By definition, for any efficient point, there exists no direction in which we can improve both objectives at the same time. For each $k,k'\in \{1,2\},k\ne k'$, let the set of descent directions on objective k at ${ {z}}(\beta )\in {\mathcal {E}}$ be denoted by

$$\begin{aligned}{\mathcal {H}}_k({ {z}}(\beta )){:}{=}\{{ {x}}\in \mathbb {R}^q:\langle { {x}}, \nabla f_k({ {z}}(\beta ))\rangle<0\}=\{{ {x}}\in \mathbb {R}^q:\langle { {x}}, H_k({ {z}}(\beta )-{ {x}}_k^*)\rangle <0\},\end{aligned}$$

where ${\mathcal {L}}_k({ {z}}(\beta ))\subset {\mathcal {H}}_k({ {z}}(\beta ))$; the lack of common descent directions for efficient points implies ${\mathcal {H}}_k({ {z}}(\beta ))\cap {\mathcal {H}}_{k'}({ {z}}(\beta ))=\emptyset$. This fact leads to the following Lemma 12 regarding the relative positions of the points in the efficient set and whether or not their level sets intersect.

Lemma 12

For $\beta _1,\beta _2\in [0,1]$, ${\mathcal {L}}_1({ {z}}(\beta _1))\cap {\mathcal {L}}_2({ {z}}(\beta _2))=\emptyset$ if and only if $\beta _2<\beta _1$.

Proof

First, we prove that ${\mathcal {L}}_1({ {z}}(\beta _1))\cap {\mathcal {L}}_2({ {z}}(\beta _2))=\emptyset$ implies $\beta _2<\beta _1$ by contrapositive. Suppose $\beta _2\ge \beta _1$. If $\beta _1=\beta _2$, then ${ {z}}_1={ {z}}_2\in {\mathcal {L}}_1({ {z}}_1)\cap {\mathcal {L}}_2({ {z}}_1)$. If $\beta _2>\beta _1$, then $f_2({ {z}}_1)<f_2({ {z}}_2)$ and $f_1({ {z}}_1)>f_1({ {z}}_2)$. Then for $\beta _3\in (\beta _1,\beta _2)$, $f_2({ {z}}_1)<f_2({ {z}}_3)<f_2({ {z}}_2)$ and $f_1({ {z}}_2)<f_1({ {z}}_3)<f_1({ {z}}_1)$, hence ${ {z}}_3\in {\mathcal {L}}_1({ {z}}_1)\cap {\mathcal {L}}_2({ {z}}_2)$. For the other direction, suppose $\beta _2<\beta _1$. Then for $\beta _3\in (\beta _2,\beta _1)$, $f_2({ {z}}_2)<f_2({ {z}}_3)<f_2({ {z}}_1)$ and $f_1({ {z}}_2)>f_1({ {z}}_3)>f_1({ {z}}_1)$. Since ${ {z}}_3\in {\mathcal {E}}$, ${\mathcal {H}}_1({ {z}}_3)\cap {\mathcal {H}}_2({ {z}}_3)=\emptyset$. Since ${\mathcal {L}}_2({ {z}}_2)\subset {\mathcal {H}}_2({ {z}}_3)$ and ${\mathcal {L}}_1({ {z}}_1)\subset {\mathcal {H}}_1({ {z}}_3)$, then ${\mathcal {L}}_1({ {z}}_1)\cap {\mathcal {L}}_2({ {z}}_2)=\emptyset$. $\square$

Further, for each $k\in \{1,2\},k\ne k'$, [32] and the parameterization in (34) imply that for any $\beta \in [0,1]$,

$$\begin{aligned} - \beta \nabla f_1({ {z}}(\beta ))&= -\beta H_1({ {z}}(\beta )-{ {x}}^*_1) \nonumber \\&=(1-\beta )H_2({ {z}}(\beta )-{ {x}}^*_2) = (1-\beta )\nabla f_2({ {z}}(\beta )), \end{aligned}$$

(35)

and, for $H_\beta {:}{=}\beta H_1+(1-\beta )H_2$, the curve ${ {z}}(\beta )$ is a differentiable curve in $\mathbb {R}^q$ with tangent vector (defined with respect to $\beta$)

$$\begin{aligned} { {z}}'(\beta )&{:}{=}\lim _{\delta \rightarrow 0}\frac{{ {z}}(\beta +\delta )-{ {z}}(\beta )}{\delta } = \left( \frac{\partial z_1(\beta )}{\partial \beta },\ldots ,\frac{\partial z_q(\beta )}{\partial \beta }\right) ^\intercal \nonumber \\&= H_\beta ^{-1} (H_1{ {x}}_1^*-H_2{ {x}}_2^*) - H_\beta ^{-1} (H_1-H_2)H_\beta ^{-1}(\beta H_1{ {x}}_1^*+(1-\beta )H_2{ {x}}_2^*) \nonumber \\&= H_\beta ^{-1} [H_2({ {z}}(\beta )-{ {x}}_2^*)-H_1({ {z}}(\beta )-{ {x}}_1^*)] \nonumber \\&= H_\beta ^{-1} [\nabla f_2({ {z}}(\beta )) -\nabla f_1({ {z}}(\beta ))] \hbox { for }\beta \in [0,1], \end{aligned}$$

(36)

$$\begin{aligned}&= \beta ^{-1}H_\beta ^{-1} \nabla f_2({ {z}}(\beta )) = -(1-\beta )^{-1}H_\beta ^{-1} \nabla f_1({ {z}}(\beta )) \,\,\hbox { for }\beta \in (0,1). \end{aligned}$$

(37)

Importantly, from (36) and (37), it follows that $\Vert { {z}}'(\beta ) \Vert > 0$ for all $\beta \in [0,1]$.

1.2 A.2 Angles: tangent vectors and gradients

In this section, we provide results on the angles between points in the efficient set, tangent vectors, and gradients. For $k,k'\in \{1,2\}, k\ne k'$, consider a point in the efficient set, ${ {z}}_k={ {z}}(\beta _k)\in {\mathcal {E}}$, $\beta _k\in [0,1]$ and any sequence of efficient points

$$\begin{aligned} \{{ {z}}_{k',n} = { {z}}(\beta _{k',n}), n\ge 0\} \end{aligned}$$

(38)

such that ${ {z}}_{k',n}\rightarrow { {z}}_k$ as $n\rightarrow \infty$, ${ {z}}_{k',n}\ne { {z}}_k$ for all n, $\beta _{k',n}\in [0,1]$ for all n, and ${ {z}}_{k',n}$ approaches ${ {z}}_k$ exclusively from one side; that is, $\beta _1<\beta _{2,n}$ or $\beta _{1,n}>\beta _2$ for all n. Define the tangent vector at ${ {z}}_k$ with respect to the direction of approach as

$$\begin{aligned} { {\tau}}({ {z}}_k) {:}{=}\lim _{n\rightarrow \infty }\frac{{ {z}}_{k',n}-{ {z}}_k}{\Vert { {z}}_{k'n}-{ {z}}_k \Vert }, \end{aligned}$$

(39)

and notice that $\Vert { {\tau}}({ {z}}_k) \Vert =1$. Then from (36) and (37),

$$\begin{aligned} { {\tau}}({ {z}}_1) =\frac{ { {z}}'(\beta _1)}{\Vert { {z}}'(\beta _1) \Vert } \text{ and } { {\tau}}({ {z}}_2) = \frac{-{ {z}}'(\beta _2)}{\Vert { {z}}'(\beta _2) \Vert }. \end{aligned}$$

(40)

Therefore if $\beta _k\in (0,1)$, $-\nabla f_k({ {z}}_k)/\Vert \nabla f_k({ {z}}_k) \Vert =\nabla f_{k'}({ {z}}_k)/\Vert \nabla f_{k'}({ {z}}_{k}) \Vert$ and from (37) and (40), we have

$$\begin{aligned} { {\tau}}({ {z}}_k) = \frac{-H_\beta ^{-1}\nabla f_k({ {z}}_k)}{\Vert H_\beta ^{-1}\nabla f_k({ {z}}_k) \Vert } = \frac{H_\beta ^{-1}\nabla f_{k'}({ {z}}_k)}{\Vert H_\beta ^{-1}\nabla f_{k'}({ {z}}_k) \Vert }, \quad \beta _k\in (0,1). \end{aligned}$$

(41)

If $\beta _k\in \{0,1\}$, so that ${ {z}}_1={ {x}}^*_2$ or ${ {z}}_2={ {x}}^*_1$, then from (40) and using $\beta =0$ and $\beta =1$ in (36), we have

$$\begin{aligned} { {\tau}}({ {z}}_k) = { {\tau}}({ {x}}^*_{k'}) = \frac{ -H_{k'}^{-1}\nabla f_{k}({ {x}}^*_{k'}) }{\Vert H_{k'}^{-1}\nabla f_{k}({ {x}}^*_{k'}) \Vert }, \quad \beta _k\in \{0,1\}. \end{aligned}$$

(42)

Figure 4 demonstrates the relative positions of the gradient and tangent vectors at the minimizers for an example in which $q=2$. For general decision space dimension q, [4, Theorem 3.2, p. 369] asserts that ${\mathcal {E}}=\{{ {z}}(\beta ):\beta \in [0,1]\}$ forms a finite arc of a hyperbola whose extreme points are the minimizers ${ {x}}^*_1$ and ${ {x}}^*_2$.

Next, we present Lemma 13 which provides results on the angles between the efficient points, gradients, and tangent vectors at the minimizers.

Lemma 13

Let $k,k'\in \{1,2\},k\ne k'$ and let $\{{ {z}}_{k',n} = { {z}}(\beta _{k',n}), n\ge 0\}$ be the sequence of efficient points converging to ${ {z}}_k={ {z}}(\beta _k)$ defined in (38). The following hold:

• 1. If $\theta _{k,n}$ is the angle between ${ {z}}_{k',n}-{ {z}}_k$ and $-\nabla f_{k}({ {z}}_k)$, then $\cos \theta _{k,n}>0$ for all n.

• 2. If $\theta _k$ is the angle between ${ {\tau}}({ {z}}_{k})$ and $-\nabla f_{k}({ {z}}_k)$, then $\cos \theta _k\ge (\kappa (H_\beta ))^{-1}$.

• 3. There exists a constant $c_0\in [1,\infty )$ such that $\cos \theta _{k,n}\ge c_0^{-1}>0$ for all n.

Proof

Part 1: First, by the definition of $\{{ {z}}_{k',n},n\ge 0\}$ in (38), we have ${ {z}}_{k',n}\notin \{{ {z}}_k, { {x}}^*_k\}$ for all n and ${ {z}}_{k}\ne { {x}}^*_{k}$. Then this result holds because under Assumption 1, for $k,k'\in \{1,2\},k'\ne k$, ${ {z}}_{k'}-{ {z}}_k$ is strictly a descent direction on objective k by (34). Therefore, it must make an acute angle with the steepest descent direction. More formally,

$$\begin{aligned} \cos \theta _{k,n}&=\frac{\langle { {z}}_{k',n}-{ {z}}_{k} , -\nabla f_{k}({ {z}}_k) \rangle }{\Vert { {z}}_{k',n}-{ {z}}_{k} \Vert \Vert \nabla f_{k}({ {z}}_k) \Vert } = \frac{ ({ {z}}_{k}-{ {z}}_{k',n}) ^\intercal H_k ({ {z}}_k-{ {x}}^*_k) }{\Vert { {z}}_{k',n}-{ {z}}_{k} \Vert \Vert \nabla f_{k}({ {z}}_k) \Vert } >0 \hbox { for all }n. \end{aligned}$$

(43)

Part 2: From (43), by continuity of the relevant functions, (41), (42), and the facts that for a square positive definite matrix $H=A^\intercal A$, $\Vert H{ {x}} \Vert \le \Vert H \Vert \Vert { {x}} \Vert$ for vector ${ {x}}$, $\Vert H \Vert =\Vert A \Vert ^2=\Vert Q\Lambda Q^\intercal \Vert =\lambda _q(H)$, and $\Vert H^{-1} \Vert =\Vert Q\Lambda ^{-1}Q^\intercal \Vert =\lambda _q(H^{-1})=1/\lambda _1(H)$,

$$\begin{aligned} \cos \theta _k&= \lim _{n\rightarrow \infty }\cos \theta _{k,n} = \frac{\langle { {\tau}}({ {z}}_{k}) , -\nabla f_{k}({ {z}}_k) \rangle }{\Vert \nabla f_{k}({ {z}}_k) \Vert } = \frac{\Vert \nabla f_{k}({ {z}}_k) \Vert }{\Vert H_\beta ^{-1}\nabla f_k({ {z}}_k) \Vert } \frac{\langle H_\beta ^{-1}\nabla f_k({ {z}}_k) , \nabla f_{k}({ {z}}_k) \rangle }{\Vert \nabla f_{k}({ {z}}_k) \Vert ^2} \\&= \frac{\Vert \nabla f_{k}({ {z}}_k) \Vert }{\Vert H_\beta ^{-1}\nabla f_k({ {z}}_k) \Vert } R(H_\beta ^{-1}, \nabla f_k({ {z}}_k) ) \ge \frac{1 }{\Vert H_\beta ^{-1} \Vert } R(H_\beta ^{-1}, \nabla f_k({ {z}}_k) ) \\&\ge \lambda _1(H_\beta )/ \lambda _q(H_\beta ) \ge 1/\kappa (H_\beta ) >0. \end{aligned}$$

Part 3: This part follows from the first two. $\square$

1.3 A.3 Osculating spheres and inscribed q-Balls

Finally, we complete the supporting results with Lemmas 14–16. First, Lemma 14 gives conditions on the lengths of the major axes of the sublevel sets for a $t$-radius ball touching the boundary to fit inside the sublevel set. Then, Lemma 15 gives conditions on the required distance between two efficient points for a $t$-radius ball to fit inside the intersection of the relevant sublevel sets. Finally, Lemma 16 uses several inscribed balls to determine the distance between a point in $B({\mathcal {E}},t)\cap {\mathcal {J}}_{k'}({ {x}}^*_k)$ and the tangent vector ${\mathcal {T}}_{\mathcal {L}}({ {x}}^*_k)$ for small enough t.

Lemma 14

Let $k\in \{1,2\}$ and ${ {x}}_0\in {\mathcal {X}}$. If $\hbox { diam(}{\mathcal {L}}_k({ {x}}_0)\hbox {)}\ge 2\kappa ^*t$ and ${ {x}}_1\in {\mathcal {L}}_k({ {x}}_0)$ is such that ${ {x}}_1 = { {x}}_0-t(\nabla f_k({ {x}}_0)/\Vert \nabla f_k({ {x}}_0) \Vert )$, then $B({ {x}}_1,t)\subset {\mathcal {L}}_k({ {x}}_0)$.

Proof (Sketch)

We show only $k=2$; $k=1$ holds by a similar process. Let ${ {x}}_0\in {\mathcal {X}}$, $\lambda _{2i}{:}{=}\lambda _i(H_2)$, $i=1,\ldots ,q$, and notice ${ {x}}_1={ {x}}_0-t(H_2{ {x}}_0/\Vert H_2{ {x}}_0 \Vert )$ is a step of length $t$ from ${ {x}}_0$ in the direction of steepest descent on objective 2. Thus, ${ {x}}_0\in \hbox { bd(}B({ {x}}_1,t)\hbox {)}$. To simplify calculations, for ${\mathcal {L}}_2({ {x}}_0)$, apply the rotation ${ {w}}=Q_2^\intercal { {x}}, { {a}}_0=Q_2^\intercal { {x}}_0$ from the proof of Lemma 5. Define ${\mathcal {L}}_{2r}({ {a}}_0) {:}{=}\{{ {w}}\colon{ {w}}^\intercal \Lambda _2{ {w}}\le { {a}}_0^\intercal \Lambda _2{ {a}}_0\}$, ${\mathcal {J}}_{2r}({ {a}}_0){:}{=}\hbox { bd(}{\mathcal {L}}_{2r}({ {a}}_0) \hbox {)}$. Now, we consider ${ {a}}_1 = {{ {a}}_0-t(\Lambda _2{ {a}}_0)/\Vert \Lambda _2{ {a}}_0 \Vert }$, where ${ {a}}_0\in \hbox { bd(}B({ {a}}_1,t)\hbox {)}$.

To ensure $B({ {a}}_1,t)\subset {\mathcal {L}}_{2r}({ {a}}_0)$, the maximum curvature at any point on ${\mathcal {J}}_{2r}({ {a}}_0)$ must be less than or equal to that at any point on a $t$-radius sphere. The maximum curvature on an ellipse is achieved at a vertex along the major axis; see, e.g., [31]. To determine the maximum curvature at a point on ${\mathcal {J}}_{2r}({ {a}}_0)$, it is sufficient to consider the intersection of ${\mathcal {J}}_{2r}({ {a}}_0)$ with the $w_1$-$w_q$ plane (see [18, p. 43], [15, p. 33]) and let ${ {a}}_0=(a_{01},0,\ldots ,0)$, resulting in the ellipse $\{(w_1,w_q)\colon\lambda _{21}w_1^2+\lambda _{2q} w_q^2 = \lambda _{21} a_{01}^2 \}=\{(w_1,w_q)\colon w_1^2/a_{01}^2 + w_q^2/(a_{01}^2/\kappa _2) = 1 \}$. Write this ellipse as the plane curve $g(s)=(a_{01}\cos (s),(a_{01}/\sqrt{\kappa _2})\sin (s))$. The length of the semi-major axis is $a_{01}$, the length of the semi-minor axis is $(a_{01}/\sqrt{\kappa _2})$, and the maximum curvature is $a_{01}/(a_{01}^2/\kappa _2)=\kappa _2/a_{01}$ at $s=0$ [31, p. 77]. Then for an osculating circle at $(a_{11},a_{1q})=(a_{01}-t,0)$ having radius $t$, curvature $1/t$, and passing through $(a_{01},0)$ to fit inside the ellipse, we require $(1/t)\ge \kappa _2/a_{01}$, hence $a_{01}\ge t\kappa _2$. Now since $a_{01}=\hbox { diam(}{\mathcal {L}}_{2r}({ {a}}_0)\hbox {)}/2 =\hbox { diam(}{\mathcal {L}}_{2}({ {x}}_0)\hbox {)}/2\ge t\kappa _2$, then $B({ {a}}_1,t)\subset {\mathcal {L}}_{2r}({ {a}}_0)$ and $B({ {x}}_1,t)\subset {\mathcal {L}}_2({ {x}}_0)$. $\square$

Lemma 15

Let ${ {z}}_1,{ {z}}_2\in {\mathcal {E}}, \beta _1,\beta _2\in [0,1], \beta _1<\beta _2$ be efficient points. There exists a constant $\eta _0\in [1,\infty )$, dependent on $\kappa ^*$, such that if $\Vert { {z}}_1-{ {z}}_{2} \Vert > 2t\kappa ^*\eta _0,$ then there exists ${ {x}}_0\in \hbox { conv}(\{{ {z}}_1,{ {z}}_2\})$ such that $B({ {x}}_0,t)\subset \hbox { int}({\mathcal {L}}_1({ {z}}_1)\cap {\mathcal {L}}_{2}({ {z}}_{2}))$.

Proof

The main idea for the proof is that for each level set ${\mathcal {L}}_k({ {z}}_k)$, $k\in \{1,2\}$, we can use Lemma 14 to fit a (relatively large) osculating ball at ${ {z}}_k$. Then we fit a smaller t-radius ball, centered at a point ${ {w}}_k$ along $\hbox { conv(}\{{ {z}}_1,{ {z}}_2\}\hbox {)}$, inside this larger osculating ball. As long as $\Vert { {z}}_1-{ {z}}_2 \Vert$ is sufficiently large and the respective centers ${ {w}}_1$ and ${ {w}}_2$ of the t-radius balls are appropriately spaced along $\hbox { conv(}\{{ {z}}_1,{ {z}}_2\}\hbox {)}$, the lemma holds.

To begin, let the postulates hold. Since $\beta _1<\beta _2$, ${\mathcal {L}}_1({ {z}}_1)\cap {\mathcal {L}}_2({ {z}}_2)\ne \emptyset$ by Lemma 12. To fit the osculating balls inside the level sets, first, for each $k\in \{1,2\}$, let ${ {x}}_k$ be the point resulting from a step of size $u_{k}(t)$ from ${ {z}}_k$ in the direction of steepest descent on objective k, so that

$$\begin{aligned} { {x}}_k={ {z}}_k-u_{k}(t)(\nabla f_k({ {z}}_k)/\Vert \nabla f_k({ {z}}_k) \Vert ), \end{aligned}$$

and $\Vert { {x}}_k-{ {z}}_k \Vert =u_{k}(t)$. By Lemma 14, if

$$\begin{aligned} \hbox { diam(}{\mathcal {L}}_k({ {z}}_k)\hbox {)} \ge 2\kappa ^* u_{k}(t) \end{aligned}$$

(44)

then $B({ {x}}_k, u_{k}(t))\subset {\mathcal {L}}_k({ {z}}_k)$. Let ${ {w}}_k$ be the orthogonal projection of ${ {x}}_k$ onto the line segment $\hbox { conv(}\{{ {z}}_1,{ {z}}_2\}\hbox {)}$, and let $\zeta _k$ be the angle between ${ {w}}_k-{ {z}}_k,{ {x}}_k-{ {z}}_k$. Since $\zeta _1,\zeta _2$ are also the angles between ${ {z}}_2-{ {z}}_1, -\nabla f_1({ {z}}_1)$ and between ${ {z}}_1-{ {z}}_2, -\nabla f_2({ {z}}_2)$, respectively, the results in Lemma 13 apply. Thus, $\cos \zeta _k=\sin (\pi /2-\zeta _k)\ge c_0^{-1}>0$ implies $\zeta _k\in [0,\pi /2)$, and hence there exists $\tilde{c}_0\in [1,\infty )$ such that $1-\sin \zeta _k\ge (\tilde{c}_0)^{-1}>0$ regardless of t and the choice of ${ {z}}_1,{ {z}}_2$. To ensure $B({ {w}}_k,t)\subset B({ {x}}_k,u_{k}(t))$, we require

$$\begin{aligned} u_{k}(t) - \Vert { {w}}_k-{ {x}}_k \Vert = u_{k}(t) - \Vert { {x}}_k-{ {z}}_k \Vert \sin \zeta _k = u_{k}(t)(1 - \sin \zeta _k) \ge t. \end{aligned}$$

(45)

Finally, for each $k,k'\in \{1,2\},k'\ne k$, by the definition of ${ {w}}_k$, we have $\Vert { {z}}_k-{ {w}}_k \Vert +\Vert { {w}}_k-{ {z}}_{k'} \Vert =\Vert { {z}}_1-{ {z}}_2 \Vert$. Then to ensure $\Vert { {z}}_k-{ {w}}_k \Vert \le \Vert { {w}}_k-{ {z}}_{k'} \Vert$, we require

$$\begin{aligned} 2\Vert { {z}}_k-{ {w}}_k \Vert = 2\Vert { {x}}_k-{ {z}}_k \Vert \cos \zeta _k = 2u_{k}(t)\cos \zeta _k \le \Vert { {z}}_1-{ {z}}_2 \Vert . \end{aligned}$$

(46)

Combining the requirements of (44), (45), and (46), it follows that

$$\begin{aligned} \frac{t}{1-\sin \zeta _k} \le u_{k}(t) \le \min \left\{ \frac{\hbox { diam(}{\mathcal {L}}_k({ {z}}_k)\hbox {)} }{ 2\kappa ^*},\, \frac{\Vert { {z}}_1-{ {z}}_2 \Vert }{2\cos \zeta _k} \right\} . \end{aligned}$$

(47)

If we can find values of $u_1(t), u_2(t)$, and a requirement on the distance $\Vert { {z}}_1-{ {z}}_2 \Vert$ such that (47) is satisfied, then for each $k,k'\in \{1,2\},k'\ne k$, we have $B({ {w}}_k,t)\subseteq B({ {x}}_k,u_k(t))\subset {\mathcal {L}}_k({ {z}}_k)$ and $\Vert { {z}}_1-{ {z}}_2 \Vert = \Vert { {z}}_1-{ {w}}_1 \Vert +\Vert { {w}}_1-{ {w}}_2 \Vert +\Vert { {w}}_2-{ {z}}_2 \Vert$, which implies that $B({ {w}}_k,t)\subset {\mathcal {L}}_{k'}({ {z}}_{k'})$. Since ${\mathcal {L}}_1({ {z}}_1)\cap {\mathcal {L}}_2({ {z}}_2)$ is convex, then $\hbox { conv(}B({ {w}}_1,t)\cup B({ {w}}_2,t)\hbox {)}\subset {\mathcal {L}}_1({ {z}}_1)\cap {\mathcal {L}}_2({ {z}}_2)$. Thus, $\exists \,{ {x}}_0\in \hbox { conv(}\{{ {w}}_1,{ {w}}_2\}\hbox {)}$ such that the result holds.

Finally, we find $u_1(t), u_2(t)$, and a requirement on $\Vert { {z}}_1-{ {z}}_2 \Vert$ such that (47) is satisfied. First, for each $k\in \{1,2\}$, $\beta _1<\beta _2$ and the triangle inequality imply

$$\begin{aligned} \Vert { {z}}_1-{ {z}}_2 \Vert \le \Vert { {z}}_1-{ {x}}^*_k \Vert +\Vert { {z}}_2-{ {x}}^*_k \Vert \le 2 \Vert { {z}}_k-{ {x}}^*_k \Vert \le \hbox { diam(}{\mathcal {L}}_k({ {z}}_k)\hbox {)}. \end{aligned}$$

Since $\kappa ^*\ge 1$ while $\cos \zeta _k\le 1$, and since $1-\sin \zeta _k\ge (\tilde{c}_0)^{-1}$, we satisfy (47) if

$$\begin{aligned} \frac{t}{1-\sin \zeta _k} \le t\tilde{c}_0 \le u_{k}(t) \le \frac{\Vert { {z}}_1-{ {z}}_2 \Vert }{ 2\kappa ^*}. \end{aligned}$$

(48)

Now select $u_k(t)=t\tilde{c}_0$ for each $k\in \{1,2\}$ and let $\eta _0{:}{=}\tilde{c}_0$. Since $\Vert { {z}}_1-{ {z}}_2 \Vert > 2t\kappa ^*\eta _0$, the result holds. $\square$

Lemma 16

Let $k\in \{1,2\}$, and let ${ {y}}$ be a point in the intersection of the t-expansion of ${\mathcal {E}}$ with the boundary of the sublevel set ${\mathcal {L}}_{k'}({ {x}}^*_k)$; that is, ${ {y}}\in B({\mathcal {E}},t)\cap {\mathcal {J}}_{k'}({ {x}}^*_k)$. Then there exists $t_1'>0$ and a constant $c_1\in [1,\infty )$, both dependent on $\kappa ^*$, such that for all $t\le t_1'$, ${\text{ dist}}({ {y}},{\mathcal {T}}_{\mathcal {L}}({ {x}}^*_k))\le t c_1$.

Proof

We find an upper bound on ${\text{dist}}({ {y}},{\mathcal {T}}_{\mathcal {L}}({ {x}}^*_k))$ by fitting a series of three balls inside the sublevel set ${\mathcal {L}}_{k'}({ {x}}^*_k)$. The first ball is the largest ball, constructed as follows. Take a step of size $\ell /\kappa _{k'}$ from ${ {x}}^*_{k}$ in the steepest descent direction,

$$\begin{aligned} { {x}}_1={ {x}}_{k}^*-(\ell /\kappa _{k'})\nabla f_{k'}({ {x}}^*_k)/\Vert \nabla f_{k'}({ {x}}^*_k) \Vert . \end{aligned}$$

Then Lemma 14 implies $B({ {x}}_1,\ell /\kappa _{k'})\subset {\mathcal {L}}_{k'}({ {x}}^*_k)$. Next, we construct the second ball by taking a step $s_k(t)<\Vert { {x}}_1-{ {x}}^*_k \Vert =\ell /\kappa _{k'}$ from ${ {x}}^*_k$ in the steepest descent direction,

$$\begin{aligned} { {x}}_2={ {x}}_{k}^*-s_k(t)\nabla f_{k'}({ {x}}^*_k)/\Vert \nabla f_{k'}({ {x}}^*_k) \Vert . \end{aligned}$$

Let ${ {z}}_3({ {x}}_2)$ be the efficient point such that ${ {x}}_2$ is its orthogonal projection onto the line $\hbox { conv(}\{{ {x}}_1,{ {x}}_k^*\}\hbox {)}$; such a point exists for small enough $s_k(t)$. We choose $s_k(t)$ so that the third ball, $B({ {z}}_3({ {x}}_2),t)$, is a subset of the first ball. That is, we find $s_k(t)$ such that $B({ {z}}_3({ {x}}_2),t)\subset$ $B({ {x}}_1,\ell /\kappa _{k'}) \subset$ ${\mathcal {L}}_{k'}({ {x}}^*_k)$. To find $s_k(t)$, first, notice that if we ensure

$$\begin{aligned} \ell /\kappa _{k'}-\Vert { {z}}_3({ {x}}_2)-{ {x}}_1 \Vert \ge t, \end{aligned}$$

(49)

it follows that $B({ {z}}_3({ {x}}_2),t)\subset B({ {x}}_1,\ell /\kappa _{k'})$. Let $\theta _{3,t}$ be the angle between ${ {z}}_3({ {x}}_2)-{ {x}}_k^*$ and ${ {x}}_2-{ {x}}_k^*$. By Lemma 13, $\cos \theta _{3,t}\ge c_0^{-1}$ for all t. Then to ensure (49) holds, we need $s_k(t)>t$ such that $\Vert { {z}}_3({ {x}}_2)-{ {x}}_1 \Vert ^2 = (\ell /\kappa _{k'}-s_k(t))^2+(s_k(t)\tan \theta _{3,t})^2 \le (\ell /\kappa _{k'}-t)^2;$ re-arranging terms implies

$$\begin{aligned} \frac{s_k(t)^2(1+\tan ^2\theta _{3,t})-t^2}{s_k(t)-t} = \frac{s_k(t)^2/\cos ^2\theta _{3,t}-t^2}{s_k(t)-t} \le 2\ell /\kappa _{k'}. \end{aligned}$$

(50)

Choose $s_k(t)=t(\kappa _{k'}+1)$ and plug into (50) to see that (49) holds for all

$$\begin{aligned}t\le \frac{2\ell c_0^{-2}}{(\kappa ^*+1)^2-1} \le \frac{2\ell \cos ^2\theta _{3,t}}{(\kappa _{k'}+1)^2-\cos ^2\theta _{3,t}}. \end{aligned}$$

Now for t small enough and $s_k(t)= t(\kappa _{k'}+1)$, we have $B({ {z}}_3({ {x}}_2),t)$ $\subset$ $B({ {x}}_1,\ell /\kappa _{k'})$ $\subset$ ${\mathcal {L}}_{k'}({ {x}}^*_k)$. Let ${ {y}}_3$ be any point in $B({ {z}}_3({ {x}}_2),t)\subset B({\mathcal {E}},t)$ such that ${\text{ dist}}({ {y}}_3,{\mathcal {T}}_{\mathcal {L}}({ {x}}^*_k))\ge {\text{ dist}}({ {y}},{\mathcal {T}}_{\mathcal {L}}({ {x}}^*_k))$; such a point exists by construction. Now, the result holds since for all t small enough and $c_1{:}{=}(1+c_0(\kappa ^*+1))$, we have

$$\begin{aligned} {\text{ dist}}({ {y}},{\mathcal {T}}_{\mathcal {L}}({ {x}}^*_k))&\le {\text{ dist}}({ {y}}_3,{\mathcal {T}}_{\mathcal {L}}({ {x}}^*_k)) \le t + \Vert { {z}}_3({ {x}}_2)-{ {x}}^*_k \Vert \\&\le t+s_k(t)/\cos \theta _{3,t} \le t+c_0(\kappa _{k'}+1)t \le t c_1. \end{aligned}$$

$\square$