The generalized trust region subproblem

Abstract

The interval bounded generalized trust region subproblem (GTRS) consists in minimizing a general quadratic objective, q ₀(x)→min, subject to an upper and lower bounded general quadratic constraint, ℓ≤q ₁(x)≤u. This means that there are no definiteness assumptions on either quadratic function. We first study characterizations of optimality for this implicitly convex problem under a constraint qualification and show that it can be assumed without loss of generality. We next classify the GTRS into easy case and hard case instances, and demonstrate that the upper and lower bounded general problem can be reduced to an equivalent equality constrained problem after identifying suitable generalized eigenvalues and possibly solving a sparse system. We then discuss how the Rendl-Wolkowicz algorithm proposed in Fortin and Wolkowicz (Optim. Methods Softw. 19(1):41–67, 2004) and Rendl and Wolkowicz (Math. Program. 77(2, Ser. B):273–299, 1997) can be extended to solve the resulting equality constrained problem, highlighting the connection between the GTRS and the problem of finding minimum generalized eigenvalues of a parameterized matrix pencil. Finally, we present numerical results to illustrate this algorithm at the end of the paper.

Appendix: Proof of Theorem 2.1(ii)

Proposition A.1

Suppose that b=0 and Items 1, 2 and 3 of Assumption 2.1 are satisfied. If GTRS (1.1) is bounded below, then D-GTRS (2.2) is feasible.

Proof

We first consider the case when B is positive semidefinite. Then there exists an invertible matrix P such that

$$ B = P \left[ \begin{array}{c@{\quad }c} 0&0\\ 0&I \end{array}\right] P^T. $$

Thus, after a change of variables y=P ^T x, GTRS (1.1) can be equivalently written as

$$ \begin{array}{r@{\quad }l} \inf& q_0(y):= \begin{bmatrix} y_1\\y_2 \end{bmatrix} ^T \left[ \begin{array}{c@{\quad }c} \bar{A}_1&\bar{A}_2^T\\ \bar{A}_2& \bar{A}_3 \end{array}\right] \begin{bmatrix} y_1\\y_2 \end{bmatrix} - 2 \begin{bmatrix} \bar{a}_1\\ \bar{a}_2 \end{bmatrix} ^T \begin{bmatrix} y_1\\y_2 \end{bmatrix} \\ \mbox{s.t.} & \ell\leq\|y_2\|^2 \leq u, \end{array} $$

(A.1)

where

$$ y = \begin{bmatrix} y_1\\y_2 \end{bmatrix},\quad \bar{A} = \left[ \begin{array}{c@{\quad }c} \bar{A}_1&\bar{A}_2^T\\ \bar{A}_2& \bar{A}_3 \end{array}\right] = P^{-1}AP^{-T},\quad \bar{a} = \begin{bmatrix} \bar{a}_1\\ \bar{a}_2 \end{bmatrix} = P^{-1}a. $$

(A.2)

By our assumptions, the program (A.1) is bounded below. Since

$$q_0(y) = y_1^T\bar{A}_1y_1 + 2y_2^T\bar{A}_2y_1 + y_2^T\bar{A}_3y_2 - 2\bar{a}_1^Ty_1 - 2\bar{a}_2^T y_2, $$

and the constraint does not involve y ₁, we must have

$$ \bar{A}_1\succeq0,\qquad \bar{A}_2^Ty_2 - \bar{a}_1\in \operatorname{{Range}}(\bar{A}_1),\quad \forall\ \ell\le \|y_2\|^2\le u. $$

(A.3)

Notice that u>0 since the RICQ (2.6) holds. Hence, the second relation in (A.3) implies

$$ \operatorname{{Range}}\bigl(\bar{A}_2^T\bigr)\subseteq \operatorname{{Range}}(\bar{A}_1)\quad \mathrm{and}\quad \bar{a}_1\in \operatorname{{Range}}(\bar{A}_1). $$

(A.4)

By (A.3), (A.4) and a consideration of the Schur complement, we conclude that for all sufficiently large λ>0, $\bar{A}_{3}+\lambda I \succ0$ and

$$ \bar{A} + \lambda \left[ \begin{array}{c@{\quad }c} 0&0\\ 0&I \end{array}\right] = \left[ \begin{array}{c@{\quad }c} \bar{A}_1&\bar{A}_2^T\\ \bar{A}_2& \bar{A}_3+\lambda I \end{array} \right] \succeq0. $$

(A.5)

Moreover, from (A.4), we see that, for all sufficiently large λ>0,

$$\operatorname{{Range}}\bigl(\bar{A}_1 - \bar{A}_2^T[\bar{A}_3+\lambda I]^{-1}\bar{A}_2\bigr) = \operatorname{{Range}}(\bar{A}_1), $$

and hence $\bar{a}_{1}\in \operatorname{{Range}}(\bar{A}_{1} - \bar{A}_{2}^{T}[\bar{A}_{3}+\lambda I]^{-1}\bar{A}_{2})$. Thus, for sufficiently large λ>0, the system of equations

$$ \left[ \begin{array}{c@{\quad }c} \bar{A}_1 - \bar{A}_2^T[\bar{A}_3+\lambda I]^{-1}\bar{A}_2 & 0\\ 0& \bar{A}_3+\lambda I \end{array} \right] \left[ \begin{array}{c} w_1\\ w_2 \end{array} \right] = \left[ \begin{array}{c} \bar{a}_1 - \bar{A}_2^T[\bar{A}_3+\lambda I]^{-1}\bar{a}_2\\ \bar{a}_2 \end{array} \right] $$

(A.6)

is consistent. Since

$$\begin{aligned} &\left[ \begin{array}{c@{\quad }c} \bar{A}_1 & \bar{A}_2^T\\ \bar{A}_2 & \bar{A}_3+\lambda I \end{array}\right] \\ & = \left[ \begin{array}{c@{\ \ }c} I & \bar{A}_2^T[\bar{A}_3\,{+}\,\lambda I]^{-1}\\ 0 & I \end{array}\right] \left[ \begin{array}{c@{\ \ }c} \bar{A}_1 \,{-}\, \bar{A}_2^T[\bar{A}_3\,{+}\,\lambda I]^{-1}\bar{A}_2 & 0\\ 0& \bar{A}_3\,{+}\,\lambda I \end{array}\right] \left[ \begin{array}{c@{\ \ }c} I & 0\\ {[}\bar{A}_3\,{+}\,\lambda I]^{-1}\bar{A}_2 & I \end{array}\right], \end{aligned}$$

and

$$\begin{aligned} \left[ \begin{array}{c@{\quad }c} I & \bar{A}_2^T[\bar{A}_3+\lambda I]^{-1}\\ 0 & I \end{array} \right] ^{-1} \left[ \begin{array}{c} \bar{a}_1\\ \bar{a}_2 \end{array}\right] =& \left[ \begin{array}{c@{\quad }c} I & -\bar{A}_2^T[\bar{A}_3+\lambda I]^{-1}\\ 0 & I \end{array} \right] \left[ \begin{array}{c} \bar{a}_1\\ \bar{a}_2 \end{array} \right]\\ =& \left[ \begin{array}{c} \bar{a}_1 - \bar{A}_2^T[\bar{A}_3+\lambda I]^{-1}\bar{a}_2\\ \bar{a}_2 \end{array}\right], \end{aligned}$$

we see from the consistency of (A.6) that for sufficiently large λ>0. From the definition of $\bar{a}$ and $\bar{A}$ in (A.2), we obtain immediately that $a\in \operatorname{{Range}}(A + \lambda B)$ for sufficiently large λ>0. This together with (A.5) proves that D-GTRS is feasible.

The case when B is negative semidefinite can be tackled similarly.

Next, we consider the case when B is indefinite. Suppose to the contrary that D-GTRS is infeasible. Then either A−λB is not positive semidefinite for any $\lambda\in {\mathbb{R}}$, or there exists λ with A−λB⪰0 but all such λ satisfy $a\notin \operatorname{{Range}}(A - \lambda B)$.

Case 1:

A−λB is not positive semidefinite for all $\lambda\in {\mathbb{R}}$. By [24, Theorem 2.3], there exists u ^∗ satisfying u ^{∗ T} Au ^∗<0 and u ^{∗ T} Bu ^∗=0.

Fix any s∈[ℓ,u]. For the above u ^∗ and each t>0, we would like to find a solution x _t of (x+tu ^∗)^T B(x+tu ^∗)=s that is uniformly bounded in t. For s=0, we just take x _t =0. In the case when s>0, the x we seek has to satisfy

$$ x^TBx + 2tx^TBu^*=s. $$

Let B=PDP ^T for some orthogonal matrix P and diagonal matrix D, and let β _i denote the ith diagonal entry of D. Also, let $\tilde{u}^{*}$ denote P ^T u ^∗. Since B is indefinite, there exists an index i ₀ such that $\beta_{i_{0}}>0$. Consider the quadratic equation

$$ \beta_{i_0} r^2 + 2 t\beta_{i_0}{\tilde{u}^*_{i_0}}r - s = 0. $$

It is clear that the solutions of the above quadratic equation are

$$\begin{aligned} r^1_t = \frac{-2 t\beta_{i_0}{\tilde{u}^*_{i_0}}+\sqrt{(2 t\beta _{i_0}{\tilde{u}^*_{i_0}})^2 + 4s\beta_{i_0}}}{ 2\beta_{i_0}}=\frac{2s}{2 t\beta_{i_0}{\tilde{u}^*_{i_0}}+\sqrt{(2 t\beta_{i_0}{\tilde{u}^*_{i_0}})^2 + 4s\beta_{i_0}}}, \\ r^2_t = \frac{-2 t\beta_{i_0}{\tilde{u}^*_{i_0}}-\sqrt{(2 t\beta _{i_0}{\tilde{u}^*_{i_0}})^2 + 4s\beta_{i_0}}}{2\beta_{i_0}} =\frac{2s}{2 t\beta_{i_0}{\tilde{u}^*_{i_0}}-\sqrt{(2 t\beta _{i_0}{\tilde{u}^*_i})^2 + 4s\beta_{i_0}}}. \end{aligned}$$

Moreover, it is easy to see that $r^{1}_{t}$ is uniformly bounded for t>0 when $\tilde{u}^{*}_{i_{0}}\ge0$, while $r^{2}_{t}$ is uniformly bounded for t>0 when $\tilde{u}^{*}_{i_{0}}\le0$. Define

$$x_t = \begin{cases} r^1_tPe_{i_0}& \mathrm{if}\ \tilde{u}^*_{i_0}\ge0,\\ r^2_tPe_{i_0}& \mathrm{otherwise}, \end{cases} $$

where $e_{i_{0}}$ is the vector which is one at the i ₀th entry and is zero otherwise. Then {x _t } is uniformly bounded in t>0 and satisfies $x_{t}^{T}Bx_{t} + 2tx_{t}^{T}Bu^{*}=s$. The case when s<0 can be considered similarly, by picking the index such that β _i <0.

Next, since {x _t } is bounded for t>0 and u ^{∗ T} Au ^∗<0, we see that

$$\begin{aligned} &\bigl(x_t+tu^*\bigr)^TA\bigl(x_t+tu^* \bigr)-2a^T\bigl(x_t+tu^*\bigr)\\ &\quad {} = x_t^TAx_t +2tx_t^TAu^* + t^2 {u^*}^TAu^*-2a^T \bigl(x_t+tu^*\bigr)\rightarrow-\infty \end{aligned}$$

as t→∞. This together with the feasibility of x _t +tu ^∗ for all t>0 contradicts the boundedness of GTRS.

Case 2:

There exists λ with A−λB⪰0 but all such λ satisfy $a\notin \operatorname{{Range}}(A - \lambda B)$. In this case, we apply [4, Theorem A.2] to derive a contradiction. To this end, let s∈[ℓ,u] be such that the set {x: x ^T Bx=s} is nonempty. Then, by assumption, we see that the equality constrained GTRS (3.1), with this s, is bounded below. Thus, f ₂(x):=x ^T Ax−2a ^T x−L is nonnegative on the set {x: f ₁(x)=0}, where L is a lower bound of the optimal value of the equality constrained GTRS, and f ₁(x):=x ^T Bx−s. Furthermore, since B is indefinite, there exist x ₁, x ₂ such that

$$ f_1(x_1)> 0\ \ \mathrm{and}\ \ f_1(x_2)<0. $$

(A.7)

Using these and invoking [4, Theorem A.2], we conclude that there exists μ such that

$$ \left( \begin{array}{c@{\quad }c} -L +\mu s& -a^T\\ -a& A -\mu B \end{array} \right) \succeq0. $$

From Schur complement, this implies that A−μB⪰0 and $a\in \operatorname{{Range}}(A - \mu B)$. Thus, D-GTRS is feasible, a contradiction to our assumption.

Combining Cases 1 and 2, we conclude that D-GTRS is also feasible when B is indefinite. □