Optimal service rate perturbations of many server queues in heavy traffic

Abstract

An optimal control problem for a single customer class, many server queueing system of the type $G/M/n+GI$ is considered, where the control corresponds to the service rate. An infinite horizon discounted cost functional which consists of a convex control cost, linear delay and idle server costs, and a linear abandonment cost is formulated. We study this problem in the heavy traffic regime originally proposed by Halfin and Whitt, where the arrival rates and the number of servers grow to infinity in concert. First we address the diffusion control problem (DCP) associated with the heavy traffic limit. By constructing a smooth solution to the associated Hamilton–Jacobi–Bellman equation, we obtain a feedback type optimal control for the DCP. We show that the value function of the DCP is an asymptotic lower bound for the value functions of the corresponding queueing control problems. We use the optimal control of the DCP to obtain an asymptotically optimal control policy for the queueing control problem.

Appendix: Proof of Proposition 3.1

Here we furnish a proof for Proposition 3.1, which guarantees a smooth solution to the HJB equation (3.13). This proof involves several technical results, but the idea is quite simple: we consider the non-linear differential equation (3.13) as an initial value problem separately on each of the intervals $(-\infty , 0]$ and $[0, \infty )$. On $(-\infty , 0]$, for each $a \ge \dfrac{-r_0}{\alpha + \mu _0},$ we consider a solution $Q$ for (3.13) with initial data $Q'(0)=a$ and $Q''(0)=r$ where $r \ge 0$. Such a solution exists only on an interval $(-\omega , 0],$ where $-\omega < 0$ is the explosion point of the solution $Q.$ Then we show that for each such $a$, there is a unique $\theta _a \ge 0$ such that (3.13) has a solution $Q_a$ which exists on the whole interval $(-\infty , 0]$ with the initial data $Q_a'(0)=a$ and $Q_a''(0)=\theta _a.$ Moreover, $Q'_a$ is bounded, strictly increasing and $\lim \nolimits _{x \rightarrow -\infty } Q'_a(x) = \dfrac{-r_0}{\alpha + \mu _0}.$ Similarly, we show that for each $a ,$ there is a unique constant $\eta _a$ and a solution $\tilde{Q}_a$ to (3.13) on the interval $[0, \infty ),$ such that $\tilde{Q}_a(0) =a,\,\tilde{Q}'_a(0)=\eta _a$ and the derivative $\tilde{Q}'_a$ is bounded on $[0, \infty )$. In the final step, we observe that the mapping $a \rightarrow (\theta _a, \eta _a)$ is continuous and we establish that there exists a point $a^*$ so that $\theta _{a^*}=\eta _{a^*}$. Then we can smoothly paste the two solutions $Q_{a^*}$ and $\tilde{Q}_{a^*}$ at the origin to obtain a smooth solution to the HJB equation (3.13) as required in Proposition 3.1. Moreover, the derivative of this smooth solution is bounded on $\mathbf {R}$.

In the following proof, we establish all the steps of the above described procedure to obtain a smooth solution to the HJB equation (3.13) which satisfies the conditions described in Proposition 3.1. Notice that in (3.13), $\phi$ is a continuously differentiable, non-negative convex function, $h$ and $H_0$ are locally Lipschitz continuous functions. The following basic facts are quite useful for our proof:

• Given initial data $Q(0)$ and $Q'(0)$, there exists a $\delta _0 >0$ so that a unique solution to (3.13) exists on the interval $(-\delta _0, \delta _0).$ This follows from the theory of differential equations using the fact that $\phi ,\,h$ and $H_0$ are locally Lipschitz continuous functions.

• If $Q$ is a solution to (3.13) on an interval $(-\delta _0, \delta _0)$ and $W(x)=Q'(x)$, then by (3.13), $\alpha Q(0)= \dfrac{\sigma ^2}{2} W'(0) -\mu _0 \beta _0W(0) -\phi (\mu _0 W(0)).$ Therefore, if $W$ exists in an interval around the origin, then we can recover the function $Q$ on the same interval.

• If $Q$ is a solution to (3.13) on $(-\delta _0, \delta _0)$ and if $W(x)=Q'(x)$, then by differentiating (3.13), we observe that W satisfies

  $$\begin{aligned}&\dfrac{\sigma ^2}{2}W''(x)-\mu _0(\beta _0 +x)W'(x) - \mu _0 \psi (\mu _0 W(x)) W'(x)\nonumber \\&\quad -(\alpha + \mu _0)W(x) -r_0 = 0, \end{aligned}$$

  (6.1)

  on the interval $(-\delta _0, 0)$, and

  $$\begin{aligned}&\dfrac{\sigma ^2}{2}W''(x)-\mu _0\Big (\beta _0 +H_0(x)\Big )W'(x) - \mu _0\psi \Big (\mu _0 W(x)\Big ) W'(x) \nonumber \\&\quad - \Big (\alpha +\mu _0 H_0'(x)\Big )W(x) +b_0 + p \mu _0 H_0'(x) = 0, \end{aligned}$$

  (6.2)

  on $(0, \delta _0).$

• $\tilde{Y}(x)= \dfrac{-r_0}{\alpha + \mu _0}$ is a stationary solution for the Eq. (6.1) on the interval $(-\infty , 0).$

• Since $\alpha Q(x)= \dfrac{\sigma ^2}{2}W'(0)-\mu _0\beta _0 W(0) -\phi (\mu _0 W(0))+ \alpha \int _{0}^{x} W(u) \mathrm{{d}}u,$ when $x<0$, the equation (3.13) can be written as

  $$\begin{aligned}&\dfrac{\sigma ^2}{2} W'(x) - \mu _0(\beta _0 +x)W(x) -\phi \Big (\mu _0 W(x)\Big ) + r_0 x^- \nonumber \\&\quad =\dfrac{\sigma ^2}{2}W'(0)-\mu _0 \beta _0 W(0) -\phi \Big (\mu _0 W(0)\Big ) + \alpha \int _{0}^{x} W(u) \mathrm{{d}}u. \end{aligned}$$

  (6.3)

1.1 Construction on $(-\infty , 0]$

We intend to establish the following proposition and its proof will require several lemmas.

Proposition 6.1

For each $a > \dfrac{-r_0}{\alpha + \mu _0},$ there exists a unique pair $(W_a, \theta _a)$ such that $W_a$ is a function defined on $(-\infty , 0]$ and $\theta _a \ge 0$ is a real number satisfying the following:

• (i) $W_a(0)=a,\,W'_a(0)=\theta _a,\,W_a$ satisfies (6.3) on $(-\infty , 0]$.

• (ii) $W_a(\cdot )$ is bounded, strictly increasing on $(-\infty , 0]$, and $\lim \nolimits _{x \rightarrow -\infty } W_a(x) = \dfrac{-r_0}{\alpha + \mu _0}.$

Moreover, let the function $Q_a$ be defined on $(-\infty , 0]$ by $\alpha Q_a(x)= \dfrac{\sigma ^2}{2}W'_a(0)-\mu _0\beta _0 W_a(0) -\phi (\mu _0 W_a(0))+ \alpha \int _{0}^{x} W_a(u) \mathrm{{d}}u.$ Then $Q_a$ satisfies (3.13) on $(-\infty , 0]$.

Note that if $a = \dfrac{-r_0}{\alpha + \mu _0},$ then one can take the above described stationary solution $\tilde{Y}$ as $W_a$ and, in this case, $\theta _a =0$. The proof of the above proposition requires several lemmas as we prove them below. Throughout, we keep the initial value $a$ fixed and $a > \dfrac{-r_0}{\alpha + \mu _0}.$

For each $r \ge 0$, let $Y_r$ be the solution to (6.1) on the interval $(-\omega _r, 0]$ with the boundary data $Y_r(0)=a$ and $Y'_r(0)= r$. Here $-\omega _r$ is the explosion time of the solution $Y_r$ and such a solution exists as explained above (6.1). In the proof of the next several lemmas, we keep the value of $a$ fixed and analyze the above described family of solutions $(Y_r)_{r\ge 0}$.

Lemma 6.2

Let $Y_r$ be the solution to (6.1) on the interval $(-\omega _r, 0]$ with $Y_r(0)=a$ and $Y'_r(0)= r,$ where $-\omega _r$ is the explosion time. Then, $Y_{r_2}(x) > Y_{r_1}(x)$ holds for all $-(\omega _{r_1}\wedge \omega _{r_2})<x <0$ whenever $r_1 > r_2 >0.$

Proof

Since $Y_{r_1}(0)= Y_{r_2}(0)=a$ and $Y'_{r_1}(0)> Y'_{r_2}(0)$, there exists an interval $(-\delta , 0)$ such that $Y_{r_2}(x) > Y_{r_1}(x)$ for all $-\delta <x<0$. Now suppose the assertion is not true. Then there exists a point $c$ such that $-(\omega _{r_1}\wedge \omega _{r_2})<c <0,\,Y_{r_2}(c) =Y_{r_1}(c)$ and $Y_{r_2}(x) > Y_{r_1}(x)$ for all $c <x<0$. Moreover, $\phi ( Y_{r_2}(c) ) = \phi (Y_{r_1}(c))$, and $Y'_{r_2}(c) \ge Y'_{r_1}(c).$ Using this data with the Eq. (6.3), we obtain

$$\begin{aligned} \dfrac{\sigma ^2}{2} \Big (Y'_{r_1}(c) - Y'_{r_2}(c)\Big ) = \dfrac{\sigma ^2}{2} (r_1 -r_2) - \alpha \int _{c}^{0} \Big (Y_{r_1}(x) -Y_{r_2}(x)\Big ) \mathrm{{d}}x. \end{aligned}$$

With the above information, the left hand side of the above equation is non-positive, while the right hand side is strictly positive. This is a contradiction and hence the proof is complete. $\square$

In the next lemma, we characterize the possible local extrema of $Y_r$ when $Y_r(0)=a > \dfrac{-r_0}{\alpha + \mu _0}.$

Lemma 6.3

Let $a > \dfrac{-r_0}{\alpha + \mu _0}$ and $Y_r$ be the solution to (6.1) on the interval $(-\omega _r, 0]$ with $Y_r(0)=a$ and $Y'_r(0)= r$, where $-\omega _r$ is the explosion time. Let $Y'_r(c)=0$ for some $c<0$. Then the following statements hold:

• (a) If $Y_r(c) > \dfrac{-r_0}{\alpha + \mu _0}$ then $x=c$ is a strict local minimum.

• (b) If $Y_r(c) < \dfrac{-r_0}{\alpha + \mu _0}$ then $x=c$ is a strict local maximum.

• (c) The case $Y_r(c) = \dfrac{-r_0}{\alpha + \mu _0}$ is not possible.

Consequently, when $r>0$, each $Y_r$ is either strictly increasing or else can have a unique local minimum, but it cannot have any local maxima. Moreover, if $r=0$, then $Y_0$ is a strictly decreasing function on its domain $(-\omega _0, 0]$.

Proof

Since $Y'_r(c)=0$ for some $c<0$, using (6.1) at $x=c$ we obtain

$$\begin{aligned} \dfrac{\sigma ^2}{2} Y''_r(c)= (\alpha + \mu _0) \Big ( Y_r(c) + \dfrac{r_0}{(\alpha + \mu _0)}\Big ). \end{aligned}$$

Now, parts (a) and (b) are an immediate consequence of this equation. For part (c), if $Y_r(c) = \dfrac{-r_0}{\alpha + \mu _0}$ and $Y'_r(c) =0$, by the uniqueness of solutions, $Y_r$ agrees with the stationary solution $\tilde{Y}(x)= \dfrac{-r_0}{\alpha + \mu _0}$ for all $x\le 0$. Consequently, $a=Y_r(0)= \dfrac{-r_0}{\alpha + \mu _0}$ and this is a contradiction. Part (c) follows.

When $r >0,\,Y_r$ is strictly increasing on an interval $(-\delta , 0]$. Thus, by parts (a) and (b), it follows that if $Y_r$ is not strictly increasing, the only possibility is that it can have a unique local minimum, but no local maxima.

If $r=0,\,Y_0(0)=a > \dfrac{-r_0}{(\alpha + \mu _0)},\,Y'_0(0)=0$ and by (6.1), $Y''_0(0) >0.$ Thus, $Y_0$ is strictly decreasing in an interval $(-\delta , 0]$. But part (a) prevents $Y_0$ having any local maxima. Thus $Y_0$ is strictly decreasing and has its absolute minimum at $x=0$. This completes the proof. $\square$

Lemma 6.4

Consider the above described family of solutions $(Y_r)_{r\ge 0}.$ Then,

$$\begin{aligned} \lim \limits _{r \rightarrow \infty } \inf \limits _{-\omega _r <x<0} Y_r(x) = -\infty . \end{aligned}$$

(6.4)

Proof

We establish this result in two steps. Let $r >0$. Introduce $c_r$ by $c_r=\inf \{ x: Y'_r(x) >0 \}.$ Since $Y'_r(0)=r >0$ and $Y'_r$ is continuous, the quantity $c_r$ is well defined and $c_r <0$.

Step 1

We first show that $c_r <-1$ for large $r$. If $c_r=-\omega _r$ and $c_r$ is finite, then $Y_r$ is bounded and as a consequence of (6.3), $Y'_r$ is also bounded on $(-\omega _r, 0]$. Using Lemma 6.3, $Y_r$ is monotone in an interval $(-\omega _r, -\omega _r +\delta )$ for some $\delta >0$. Hence the limits $Y_r(-\omega _{r}+)$ and $Y'_r(-\omega _{r}+)$ are finite and thus the solution $Y_r$ can be extended to an interval $(-\omega _r - \epsilon , 0]$ for some $\epsilon > 0.$ This contradicts the definition of the explosion time. Thus, if $c_r=-\omega _r$, the only possibility is $c_r=-\omega _r= -\infty .$

If $c_r > -\omega _r$, then $Y'_r(c_r)=0$ and $x=c_r$ is necessarily a local minimum. Using (6.3) and Lemma 6.3, we obtain $\dfrac{-r_0}{(\alpha + \mu _0)} < Y_r(x) <a$ for all $c_r \le x \le 0$. Consequently, we can find a constant $K >0$ so that $|Y_r(x)| < K$ for all $c_r \le x \le 0.$ This also implies that $\alpha | \int _{c_r}^{0} Y_r(u) \mathrm{{d}}u| < K \alpha |c_r|.$ Now, we use (6.3) (evaluated at $x=c_r$) together with these estimates to obtain $\dfrac{\sigma ^2}{2} r \le K_1 |c_r| + K_2,$ where $K_1 >0$ and $K_2 >0$ are generic constants independent of $r$ and $c_r$. Hence, we can find $r_1>0$ so that the estimate $|c_r| >1$ holds for all $r>r_1$. Thus, the proof of step 1 is complete.

Step 2

For large $r$, using the result from step 1, we have $Y'_r(x) >0$ on $[-1, 0].$ Hence, $|Y_r(x)| < K$ on $[-1, 0]$ and $\alpha | \int _{-1}^{0} Y_r(u) \mathrm{{d}}u| < K \alpha$ where $K>0$ is a constant independent of $r$. Again using these estimates in (6.3), we obtain $\dfrac{\sigma ^2}{2} Y'_r(x) > \dfrac{\sigma ^2}{2} r - K \alpha - K_2$ on the interval $[-1, 0]$, where $K$ and $K_2$ are positive generic constants independent of $r$. Hence, we can find $r_2 > 0$ so that $Y'_r(x) > \frac{1}{2} r$ on the interval $[-1, 0]$ for all $r >r_2$. By integration, we obtain $a-\dfrac{r}{2} > Y_r(-1)$ whenever $r >r_2$. Hence, $\lim \nolimits _{r \rightarrow \infty } Y_r(-1) =-\infty$ and this completes the proof of (6.4). $\square$

Lemma 6.5

There exists a $\delta _0 >0$ such that for every $0<r<\delta _0$,

$\inf \limits _{-\omega _r <x <0} Y_r(x) > \dfrac{-r_0}{(\alpha + \mu _0)}.$

Proof

By Lemma 6.3, $Y_0$ is a strictly decreasing function on $(-\omega _0, 0).$ We pick $\epsilon _0 >0$ so that $\epsilon _0< \omega _0$. Then $Y_0(-\epsilon _0) >Y_0(0)=a.$ By the basic theory of differential equations, $Y_r(x)$ is jointly continuous in the variables $(r, x)$ (see Chapter 5 of [19]). Hence, we can find a $\delta _0 >0$ such that $Y_r(-\epsilon _0) >Y_r(0)=a$ for all $0 \le r <\delta _0$. But $Y'_r(0)=r >0$ for any such $0< r <\delta _0,\,Y_r(0)=Y_0(0)=a$, and hence this leads to a local minimum for $Y_r$ on the interval $(-\epsilon _0, 0)$, say, at $x=c.$ By Lemma 6.3, $Y_r(c)$ also becomes the absolute minimum, $Y_r(c)> \dfrac{-r_0}{(\alpha + \mu _0)}$ and each such $Y_r$ cannot have any local maxima. Thus, we can conclude that $Y_r(c)=\inf \limits _{-\omega _r <x <0} Y_r(x) > \dfrac{-r_0}{(\alpha + \mu _0)}.$ This completes the proof. $\square$

With the above lemmas in hand, now we can complete the proof of Proposition 6.1.

Proof

(proof of Proposition 6.1)

We introduce the set

$$\begin{aligned} S=\Big \{r >0: \inf \limits _{-\omega _r <x <0} Y_r(x) < \dfrac{-r_0}{(\alpha + \mu _0)} \Big \}. \end{aligned}$$

(6.5)

By Lemma 6.4, the above set is non-empty. Introduce $\theta _a$ by

$$\begin{aligned} \theta _a=\inf S \end{aligned}$$

(6.6)

By Lemma 6.4, $\theta _a$ is finite, and $\theta _a \ge \delta _0 >0$ by Lemma 6.5. We consider the solution $Y_{\theta _a}$.

Our aim is to show that $Y_{\theta _a}$ satisfies all the requirements listed for $W_a$ in Proposition 6.1. Then we intend to relabel $Y_{\theta _a}$ as $W_a$. Suppose that $Y_{\theta _a}(x_0) < \dfrac{-r_0}{(\alpha + \mu _0)}$ for some $x_0 <0$. Since $Y_r$ is continuous in the variable $r$, we can find $\epsilon >0$ so that $Y_{r}(x_0) < \dfrac{-r_0}{(\alpha + \mu _0)}$ for all $\theta _a - \epsilon < r < \theta _a.$ This contradicts (6.6) and hence $Y_{\theta _a}(x) \ge \dfrac{-r_0}{(\alpha + \mu _0)}$ for all $x$. Now suppose that $Y_{\theta _a}(x_1)= \dfrac{-r_0}{(\alpha + \mu _0)}$ for some $x_1$. Then $x =x_1 <0$ is a local minimum and, by part (c) of Lemma 6.3, this is not possible. Hence, $Y_{\theta _a}(x) > \dfrac{-r_0}{(\alpha + \mu _0)}$ for all $-\omega _{\theta _a} <x <0.$ Next, we introduce $c_a$ by

$$\begin{aligned} c_a= \inf \Big \{ x <0 : \inf \limits _{x <u <0} Y'_{\theta _a}(u) > 0 \Big \}. \end{aligned}$$

Since $Y'_{\theta _a}$ is continuous and $Y'_{\theta _a}(0) >0$, we have $c_a <0$. We intend to show that $c_a =-\infty .$ Suppose $c_a$ is finite, Then $Y_{\theta _a}$ is bounded on $(c_a, 0]$. Now mimicking the arguments in step 1 of the proof of Lemma 6.4, we can obtain that $-\omega _{\theta _a} < c_a$ and thus $Y'_{\theta _a}(c_a)=0.$ By Lemma 6.3, $x=c_a$ is a strict local minimum of $Y_{\theta _a}$. Since $Y_r$ is continuous in the variable $r$, when $|r- \theta _a |$ is very small, $Y_r$ also has a local minimum in the vicinity. Thus, by Lemma 6.2, when $|r- \theta _a |$ is very small and $r > \theta _a$, we have $\inf \limits _{-\omega _r <x <0} Y_r(x) > \dfrac{-r_0}{(\alpha + \mu _0)}.$ This contradicts (6.6). Hence, $c_a$ is infinite and as a consequence, $-\omega _{\theta _a}$ also infinite. Moreover, $Y'_{\theta _a}(x) >0$ and $Y_{\theta _a}(x) > \dfrac{-r_0}{(\alpha + \mu _0)}$ for all $x <0$. Since $Y_{\theta _a}$ is strictly increasing on $(-\infty , 0]$ and is bounded below by $\dfrac{-r_0}{(\alpha + \mu _0)},\,\lim \nolimits _{x \rightarrow -\infty } Y_{\theta _a}(x) \equiv l$ exists and $l \ge \dfrac{-r_0}{(\alpha + \mu _0)}.$ We use the facts that $Y'_{\theta _a}(x) >0$, and the functions $Y_{\theta _a}$ and $\phi (Y_{\theta _a})$ are bounded on $(-\infty , 0]$ in Eq. (6.3) to obtain the inequality $\alpha \int _{x}^{0} Y_{\theta _a}(u) \mathrm{{d}}u \le \mu _0 (\beta _0 +x) Y_{\theta _a}(x) +r_0 x +K_0$, for all $x<0$. Here $K_0>0$ is a constant independent of $x$. Now divide both sides of the inequality by $-x$ and let $x$ tend to $-\infty$, and then we obtain $l \le \dfrac{-r_0}{(\alpha + \mu _0)}.$ Consequently, $\lim \nolimits _{x \rightarrow -\infty } Y_{\theta _a}(x)= \dfrac{-r_0}{(\alpha + \mu _0)}$. Finally, we relabel $Y_{\theta _a}$ as $W_a$ which thus satisfies all the required conditions listed in Proposition 6.1.

Introduce the function $Q_a$ on $(-\infty , 0]$ by $\alpha Q_a(x)= \dfrac{\sigma ^2}{2}W'_a(0)-\mu _0\beta _0 W_a(0) -\phi (\mu _0 W_a(0))+ \alpha \int _{0}^{x} W_a(u) \mathrm{{d}}u.$ Then, it is evident that $Q_a$ satisfies (3.13) on $(-\infty , 0]$. This completes the proof. $\square$

Our next step is to show that $\theta _a$ is continuous as a function of $a$. First we show that the function $W_a$ and $\theta _a$ are unique for a given value of $a$.

Lemma 6.6

• (i) For any $a > \dfrac{-r_0}{(\alpha + \mu _0)}$, the function $W_a$ in Proposition 6.1 is unique.

• (ii) If $a_1 >a_2 > \dfrac{-r_0}{(\alpha + \mu _0)}$, then $W_{a_1}(x) > W_{a_2}(x)$ for all $x <0$.

Proof

Let $a > \dfrac{-r_0}{(\alpha + \mu _0)}$ and suppose the above assertion is not true. Then there exist two strictly increasing functions $W_1$ and $W_2$ satisfying the conditions described in Proposition 6.1. Thus, $W_1(0)=W_2(0)=a$, and without loss of generality, let $W'_1(0)=r_1 > W'_2(0)=r_2 >0$. By Lemma 6.2, $W_2(x) > W_1(x)$ for all $x<0$ and $\lim \nolimits _{x \rightarrow -\infty } W_i(x)= \dfrac{-r_0}{(\alpha + \mu _0)}$ for $i=1, 2.$ Then, by considering the function $W_2-W_1,$ we can find a strictly decreasing sequence $(x_n)$ which tends to negative infinity, satisfying the following: $W_i(x_n) <0$ for all $n \ge 1,\,i=1,2,$ and $\lim \nolimits _{n \rightarrow \infty } W'_i(x_n)= 0$, for $i=1, 2.$ We use (6.3) and notice that $\phi (\mu _0W_2(x_n))= \phi (\mu _0W_1(x_n))=0$, and the term $\mu _0(\beta _0 +x_n)(W_2(x_n) - W_1(x_n))$ is negative for large $n$, to obtain

$$\begin{aligned} \dfrac{\sigma ^2}{2} \Big (W'_2(x_n)-W'_1(x_n)\Big ) \le \dfrac{\sigma ^2}{2}(r_2 -r_1) - \alpha \int _{x_n}^{0} \Big (W_2(u) - W_1(u)\Big ) \mathrm{{d}}u, \end{aligned}$$

(6.7)

By letting $n$ tend to infinity, the left hand side of (6.7) goes to zero, while the right hand side is strictly less than the negative quantity $\frac{\sigma ^2}{2}(r_2 -r_1)$. This is a contradiction and hence the proof of part (i) is complete.

To obtain part (ii), let us assume the contrary. Then there exists a point $c$ so that $W_{a_1}(c)= W_{a_2}(c).$ By the uniqueness of the solutions to the differential equations, $W'_{a_1}(c)$ cannot be equal to $W'_{a_2}(c).$ Now we can essentially follow the proof of part (i) on $(-\infty , c)$ to obtain a contradiction. This completes the proof of part (ii). $\square$

Next we show that $\theta _a \equiv W'_a(0)$ is continuous in the variable $a$.

Proposition 6.7

Let $l_0= \dfrac{-r_0}{(\alpha + \mu _0)}$ and $f: (l_0, \infty ) \rightarrow (0, \infty )$ be the function defined by $f(a)= \theta _a$. Then

• (i) $f$ is continuous on its domain $(l_0, \infty )$, and

• (ii) $\lim \nolimits _{a \rightarrow l_0} f(a) =0.$

Proof

Let $a>l_0$ be fixed. Consider a sequence $(a_n)$ decreasing to $a$. For simplicity, we relabel the pair $(\theta _{a_n}, W_{a_n})$ by $(\theta _n, W_n)$ in this proof. We intend to show that $\lim \nolimits _{n \rightarrow \infty } \theta _n = \theta _a$. For each $x \le 0$, by Lemma 6.6, $(W_n(x))$ is decreasing and $l_0< W_n(x) \le a_1$. Hence, $\lim \nolimits _{n \rightarrow \infty } W_n(x) \equiv W_{\infty }(x)$ exists for each $x\le 0$. By integrating (6.3) on the interval $[x, 0]$, we obtain

$$\begin{aligned} -\dfrac{\sigma ^2}{2} \theta _n x&= - \dfrac{\sigma ^2}{2} \Big (a_n -W_n(x)\Big ) -\Big (\mu _0 \beta _0 a_n+ \phi (\mu _0 a_n)\Big )x - \alpha \int _{x}^{0} \int _{y}^{0} W_n(u)\mathrm{{d}}u \mathrm{{d}}y \nonumber \\&+ \int _{x}^{0} \Big [ \mu _0 (\beta _0 +u) W_n(u) +\phi (\mu _0 W_n(u))\Big ] \mathrm{{d}}u - \dfrac{r_0}{2} x^2, \end{aligned}$$

(6.8)

for each $x <0$. Since $\lim \nolimits _{n \rightarrow \infty } a_n =a$ and $\lim \nolimits _{n \rightarrow \infty } W_n(x) = W_{\infty }(x)$ exist, it follows that the right hand side of the above equation is convergent. Consequently, $\lim \nolimits _{n \rightarrow \infty } \theta _n \equiv \theta _{\infty }$ exists. Moreover, it satisfies

$$\begin{aligned} -\dfrac{\sigma ^2}{2} \theta _{\infty } x&= -\dfrac{\sigma ^2}{2} \Big (a -W_{\infty }(x)\Big ) -\Big (\mu _0 \beta _0 a+ \phi (\mu _0 a)\Big )x - \alpha \int _{x}^{0} \int _{y}^{0} W_{\infty }(u)\mathrm{{d}}u \mathrm{{d}}y \\&+ \int _{x}^{0} \Big [ \mu _0 (\beta _0 +u) W_{\infty }(u) +\phi (\mu _0 W_{\infty }(u))\Big ] \mathrm{{d}}u - \dfrac{r_0}{2} x^2. \end{aligned}$$

Thus, $W_{\infty }$ is differentiable and by differentiating this, we see that it satisfies (6.3) with $W_{\infty }(0)=a$ and $W'_{\infty }(0)=\theta _{\infty }$. Hence, by Lemma 6.6, $W_{\infty }$ coincides with $W_a$ and $\theta _{\infty } \equiv \theta _a$. Following a very similar argument, we can derive the same conclusion when the sequence $(a_n)$ is increasing to $a$. Consequently, the function $f$ is continuous on $(l_0, \infty )$ and this completes part (i).

For part (ii), let the sequence $(a_n)$ be decreasing to $l_0$. Again, we replace $(\theta _{a_n}, W_{a_n})$ by $(\theta _n, W_n)$ here. Then by Proposition 6.1, $|W_{n}(x) -l_0 | \le |a_n -l_0|$ for all $x\le 0$. Hence, $\lim \nolimits _{n \rightarrow \infty } W_n(x) = l_0$ for each $x \le 0.$ Now we substitute $x=-1$ in (6.8) and let $n$ tend to infinity to observe that the right hand side of (6.8) converges to $\dfrac{1}{2}[ (\alpha + \mu _0)l_0 +r_0]$. By the definition of $l_0$, this quantity is zero. Hence, $\lim \nolimits _{n \rightarrow \infty } \theta _n = 0$. Thus, we obtain part (ii) and the proof is complete. $\square$

1.2 Construction on $[0, \infty )$

To construct a smooth solution to (3.13), here we analyze the situation on the domain $[0, \infty ).$ Given the initial data $Q(0)$ and $Q'(0)$, we know that (3.13) has a solution $Q$ on an interval $(-\delta , \delta )$ for some $\delta >0,$ as explained above (6.1). If $Q$ is a solution to (3.13) on $(-\delta , \delta )$, and if $\tilde{W}(x)=Q'(x)$ for all $x$, then following the derivation of (6.3), we can obtain $\alpha Q(x)= \dfrac{\sigma ^2}{2}\tilde{W}'(0)-\mu _0\beta _0 \tilde{W}(0) -\phi (\mu _0 \tilde{W}(0))+ \alpha \int _{0}^{x} \tilde{W}(u) \mathrm{{d}}u,$ and the Eq. (3.13) can be written as

$$\begin{aligned}&\dfrac{\sigma ^2}{2} \tilde{W}'(x) - \mu _0\Big (\beta _0 +H_0(x)\Big )\tilde{W}(x) -\phi \Big (\mu _0 \tilde{W}(x)\Big )+b_0 x + p \mu _0 H_0(x) \nonumber \\&\quad =\dfrac{\sigma ^2}{2}\tilde{W}'(0)-\mu _0 \beta _0 \tilde{W}(0) -\phi \Big (\mu _0 \tilde{W}(0)\Big ) + \alpha \int _{0}^{x} \tilde{W}(u) \mathrm{{d}}u, \end{aligned}$$

(6.9)

for $x> 0.$ Moreover, $\tilde{W}$ satisfies (6.2), but unlike the previous case, (6.2) does not necessarily have a stationary solution on $[0, \infty ).$ Therefore, our analysis and estimates are somewhat different from the previous subsection. First, we introduce a non-negative function $\Gamma$ defined on $[0, \infty )$ which will play a key role in the proofs. Let

$$\begin{aligned} \Gamma (x)= \dfrac{b_0+p\mu _0 H'_0(x)}{( \alpha +\mu _0 H'_0(x))} \end{aligned}$$

(6.10)

for all $x \ge 0$. The function $\Gamma$ is bounded. To see this, we pick a constant $M_0 >0$ so that $M_0 > \max \{ \frac{b_0}{\alpha }, p \}.$ Then it is evident that

$$\begin{aligned} 0 \le \Gamma (x) < M_0 \end{aligned}$$

(6.11)

for all $x \ge 0$. The following proposition is analogous to Proposition 6.1 and the constant $p>0$ is as in the HJB equation (3.13).

Proposition 6.8

For each $a$ in $\mathbf {R}$, there exists a unique pair $(\tilde{W}_a, \eta _a)$ where $\tilde{W}_a$ is a bounded function defined on $[0, \infty )$ and $\eta _a$ is a real number satisfying the following conditions:

the function $\tilde{W}_a$ satisfies (6.9) on $[0, \infty ),$ with the initial data $\tilde{W}_a(0)=a,\,\tilde{W}'_a(0)=\eta _a$, and $| \tilde{W}_a(x) | \le \max \{ |a|, M_0 \}$ for all $x \ge 0$, where $M_0$ is given in (6.11).

Moreover, let the function $Q_a$ be defined on $[0, \infty )$ by $\alpha Q_a(x)= \dfrac{\sigma ^2}{2}\tilde{W}'_a(0)-\mu _0\beta _0 \tilde{W}_a(0) -\phi (\mu _0 \tilde{W}_a(0))+ \alpha \int _{0}^{x} \tilde{W}_a(u) \mathrm{{d}}u.$ Then $Q_a$ satisfies (3.13) on $(0, \infty )$.

To prove this proposition, we need several basic results. We obtain them in the next few lemmas. Throughout, we keep the initial value $a$ fixed. For each real number $r$, let $Y_r$ be the solution to (6.2) on the interval $[0, \omega _r),$ with the boundary data $Y_r(0)=a$ and $Y'_r(0)= r$. Here, $\omega _r$ is the explosion time of the solution $Y_r$. This solution $Y_r$ also satisfies (6.9) on $[0, \omega _r)$. It should be noted that, given $Y_r(0)$ and $Y'_r(0)$, such a solution exists on an interval $[0, \delta )$, for some $\delta >0$ as explained above (6.1) and consequently $\omega _r \ge \delta >0.$ In the next few lemmas, we intend to analyze this family $(Y_r)_{r \ge 0}$ to obtain the required solution $\tilde{W}_a.$

Our first lemma is a comparison result.

Lemma 6.9

Let $Y_r$ be the solution to (6.9) on the interval $[0, \omega _r)$ with $Y_r(0)=a$ and $Y'_r(0)= r,$ where $\omega _r$ is the explosion time. Then, the following results hold:

• (i) If $r_1 > r_2$, then $Y_{r_1}(x) > Y_{r_2}(x)$ holds for all $0 < x < (\omega _{r_1}\wedge \omega _{r_2}),$ and

• (ii) $Y_{r_1}(x) - Y_{r_2}(x) \ge (r_1 -r_2) \mathrm{{e}}^{I(x)} \int _{0}^{x} \mathrm{{e}}^{-I(u)} \mathrm{{d}}u$ holds for all $0 \le x < (\omega _{r_1}\wedge \omega _{r_2}),$ where $I(x) = \frac{2 \mu _0}{\sigma ^2} \int _{0}^{x} (\beta _0 +H_0(u)) \mathrm{{d}}u$ for all $x \ge 0.$

Proof

Since $Y_{r_1}(0)= Y_{r_2}(0)=a$ and $Y'_{r_1}(0)= r_1 > Y'_{r_2}(0)=r_2$, we can find a $\delta _1 >0$ so that $Y_{r_1}(x) > Y_{r_2}(x)$ for all $0 <x < \delta _1 \le (\omega _{r_1}\wedge \omega _{r_2}).$ Now suppose the assertion in part (i) is not true. Then, there exists $x_1 >0$ such that $Y_{r_1}(x_1) < Y_{r_2}(x_1).$ Hence, we can find a $c$ so that $0 < \delta _1 \le c \le x_1,\,Y_{r_1}(c) = Y_{r_2}(c)$ and $Y_{r_1}(x) > Y_{r_2}(x)$ holds for all $0 < x < c$. Consequently, $Y'_{r_1}(c) \le Y'_{r_2}(c).$ Using (6.9) together with these facts, we obtain

$$\begin{aligned} \dfrac{\sigma ^2}{2} \Big ( Y'_{r_1}(c)- Y'_{r_2}(c)\Big ) = \dfrac{\sigma ^2}{2}(r_1 -r_2) + \alpha \int _{0}^{c} \Big (Y_{r_1}(u) - Y_{r_2}(u)\Big ) \mathrm{{d}}u. \end{aligned}$$

Now the right hand side of the above equation is greater than the positive quantity $\dfrac{\sigma ^2}{2}(r_1 -r_2)$ while the left hand side is less than or equal to zero. This is a contradiction. Hence, part (i) follows.

For part (ii), we can use the result in part (i) to deduce $\phi (\mu _0 Y_{r_1}(x)) \ge \phi (\mu _0 Y_{r_2}(x))$ for all $0 \le x < (\omega _{r_1}\wedge \omega _{r_2})$. This, together with (6.9), yields

$$\begin{aligned}&\dfrac{\sigma ^2}{2} \Big ( Y'_{r_1}(x)- Y'_{r_2}(x)\Big ) - \mu _0 \Big (\beta _0 + H_0(x)\Big ) \Big ( Y_{r_1}(x)- Y_{r_2}(x)\Big )\nonumber \\&\quad \ge \dfrac{\sigma ^2}{2}(r_1 -r_2) + \alpha \int _{0}^{x} \Big (Y_{r_1}(u) - Y_{r_2}(u)\Big ) \mathrm{{d}}u, \end{aligned}$$

(6.12)

for all $0 \le x < (\omega _{r_1}\wedge \omega _{r_2})$. We let $R(x) \equiv ( Y_{r_1}(x)- Y_{r_2}(x))$ and hence, using part (i) again, we obtain $\dfrac{d}{\mathrm{{d}}x}R(x) - \dfrac{2 \mu _0}{\sigma ^2} (\beta _0 +H_0(x)) R(x) \ge (r_1 -r_2)$ for all $0 \le x < (\omega _{r_1}\wedge \omega _{r_2})$. Then it is straightforward to obtain

$$\begin{aligned} \dfrac{d}{\mathrm{{d}}x}\Big [ \mathrm{{e}}^{-I(x)}R(x)\Big ] \ge (r_1 -r_2) \mathrm{{e}}^{-I(x)} \end{aligned}$$

(6.13)

for all $0 \le x < (\omega _{r_1}\wedge \omega _{r_2})$, where $I(x) = \dfrac{2 \mu _0}{\sigma ^2} \int _{0}^{x} (\beta _0 +H_0(u)) \mathrm{{d}}u$ for all $x \ge 0.$ Since $R(0)=0$, by integrating this inequality on the interval $[0, x]$, part (ii) follows. $\square$

Next, we discuss the nature of the local extrema of $Y_r$ in the next lemma, which is similar to Lemma 6.3.

Lemma 6.10

Let $Y_r$ be the solution to (6.9) on the interval $[0, \omega _r)$ with $Y_r(0)=a$ and $Y'_r(0)= r$, where $\omega _r$ is the explosion time. Let $Y'_r(c)=0$ for some $c>0$ and the function $\Gamma$ be defined by (6.10). Then the following statements hold:

• (a) If $Y_r(c) > \Gamma (c)$ then $x=c$ is a strict local minimum.

• (b) If $Y_r(c) < \Gamma (c)$ then $x=c$ is a strict local maximum.

Proof

Since $Y'_r(c)=0$ for some $c<0$, using (6.2) at $x=c$ we obtain

$$\begin{aligned} \dfrac{\sigma ^2}{2} Y''_r(c)= \Big (\alpha + \mu _0 H'_0(c)\Big ) \Big ( Y_r(c) - \Gamma (c)\Big ). \end{aligned}$$

Now, parts (a) and (b) are an immediate consequence of this equation. $\square$

In the next step, we show that the explosion time is infinite for any bounded solution $Y_r$.

Lemma 6.11

Let $Y_r$ be the solution to (6.9) on $[0, \omega _r)$ with the initial data $Y_r(0)=a$ and $Y'_r(0)= r,$ where $\omega _r$ is the explosion time. If $Y_r$ is bounded then its explosion time $\omega _r$ is infinite.

Proof

Let $Y_r$ be a bounded solution to (6.9) on the interval $[0, \omega _r).$ Suppose that $\omega _r$ is finite. Then by (6.9) it follows that $Y'_r$ is also bounded and continuous on $[0, \omega _r).$ Next, we use the following fact: If $f$ is a bounded continuous function on the half-open interval $[0, a)$, where $a$ is finite, and $F$ is an anti-derivative of $f$ on $[0, a),$ then $\lim \nolimits _{x \rightarrow a-} F(x)$ exists and finite. The proof of this fact is easy to see by using the decomposition $f=f^+ -f^-$ and writing $F$ as an integral of $f$ plus a constant. Hence, we omit it. Using this with $Y_r$ and $Y'_r$, we can conclude that $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x)$ exists and is finite. Using (6.9), we can deduce that $\lim \nolimits _{x \rightarrow \omega _r} Y'_r(x)$ also exists and is finite. Hence, as in the proof of step 1 in Lemma 6.4, we can extend $Y_r$ to an interval $[0, \omega _r+ \epsilon )$ as a solution to (6.9), for some $\epsilon >0.$ This contradicts the fact that $\omega _r >0$ is the explosion time. Hence, $\omega _r$ is infinite and this completes the proof. $\square$

Lemma 6.12

Given the initial value $Y_r(0)=a$, there is at most one $r$ so that (6.9) has a bounded solution with the initial data $Y_r(0)=a$ and $Y'_r(0)= r$.

Proof

Suppose $Y_1$ and $Y_2$ are two bounded solutions to (6.9) such that $Y_1(0)=Y_2(0)=a,\,Y'_1(0)=r_1$ and $Y'_2(0) =r_2$. By the uniqueness of solutions to (6.9) we deduce that $r_1$ and $r_2$ are not equal. Without loss of generality, let $r_1 >r_2$. By Lemma 6.11, $Y_1$ and $Y_2$ are both solutions to (6.9) on $(0, \infty ),$ and by Lemma 6.8, $Y_1(x) >Y_2(x)$ for all $x>0$. We can further deduce that $\phi (\mu _0 Y_1(x)) >\phi (\mu _0 Y_2(x))$ for all $x>0$. Moreover, using (6.13), the function $\mathrm{{e}}^{-I(x)}R(x)$ is strictly increasing and hence we can derive $R(x) > (\mathrm{{e}}^{-I(1)}R(1)) \mathrm{{e}}^{I(x)}$ for all $x >1$, where the function $I(\cdot )$ is described in part (ii) of Lemma 6.9. By our assumptions on the function $H(\cdot )$ (see above (2.10)), $H(x)$ is increasing to infinity as $x$ tends to infinity. Hence $\lim \nolimits _{x \rightarrow \infty } I(x) = \infty$, and as a consequence, we obtain $\lim \nolimits _{x \rightarrow \infty } R(x) = \infty .$ This contradicts the fact that the functions $Y_1$ and $Y_2$ are bounded. This completes the proof. $\square$

Next, we show that when $r$ is very large, $Y_r$ tends to infinity, and when $r$ is small, $Y_r$ tends to negative infinity.

Lemma 6.13

Let $a$ be fixed and $M > \max \{ |a|, M_0 \}$ be a constant, where $M_0>0$ is given in (6.11). Then there exists a pair $(r_1, r_2)$ such that $r_1 > r_2$ and satisfying the following:

• (i) For every $r>r_1$, if $Y_r$ is the solution to (6.9) with boundary data $Y_r(0)=a$ and $Y'_r(0)=r$, then $Y_r(x) >M$ for some $x$. Moreover, $Y_r$ is increasing for large $x$ and $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x) = \infty .$

• (ii) For every $r<r_2$, if $Y_r$ is the solution to (6.9) with boundary data $Y_r(0)=a$ and $Y'_r(0)=r$, then $Y_r(x) < -M$ for some $x$. Moreover, $Y_r$ is decreasing for large $x$ and $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x) = -\infty .$

Proof

In our first step, we show that we can find $r_1$ so that $Y_{r_1}(x) >M$ for some $x$. Using Lemma 6.12, we can easily pick a value $\tilde{r}$ so that the corresponding solution $Y_{\tilde{r}}$ is unbounded. If $Y_{\tilde{r}}$ exceeds the value $M$, then we can simply take $r_1 = \tilde{r}.$ Otherwise, $Y_{\tilde{r}}$ has to attain values below $-M.$ Let $x_0$ be fixed so that $Y_{\tilde{r}}(x_0) < -M.$ Let us pick $r_1 > \tilde{r}$ so that $Y_{\tilde{r}}(x_0) +(r_1 - \tilde{r}) \mathrm{{e}}^{ I(x_0)} \int _{0}^{x_0} \mathrm{{e}}^{-I(u)} \mathrm{{d}}u >M.$ Now consider the solution $Y_{r_1}$ which has explosion time $\omega _{r_1}$. If $\omega _{r_1} > x_0$, then by part (ii) of Lemma 6.9, $Y_{r_1}(x_0) >M$ as desired. Next consider the case $\omega _{r_1} \le x_0.$ Suppose that the solution $Y_{r_1}$ is bounded above by the constant $M$. By Lemma 6.9, we have $Y_{r_1}(x) > Y_{\tilde{r}}(x)$ for all $0< x < \omega _{r_1} \le x_0$. Consequently, $Y_{r_1}$ is a bounded solution and hence, using Lemma 6.11, we can conclude that $\omega _{r_1}$ is infinite. This is a contradiction, since $\omega _{r_1} \le x_0$ and $x_0$ is finite. Hence, $\sup \nolimits _{[0, \omega _{r_1})} Y_{r_1}(x) > M$ and thus we can find $x_1$ so that $Y_{r_1}(x_1) >M$. Since $M > M_0 > \Gamma (x)$ for all $x \ge 0$, using Lemma 6.10, we can easily deduce that $Y'_{r_1}(x_1) >0.$ Otherwise, there will be a local maximum of $Y_{r_1}$ with its value exceeding $M.$ This is not possible by Lemma 6.10. Since $Y'_{r_1}(x_1) >0$ and $Y'_{r_1}$ is continuous, we can also conclude that it remains positive on $[x_1, \omega _{r_1})$ by the same argument above. Hence, $Y_{r_1}$ is strictly increasing on this interval. Similarly, it is evident that $Y_{r_1}$ remains above $-M$, otherwise there will be a local minimum with value below $-M$ which contradicts Lemma 6.10.

In the next step, we let $r>r_1$, and consider $Y_r.$ By Lemma 6.9, $Y_r(x)>Y_{r_1}(x)$ for $0<x< (\omega _r \wedge \omega _{r_1}).$ We consider two cases here: $\omega _r < \omega _{r_1}$ and $\omega _r \ge \omega _{r_1}.$ By following an argument similar to above, it is easy to show that if $\omega _r < \omega _{r_1},$ then necessarily $Y_r$ is an unbounded solution and hence $Y_r(\hat{x}) >M$ for some $\hat{x}$. Moreover, $Y'_r(x) >0$ for all $x \ge \hat{x}.$ Thus $Y_r$ is strictly increasing on $[ \hat{x}, \omega _r).$ If $\omega _r \ge \omega _{r_1},$ then $Y_r(x_1) > Y_{r_1}(x_1) >M,$ and again $Y_r$ is strictly increasing on $[x_1, \omega _r).$ In both cases, $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x)$ exists and is finite or it is infinity.

If $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x)$ is finite, then $Y_r$ is a bounded solution. Hence, by Lemma 6.11, $\omega _r$ is infinite. Consequently, $Y_{r_1}$ is also a bounded solution with infinite explosion time. Thus, $Y_{r}$ and $Y_{r_1}$ are two distinct bounded solutions to (6.9) and this contradicts Lemma 6.12. Hence, if $\omega _r$ is finite, then we conclude $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x) = \infty .$ In the case $\omega _r$ is infinite, suppose that $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x) = l$ is finite. Since, $Y_{r}(x_1)$ is continuous in $r$, we can find $r_3$ so that $r_3 < r_1,\,r_1 -r_3$ is sufficiently small and $Y_{r_3}(x_1) >M.$ With the same arguments, $\lim \nolimits _{x \rightarrow \omega _r} Y_{r_{3}}(x)$ exists and less than or equal to $l$. Thus, $Y_{r}$ and $Y_{r_{3}}$ are bounded solutions, and this again contradicts Lemma 6.12. Therefore, we can conclude that $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x) = \infty .$ This completes the proof of part (i).

We omit the proof of part (ii), since one can employ very similar arguments to establish it. The conclusion $r_1 >r_2$ is a direct consequence of Lemma 6.9. $\square$

Remark

In the case where $Y_r(0)=a>0$, following the same arguments above, we can conclude that if $Y_r(c) <0$ for some $c >0,$ then $Y_r$ is strictly decreasing and $\lim \nolimits _{x \rightarrow \omega _r} Y_r(x) = -\infty .$

The following proof is somewhat parallel to that of Proposition 6.1 and the constant $M > \max \{ |a|, M_0 \}$ is fixed.

Proof

(Proof of Proposition 6.8)

Let $Y_r$ be the solution to (6.9) with $Y_r(0)=a$ fixed and $Y'_r(0)=r.$ We introduce the set

$$\begin{aligned} S=\Big \{r >0: \sup \limits _{0<x < \omega _r } Y_r(x) > M \Big \}. \end{aligned}$$

(6.14)

By Lemma 6.13, the above set is non-empty, and $r_2$ described in part (ii) of the same lemma is a lower bound. Introduce $\eta _a$ by

$$\begin{aligned} \eta _a=\inf S. \end{aligned}$$

(6.15)

Thus, $\eta _a$ is well defined and $\eta _a \ge r_2$. We consider the solution $Y_{\eta _a}$. Suppose that $Y_{\eta _a}(x)>M$ for some $x$. Since $Y_r(x)$ is continuous in the variable $r$, we can find $r$ close to $\eta _a$ such that $\eta _a >r$ and $Y_r(x) > M.$ This contradicts (6.15) and hence, $Y_{\eta _a}(x) \le M$ for all $x < \omega _{\eta _a}$. A similar argument can be used to prove $Y_{\eta _a}(x) \ge -M$ for all $x < \omega _{\eta _a}$. Therefore, $Y_{\eta _a}$ is a bounded solution to (6.9). Thus, $\omega _{\eta _a}$ is infinite by Lemma 6.11. Finally, we relabel the function $Y_{\eta _a}$ as $\tilde{W}_a$. Thus, $|\tilde{W}_a(x) | \le M$ for each $M > \max \{ |a|, M_0 \},$ where $M_0>0$ is given in (6.11). Consequently, $|\tilde{W}_a(x) | \le \max \{ |a|, M_0 \}.$ By Lemma 6.12, $\eta _a$ is unique. Hence, the unique pair $(\tilde{W}_a, \eta _a)$ satisfies all the conditions described in Proposition 6.8. It is also evident that the function $Q_a$ defined on $[0, \infty )$ by $\alpha Q_a(x)= \dfrac{\sigma ^2}{2}\tilde{W}'_a(0)-\mu _0\beta _0 \tilde{W}_a(0) -\phi (\mu _0 \tilde{W}_a(0))+ \alpha \int _{0}^{x} \tilde{W}_a(u) \mathrm{{d}}u,$ satisfies (3.13) on $(0, \infty )$. $\square$

Following the remark just above the proof of Proposition 6.8, when $a>0,$ we can establish that $0 \le \tilde{W}_a(x) \le M$ for all $x \ge 0$ using the same arguments in the above proof. Next we show that $\theta _a \equiv \tilde{W}'_a(0)$ is continuous in the variable $a$.

Proposition 6.14

Let $g: \mathbf {R} \rightarrow \mathbf {R}$ be the function defined by $g(a)= \eta _a$. Then

• (i) $g$ is continuous on $\mathbf {R}$.

• (ii) $g(a) \le 0$ for each $a > M_0$, and $g(a) >0$ for each $a<0,$ where $M_0>0$ is given in (6.11).

Proof

Let $a$ be fixed and consider a sequence $(a_n)$ converging to $a$. For simplicity, we relabel the pair $(\eta _{a_n}, \tilde{W}_{a_n})$ by $(\eta _n, \tilde{W}_n)$ in this proof. We intend to show that $\lim \nolimits _{n \rightarrow \infty } \eta _n = \eta _a$. By integrating (6.9) on an interval $[0, x]$, we obtain

$$\begin{aligned} -\dfrac{\sigma ^2}{2} \eta _n x&= \dfrac{\sigma ^2}{2} \Big (a_n -\tilde{W}_n(x)\Big ) -\Big (\mu _0 \beta _0 a_n+ \phi (\mu _0 a_n)\Big )x + \alpha \int _{0}^{x} \int _{0}^{y} \tilde{W}_n(u)\mathrm{{d}}u \mathrm{{d}}y \nonumber \\&+ \int _{0}^{x} \Big [ \mu _0 (\beta _0 \!+\!H_0(u)) \tilde{W}_n(u) \!+\!\phi (\mu _0 {W}_n(u)) \!-\!b_0 u-p \mu _0 H_0(u)\Big ] \mathrm{{d}}u, \nonumber \\ \end{aligned}$$

(6.16)

for each $x >0$. Since $(a_n)$ is bounded, we can pick a constant $M$ so that $M > \max \{ \sup \nolimits _{n} |a_n|, |a|, M_0 \}.$ In (6.16), we use the conclusion of Proposition 6.8 to obtain $\sup \nolimits _{n \ge 1} \sup \nolimits _{x \ge 0} |\tilde{W}_n(x)| < M$. Next, we simply take $x=1$ in (6.16) and observe that each term in the right hand side of (6.16) is bounded. Therefore, the sequence $(\eta _n)$ is bounded.

To establish the convergence of $(\eta _n)$, here we show that every convergent subsequence has the same limit. Since $(\eta _n)$ is bounded, we pick a convergent subsequence and, with a slight abuse of the notation, we relabel it as the original sequence $(\eta _n)$ with the limit $m_0$. We pick an arbitrarily large $N >1$ and use (6.9) on the interval $[0, N]$ to obtain a constant $K_N>0$ such that $\sup \nolimits _{n \ge 1} \sup \nolimits _{0 \le x \le N} |\tilde{W}'_n(x)| <K_N.$ Consequently, $(\tilde{W}_n)$ is equi-continuous on the interval $[0, N]$. Hence, we can use the Arzela-Ascolli Theorem to obtain a convergent subsequence of $(\tilde{W}_n)$ on $[0, N]$ using the sup norm. We relabel this further subsequence as the original sequence and let this limiting function be $\tilde{W}_{\infty }.$ Then, by letting $n$ tend to infinity in (6.16), we see that $\tilde{W}_{\infty }$ satisfies the equation

$$\begin{aligned} -\dfrac{\sigma ^2}{2} m_0 x&= \dfrac{\sigma ^2}{2} \Big (a -\tilde{W}_{\infty }(x)\Big ) -\Big (\mu _0 \beta _0 a+ \phi (\mu _0 a)\Big )x + \alpha \int _{0}^{x} \int _{0}^{y} \tilde{W}_{\infty }(u)\mathrm{{d}}u \mathrm{{d}}y \nonumber \\&+ \int _{0}^{x} \Big [ \mu _0 (\beta _0 \!+\!H_0(u)) \tilde{W}_{\infty }(u) \!+\!\phi (\mu _0 {W}_{\infty }(u)) \!-\! b_0 u\!-\!p \mu _0 H_0(u)\Big ] \mathrm{{d}}u, \nonumber \\ \end{aligned}$$

(6.17)

for each $0<x<N$. Consequently, $\tilde{W}_{\infty }$ is differentiable on $[0, N]$ and by differentiating (6.17), we observe that it satisfies (6.9) with initial data $\tilde{W}_{\infty }(0)=a$ and $\tilde{W}'_{\infty }(0)=m_0$. Notice that by the uniqueness of the solutions to the differential equation (6.9) with initial data $\tilde{W}_{\infty }(0)=a$ and $\tilde{W}'_{\infty }(0)=m_0$, the solutions overlap as $N$ increases to infinity. Hence, $\tilde{W}_{\infty }$ satisfies (6.9) on the interval $[0, \infty )$ with initial data $\tilde{W}_{\infty }(0)=a$ and $\tilde{W}'_{\infty }(0)=m_0.$ It also satisfies the bound $| \tilde{W}_{\infty }(x) | \le M$ for all $x \ge 0$, since each $\tilde{W}_n$ satisfies the same bound. Hence, we can conclude that $\tilde{W}_{\infty }$ is identical to the function $\tilde{W}_{a}$ on $[0, \infty )$ and $m_0= \eta _a$. Therefore, every convergent subsequence of the sequence $(\eta _n)$ has the same limit $\eta _a$. Consequently, $\lim \nolimits _{n \rightarrow \infty } \eta _n = \eta _a$ and the function $g$ is continuous on $\mathbf {R}$.

For part (ii), let $a > M_0 > 0$. From Proposition 6.8, it follows that $\tilde{W}_a(x) \le a$ for all $x \ge 0.$ Consequently, $g(a)=\eta _a \le 0.$ Next, we consider the case $a <0.$ Suppose that $\eta _a <0.$ Following the arguments in Lemma 6.13, we can show that $\tilde{W}'_a(x) < 0$ for all $x>0$ and $\tilde{W}_a$ decreases to $- \infty .$ This is a contradiction and hence $\eta _a \ge 0.$ Next, suppose $\eta _a =0$. Then, similar to Lemma 6.10, $\tilde{W}''_a(0) <0$ and again $\tilde{W}'_a(x) <0$ for $0 <x < \delta$ for some $\delta >0.$ Since, $\tilde{W}_a$ cannot have any negative local minima by Lemma 6.10, it is strictly decreasing. Following an argument similar to Lemma 6.13 again, we obtain $\lim \nolimits _{x \rightarrow \infty } \tilde{W}_a(x) = -\infty .$ This is a contradiction and hence $g(a)= \eta _a >0$ when $a<0$. This completes the proof. $\square$

1.3 A smooth solution to the HJB equation

Now we have all the technical results needed to complete the proof of Proposition 3.1. In the following lemma we obtain a result which shows that “smooth-pasting” of two solutions on $(-\infty , 0]$ and $[0, \infty )$ is possible.

Lemma 6.15

Let $l_0= \dfrac{-r_0}{(\alpha + \mu _0)}$ and $f: ( l_0, \infty ) \rightarrow (0, \infty )$ be the continuous function given in Proposition 6.7. Next, let $g: \mathbf {R} \rightarrow \mathbf {R}$ be the continuous function obtained in Proposition 6.14. Then there exists a point $a^* > l_0$ such that $f(a^*)=g(a^*).$

Proof

Since $\lim \nolimits _{a \rightarrow l_0} f(a) =0$ as in Proposition 6.7, we can define $f(l_0)=0$ so that the function $f$ is continuous on $[l_0, \infty ).$ Clearly, the function $g$ is continuous on $[l_0, \infty )$ and $g(l_0) > f(l_0)=0$ by Proposition 6.14. Next, we take $a > M_0 >0$, where $M_0$ is given in (6.11). Then $g(a) \le 0$ by Proposition 6.14. But, $f(a) >0$ from Proposition 6.7. Since both $f$ and $g$ are continuous, we can use the intermediate value theorem to conclude that there exists a point $a^* > l_0$ such that $f(a^*)=g(a^*).$ This completes the proof of the lemma. $\square$

Proof of Proposition 3.1.

Proof

Consider the point $a^*$ obtained in the previous lemma and simply label $\theta ^* \equiv f(a^*)=g(a^*).$ We have the solution $W_{a^*}$ satisfying (6.3) on $(-\infty , 0)$ with the boundary data $W_{a^*}(0)= a^*$ and $W'_{a^*}(0)= \theta ^*$ as described in Proposition 6.1. On the interval $(0, \infty )$, we have the solution $\tilde{W}_{a^*}$ satisfying (6.9) with $\tilde{W}_{a^*}(0)= a^*$ and $\tilde{W}'_{a^*}(0)= \theta ^*$ as obtained in Proposition 6.8. Since $W'_{a^*}(0)= \tilde{W}'_{a^*}(0)= \theta ^*$, it is evident that these two functions $W_{a^*}$ and $\tilde{W}_{a^*}$ can be smoothly pasted at the origin. Hence, we define the function $W:\mathbf {R} \rightarrow \mathbf {R}$ by

$$\begin{aligned} W(x)= {\left\{ \begin{array}{ll} W_{a^*}(x) &{}\quad \hbox {if}\quad x < 0, \\ \tilde{W}_{a^*}(x)&{}\quad \hbox {if}\quad x \ge 0. \\ \end{array}\right. } \end{aligned}$$

(6.18)

The function $W$ is continuously differentiable on $\mathbf {R}$ and now we can define the function $Q$ as described in the discussion above (6.1) so that $Q'(x)=W(x)$ for all $x.$ By Proposition 6.1, $l_0< W_{a^*}(x) \le a^*$ for $x \le 0$ and by Proposition 6.8, $|\tilde{W}_{a^*}(x)| \le \max \{ |a^*|, M_0 \}$ for $x \ge 0.$ Therefore, $Q'$ is bounded on $\mathbf {R}$. Moreover, we let $Q(0)$ satisfy $\alpha Q(0)= \frac{\sigma ^2}{2} \theta ^* -\mu _0 \beta _0a^* -\phi (\mu _0 a^*).$ Hence, we can write

$$\begin{aligned} Q(x)= \frac{1}{\alpha }\Big [ \frac{\sigma ^2}{2} \theta ^* -\mu _0 \beta _0a^* -\phi (\mu _0 a^*)\Big ] + \int _{0}^{x} W(u) \mathrm{{d}}u, \end{aligned}$$

for all $x$ in $\mathbf {R}.$ It is straightforward to check that $Q$ is twice continuously differentiable, $Q'$ is bounded and it is a solution to the HJB equation (3.13) on $\mathbf {R}$. This completes the proof. $\square$