On the single-leg airline revenue management problem in continuous time

Abstract

We consider the single-leg airline revenue management problem in continuous time with Poisson arrivals. Earlier work on this problem generally uses the Hamilton–Jacobi–Bellman equation to find an optimal policy whenever the value function is differentiable and is a solution to this equation. In this paper, we employ a different probabilistic approach, which does not rely on the smoothness of the value function. Instead, we use a continuous-time discrete-event dynamic programming operator to construct the value function and study its properties. A by-product of this approach is the analysis of the differentiability of the value function. We show that differentiability may break down for example with discontinuous arrival intensities. Therefore, one should exercise caution in using arguments based on the differentiability of the value function and the Hamilton–Jacobi–Bellman equation in general.

Appendix: Auxiliary results and other proofs

Remark 5.1

We have

$$\begin{aligned} {\mathbb {E}}^{(t,\cdot )} \sum _{j = n+1}^{\infty } 1_{ \{ T_j\le u \} } \le \frac{ [\Lambda _t (u)]^{n+1} }{(n+1)!}, \qquad \hbox {for}\quad n \ge 1, t \le T,\hbox { and }u \ge 0. \end{aligned}$$

Proof

Note that (due to monotone convergence theorem) the left hand side above equals

$$\begin{aligned} \sum _{j = n+1}^{\infty } {\mathbb {P}}^{(t,\cdot )} \{N_u \ge j \} = \sum _{j = n+1}^{\infty } \sum _{\ell = j}^{\infty } {\mathbb {P}}^{(t,\cdot )} \{N_u = \ell \} = \sum _{j = n+1}^{\infty } \sum _{\ell = j}^{\infty } e^{-\Lambda _t (u)} \frac{[\Lambda _t (u)]^\ell }{\ell !}. \end{aligned}$$

Using the tail sum

$$\begin{aligned} \sum _{k = m}^{\infty } \frac{x^k}{k!} = e^{x} \int _0^x e^{-y} \frac{ y^{m -1} }{(m -1)!} \, dy \end{aligned}$$

(48)

successively, we obtain

$$\begin{aligned} {\mathbb {E}}^{(t,\cdot )} \sum _{j = n+1}^{\infty } 1_{ \{ T_j\le u \}}&= \sum _{j = n+1}^{\infty } \int _0^{\Lambda _t (u)} \!e^{-y} \frac{ y^{j -1} }{(j -1)!} dy = \int _0^{\Lambda _t (u)} \!e^{-y} \sum _{j = n+1}^{\infty } \frac{ y^{j -1} }{(j -1)!}dy \\&= \int _0^{\Lambda _t (u)} \int _0^{y} e^{-u}\frac{ u^{n -1} }{(n -1)!} du \, dy \le \int _0^{\Lambda _t (u)} \int _0^{y}\frac{ u^{n -1} }{(n -1)!} du dy \\&= \int _0^{\Lambda _t (u)} \frac{ y^{n} }{n!} dy = \frac{ [\Lambda _t (u)]^{n+1} }{(n+1)!}. \end{aligned}$$

$\square$

Proof of the inequality in

(7) Let ${\mathcal {A}}= (A_i )_{i \in {\mathbb {N}}}$ be an admissible policy for the problem without an overbooking limit. Under the policy ${\mathcal {A}}$, among the first $N_T \wedge \bar{P}$-many requests some of them may be rejected. Among those which are accepted, let $C^{(\bar{P})}_T$ and $S^{(\bar{P})}_T$ denote respectively the number of cancellations and number of remaining reservations as of the departure time. Note that $C^{(\bar{P})}_T \le C_T$, $S^{(\bar{P})}_T \le S_T$ and $C^{(\bar{P})}_T + S^{(\bar{P})}_T = \sum _{i= 1}^{N_T \wedge \bar{P}} A_i$. Then we can decompose the expected revenue associated with ${\mathcal {A}}$ as

$$\begin{aligned}&\!\! {\mathbb {E}}\left[ \sum _{i=1}^{N_T} A_i \, r(L_i) - \kappa C_T - \gamma \left( \sum _{i=1}^{S_T} B_i - P \right) ^+ \right] \\&\!\!\quad = {\mathbb {E}}\left[ \sum _{i=1}^{N_T \wedge \bar{P}} A_i \, r(L_i) - \kappa C_T - \gamma \left( \sum _{i=1}^{S_T} B_i - P \right) ^+ \right] + {\mathbb {E}}\left[ \sum _{i=\bar{P}+1}^\infty 1_{ \{ T_i \le T \}} A_i \, r(L_i) \right] \\&\!\!\quad \le {\mathbb {E}}\left[ \sum _{i=1}^{N_T \wedge \bar{P}} A_i \, r(L_i) - \kappa C^{\bar{P}}_T - \gamma \left( \sum _{i=1}^{S^{\bar{P}}_T} B_i - P \right) ^+ \right] + {\mathbb {E}}\left[ \sum _{i=\bar{P}+1}^\infty 1_{ \{ T_i \le T \}} A_i \, r(L_i) \right] \\&\!\!\quad \le V^{(\bar{P})} (T,0) + r_m \,{\mathbb {E}}\left[ \sum _{i=\bar{P}+1}^\infty 1_{ \{ T_i \le T \}} \right] \le V^{(\bar{P})} (T,0) + r_m \frac{ [ \Lambda (T) ]^{ \bar{P}+1 } }{(\bar{P}+1 )!} \end{aligned}$$

where the last inequality follows from Remark 5.1 above. Since this upper bound holds for any policy ${\mathcal {A}}$ (feasible for the problem with no overbooking) we have

$$\begin{aligned} V^{(\infty )} (T,0) \le V^{(\bar{P})} (T,0) + r_m \frac{ [ \Lambda (T) ]^{ \bar{P}+1 } }{(\bar{P}+1 )!}. \end{aligned}$$

$\square$

Proof of Lemma 3.2

• (i) It is easy to see that the expressions in (17) and (18) are respectively non-decreasing and strictly increasing in $s$ for all $t \in [0, T]$. Moreover, if $s \mapsto f(\cdot , s)$ is non-increasing, so are $M_r[f](\cdot ,s)$ and $M_r[f](t - T_1,S_{T_1-}) = M_r[f](t - T_1,s -C_{T_1-})$. Therefore, $s \mapsto {\mathcal {L}}[f](\cdot ,s)$ in (15) is strictly decreasing for each $t \le T$. To prove the result on concavity, we first observe that if $s \mapsto f(\cdot ,s)$ is concave, then $s \mapsto M_{\cdot } [f] (\cdot , s )$ is also concave; see for example Lippman and Stidham (1977, Lemma 2). In Aydin et al. (2013, Lemma B.2) it is shown that if we compose a concave function with a binomially distributed random variable with parameters $s$ and $q$ (where $s$ denotes the number of Bernoulli trials and $q$ is the constant success probability in each trial), then the expected value of the composition is again concave in $s$. In (15), conditioned on $T_1$, the random variable $S_{T_1-}$ has the binomial distribution with parameters $s$ and $e^{-\mu T_1}$ on the event $\{ T_1 \le t \}$. Therefore, the conditional expectation of ${\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-})$ conditioned on $T_1$ is concave in $s$ on the event $\{ T_1 \le t \}$ almost surely, and this further implies that the unconditional expectation of $1_{ \{ T_1 \le t \}} \cdot {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} )$ is again concave in $s$. Next, note that $x \mapsto - (x-P)^+$ is a concave function, and conditioned on $T_1$ the sum $\sum _{i=1}^{S_t} B_i$ has the binomial distribution with parameters $s$ and $e^{-\mu t}p$ on the event $\{T_1 > t\}$. Then, similarly, we can conclude that the expectation of $- 1_{ \{ T_1 > t \}} \left( \sum _{i=1}^{S_t} B_i - P\right) ^+$ is also concave in $s$. Finally since the expression in (17) is linear in $s$, it follows that all the terms in (15) are concave in $s$ and this establishes the concavity.

• (ii) Under the transformation $y = T- t+ u$, the integral in (17) can be written as

  $$\begin{aligned} \int _0^t e^{-\mu u} F^{(t)} (du) = e^{\, \mu \cdot (T-t)} e^{\Lambda (T-t)} \int _{T-t}^T e^{\mu y} \, F^{(0)} (dy), \end{aligned}$$

  which is continuous in $t$. Clearly, the expressions in (18) is also continuous in $t$ for fixed $s$. To show the continuity of the last term in (15) we note that

  $$\begin{aligned}&{\mathbb {E}}^{(t,s)} \Big [ 1_{ \{ T_1 \le t \}} \cdot {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} ) \Big ]\\&\qquad = E^{(t,s)} \bigg [ 1_{ \{ T_1 \le t \}} \sum _{i = 0}^{s} {s \atopwithdelims ()i} (e^{-\mu T_1})^i (1- e^{-\mu T_1})^{s-i} \; \, M_{r(L_1)}[f](t- T_1, i) \bigg ]\\&\qquad = \sum _{i = 0}^{s} \sum _{j = 1}^{m} {s \atopwithdelims ()i} \int _0^t (e^{-\mu u})^i (1- e^{-\mu u})^{s-i} \; M_{r_j}[f](t- u, i) \frac{\lambda _j^{(t)} (u) }{\lambda ^{(t)} (u)} \, F^{(t)} (du). \end{aligned}$$

  Using the transformation $y = T- t+ u$ as above, the last expression can be rewritten as

  $$\begin{aligned}&\sum _{i = 0}^{s} \sum _{j = 1}^{m} {s \atopwithdelims ()i} e^{\mu i (T-t)} e^{\Lambda (T-t)} \int _{T-t}^T e^{\mu y i} \big (1- e^{-\mu [y- (T-t)]} \big )^{s-i}\\&\quad M_{r_j}[f](T-y, i) \frac{\lambda _j (y) }{\lambda (y)} F^{(0)} (dy). \end{aligned}$$

  For fixed $s,i$ and $j$, the integral above has the form $\int _{T-t}^T g_1(t,y) g_2(y) F^{(0)} (dy)$ where $g_1 (t, y) = (1- e^{-\mu [y- (T-t)]})^{s-i}$ is a uniformly continuous function on $[0,T ] \times [0,T]$, and the remaining term $g_2(\cdot )$ bounded on $[0,T]$. As a result, each integral and therefore the double sum above is continuous in $t$, for fixed $s$.

• (iii) For fixed $(t,s) \in [0,T]\times \{1, \ldots , \bar{P}-1\}$, we can decompose the difference as

  $$\begin{aligned} {\mathcal {L}}[f](t, s) - {\mathcal {L}}[f](t, s+1) = I_1(t,s) + I_2 (t,s) + I_3(t,s) \end{aligned}$$

where

$$\begin{aligned} I_1(t,s)&= -\kappa {\mathbb {E}}^{(t,s)} \left[ C_{t \wedge T_1} \right] + \kappa {\mathbb {E}}^{(t,s+1)} \left[ C_{t \wedge T_1}\right] \\ I_2(t,s)&= -\gamma \;{\mathbb {E}}^{(t,s)} \left[ 1_{\{ T_1 > t\}}\bigg ( \sum _{i=1}^{S_t} B_i - P \bigg )^+\right] \\&+\,\gamma {\mathbb {E}}^{(t,s+1)} \left[ 1_{\{ T_1 > t\}}\bigg ( \sum _{i=1}^{S_t} B_i - P \bigg )^+\right] \\ I_3(t,s)&= {\mathbb {E}}^{(t,s)} \left[ 1_{ \{ T_1 \le t \}} \cdot {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} ) \right] \\&-\,\, {\mathbb {E}}^{(t,s+1)} \left[ 1_{ \{ T_1 \le t \}} \cdot {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} ) \right] . \end{aligned}$$

It is easy to see that

$$\begin{aligned} I_1(t,s)&= -\kappa s {\mathbb {E}}^{(t,\cdot )} \left[ 1-e^{\mu (t \wedge T_1) } \right] + \kappa (s+1) {\mathbb {E}}^{(t,\cdot )} \left[ 1-e^{\mu (t \wedge T_1)} \right] \nonumber \\&= \kappa {\mathbb {E}}^{(t,\cdot )} \left[ 1-e^{\mu (t \wedge T_1)}\right] . \end{aligned}$$

(49)

Also note that $I_2 (t, s)$ is non-negative and can be rewritten as

$$\begin{aligned} I_2(t,s)&= -\gamma {\mathbb {E}}^{(t,s)} \left[ 1_{\{ T_1 > t\}}\bigg ( \sum _{i=1}^{S_t} B_i - P \bigg )^+\right] \\&+ \gamma {\mathbb {E}}^{(t,s)} \left[ 1_{\{ T_1 > t\}}\bigg (\sum _{i=1}^{S_t} B_i + Z - P \bigg )^+\right] \end{aligned}$$

in terms of an independent Bernoulli random variable $Z$ with parameter $p e^{-\mu t}$. Then we have

$$\begin{aligned} I_2(t,s)&\le - \gamma {\mathbb {E}}^{(t,s)} 1_{\{ T_1 > t\}} \bigg ( \sum _{i=1}^{S_t} B_i - P \bigg )^+ + \gamma {\mathbb {E}}^{(t,s)} 1_{\{ T_1 > t\}} \bigg ( \sum _{i=1}^{S_t} B_i - P \bigg )^+ \nonumber \\&\quad + \gamma {\mathbb {E}}^{(t,s)} 1_{\{ T_1 > t\}} Z = \gamma p e^{-\mu t}{\mathbb {E}}^{(t,s)} 1_{\{ T_1 > t\}}. \end{aligned}$$

(50)

As for the last term we observe that

$$\begin{aligned} I_3(t,s) \!=\! {\mathbb {E}}^{(t,s)} 1_{ \{ T_1 \le t \}} \cdot \left[ {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} ) - {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} + W ) \right] \end{aligned}$$

for a conditionally independent Bernoulli random variable $W$ with parameter $e^{-\mu T_1}$. This observation further gives

$$\begin{aligned} I_3(t,s)&= {\mathbb {E}}^{(t,s)} 1_{ \{ T_1 \le t \}} 1_{ \{ W =1 \}} \cdot \left[ {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} )\right. \\&\left. - {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} + 1 ) \right] \\&= {\mathbb {E}}^{(t,s)} 1_{ \{ T_1 \le t \}} e^{-\mu T_1 } \cdot \left[ {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} )\right. \\&\left. - {\mathcal {M}}_{ r(L_1)} [f] (t-T_1 , S_{T_1-} + 1 ) \right] . \end{aligned}$$

Note that the operator ${\mathcal {M}}$ preserves the upper and lower bounds given in the statement of Lemma 3.2(iii). Therefore, we have

$$\begin{aligned}&{\mathbb {E}}^{(t,s)} 1_{ \{ T_1 \le t \}} e^{-\mu T_1 } \kappa (1-e^{-\mu (t-T_1)}) \le I_3(t,s)\nonumber \\&\qquad \le {\mathbb {E}}^{(t,s)} 1_{ \{ T_1 \le t \}} e^{-\mu T_1 } \left[ \kappa (1-e^{-\mu (t-T_1)}) + \gamma p e^{-\mu (t-T_1)} \right] . \end{aligned}$$

(51)

Finally, when we combine the identity in (49) with the upper and lower bounds in (50) and (51), straightforward algebra yields

$$\begin{aligned} \kappa (1-e^{-\mu t}) \le I_1 (t,s) + I_2 (t,s) + I_3 (t,s) \le \kappa (1-e^{-\mu t}) + \gamma p e^{-\mu t}. \end{aligned}$$

$\square$