skip to main content
research-article
Public Access

Optimal Pricing in a Single Server System

Published: 11 August 2023 Publication History

Abstract

We study optimal pricing in a single server queue when the customers valuation of service depends on their waiting time. In particular, we consider a very general model, where the customer valuations are random and are sampled from a distribution that depends on the queue length. The goal of the service provider is to set dynamic state dependent prices in order to maximize its revenue, while also managing congestion. We model the problem as a Markov decision process and present structural results on the optimal policy. We also present an algorithm to find an approximate optimal policy. We further present a myopic policy that is easy to evaluate and present bounds on its performance. We finally illustrate the quality of our approximate solution and the myopic solution using numerical simulations.

1 Introduction

The problem of allocating limited resources amongst competing users is traditionally studied using queueing theory and stochastic networks, with a focus on metrics such as throughput and delay. Users and system operators in real world respond to prices and incentives in addition to the queueing metrics. The focus of this article is to study such interplay of queueing and pricing. A user has certain valuation for the service, and the user enters the system only if the price is smaller than their valuation. However, the valuation of the user can decrease based on quality of experience, such as delay and other queueing metrics. The service provider, therefore, employs dynamic pricing and tunes prices in order to use them as control levers for admission control and to manage congestion in the system. Moreover, the prices affect the service provider’s revenue, which they want to maximize. Examples of such systems include spot pricing for the cloud services [28], surge pricing in ridehailing systems such as Uber [8], auctions for web advertisements [15], and highway tolls with increased peak-hour rates [18].
In this article, we abstract away the resources and model them by a single server queue. Several pricing questions in such a simple system are still unanswered in the literature. Such a system can be used to model the aggregate service capacity of a shared service system such as a data center. Customers arrive into this system according to a Poisson process and see the queue length ahead. A customer’s valuation of the service depends on the queue length. Given the queue length, the customer samples its valuation in an independent and identically distributed (i.i.d.) manner from a distribution. The service provider posts a price at each time that depends on the current queue length. If the customer’s valuation is larger than the price, she pays the price, joins the queue, and leaves only after service completion; otherwise, she does not join the queue. All customers in the queue are served in an FCFS manner, and each customer needs an i.i.d. exponential amount of service. The service provider is not aware of the individual valuations of the customers, but knows the distributions of the queue length dependent valuations. We study the problem from the perspective of the service provider, aiming to maximize long run average revenue.
The following are the main contributions of our work.
We formulate the revenue maximization problem as a continuous time Markov decision process (MDP) and present structural results on the optimal pricing policy and the optimal revenue.
Since a queue has infinite states, finding the exact solution of this MDP involves solving an infinite system of equations. By truncating the state space appropriately, we propose a finite system of equations to obtain an approximate solution and present its structural properties.
We present an algorithm to find the approximate solution, and present guarantees on its convergence. We arrive at this algorithm following intricate arguments that translate Bellman’s equation to a fixed point equation in single variable, namely the optimal revenue rate. Counterintuitively, the optimal prices may be non-monotonic in queue lengths.
We consider a special case when the valuations are a deterministic function of the queue lengths and present explicit closed form expression for the optimal dynamic prices. Such deterministic valuations were considered in prior work [5, 7] which focused on only specific functions. Our model is therefore, more general, since we not only consider general deterministic valuations, but also allow randomized valuations.
We present a myopic policy, prices of which are much simpler to obtain. We show that even such a simple policy has good performance, and present a lower bound on the ratio of average revenue rates under the myopic policy and the optimal policy. We further present numerical simulations to compare the two policies. The simulations also illustrate rich tradeoffs that occur due to interplay of customers’ preference towards smaller delay and the provider’s goals of maximizing revenue and managing congestion.

1.1 Related Work

Since the queue pricing problem arises naturally in a number of situations, it has been studied extensively. The earliest work on pricing in a queue is the seminal paper by Naor [22], in which the entry of customers to a queue was regulated using tolls. After observing the queue size, customers decide to join (or not join) the queue. Such systems are called observable. Customers join if the difference between their valuation of the job and the cost of waiting exceeds the admission price to the queue. This is equivalent to a threshold type policy—if the queue length is smaller than the threshold, the customers join; otherwise they leave. The optimal threshold varies, depending on whether we are maximizing the social utility or the revenue. In [22], it was demonstrated that the socially optimal threshold was higher than the revenue maximization threshold. A number of subsequent works have looked at the queue pricing problem under different constraints and assumptions, and study the effects of different system parameters. An exhaustive survey of this literature is available in [13, 14].
We are interested specifically in the question of maximization of revenue. In [20], the author studies optimal pricing for an \(M/M/s\) queue with finite waiting room. The author shows that the optimal prices are monotonically increasing in the number of customers waiting in the system. Such structural results for the optimal prices are a common result in a number of works. For example, [23] shows that optimal price is monotonically increasing in the number of customers, for a multi server system with no waiting room. Proceeding further along these lines, [7] is interested in maximizing the expected discounted revenue, while keeping the queueing model of [22]. They obtain a revenue optimizing threshold queue length beyond which entries are not allowed into the queue. This threshold can be computed numerically. All customers who see a waiting queue length smaller than this threshold pay a price equal to the difference between their valuation and waiting cost. In [5], an explicit form is derived for the threshold obtained in the previous work, and they characterize the earning rate asymptotically. However, both aforementioned works provide explicit solutions in the case of fixed service valuation (or simple valuation distributions, such as a valuation which takes two values). They do not provide explicit solutions for valuations with continuous support and general distributions.
In [30], the authors look at optimal pricing in finite server queueing systems, but restrict themselves to the sub-optimal class of static prices. These prices are not dependent on the system state (queue length). They study the variation of optimal price with number of servers. While static prices are sub-optimal, they can be close to optimal in some systems. For example, in [4], the authors prove the existence of a static pricing policy that obtains \(78.9\%\) of the optimal profit in a system with multiple reusable resources. This result holds under the assumption that the revenue rate is a concave function of the arrival rate. In [21], the authors obtain the revenue maximizing policy for a queueing system under the assumption that the generalized hazard rate of the valuation distribution, which is the ratio of price times the density of valuation, to the complementary cumulative distribution of valuation is strictly increasing in price. This assumption, however, does not hold in general. There are a number of similar works that provide existential results and structural results for optimal policies, after modelling the revenue maximization problem as a Markov decision problem (MDP) [10, 27, 29]. There are also many works that study the pricing problem in queues in different asymptotic regimes. In [1], an asymptotically optimal price is obtained for customers with fixed valuations. In [16], the authors show that the revenue loss due to randomness is lower for dynamic pricing than static pricing. An approximately optimal price is obtained by solving a diffusion equation in [6].
In our earlier works [17, 25], we study optimal service pricing in multi-server systems in which the service provider charges a time varying service fee aiming at maximizing its revenue rate. The customers that find free servers and service fees lesser than their valuation join for the service else they leave without waiting. We solve the optimal pricing problems using the framework of MDPs and show that the optimal prices depend on the number of free servers. We propose algorithms to compute the optimal prices. We also establish several properties of the optimal prices and the corresponding revenue rates in the case of Poisson customer arrivals.
In this article, we proceed to model the control problem as an MDP and proceed to solve it. We are interested in the optimal prices themselves, rather than structural properties, as in [10, 27, 29]. We obtain the optimal solution as the solution of a fixed point equation. In all the works discussed, the optimal prices are obtained under restrictive assumptions [4, 21], or assume non-random (or less random) service valuations [5, 7]. We also assume a more general structure for the valuation distribution, which can be viewed as a generalization of valuation models such as in [22] and [19]. Hence, the results in this article are more general.

1.2 Organization of the Article

The rest of this article is organized as follows. In Section 2, we describe the system model, formulate the problem as an MDP, and write the Bellman’s equations corresponding to the optimal policy. In Section 3, we solve the Bellman’s equations after truncating the equations. In Section 4, we obtain a myopic policy that is easy to implement and serves as a benchmark for comparison. In Section 5, we solve the Bellman’s equations for the special case when the valuation function is deterministic. Numerical simulations are presented in Section 6. For ease of reading, we have moved some of the lengthier proofs into a separate section, Section 7.

2 System Model and Preliminaries

We begin with description of the system model, specifically of the stochastic valuation functions. Subsequently, we frame the optimal pricing problem as a continuous time Markov decision problem (CTMDP). We write the Bellman’s equations and derive several properties.

2.1 System Model

We model the system as a single server queue with infinite buffer size. Customers arrive as a Poisson process of rate \(\lambda\). Each arriving customer has a random service time requirement S. We will assume that the service requirements of different customers are i.i.d. exponential with mean \(\frac{1}{\mu }\). The admission price is a function of the number of customers present in the system. We model the service valuation as a non-negative quantity, stochastically decreasing with the number of waiting customers, motivated by the fact that the mean waiting time of a customer is directly proportional to the number of waiting customers. As such, two customers who see the same queue length on arrival will have i.i.d. valuations. If two customers see queue lengths \(i_1\) and \(i_2\) on arrival, where \(i_1\gt i_2\), their valuations \(V_{i_1}\) and \(V_{i_2}\) will be stochastically ordered, i.e.,
\begin{equation} \mathbb {P}[V_{i_1}\leqslant x]\geqslant \mathbb {P}[V_{i_2}\leqslant x],~x \geqslant 0. \end{equation}
(1)
Condition (1) implies that on average, customers seeing a higher queue length on arrival will have lower valuation. Let \(G_i(\cdot)\) denote the value distribution for a customer seeing i customers on arrival, i.e., \(G_i(x)=\mathbb {P}[V_i\leqslant x]\). We denote the complementary valuation distributions by \(\overline{G}_i \triangleq 1-G_i\) for each i. We have succinctly represented our system model in Figure 1.
Fig. 1.
Fig. 1. A single server queueing system where customers arrive as a Poisson process of rate \(\lambda\) with i.i.d. exponential service times of rate \(\mu\). An arriving customer that sees i existing customers in service has a random valuation \(V_i\) with distribution \(G_i\) independent of everything else.
We assume that the service provider selects a deterministic service fee of \(u_i\) for an arriving customer that sees i queued customers at its arrival. If the valuation \(V_i\) exceeds this fee, then the customer pays this service fee \(u_i\) to the provider and joins the system; else he leaves without joining the system. That is, given that an incoming arrival sees i queued customers, its probability of joining the system is \(\overline{G}_i(u_i)\) independent of everything else. We denote the sequence of prices as \(\mathbf {u}= (u_0, u_1, \dots)\). Our objective is to find the stationary price sequence that maximizes the long term average revenue generated by the service provider.

2.1.1 Discussion on the Valuation Model.

The valuation model we present here subsumes as a special case other waiting cost models studied in the literature, as the following examples illustrate.
Example 1 (Valuation Model of [7]).
Let the random valuation of arriving customers be i.i.d., and denoted by V for a typical customer. Let the price of admission be u. Let \(\tau\) be the random waiting time until service completion for a typical customer, c be the cost per unit time of waiting, and \(\gamma \gt 0\) be some discount factor. An arriving customer joins the system if
\begin{equation} V-\mathbb {E}\left[c\int _0^{\tau }e^{-\gamma t}{\rm d} t\right]\gt u. \end{equation}
(2)
If the service times of customers are i.i.d. exponential with mean \(\frac{1}{\mu }\) and there are i existing customers in the system at the time of the arrival, \(\tau\) is equivalent to the sum of \(i+1\) exponential random variables. Then, (2) is equivalent to
\begin{equation} V+\frac{c}{\gamma }\left(\frac{\mu }{\mu +\gamma }\right)^{i+1}-\frac{c}{\gamma }\gt u. \end{equation}
(3)
Thus, the effective valuation of the job from the perspective of the customer who sees n people queued ahead is \(V+\frac{c}{\gamma }(\frac{\mu }{\mu +\gamma })^{i+1}-\frac{c}{\gamma }\), which is a decreasing function of the queue length i seen on arrival. If the discount factor \(\gamma =0\), the queue length dependent valuation can be seen to be \(V-\frac{c(i+1)}{\mu }\). From this example, it makes sense to consider mean valuation decreasing in queue length. Such an assumption generalizes the above model and accommodates (2) for all non-negative values of \(\gamma\). It also includes other models where the cost of delay may have a different functional behavior, though the mean cost of delay increases in queue length.
Example 2 (Valuation Model of [19]).
Let the valuation of a customer who sees i customers on arrival be given by \(V_i=X-iY\), where X and Y are proper non-negative random variables. For \(i_1\gt i_2\), it is easy to check that
\begin{equation*} \mathbb {P}[V_{i_1}\leqslant x]\geqslant \mathbb {P}[V_{i_2}\leqslant x]. \end{equation*}
Thus, this is a special case of our valuation model.

2.2 MDP Formulation

We formulate the pricing problem as a CTMDP [3, chapter 5]. The state of the CTMDP is the number of customers \(Q(t)\) present in the system at time t. This state evolves over the state space \(\mathcal {Q}\triangleq \left\lbrace 0,1,2,3,\dots \right\rbrace\). The action in each state i is to pick the corresponding price \(u_i\), and we seek to find a stationary state dependent policy \(\mathbf {u}=(u_i,i\in \mathcal {Q})\). The state of the system evolves in a Markovian manner in continuous time. The transition rates from state i to \(i+1\) are
\begin{equation*} \lambda _i \triangleq \lambda \overline{G}_i(u_i). \end{equation*}
The transition rate from state \(i+1\) to i is always the service rate \(\mu\). The state transition diagram is given in Figure 2.
Fig. 2.
Fig. 2. State transition diagram of Markov chain \(Q(t)\).
Thus, the transition rates of the CTMDP under a policy \(\mathbf {u}\) are given by
\begin{equation} \nu _i(u_i) = {\left\lbrace \begin{array}{ll} \lambda \bar{G}_0(u_0) & \text{ if } i = 0,\\ \mu + \lambda \bar{G}_i(u_i) & \text{ if } i \geqslant 1. \end{array}\right.} \end{equation}
(4)
The pairwise transition probabilities are given by, \(p_{01}(u_0) = 1\) and for \(i \geqslant 1\),
\begin{equation} p_{ij}(u_i) = {\left\lbrace \begin{array}{ll} \frac{\lambda \bar{G}_i(u_i)}{\nu _i(u_i)} & \text{ if } j = i+1,\\ \frac{\mu }{\nu _i(u_i)} & \text{ if } j = i-1,\\ 0 & \text{ otherwise.} \end{array}\right.} \end{equation}
(5)
Customers pay the price on arrival. Therefore, in state i, the provider receives a reward of \(u_i\) if the next transition is due to an arrival that accepts the price, and does not receive any reward if the transition is due to a departure. Thus, on average, the reward \(g(i,u_i)\) received in state i under an action \(u_i\) is given by
\begin{equation} g(i,u_i) = {\left\lbrace \begin{array}{ll} u_0 & \text{ if } i = 0,\\ \frac{\lambda u_i\bar{G}_i(u_i)}{\nu _i(u_i)} & \text{ if } i \geqslant 1. \end{array}\right.} \end{equation}
(6)
Following the discussion in [3], we observe that the long term average revenue is equal to
\begin{equation} R(\mathbf {u})=\lim _{N \rightarrow \infty }\frac{1}{\mathbb {E}t_N} \mathbb {E}\sum _{n = 1}^{N} g(Q_n,u_{Q_n}), \end{equation}
(7)
where \(t_N\) is the completion time of the Nth transition of the Markov chain. We formally state our problem in the following.
Problem 1.
Find the optimal state dependent price sequence \(\mathbf {u}:\mathcal {Q}\rightarrow \mathbb {R}_+\) that maximizes long term average reward \(R(\mathbf {u})\) defined in (7), where the state evolves as a controlled CTMC with transition rates \((\nu _i(u_i): i \in \mathcal {Q})\) defined in (4) and controlled transition probabilities \((p_{ij}(u_i):i,j\in \mathcal {Q})\) for the associated jump chain defined in (5), under any price sequence \(\mathbf {u}\). That is, we can define the optimal pricing as
\begin{equation} \mathbf {u}^\ast \triangleq \arg \max R(\mathbf {u}), \end{equation}
(8)
and the optimal revenue rate as
\begin{equation} \theta ^\ast \triangleq R(\mathbf {u}^\ast). \end{equation}
(9)

2.3 Bellman’s Equations

First, we obtain Bellman’s equations corresponding to the average reward CTMDP problem defined in Problem 1, and rewrite them as a series of iterative equations. Finding the optimal solution for Problem 1 is equivalent to solving the following set of Bellman’s equations
\begin{equation} h(i) = \max _{u_i}\left\lbrace g(i,u_i) - \frac{\theta }{\nu _i(u_i)} + \sum _j p_{ij}(u_i)h(j) \right\rbrace ,\qquad i \in \mathcal {Q}. \end{equation}
(10)

2.3.1 Uniformization.

From the definition of transition rates \(\nu _i(u)\) defined in (4) and the fact that \(\bar{G}_i(u_i) \leqslant 1\), it follows that the transitions rates are uniformly bounded for all states \(i \in \mathcal {Q}\) and prices \(\mathbf {u}:\mathcal {Q}\rightarrow \mathbb {R}_+\), where
\begin{equation*} \nu _i(u_i) \lt \Lambda \triangleq \mu + \lambda . \end{equation*}
By scaling the transition rates with \(\Lambda\), we can uniformize the controlled CTMC. The state transition probabilities are redefined to allow self-transitions such that the resulting dynamics remains unchanged. Specifically, we define the transition probabilities for the jump chain of uniformized controlled CTMC as
\begin{equation} \tilde{p}_{ij}(u_i) = {\left\lbrace \begin{array}{ll}\frac{\lambda \bar{G}_i(u_i)}{\Lambda } & \text{ if } j = i+1,\\ \frac{\lambda G_i(u_i)}{\Lambda } & \text{ if } j = i,\\ \frac{\mu }{\Lambda } & \text{ if } j = i-1. \end{array}\right.} \end{equation}
(11)
Thus, we can transform the CTMDP problem defined in Problem 1 to an equivalent MDP defined below.
Problem 2.
Consider a controlled DTMC with transition probabilities \((\tilde{p}_{ij}(u_i):i,j\in \mathcal {Q})\) and per stage reward \(g(i,u_i)\) defined in (6), for any price sequence \(\mathbf {u}:\mathcal {Q}\rightarrow \mathbb {R}_+\). Find the optimal state dependent price sequence \(\mathbf {u}\) that maximizes the long term average reward
\begin{equation} R(\mathbf {u}) \triangleq \lim _{N\rightarrow \infty }\frac{1}{N}\mathbb {E}\sum _{n=1}^Ng(Q_n,u_{Q_n}). \end{equation}
(12)
The set of Bellman’s equations for the MDP defined in Problem 2 is equivalent to the set of Bellman’s equations given in (10) and are given by
\begin{equation} \tilde{h}(i) = \max _{u_i}\left\lbrace g(i,u_i)\nu _i(u_i) - \theta + \sum _j \tilde{p}_{ij}(u_i)\tilde{h}(j) \right\rbrace ,\quad i \in \mathcal {Q}. \end{equation}
(13)
Remark 1.
The pair \((\theta ,h)\) satisfies the Bellman’s equation for the original continuous time problem defined in Problem 1 if and only if the pair \((\theta ,\tilde{h})\) satisfies the Bellman’s equation for the discrete time problem defined in Problem 2, where \(\tilde{h}(i) = \Lambda h(i)\) for all i. Moreover, for all the states, the optimal actions for the two problems are identical [3, Chapter 5, Proposition 3.3].

2.3.2 Reduction of Bellman’s Equations.

Substituting the values of \(g(i,u)\) from (6) in the Bellman’s Equation (13) for the discrete-time system, we obtain
\begin{equation} \tilde{h}(i) = \max _{u_i}\left\lbrace \lambda u_i\bar{G}_i(u_i) - \theta + \sum _j \tilde{p}_{ij}(u_i)\tilde{h}(j) \right\rbrace ,\quad i \in \mathcal {Q}. \end{equation}
(14)
We define the scaled difference of \(\tilde{h}(i)\) for all \(i \in \mathcal {Q}\), as
\begin{align} \Delta (i) \triangleq (\tilde{h}(i) - \tilde{h}(i+1))/\Lambda . \end{align}
(15)
We define the following functions that help us understand the properties of optimal pricing and revenue rate for Problem 2. For all \(i \in \mathcal {Q}\), we define
\begin{equation} m_i(B) \triangleq \sup _u\left\lbrace (u-B)\bar{G}_i(u) \right\rbrace . \end{equation}
(16)
Assumption 1.
The supremum in (16) is achieved and \(m_i(B)\) is finite for all B and i. In particular, we assume that \(\lim _{u \rightarrow \infty }u\bar{G}_i(u) \lt \infty\) for all \(i \in \mathcal {Q}\).
We further define for all \(i \in \mathcal {Q}\), the pricing functions
\begin{equation} u_i(B) \triangleq \max \arg \max _u\left\lbrace (u-B)\bar{G}_i(u) \right\rbrace . \end{equation}
(17)
The terms \(u_i(B)\) and \(m_i(B)\) can be interpreted as follows. Suppose the system is in state i (i.e., there are i customers). For any price u that is set, the instantaneous revenue at the next arrival is \(u\bar{G}_i(u)\). Suppose B is the “cost” that captures reduced valuation of the future users due to increase in queue length because of admitting this user, then the expected instantaneous “profit” is \((u-B)\bar{G}_i(u)\), and \(u_i(B)\) is the price that maximizes this expected instantaneous profit and \(m_i(B)\) is the corresponding profit. We note that the cost B is a priori unknown but is yielded by the analysis in Section 3.
Substituting the values of \(\tilde{p}_{ij}(u)\) from (11), the reduced Bellman’s Equation (14) can be rewritten in terms of the functions \(m_i(\cdot)\)’s defined in (16) and scaled differences \(\Delta (\cdot)\)’s defined in (15), as
\begin{align} m_0(\Delta (0)) &= \frac{\theta }{\lambda }, \end{align}
(18)
\begin{align} m_i(\Delta (i)) &= \frac{\theta - \mu \Delta (i-1)}{\lambda },\quad i \geqslant 1. \end{align}
(19)

2.3.3 Limiting Valuation and Pricing.

We show that the valuation distribution \(G_i\)’s being stochastically ordered implies that there exists a distribution function G which is the limit of \(G_i\)’s at all continuity points. Thus, the valuations \(V_i\)’s converge in distribution to a random variable V with the distribution G.
Lemma 3.
The random valuations \(V_i\)’s converge in distribution to a limiting random variable V with distribution G that satisfies \(G(u) = \lim _{i \rightarrow \infty }G_i(u)\) at all the continuity points u of G.
Proof.
See Section 7.1.1.□
Remark 2.
Notice that \(G(\cdot)\) is identical to \(\lim _{i\rightarrow \infty }G_i(\cdot)\) at its continuity points; it may differ at the points where \(G(\cdot)\) is not continuous. In fact, the limit \(\lim _{i\rightarrow \infty }G_i\) may not be a distribution function. If the distributions \(G_i\) are all continuous and converge to a continuous function G, this distinction is not important. Furthermore, Assumption 1 implies that \(\lim _{u \rightarrow \infty }u\bar{G}(u) \lt \infty\).
Let \(C \subset [0, \infty)\) be the set of non-negative continuity points of \(G(\cdot)\), and define
\begin{align} m(B) \triangleq & \max _{u}\left\lbrace (u-B)\bar{G}(u) \right\rbrace = \max _{u \in C}\left\lbrace (u-B)\bar{G}(u) \right\rbrace , \end{align}
(20)
\begin{align} u(B) & \triangleq \max \arg \max _u\left\lbrace (u-B)\bar{G}(u) \right\rbrace . \end{align}
(21)
We have defined the limiting value \(m(B)\) and limiting price \(u(B)\) assuming that the maximization in (20) has a solution for each B. It can be shown that \(\lim _{i \rightarrow \infty } m_i(B) \geqslant m(B)\) for all B. It is not clear whether \(m_i(B)\) and \(u_i(B)\) converge to \(m(B)\) and \(u(B),\) respectively. However, it is not hard to imagine that these sequences converge under certain smoothness assumptions. We summarize these assumptions below.
Assumption 2.
The maximum in (20) is achieved, \(\lim _{i \rightarrow \infty } m_i(B) = m(B)\) and \(\lim _{i \rightarrow \infty } u_i(B) = u(B)\) for all B.
This assumption is used in Lemma 30 to argue that if the original problem is non-degenerate, then the k-truncated problems are also non-degenerate for large k.
Example 4.
Consider the exponential valuation distributions, where the complimentary distribution function of valuation is defined as \(\overline{G}_i(u) \triangleq e^{-(i+1)u}\) for \(u \geqslant 0\) and \(i \in \mathcal {Q}\). We can see that \(G_i\)’s converge to G which is zero everywhere except at \(u=0\). Correspondingly, \(m(B)=0\) and \(m_i(B)=\frac{1}{i+1}e^{-(1+B(i+1))}\) and we can see that \(m_i(B)\rightarrow m(B)\) as \(i\rightarrow \infty\).
We observe that Assumption 2 holds in this case.

3 Optimal Pricing

We have an infinite system of equations from Bellman’s equation for the problem. The properties of optimal pricing sequences depend on the limiting valuation of the customer that sees an arbitrary large number of existing customers. Thus, we need to understand the limit of valuations \(V_i\)’s and the distributions \(G_i\)’s. This is the focus of the first subsection. In the next subsection, we propose a truncation method to obtain an approximate solution. We show that as the truncation threshold gets larger, we converge to the optimal solution to Problem 2 with an infinite system of equations.

3.1 Existence of the Optimal Solution

Definition 5.
We call a pricing policy \(\mathbf {u}\) ergodic if it renders the CTMC positive recurrent.
Proposition 6.
Let \((\theta ,\Delta (0),\Delta (1),\dots)\) be a solution to (18) and (19) such that \((\Delta (i): i \in \mathcal {Q})\) are bounded. Then \(\theta\) is an upper bound on the revenue rate for any ergodic policy u, and hence an upper bound on the optimal average revenue rate \(\theta ^\ast\). Moreover, if the policy \(\bar{\mathbf {u}}:\mathcal {Q}\rightarrow \mathbb {R}_+\) is defined by
\begin{equation} \bar{u}_i \triangleq u_i(\Delta (i)) \text{ for all }i \in \mathcal {Q} \end{equation}
(22)
is ergodic, then \(\theta ^\ast = \theta\), implying that \(\bar{\mathbf {u}} = \mathbf {u}^\ast\), the optimal price sequence.
Proof.
See Section 7.2.□

Degeneracy.

Before proceeding further, we discuss a special case. Recall that \(\mathbf {u}^\ast\) is the optimal ergodic policy and \(\theta ^\ast \triangleq R(\mathbf {u}^\ast)\) is the maximum revenue rate across all the ergodic policies. Let \(\theta ^\ast \leqslant \lambda m(0) = \lambda \max _u u \bar{G}(u)\). The policy \(\mathbf {u}= (u(0),u(0),\dots)\) achieves a revenue rate \(R(\mathbf {u}) \geqslant \lambda u(0) \bar{G}(u(0))\). The inequality follows from the fact that \((V_i: i \in \mathcal {Q})\) are stochastically decreasing. This result implies that the policy \(\mathbf {u}\) achieves a revenue rate greater than or equal to the maximum revenue rate \(\theta ^\ast\) across all ergodic policies. In this case, we call our pricing problem degenerate [11]. As in [11], we will bypass the degenerate case, i.e., we will focus on the case \(\theta ^\ast \gt \lambda m(0)\).
Remark 3.
Observe that if \(\theta ^\ast \lt \lambda m(0),\) then \(\mathbf {u}\) cannot be an ergodic policy. That is, \(\lambda \bar{G}_i(u(0)) \geqslant \mu\) for infinitely many \(i \in \mathcal {Q}\). Since \(V_i\)’s are stochastically decreasing, it implies that \(\lambda \bar{G}_i(u(0)) \geqslant \mu\) for all \(i\in \mathcal {Q}\). In particular, \(\lambda \bar{G}(u(0)) \geqslant \mu\). This implies that even at arbitrary large queue-length, the valuation of incoming customers does not decrease enough to discourage them from entering the system leading to instability. On the other hand, if \(\theta ^\ast = \lambda m(0)\), then \(\mathbf {u}\) may or may not be an ergodic policy. Further analysis of this case is quite subtle, and we do not consider this case.

3.2 Truncated Problems

We recall the reduced Bellman’s Equations (18) and (19), which characterize the optimal price sequence for the revenue rate maximization problem. This being an infinite system of equations is not amenable to numerical computation. We address this issue by considering the corresponding k-truncated systems defined as follows.
Consider a system with sequence of valuation distributions \((G_i: i \in \mathcal {Q})\). For any \(k \in \mathcal {Q}\), the corresponding k-truncated system is defined as the one with sequence of valuation distributions \((G_{i\wedge k}: i \in \mathcal {Q})\). We refer to Problem 2 associated with k-truncated system as the k-truncated problem.
Remark 4.
We observe that in the original problem, the valuation distribution is non-decreasing for all states \(i \in \mathcal {Q}\). However, the valuation distribution gets fixed for all \(i \geqslant k\) in the k-truncated problem. It follows that the optimal revenue rate for k-truncated problems would be non-increasing in k, and lower bounded by the optimal revenue rate of the original problem.
In the following, we derive properties of the optimal pricing sequences and the corresponding revenue rates for the k-truncated problem. We also prove that as the truncation threshold k increases, the corresponding optimal pricing sequences when applied to the original system yield revenue rates arbitrarily close to the optimal revenue rate \(\theta ^\ast\). In Section 3.4, we develop an algorithm to derive the numerical solution to a k-truncated problem.
Remark 5.
It follows from (16) and (17) that the \(m_i\)s and \(u_i\)s for a k-truncated system are \((m_{i\wedge k}: i \in \mathcal {Q})\) and \((u_{i\wedge k}: i \in \mathcal {Q}),\) respectively. Thus, the optimal price sequence for the k-truncated system is characterized by \(k+2\) equations,
\begin{align} m_0(\Delta (0)) &= \frac{\theta }{\lambda }, \end{align}
(23)
\begin{align} m_i(\Delta (i)) &= \frac{\theta - \mu \Delta (i-1)}{\lambda },\quad 1 \leqslant i \leqslant k, \end{align}
(24)
\begin{align} \Delta (k) &= \Delta (k-1). \end{align}
(25)
We can view these \(k+2\) equations as a fixed point equation in \(\theta\) as follows. We express \(\Delta (i)\)’s successively in terms of \(\theta\). In particular, we define functions \(\phi _i:\mathbb {R}_+\rightarrow \mathbb {R}\) as
\begin{align} \phi _{-1}(\theta) &= 0, \nonumber \nonumber\\ \phi _i(\theta) &= m_i^{-1}\left(\frac{\theta -\mu \phi _{i-1}(\theta)}{\lambda }\right), \quad i \in \mathcal {Q}. \end{align}
(26)
Then, from (18) and (19), we can see that \(\Delta (i) = \phi _i(\theta)\) for all \(i \in \mathcal {Q}\) and the above system of equations reduces to \(\phi _k(\theta) = \phi _{k-1}(\theta)\). We analyze this fixed point equation in Proposition 9.
For notational convenience, we define functions \(\delta _i:\mathbb {R}_+\rightarrow \mathbb {R}\) for all \(\theta \in \mathbb {R}_+\) and \(i \in \mathcal {Q}\), as
\begin{equation} \delta _i(\theta) \triangleq \phi _i(\theta) - \phi _{i-1}(\theta). \end{equation}
(27)
Then \(\delta _0(\theta) = m_0^{-1}(\theta /\lambda)\), and for \(i \geqslant 1\),
\begin{equation} \delta _i(\theta) = m_i^{-1}\left(\frac{\theta -\mu \phi _{i-1}(\theta)}{\lambda }\right) - \phi _{i-1}(\theta) = \phi _i(\theta) - \frac{\theta - \lambda m_i(\phi _i(\theta))}{\mu }. \end{equation}
(28)
Furthermore, the desired fixed point equation for the k-truncated system reduces to \(\delta _k(\theta) = 0\). We characterize the solutions to these equations for all thresholds k in Proposition 9. However, it requires the following assumption.
Assumption 3.
For all \(B \geqslant 0\), the probability \(\bar{G}_i(u_i(B))\) of joining the queue under price \(u_i(B)\) is non-increasing in the observed state \(i \in \mathcal {Q}\).
This assumption is used in Proposition 9 and in Theorem 15 to establish that the k-truncated problems posses unique optimal revenue rates, \(\theta _k\)s, which are non-increasing in k. Consequently, all the results that require Proposition 9 rely on this assumption. It is also used in Lemma 30. This assumption is also satisfied by many valuation sequences. Following is one such instance.
Example 8.
Let \(G_i(u) \triangleq 1-e^{-(i+1)u}\) for \(u \geqslant 0\) and \(i \in \mathcal {Q}\). In this case, \(u_i(B)= B+\frac{1}{i+1}\) and \(\bar{G}_i(u_i(B))=e^{-(1+B(i+1))}\) for \(B \geqslant 0\) and \(i \in \mathcal {Q}\). This is clearly non-increasing in i.
Proposition 9.
Under Assumption 3, there exists a unique non-increasing sequence \((\theta _k: k \in \mathcal {Q})\) such that \(\delta _k(\theta _k) = 0\) for all \(k \in \mathcal {Q}\). Moreover, for functions \(\phi _i(\cdot)\) defined in (26), all \(k \in \mathcal {Q}\) and \(i \leqslant k\),
\begin{equation*} \phi _{i-1}(\theta _k) \leqslant \phi _i(\theta _k). \end{equation*}
Proof.
See Section 7.3.□
Remark 6.
If \(\theta _k \leqslant \lambda m_k(0)\), the k-truncated problem is degenerate, and uniform pricing, i.e., setting the same price for all queue length states, offers better revenue rate than any ergodic pricing policy.
The following theorem characterizes the optimal revenue rate as well as an optimal price sequence for the non-degenerate k-truncated problem.
Theorem 10.
The optimal revenue rate of the k-truncated problem is upper bounded by \(\theta _k\) defined in Proposition 9. We define the following policy in terms of \(\phi _i(\cdot)\)’s defined in (26) and \(\theta _k\), as
\begin{equation} u^k_i \triangleq {\left\lbrace \begin{array}{ll} u_i(\phi _i(\theta _k)) \ \text{for } i \in \left\lbrace 0,1,\dots ,k-1 \right\rbrace ,\\ u_k(\phi _k(\theta _k)) \ \text{for } i \geqslant k. \end{array}\right.} \end{equation}
(29)
If \(\theta _k \gt \lambda m_k(0)\), then policy \(u^k\) is ergodic, achieves revenue rate \(\theta _k\), and hence optimal for the k-truncated problem.
Proof.
See Section 7.3.2.□

3.3 Convergence to an Optimal Solution

We now show that the optimal solutions to the k-truncated problem converge monotonically to an optimal solution to the original pricing problem, as the threshold \(k \rightarrow \infty\). We also obtain a bound on the performance (i.e., the revenue rates) of the optimal policy for a k-truncated problem. First, we show that as the truncation threshold k increases, the corresponding optimal pricing sequences \((\mathbf {u}^k: k \in \mathcal {Q})\) when applied to the original system yield revenue rates \(R(\mathbf {u}^k)\) arbitrarily close to the optimal revenue rate \(\theta ^\ast\).
Remark 7.
From Theorem 10, \(\theta _k\) is the optimal revenue rate for a k-truncated problem. From Remark 4, the sequence \((\theta _k, k \in \mathcal {Q})\) is non-increasing and \(\theta _\infty \triangleq \inf _{k \in \mathcal {Q}}\theta _k \geqslant \theta ^\ast\). Since \(\phi _i(\cdot)\) is continuous and strictly decreasing (see Lemma 29 in Section 7.3), it follows that \(\phi _i(\theta _k)\) is monotonically increasing in k and converges to \(\phi ^\ast _i \triangleq \max _{k\in \mathcal {Q}}\phi _i(\theta _k) = \phi _i(\theta _\infty)\). Recall from Proposition 9 that \(\phi _i(\theta _k)\) is non-decreasing for all \(i \leqslant k\). From continuity of \(\phi _i(\cdot)\)’s, it follows that the sequence \((\phi _i^\ast , i \in \mathcal {Q})\) is non-decreasing and converges to \(\phi _\infty \triangleq \sup _{i \in \mathcal {Q}}\phi _i^\ast\).
The price function \(u_i(\cdot)\) is right-continuous and increasing (see Lemma 27 in Section 7.2). Therefore, the sequence \((u_i(\phi _i(\theta _k)), k \in \mathcal {Q})\) is increasing and hence converges to \(u^\ast _i \triangleq \sup _{k\in \mathcal {Q}} u_i(\phi _i(\theta _k))\) for all \(i \in \mathcal {Q}\). Since \(u_i\)’s need not be left-continuous, we can only infer that \(u^\ast _i \leqslant u_i(\phi ^\ast _i)\). Since the valuations are stochastically non-increasing, it follows that for all \(i \in \mathcal {Q}\),
\begin{equation*} \inf _{k \in \mathcal {Q}} \bar{G}_i(u_i(\phi _i(\theta _k))) \geqslant \bar{G}_i(u^\ast _i) \geqslant \bar{G}_i(u_i(\phi ^\ast _i)). \end{equation*}
In particular, using the definition of \(u^k_i\) in Theorem 10, \(\inf _{k \in \mathcal {Q}} \bar{G}_i(u^k_i) \geqslant \bar{G}_i(u^\ast _i).\)
If \(\theta _\infty \leqslant \lambda m(0)\), then \(\theta ^\ast \leqslant \lambda m(0)\), rendering the original pricing problem degenerate. We dispense with this case and focus on the case \(\theta _\infty \gt \lambda m(0)\).
Theorem 11.
If \(\theta _\infty \gt \lambda m(0),\) then \(\theta _\infty = \theta ^\ast\). Moreover, \(\lim _{k \rightarrow \infty } R(\mathbf {u}^k) = \theta ^\ast\).
Proof.
See Section 7.4.□
Recall that the optimal policies for the k-truncated problems converge to \(\mathbf {u}^\ast = (u^\ast _i, i \in \mathcal {Q})\). In general, the asymptotic policy \(\mathbf {u}^\ast\) needs not be optimal for the original problem. The following corollary to Theorem 11 states that if \(\lim _{k \rightarrow \infty } \bar{G}_i(u^k_i) = \bar{G}_i(u^\ast _i)\) for all \(i \in \mathcal {Q},\) then \(\mathbf {u}^\ast\) is an optimal policy for the original problem. We know from Remark 7 that, in general, \(\lim _{k \rightarrow \infty } \bar{G}_i(u^k_i) \geqslant \bar{G}_i(u^\ast _i)\).
Corollary 12.
If \(\lim _{k \rightarrow \infty } \bar{G}_i(u^k_i) = \bar{G}_i(u^\ast _i)\) for all \(i \in \mathcal {Q},\) then \(\mathbf {u}^\ast = (u^\ast _i, i \in \mathcal {Q})\) is an optimal policy for the original problem.
Proof.
See Section 7.4.2.□

3.4 Solution to the k-Truncated Problem

Under Assumption 3, we can iteratively obtain the revenue rates \(\theta _k\) and the optimal policies \(\mathbf {u}^k\) for the k-truncated problems for all \(k \in \mathcal {Q}\), as shown in Proposition 9. It warrants solving (58) for each threshold k (see Section 7.3), which is a considerable computational task. In particular, this equation resembles an intricate fixed point equation which can be solved for \(\theta _k\) via iterative methods. In this section, we develop a fixed point iteration that yields \(\theta _k\) for any given k. Note that, from Theorem 11 and Corollary 12, \(R(\mathbf {u}^k)\) and \(\mathbf {u}^k\) are close to the optimal revenue rate and the optimal policy, respectively, for large k.
Recall that the optimal price sequence for the k-truncated system is characterized by \(k+2\) equations, namely (18) for \(i = 0\), (19) for \(i \in [k]\), and \(\Delta (k) = \Delta (k-1)\). We claim the following property of any solution to these equations.
Lemma 13.
Let \((\theta , \Delta (i), 0 \leqslant i \leqslant k)\) be a solution to (18), (19) for \(i \in [k]\) and \(\Delta (k) = \Delta (k-1)\). Then \(\Delta (i), 0 \leqslant i \leqslant k\) are non-negative and non-decreasing in i.
Proof.
See Section 7.5.1.□
We again express \(\Delta (i)\)’s successively in terms of \(\theta\) and treat these \(k+2\) equations as a fixed point equation in \(\theta\). In particular, we define functions \(\psi _i:\mathbb {R}_+\rightarrow \mathbb {R}\) as follows. Let \(\psi _k(\theta)\) be a solution of the following fixed point equation in \(\Delta :\)
\begin{align} m_k(\Delta) & = \frac{\theta - \mu \Delta }{\lambda } \end{align}
(30)
and
\begin{equation} \psi _i(\theta) = \frac{\theta -\lambda m_{i+1}(\psi _{i+1}(\theta))}{\mu } \end{equation}
(31)
for \(i = 0,1,\ldots ,k-1\). Then, \(\Delta (i) = \psi _i(\theta)\) for all \(i = 0,1,\ldots ,k\) and the above system of equations reduces to \(\lambda m_0(\psi _0(\theta)) = \theta\). We first prove certain properties of (30) before proposing a fixed point iteration and analyzing it in Proposition 15. We can define \(f_k(\Delta)\) to rewrite (30) as
\begin{equation} f_k(\Delta) \triangleq \frac{\theta - \lambda m_k(\Delta)}{\mu } = \Delta . \end{equation}
(32)
Since \(m_k(\Delta)\) is non-negative, non-increasing, continuous, and convex in \(\Delta\), \(f_k(\Delta)\) is non-decreasing, continuous, and concave in \(\Delta\). Moreover, by definition, \(f_k(\Delta) \leqslant \theta /\mu\) for all \(\Delta\). Hence, for a given \(\theta\), the necessary and sufficient condition for existence of a fixed point is \(f_k(\Delta) \geqslant \Delta\) for some \(\Delta\), or alternatively, \(\lambda m_k(\Delta) + \mu \Delta \leqslant \theta\) for some \(\Delta\). So, defining
\begin{equation*} {\underline{\theta }} \triangleq \min _\Delta \lbrace \mu \Delta +\lambda m_k(\Delta)\rbrace , \end{equation*}
we see that (32) has a solution for all \(\theta \geqslant {\underline{\theta }}\) and does not have a solution for \(\theta \lt {\underline{\theta }}\). Observe that \({\underline{\theta }} \leqslant \lambda m_k(0)\). The following proposition summarizes the properties of the fixed points of (32) for \(\theta \geqslant {\underline{\theta }}\).
Proposition 14.
Let \(\rho \triangleq \lambda /\mu\). The fixed point Equation (32) has a solution under the following conditions.
(a)
If \(\rho \bar{G}_k(0) \lt 1\), then (32) has a unique solution, \(D(\theta)\), for all \(\theta\). Also, \(D(\theta)\) is increasing in \(\theta\).
(b)
If \(\rho \bar{G}_k(0)\geqslant 1\), then (32) has two solutions, \(D_1(\theta)\) and \(D_2(\theta)\), where \(D_1(\theta) \leqslant D_2(\theta)\), for \(\theta \geqslant {\underline{\theta }}\). Also, \(D_1(\theta)\) is decreasing in \(\theta\) whereas \(D_2(\theta)\) is increasing in \(\theta\).
Proof.
See Section 7.5.3.□
Remark 8.
Let us observe a few more properties of the roots \(D(\theta),D_1(\theta),\) and \(D_2(\theta)\) of (32); we use these properties in Theorem 15 to assert existence and uniqueness of the fixed point of \(\theta = \lambda m_0(\psi _0(\theta))\). If \(\theta = \lambda m_k(0)\), then \(\Delta = 0\) satisfies (32). Hence, for \(\rho \bar{G}_k(0) \lt 1\), \(D(\theta) = 0\) if \(\theta = \lambda m_k(0)\). Moreover, from Proposition 14\((a)\), \(D(\theta) \gt 0\) if \(\theta \gt \lambda m_k(0)\) and \(D(\theta) \lt 0\) if \(\theta \lt \lambda m_k(0)\). For \(\rho \bar{G}_k(0) \geqslant 1\), (1) if \(\theta \gt \lambda m_k(0)\), then \(D_1(\theta) \lt 0\) and \(D_2(\theta) \gt 0\), (2) if \(\theta = \lambda m_k(0)\), then either \(D_1(\theta) = 0\) or \(D_2(\theta) = 0\), and (3) if \(\theta \in ({\underline{\theta }}, \lambda m_k(0))\), either both \(D_1(\theta)\) and \(D_2(\theta)\) are positive or both are negative.
Remark 9.
The fact that \(f_k(\Delta)\) is non-decreasing, continuous, and concave in \(\Delta\), leads to the following monotonicity properties of solutions of (32). For \(\rho \bar{G}_k(0) \lt 1\), given \(\theta\) and \(\mu\), the unique root of (32), \(D(\theta)\), is decreasing in \(\lambda\). Also, given \(\theta\) and \(\lambda\), \(D(\theta)\) is decreasing in \(\mu\) if \(\theta \gt \lambda m_k(0)\) and is increasing in \(\mu\) if \(\theta \lt \lambda m_k(0)\). For \(\rho \bar{G}_k(0) \geqslant 1\) also the larger root of (32), \(D_2(\theta)\), exhibits these properties.
Notice that if \(\rho \bar{G}_k(0) \gt 1\), then (32) has two solutions, \(D_1(\theta)\) and \(D_2(\theta)\). However, \(D_1(\theta) = D_2(\theta)\) for \(\theta = {\underline{\theta }}\); we refer to the common value as \(\Delta ({\underline{\theta }})\). We set
\begin{equation*} \psi _k(\theta) \triangleq {\left\lbrace \begin{array}{ll} D(\theta) \text{ if } \rho \bar{G}_k(0) \lt 1 \\ D_1(\theta) \text{ if } \rho \bar{G}_k(0) \geqslant 1, \Delta ({\underline{\theta }}) \gt 0 \text{ and } \lambda m_0(\psi _0({\underline{\theta }})) \leqslant {\underline{\theta }}\\ D_2(\theta) \text{ otherwise.} \end{array}\right.} \end{equation*}
For this setting, we establish existence and uniqueness of fixed point of \(\theta = \lambda m_0(\psi _0(\theta))\) in Theorem 15. As a part of the proof of the theorem, we also show that this fixed point equation does not have a solution for other potential choices of \(\psi _k(\theta)\). As in Section 2.2, we let \(\theta _k\) denote the unique solution of \(\lambda m_0(\psi _0(\theta)) = \theta\). Recall that if \(\theta _k \le \lambda m_k(0)\), then the policy \(u = (u_k(0),u_k(0),\dots)\) provides at least as much revenue rate as any ergodic policy. We will distinguish this degenerate case and the case \(\theta _k \gt \lambda m_k(0)\) in what follows. Note that if \(\theta \gt \lambda m_k(0)\), then (32) has unique positive root which is increasing in \(\theta\); it is \(D(\theta)\) if \(\rho \bar{G}_k(0) \lt 1\) and \(D_2(\theta)\) if \(\rho \bar{G}_k(0) \ge 1\). We present a fixed point iteration that solves \(\lambda m_0(\psi _0(\theta)) = \theta\) when \(\psi _k(\theta) = D(\theta)\) or \(\psi _k(\theta) = D_2(\theta)\). In particular, the algorithm yields \(\theta _k\) whenever it exceeds \(\lambda m_k(0)\).

3.4.1 An Iterative Algorithm.

Algorithm 1, displayed below, is an iterative algorithm that generates two sequences \({\underline{\theta }}_n, n \geqslant 0\) and \(\bar{\theta }_n, n \geqslant 0\) starting with \({\underline{\theta }}_0 = \lambda m(0)\) and \(\bar{\theta }_0 = \lambda m_0(\psi _0(\lambda m(0)))\), respectively.
Theorem 15.
Under Assumption 3, the following results hold.
(a)
The fixed point equation \(\theta = \lambda m_0(\psi _0(\theta))\) has unique solution \(\theta _k\). Moreover, \(\theta _k \ge \lambda m_k(0)\) when \(\psi _k(\theta) = D(\theta)\) and \(\theta _k \in [{\underline{\theta }}, \lambda m_k(0)]\) when \(\psi _k(\theta) = D_1(\theta)\).
(b)
In Algorithm 1, \({\underline{\theta }}_n \uparrow \theta _k\) and \({\underline{\theta }}_n \downarrow \theta _k\) when \(\psi _k(\theta) = D(\theta)\) or \(\psi _k(\theta) = D_2(\theta)\).
Proof.
See Section 7.5.4.□
Remark 10.
When \(\theta _k \gt \lambda m_k(0)\) the policy \(u^k\) defined as
\begin{equation} u^k_i \triangleq {\left\lbrace \begin{array}{ll} u_i(\psi _i(\theta _k)) \ \text{for } i \in \left\lbrace 0,1,\dots ,k-1 \right\rbrace ,\\ u_k(\psi _k(\theta _k)) \ \text{for } i \geqslant k. \end{array}\right.} \end{equation}
(33)
is an optimal ergodic policy. Recall from Proposition 10 that \(\theta _k \gt \lambda m_k(0)\) implies \(\lambda \bar{G}_k(u_k(\psi _k(\theta _k))) \lt \mu\), as required for ergodicity of \(u^k\).

4 A Myopic Policy

We now proceed to construct a simple myopic policy that is easy to implement. This will serve as a performance benchmark and help us characterize the loss of revenue due to sub-optimality. Towards this, we first have the following result about the stability of the Markov chain \((Q(t),t\geqslant 0)\) introduced in Section 2. All the results in this section are subject to Assumption 1. Let the load factor be defined as
\begin{align} \rho \triangleq \frac{\lambda }{\mu }. \end{align}
(34)
Proposition 16.
For any load factor \(\rho\) and the pricing vector \(\mathbf {u}\), define
\begin{align} \sigma =1+\sum _{i=1}^{\infty }\rho ^{i}\prod _{j=0}^{i-1}\bar{G}_j(u_j). \end{align}
(35)
If \(\sigma \lt \infty\), the Markov chain \((Q(t),t\geqslant 0)\) is positive recurrent with stationary distribution \(\pi (\mathbf {u})\), where
\begin{equation} \pi _{i}(\mathbf {u}) \triangleq {\left\lbrace \begin{array}{ll} \sigma ^{-1},& i=0, \\ \sigma ^{-1}\rho ^{i}\prod _{j=0}^{i-1}\bar{G}_j(u_j), & i \geqslant 1. \end{array}\right.} \end{equation}
(36)
This is a standard result from the theory of Markov chains [12, Section 6.11]. From this result, we deduce the following result regarding the stability of our model.
Corollary 17.
For all load factors \(\rho\) and any price vector \(\mathbf {u}\) that satisfies
\begin{equation*} \lim _{i\rightarrow \infty }\rho \overline{G}_{i}(u_{i})\lt 1, \end{equation*}
the Markov chain \(Q(t)\) is positive recurrent.
Proof.
Applying the ratio test [24, Section 3.34] for the convergence of the series \(\sigma\) defined above, we see that if \(\lim _{i\rightarrow \infty }\rho \overline{G}_{i}(u_{i})\lt 1,\) then \(\sigma \lt \infty\).□
Example 18.
If the valuations are \(\overline{G}_{i}(u)=\mathbb {P}[\max (X-iY,0)\gt u]\), for proper, non-negative random variables X and Y, we see that any non-decreasing price vector will result in a positive recurrent Markov chain. In particular, this is true for the constant price vector \(\mathbf {u}=(u,u,u,\dots)\).
For any price vector \(\mathbf {u}\) that results in the Markov chain \((Q(t),t\geqslant 0)\) positive recurrent, we can rewrite the revenue rate in the following form.
Proposition 19.
Let \(\mathbf {u}\) be a pricing vector under which the Markov chain \((Q(t),t\geqslant 0)\) is positive recurrent with stationary distribution \(\pi (\mathbf {u})\). Then, the revenue rate is given by
\begin{equation*} R(\mathbf {u}) =\sum _{i=0}^{\infty } \pi _{i}(\mathbf {u})\lambda _{i} u_{i}. \end{equation*}
Proof.
Recall that \(\hat{R}(t)\) denotes the cumulative reward till time t. Denote the number of arrivals till time t by \(N(t)\), and the jump chain associated with the CTMC \((Q(t),t\geqslant 0)\) by \((Q_n,n\in \mathbb {Z}_+)\). We can write
\begin{align} \hat{R}(t)=\sum _{n=1}^{N(t)}r(Q_{n},Q_{n+1}), \end{align}
(37)
where \(r(i,j)\) is non-zero only if \(j=i+1\), and is given by \(r(i,i+1)=u_{i}\). Recall from Section 2 that the rate of this transition is \(\lambda _{i}\). From [26, Equations 4.32 and 4.33], it follows that, almost surely,
\begin{align} \lim _{t\rightarrow \infty }\frac{\hat{R}(t)}{t}=\sum _{i=0}^{\infty } \pi _{i}(\mathbf {u})\lambda _{i} u_{i}. \end{align}
(38)
From (37), we can see \(\hat{R}(t)/t \leqslant (\sum _{n=1}^{N_t}V_{Q_n})/t\). Applying [26, Theorem 45] to \((\sum _{n=1}^{N_t}V_{Q_n})/t\), we see that it converges to \(\sum _{i}\pi _{i}(\mathbf {u})\mathbb {E}V_{i}\) almost surely. Therefore, using the dominated convergence theorem [2, Theorem 2.3.11], we have
\begin{equation*} \lim _{t\rightarrow \infty }\frac{\mathbb {E}\hat{R}(t)}{t}=\mathbb {E}\lim _{t\rightarrow \infty }\frac{\hat{R}(t)}{t} =\sum _{i=0}^{\infty } \pi _{i}(\mathbf {u})\lambda _{i} u_{i}. \end{equation*}
Following Proposition 19, we can rewrite the optimal pricing problem defined in (8) as
\begin{equation} \mathbf {u}^\ast =\arg _{\mathbf {u}}\max \sum _{i=0}^{\infty } \pi _{i}(\mathbf {u})\lambda _{i} u_{i}. \end{equation}
(39)
Obtaining an analytical solution to the optimal pricing problem in (39) is difficult in general because the stationary distribution has a complex structure, as seen in (36). Define the myopic pricing vector \(\tilde{\mathbf {u}}=(\tilde{u}_0,\tilde{u}_1,\tilde{u}_2,\dots)\) such that per state price is the maximizer of the expected revenue in each state. That is,
\begin{equation} \tilde{u}_i \triangleq \arg _u\max u\overline{G}_i(u),\quad i \in \mathcal {Q}. \end{equation}
(40)
While this pricing vector is sub-optimal, we obtain the following bound that shows that the revenue \(R(\tilde{\mathbf {u}})\) under myopic pricing is greater than a fixed fraction of the optimal revenue \(R(\mathbf {p}^\ast)\), for all values of \(\rho\) for which a stationary distribution exists. Following the bound, we show by numerical examples that in some cases, the fraction of revenue of the optimal that can be obtained by the myopic policy is substantial. We observe that the lower bound on the revenue fraction depends only on the myopic policy, and hence, is easily computable.
Theorem 20.
For the pricing vector \(\tilde{\mathbf {u}}\), we define a sequence \(\alpha\) such that \(\alpha _{i}(\tilde{\mathbf {u}}) \triangleq \frac{\tilde{u}_{i}\overline{G}_{i}(\tilde{u}_{i})}{\tilde{u}_0\overline{G}_0(\tilde{u}_0)}\) for all \(i\in \mathcal {Q}\). Let \(\pi (\tilde{\mathbf {u}})\) denote the stationary distribution under the pricing vector \(\tilde{\mathbf {u}}\). Then, \(\alpha _{i}(\tilde{\mathbf {u}}) \leqslant 1\) for all \(i \in \mathcal {Q}\), and
\begin{equation} R(\tilde{\mathbf {u}})\geqslant \Big (\sum _{i\in \mathcal {Q}}\pi _{i}(\tilde{\mathbf {u}})\alpha _{i}(\tilde{\mathbf {u}})\Big)R(\mathbf {u}^\ast). \end{equation}
(41)
Proof.
We fix an \(i \in \mathcal {Q}\) and observe that \(\overline{G}_{i+1}(u)\leqslant \overline{G}_{i}(u)\) from the assumption on the service valuation distribution. Therefore,
\begin{equation*} \tilde{u}_{i+1}\overline{G}_{i+1}(\tilde{u}_{i+1}) \leqslant \tilde{u}_{i+1}\overline{G}_{i}(\tilde{u}_{i+1}) \leqslant \tilde{u}_{i}\overline{G}_{i}(\tilde{u}_{i}), \end{equation*}
where the last inequality follows from the definition of \(\tilde{u}_{i}\). Hence, we have shown that for all \(i\in \mathcal {Q}\),
\begin{equation} \tilde{u}_{i}\overline{G}_{i}(\tilde{u}_{i})\geqslant \tilde{u}_{i+1}\overline{G}_{i+1}(\tilde{u}_{i+1}). \end{equation}
(42)
It follows that for any price u and any \(i \in \mathcal {Q}\), \(u\overline{G}_{i}(u) \leqslant \tilde{u}_{i}\overline{G}_{i}(\tilde{u}_{i}) \leqslant \tilde{u}_0\overline{G}_0(\tilde{u}_0).\) This implies that \(R(\mathbf {u}^\ast)\leqslant \lambda \tilde{u}_0\overline{G}_0(\tilde{u}_0)\), and hence we have
\begin{equation*} \frac{R(\tilde{\mathbf {u}})}{R(\mathbf {u}^\ast)} \geqslant \sum _{i \in \mathcal {Q}} \pi _{i}(\tilde{\mathbf {u}}) \frac{\tilde{u}_{i}\overline{G}_{i}(\tilde{u}_{i})}{\tilde{u}_0\overline{G}_0(\tilde{u}_0)}. \end{equation*}
The myopic pricing scheme is guaranteed to yield at least a positive fraction of the optimal revenue rate. Note that this lower bound was obtained by upper bounding the optimal revenue rate \(R(\mathbf {u}^\ast)\), by using the inequality \(u_{i}^\ast \overline{G}_{i}(u_{i}^\ast) \leqslant \tilde{u}_0\overline{G}_0(\tilde{u}_0).\) Hence, the bound being close to zero may not indicate that the myopic policy is bad, but that the gap between \(u_{i}^\ast \overline{G}_{i}(u_{i}^\ast)\) and \(\tilde{u}_0\overline{G}_0(\tilde{u}_0)\) is large for states \({i}\) which have a high value of \(\pi _{i}(\mathbf {u}^\ast).\) Characterizing this gap would require an analytical description of the behavior of \(\mathbf {u}^\ast\), which is non-trivial in general, as seen from the preceding sections.
We obtain the value of this fraction for an example value function, below. In this case the myopic pricing performs well, yielding more than \(78\%\) of the optimal revenue for all \(\rho \lt 1\).
Example 21.
Let \(a\in \mathbb {R}_+^\mathbb {N}\) be positive sequence such that \(0 \lt \alpha \leqslant a_{i} \leqslant \alpha ({i}+1)\), and let the service valuations be exponential with \(\overline{G}_{i}(u) \triangleq e^{-a_{i}u}\) for all \(u \geqslant 0\) and \({i} \in \mathcal {Q}\). For this sequence of service valuations, the myopic price vector is \(\tilde{\mathbf {u}}= (1/a_0, 1/a_1, \dots)\). It follows that \(\overline{G}(\tilde{u}_{i})= 1/e\), and the stationary distribution exists for all \(\rho \lt e\), and is given by
\begin{equation*} \pi _{i}(\tilde{\mathbf {u}}) = \Big (1-\frac{\rho }{e}\Big)\Big (\frac{\rho }{e}\Big)^{i}\text{ for all }{i} \in \mathcal {Q}. \end{equation*}
Since \(1 \lt a_{i}/a_0 \leqslant {i}+1\) for all \({i} \in \mathcal {Q}\), we can bound the ratio between revenue rate under myopic and optimal pricing as
\begin{equation*} \frac{R(\tilde{\mathbf {u}})}{R(\mathbf {u}^\ast)} \geqslant \Big (1-\frac{\rho }{e}\Big)\sum _{{i} \in \mathcal {Q}}\Big (\frac{\rho }{e}\Big)^{i}\frac{a_0}{a_{i}} \geqslant \Big (1-\frac{\rho }{e}\Big)\sum _{{i} \in \mathcal {Q}}\Big (\frac{\rho }{e}\Big)^{i}\frac{1}{{i}+1}. \end{equation*}
Using the fact that \(\sum _{n\geqslant 1}\frac{x^n}{n} =-\ln (1-x)\) for \(x\lt 1\), we can write
\begin{equation*} \frac{R(\tilde{\mathbf {u}})}{R(\mathbf {u}^\ast)} \geqslant \Big (1-\frac{e}{\rho }\Big)\ln \Big (1-\frac{\rho }{e}\Big). \end{equation*}
We plot this lower bound in Figure 3. For small values of load \(\rho ,\) this is quite close to 1. As we increase \(\rho\) to 1, this decreases to 0.78. However, for loads \(\rho \gt 1\), the bound decreases further, and is zero when \(\rho =e\). Thus, the myopic policy performs well when the value exponent decays slower than a linear function, and at lower values of load \(\rho\). Also note that, in this example, the myopic policy can only stabilize the system for loads \(\rho \lt e\). It turns out that at higher load values, we can do better than the myopic policy and stabilize the system using simple policies, which are possibly non-optimal, as demonstrated in the next example.
Fig. 3.
Fig. 3. Lower bound for the ratio of revenues of myopic and optimal policies in Example 21.
Example 22.
Consider the service valuations \(\overline{G}_i(u) \triangleq e^{-(i+1)u}\) for all \(i \in \mathcal {Q}\). This sequence satisfies the conditions of Example 21 with \(\alpha =1\). In this case, the myopic pricing is given by \(\tilde{u}_i=\frac{1}{i+1}\), and the revenue under myopic pricing is given by (for all \(\rho \lt e\))
\begin{align} R(\tilde{\mathbf {u}}) &=\lambda \sum _{n\in \mathcal {Q}}\pi _i(\tilde{\mathbf {u}})\overline{G}_i(\tilde{u}_i)\tilde{u}_i \end{align}
(43)
\begin{align} &=\lambda \Big (1-\frac{\rho }{e}\Big)\sum _{i\in \mathcal {Q}}\Big (\frac{\rho }{e}\Big)^ie^{-1}\frac{1}{i+1} \end{align}
(44)
\begin{align} &=-\frac{\lambda }{\rho }\left(1-\frac{\rho }{e}\right)\ln \left(1-\frac{\rho }{e}\right). \end{align}
(45)
Now consider the pricing given by \(u_i=\frac{K}{i+1}\), for some \(K\geqslant 1\). It is easy to check that this policy stabilizes the system for loads \(\rho \lt e^K\). Denoting the revenue of such a system by \(R^K\), we can obtain, as before,
\begin{align} R^K &=-\frac{K\lambda }{\rho }\left(1-\frac{\rho }{e^K}\right)\ln \left(1-\frac{\rho }{e^K}\right). \end{align}
(46)
Note that \(R^1=R(\tilde{\mathbf {u}})\). We plot the value of \(R^K\) for three values of \(K=1,2,3\) in Figure 4, with \(\lambda =1\). Note that the revenues are plotted only for the stability region for each policy, which, respectively, correspond to \(\rho \lt e\), \(\rho \lt e^2\), and \(\rho \lt e^3\). It is easy to see that \(R^2\) dominates \(R^1\) (the myopic policy) after a certain \(\rho\), and similarly, \(R^3\) dominates \(R^2\) after a point. Also note that the revenue of the myopic policy, as well as of the other policies, goes to zero as the load increases. This example points to the possibility that the optimal policy can do better than the myopic policy.
Fig. 4.
Fig. 4. Revenue of myopic policy versus other policies.
Next, we consider the special case when all the value distributions are identical, i.e., \(G_{i} = G\) for all \({i} \in \mathcal {Q}\). In this case, the customer is not sensitive to waiting time. The optimal price in this case turns out to be the myopic price.
Corollary 23.
When service valuation distribution does not depend on the system state, i.e., \(G_{i} = G\) for all \({i} \in \mathcal {Q}\), then the pricing \(\mathbf {u}^\ast = u^\ast \mathbf {1}= (u^\ast , u^\ast , \dots)\) is optimal, where
\begin{equation*} u^\ast \triangleq \arg _u\max u\overline{G}(u) \end{equation*}
for all \(\rho \lt \frac{1}{\overline{G}(u^*)}\).
Proof.
For any load factor \(\rho \lt \frac{1}{\overline{G}(u*)}\), we see that (35) converges and hence, \(\sigma \lt \infty\). Hence a stationary distribution exists under the pricing \(\mathbf {u}^*\). For any other price vector \(\mathbf {u}\) with stationary distribution \(\pi (\mathbf {u})\), we have \(u_{i}\overline{G}(u_{i}) \leqslant u^\ast \overline{G}(u^\ast)\). For this price vector \(\mathbf {u}\), the revenue rate is given by
\begin{equation*} R(\mathbf {u}) = \sum _{{i}\in \mathcal {Q}}\pi _{i}(\mathbf {u})\lambda u_{i}\overline{G}(u_{i}) \leqslant \lambda u^\ast \overline{G}(u^\ast) = R(u^\ast \mathbf {1}). \end{equation*}
The second inequality follows from the definition of \(u^\ast\) and the fact that \(\sum _{{i}\in \mathcal {Q}}\pi _{i}(\mathbf {u}) = 1\).□
Intuitively, if the valuations are insensitive to the queue length, then the operator could just maximize the average reward for each arrival, resulting in a myopic optimal policy. This result suggests that a myopic policy is good in situations where the valuations do not change much with the queue length. We now proceed to solve these equations for the special case of deterministic service valuations.

5 Deterministic Service Valuation

We now consider the special case where the customers’ valuation of the service is a deterministic function of the number of customers in the queue. We assume that the customer valuation for the service is \(v_i\) when there are i jobs in the queue, and the valuation sequence \((v_i, i \in \mathcal {Q})\) is monotonically decreasing. Equivalently, sequence of valuation distribution functions \((G_i(\cdot), i \in \mathcal {Q})\) are Dirac measures with respective atoms at \((v_i, i \in \mathcal {Q})\). The Bellman’s equations are identical to Equations (18) and (19), where for all \(i \in \mathcal {Q}\)
\begin{align*} m_i(B) &= \max _{u \geqslant 0}\lbrace (u-B)\mathbb {1}_{\left\lbrace u \leqslant v_i \right\rbrace }\rbrace = (v_i -B)^+, \\ \text{and} \ u_i(B) & = {\left\lbrace \begin{array}{ll}v_i & \text{ if } B \leqslant v_i,\\ (v_i,\infty) & \text{ otherwise}.\end{array}\right.} \end{align*}
In particular, \(m_i(0) = v_i\) for all \(i \in \mathcal {Q}\). Equations (18) and (19) now simplify to
\begin{align} (v_0 - \Delta (0))^+ &= \frac{\theta }{\lambda }, \end{align}
(47a)
\begin{align} (v_i - \Delta (i))^+ &= \frac{\theta -\mu \Delta (i-1)}{\lambda }, \text{ for all }i \geqslant 1 . \end{align}
(47b)
Unlike Section 3 where we proposed an algorithm to obtain the optimal policy for the k-truncated version of the problem, we now derive the optimal policy for the deterministic valuation problem without the need for any truncation. Consider the limiting equation \((v-\Delta)^+ = \frac{\theta - \mu \Delta }{\lambda }\) involving the asymptotic valuation \(v=\lim _{i\rightarrow \infty } v_i\). Defining \(g(\Delta) \triangleq \frac{\theta - \lambda (v-\Delta)^+}{\mu }\), it can be rewritten as
\begin{equation} g(\Delta) = \Delta . \end{equation}
(48)
Let us observe a few properties of the fixed point Equation (48). Since \((v-\Delta)^+\) is non-negative, non-increasing, continuous, and convex in \(\Delta\), it follows that \(g(\Delta)\) is non-decreasing, continuous, and concave in \(\Delta\). From the definition, it follows that \(g(\Delta) \leqslant \theta /\mu\) for all \(\Delta\). In particular, we have \(g(\Delta)-\Delta \leqslant 0\) for all \(\Delta \geqslant \theta /\mu\). If there exists a \(\Delta\) such that \(g(\Delta)-\Delta \geqslant 0\), then from the continuity of \(g(\Delta)-\Delta\) it follows that there must exists a \(\Delta\) such that \(g(\Delta)-\Delta = 0\). Hence, for a given \(\theta\), the necessary and sufficient condition for existence of a fixed point is \(g(\Delta) \geqslant \Delta\) for some \(\Delta\), or equivalently, \(\lambda (v-\Delta)^+ + \mu \Delta \leqslant \theta\) for some \(\Delta\). So, defining \({\underline{\theta }} \triangleq \min _\Delta \lbrace \mu \Delta +\lambda (v-\Delta)^+\rbrace ,\) (48) has a solution for all \(\theta \geqslant {\underline{\theta }}\) but does not have a solution for \(\theta \lt {\underline{\theta }}\). We have the following result about the solutions of this limiting equation.
Proposition 24.
The Equation (48) has a solution for all \(\theta \geqslant {\underline{\theta }}\), but does not have a solution for \(\theta \lt {\underline{\theta }}\). The value of \({\underline{\theta }}\) is given by
\begin{align*} {\underline{\theta }}= {\left\lbrace \begin{array}{ll} -\infty & \text{ if } \ \lambda \lt \mu \\ \mu v & \text{if} \ \lambda \geqslant \mu \end{array}\right.} \end{align*}
Furthermore, the solution to (48) is given by
\begin{align*} D(\theta)= {\left\lbrace \begin{array}{ll} \frac{\theta }{\mu }\left(\frac{1- \lambda v/\min (\theta ,\mu v)}{1 - \lambda /\mu }\right) & \ \text{if}\lambda \lt \mu \\ \lbrace \Delta : \Delta \leqslant \frac{\theta }{\mu }\rbrace &\ \text{ if } \lambda =\mu \ \text{and }\theta = \mu v\\ \frac{\theta }{\mu } & \ \text{if}\lambda =\mu \ \text{and} \theta \gt \mu v\\ \left\lbrace \frac{\theta }{\mu }\left(\frac{1- \lambda v/\theta }{1 - \lambda /\mu }\right),\frac{\theta }{\mu }\right\rbrace & \ \text{if}\lambda \gt \mu \ \text{and} \theta \geqslant \mu v. \end{array}\right.} \end{align*}
Proof.
See Section 7.6.1.□

5.1 Optimal Revenue Rate

Let \((\theta , \Delta (i), i \in \mathcal {Q})\) be a solution to (47a)–(47b) and (48). Let \(K \triangleq \max \left\lbrace i \in \mathcal {Q}: \Delta (i) \leqslant v_i \right\rbrace\). It is possible that \(\Delta (i) \leqslant v_i\) for all i in which case K is infinity. We will first consider the finite K case. In the following we write the solution as \((\theta ^K, \Delta ^K(i), i \in \mathcal {Q})\) to explicitly express the dependence on K. Since \(\Delta ^K(K+1) \gt v_{K+1}\), from (47b), \(\Delta ^K(K) = \frac{\theta ^K}{\mu }.\) Furthermore, \(\Delta ^K(K-1) = \frac{\theta ^K - \lambda (v_K - \Delta ^K(K))^+}{\mu } \leqslant \frac{\theta ^K}{\mu } = \Delta ^K(K),\) and \((v_{K-1} - \Delta ^K(K-1))^+ \geqslant (v_{K-1} - \Delta ^K(K))^+ \geqslant (v_K - \Delta ^K(K))^+.\) Continuing iteratively for \(i = K-2,\dots ,0\), we can infer that
\begin{equation*} \Delta ^K(0) \leqslant \Delta ^K(1) \leqslant \cdots \leqslant \Delta ^K(K),\quad \text{and } (v_{0} - \Delta ^K(0))^+ \geqslant (v_{1} - \Delta ^K(1))^+ \geqslant (v_{K} - \Delta ^K(K))^+. \end{equation*}
Notice that \(\Delta ^K(i) \leqslant v_i\) for all \(i \leqslant K\). Furthermore, from (47a), \(\Delta ^K(0) = v_0 - \theta ^K/\lambda\), and from (47b),
\begin{equation*} \Delta ^K(i) = v_i - \frac{\theta ^K}{\lambda } + \frac{\mu \Delta ^K(i-1)}{\lambda }, \ \forall i = 1,\dots ,K. \end{equation*}
We thus obtain
\begin{equation} \Delta ^K(K) = \sum _{i=0}^K v_i\left(\frac{\mu }{\lambda }\right)^{K-i} - \frac{\theta ^K}{\lambda }\sum _{i=0}^K \left(\frac{\mu }{\lambda }\right)^i. \end{equation}
(49)
Combining it with \(\Delta ^K(K) = \theta ^K/\mu\), we get
\begin{align} \theta ^K &= \mu \left(\sum _{i=0}^K v_i\left(\frac{\mu }{\lambda }\right)^{K-i} - \frac{\theta ^K}{\lambda }\sum _{i=0}^K \left(\frac{\mu }{\lambda }\right)^i\right), \nonumber \nonumber\\ \text{or } \theta ^K &= \frac{\sum _{i=0}^K v_i\left(\frac{\mu }{\lambda }\right)^{K-i}}{\frac{1}{\mu }+\frac{1}{\lambda }\sum _{i=0}^K \left(\frac{\mu }{\lambda }\right)^i} = \frac{\mu \sum _{i=0}^K v_i\left(\frac{\mu }{\lambda }\right)^{K-i}}{\sum _{i=0}^{K+1} \left(\frac{\mu }{\lambda }\right)^i} \end{align}
(50)
\begin{align} \text{and } \Delta ^K(K) &=\frac{\sum _{i=0}^K v_i\left(\frac{\mu }{\lambda }\right)^{K-i}}{\sum _{i=0}^{K+1} \left(\frac{\mu }{\lambda }\right)^i}. \end{align}
(51)
In fact, using (47b) for \(i \gt K+1\),
\begin{equation} \Delta ^K(i) = \frac{\theta ^K}{\mu } = \Delta ^K(K), \ \forall i \gt K. \end{equation}
(52)
In view of (51), \(v_K \geqslant \Delta ^K(K)\) is equivalent to
\begin{equation*} v_K \geqslant \frac{\sum _{i=0}^{K-1} v_i\left(\frac{\mu }{\lambda }\right)^{K-1-i}}{\sum _{i=0}^K \left(\frac{\mu }{\lambda }\right)^i} = \Delta ^{K-1}(K-1). \end{equation*}
Let there also be another \(K^{\prime } \gt K\) such that \(\Delta ^{K^{\prime }}(K^{\prime }) \leqslant v_{K^{\prime }}\), or equivalently, \(v_{K^{\prime }} \geqslant \Delta ^{K^{\prime }-1}(K^{\prime }-1)\). This implies that \(v_{K^{\prime }-1} \geqslant \Delta ^{K^{\prime }-1}(K^{\prime }-1)\). Iteratively, we can show that \(v_{K+1} \geqslant \Delta ^K(K)\), also implying that \(v_{K+1} \geqslant \Delta ^K(K+1)\) (see (52)). This contradicts the assumption that \(K = \max \lbrace i:\Delta ^K(i) \leqslant v_i\rbrace\). Hence we must have
\begin{equation*} K = \max \lbrace i:\Delta ^i(i) \leqslant v_i\rbrace = \max \lbrace i:\theta ^i \leqslant \mu v_i\rbrace \end{equation*}
for \((\theta ^K, \Delta ^K(i), i \geqslant 0)\) to be a solution.
Next we focus on the case where \(\Delta (i) \leqslant v_i\) for all i and derive the revenue rate in this case. We first consider a truncated system in which \(\Delta (i) = \Delta (K)\) for all \(i \gt K\). We again write the solution as \((\theta ^K, \Delta ^K(i), i \geqslant 0)\) to explicitly express the dependence on K. Notice that \(\Delta (K)\) is still given by (49). But, using (47b) for \(i = K+1\), we also have \(\Delta ^K(K) = (\theta ^K - \lambda v_K)/(\mu - \lambda)\). Combining these two equations
\begin{equation} \theta ^K = \frac{\frac{\lambda v}{\mu -\lambda }+\sum _{i=0}^K v_i\left(\frac{\mu }{\lambda }\right)^{K-i}}{\frac{1}{\mu -\lambda }+\frac{1}{\lambda }\sum _{i=0}^K \left(\frac{\mu }{\lambda }\right)^i}. \end{equation}
(53)
To distinguish the two expressions of the revenue rates in (50) and (53), we refer to them as \(\theta ^K_1\) and \(\theta ^K_2\), respectively, in what follows.
We now derive conditions under which each of the above two solutions arise. For clarity of exposition, we make the following assumption
Assumption 4.
For \(i = 1,2, \lim _{k \rightarrow \infty } \theta ^k_i\) exist; \(\theta ^\infty _i \triangleq \lim _{k \rightarrow \infty } \theta ^k_i\).1
Following theorem yields the optimal revenue rates and the optimal prices.
Theorem 25.
(1)
\(\lambda \lt \mu :\)
\begin{equation*} \theta ^\ast = {\left\lbrace \begin{array}{ll} \theta ^\infty _2 & \text{if} \ \theta ^\infty _1 \in [\lambda v, \mu v] \\ \theta ^K_1 & \text{if} \ \theta ^\infty _1 \gt \mu v, \end{array}\right.} \end{equation*}
where \(K = \max \lbrace i: \Delta ^i(i) \leqslant v_i\rbrace\). If \(\theta ^\infty _1 \in [\lambda v, \mu v]\) then \(u_i = v_i\) for all i, and if \(\theta ^\infty _1 \gt \mu v\) then \(u_i = v_i\) for all \(i \leqslant K\) and \(u_i \in (v_i,\infty)\) for all \(i \gt K\).
(2)
\(\lambda \geqslant \mu :\)
\begin{equation*} \theta ^\ast = {\left\lbrace \begin{array}{ll} \lambda v & \text{if} \ \theta ^K_1 \in [\mu v, \lambda v] \\ \theta ^K_1 & \text{if} \ \theta ^K_1 \gt \lambda v. \end{array}\right.} \end{equation*}
where K is as defined above. If \(\theta ^K_1 \in [\mu v, \lambda v]\) then \(u_i = v_i\) for all i, and if \(\theta ^\infty _1 \gt \lambda v\) then \(u_i = v_i\) for all \(i \leqslant K\) and \(u_i \in (v_i,\infty)\) for all \(i \gt K\).
Proof.
See Section 7.6.2.□
Remark 11.
If \(\lambda \lt \mu\) and \(\theta ^\infty _1 \gt \mu v\) or \(\lambda \geqslant \mu\) and \(\theta ^K_1 \gt \lambda v\), then new jobs are not accepted provided there are K jobs in the system. If \(\lambda \lt \mu\) and \(\theta ^\infty _1 \leqslant \mu v\) or \(\lambda \geqslant \mu\) and \(\theta ^K_1 \leqslant \lambda v,\) then all the jobs are accepted. Notice that \(\lambda \lt \mu v/v_0\) or \(\mu \leqslant \lambda v/v_K\) imply this case.

6 Numerical Simulations

In all the simulations for random valuations mentioned below, the optimal prices refer to those for k-truncated systems for \(k = 1000\), and the optimal revenue rates refer to the revenue rates corresponding to these prices.
We first consider a system with fixed service rate \(\mu =5\) and valuation distribution \(G_i(u)=1-\exp (-\log (e+i)u)\). We plot the optimal prices versus queue length for three different values of arrival rate \(\lambda\), in Figure 5. The optimal prices are higher for larger arrival rates but decrease monotonically as the number of customers in the queue increases. At higher arrival rates, the service provider can charge higher prices without the risk of losing a customer, because another customer will be available soon. This leads to higher prices, and consequently, higher revenue.
Fig. 5.
Fig. 5. Optimal price vectors for different arrival rates, for value distribution \(G_i(u)=1-\exp (-\log (e+i)u)\).
Next, we plot the optimal price vector for a system with arrival rate \(\lambda =5\), service rate \(\mu =1,\) and the state dependent value distribution \(G_i(u)=1-e^{-(2-\frac{1}{i+1})u}\). We vary the number of customers in the queue from 0 through 30, in Figure 6. We see that the prices sharply decrease initially as in the previous example but then increase as the queue length further increases. As the queue length becomes larger, customers valuations become very small, and admitting and serving new customers would lower the service provider’s revenue rate. Thus the service provider increases the prices to discourage new customers so that the queue length can be kept small.
Fig. 6.
Fig. 6. Optimal price versus number of customers in queue, for value distribution \(G_i(u)=1-e^{-(2-\frac{1}{i+1})u}\) when incoming arrival sees i customers in the queue. The region in the box is zoomed in on the right, and we can see that prices are increasing.
In Figure 7(a), we plot the revenue versus arrival rate, for the optimal policy as well as the myopic policy in (40). The valuation function is \(G_i(u)=1-\exp (-\log (e+i)u)\), and the service rate \(\mu =5\). Both revenues increase monotonically with arrival rate. The myopic policy performs close to optimal at low values of arrival rate. As arrival rate increases above the service rate (i.e., \(\lambda \gt 5\) or \(\rho \gt 1\)), the two revenues begin to diverge. This is similar to what we had observed in Example 21. From Proposition 20, we can see that at low values of \(\rho\), the stationary distribution will decay quite fast with n. Hence the stationary distribution can be approximated by its first few elements. For these values, the ratio \(\alpha _n(\tilde{\mathbf {u}})\) in (41) will close to one as well. Hence, the myopic policy will give value close to the optimal. The gap between the two revenues increases as arrival rate increases. Hence, at higher arrival rates, the revenue rate can be improved by using the optimal policy.
Fig. 7.
Fig. 7. Revenue variation with arrival rate.
Finally, we also plot the optimal revenue rate for a deterministic valuation function. We consider the deterministic valuation \(v(i)=\frac{1}{\log (e+i)}\), which is the deterministic counterpart of the random value function with distribution \(G_i(u)=1-\exp (-\log (e+i)u)\). We compare the revenue rates for the random and deterministic rates in Figure 7(b). The service rate \(\mu =5\). While both revenues are increasing in arrival rate, as expected, the revenue extracted is higher in the deterministic case. This is because the service provider knows exactly how much each customer is willing to pay, and extracts it completely. There is no loss due to randomness.

7 Proofs

7.1 Proofs of Section 2

7.1.1 Proof of Lemma 3.

We first show the existence of a distribution function \(G(\cdot)\). Observe that for any given u, \(G_i(u)\) are non-decreasing and bounded, and hence \(\lim _{i \rightarrow \infty }G_i(u)\) exists. Moreover, by Helly’s selection theorem, there exists a subsequence of \(G_i(\cdot),i\geqslant 0\), \(G_{i_n}(\cdot), n \geqslant 0\), and a right-continuous, non-decreasing function \(G(\cdot)\) such that \(G(u) = \lim _{n \rightarrow \infty }G_{i_n}(u)\) at all the continuity points u of G. Clearly, \(\lim _{i \rightarrow \infty }G_i(u) = \lim _{n \rightarrow \infty }G_{i_n}(u)\) at all these points. This function has the following properties.
(1)
\(\lim _{u \rightarrow -\infty }G(u) = 0\): For any \(u \lt 0\), which is a continuity point of \(G(\cdot)\), \(G_i(u) = 0\). So \(G(u) = 0\) for any \(u \lt 0\).
(2)
\(\lim _{u \rightarrow \infty }G(u) = 1\): Let us fix an \(\epsilon \lt 1\). There exists a \(u_\epsilon\), a continuity point of \(G(\cdot)\), such that \(G_0(u) \gt 1-\epsilon\) for all \(u \geqslant u_\epsilon\). Since \(G_i(u_\epsilon) \uparrow G(u_\epsilon)\) and \(G(\cdot)\) is also non-decreasing, \(G(u) \gt 1-\epsilon\) for all \(u \geqslant u_\epsilon\). Choosing \(\epsilon\) to be arbitrarily small leads us to the desired result.

7.2 Proofs of Section 3.1

We provide the proof of Proposition 6 in this section. Recall that price \(u_i(B)\) attains the maximum value \(m_i(B)\) in (17) for all \(i \in \mathcal {Q}\). Thus, we first understand the properties of \((m_i(B), i \in \mathcal {Q})\), followed by properties of price sequence \((u_i(B), i \in \mathcal {Q})\).
Lemma 26.
The sequence of functions \((m_i(B), i \in \mathcal {Q})\) defined in (16) have the following properties.
(a)
\(m_i(B)\) is non-negative and decreasing in B.
(b)
\(m_i(B)\) is a Lipschitz-1 continuous, convex function of B with derivative \(-\bar{G}_i(u_i(B))\) wherever it exists.
(c)
For each B, the sequence \((m_i(B), i \in \mathcal {Q})\) is non-increasing and converging.
Proof.
(a)
It directly follows from (16) that \(m_i(B) \ge 0\) for all \(B \in R\). For any \(B_0, B \in R\),
\begin{align*} m_i(B) &\geqslant (u_i(B_0) - B)\bar{G}_i(u_i(B_0)) = m_i(B_0)\frac{(u_i(B_0) - B)}{u_i(B_0) - B_0} \\ &= m_i(B_0) + \frac{m_i(B_0)}{(u_i(B_0) - B_0)}(B_0 - B) = m_i(B_0) - \bar{G}_i(u_i(B_0))(B - B_0). \end{align*}
This inequality implies that \(m_i(B_0)- m_i(B) \leqslant \bar{G}_i(u_i(B_0))(B - B_0)\). Hence, for \(B_0 \gt B\), \(m_i(B_0)- m_i(B) \leqslant 0\) as desired.
(2)
It follows from the above inequality that \(m_i(\cdot)\) is convex. Since it is finite valued, it is also continuous and differentiable almost everywhere. The above inequality also implies that the derivative of \(m_i(B)\) is \(-\bar{G}_i(u_i(B))\) wherever the former exists. Finally, from the above inequality, \(m_i(B_0)- m_i(B) \leqslant B_0 - B\). We can similarly see that \(m_i(B)- m_i(B_0) \leqslant B - B_0\). These together imply \(|m_i(B)- m_i(B_0)| \leqslant |B - B_0|\), and so, Lipschitz-1 continuity of \(m_i\).
(3)
Let us fix B and consider \(i, j\) such that \(j \gt i\). Then, \((u-B)\bar{G}_i(u) \geqslant (u-B)\bar{G}_j(u)\) for all \(u \geqslant B\). This implies that \(m_i(B) \geqslant (u-B)\bar{G}_j(u),\) for all \(u \geqslant B\), and, in turn, that \(m_i(B) \geqslant m_j(B)\). It follows that the sequence \((m_i(B), i\in \mathcal {Q})\) is non-decreasing in i, and since the sequence is lower bounded by zero, it converges as desired.
Lemma 27.
The price \(u_i(B)\) defined in (17) satisfies \(u_i(B) \geqslant B\) is right-continuous, and non-decreasing in B for all \(i \in \mathcal {Q}\).
Proof.
For \(B_1, B_2 \in \mathbb {R}\), we have \((u_i(B_1) - B_2)\bar{G}_i(u_i(B_1)) \leqslant (u_i(B_2) - B_2)\bar{G}_i(u_i(B_2)),\) implying
\begin{equation} B_2(\bar{G}_i(u_i(B_2))-\bar{G}_i(u_i(B_1))) \leqslant u_i(B_2)\bar{G}_i(u_i(B_2)) - u_i(B_1)\bar{G}_i(u_i(B_1)). \end{equation}
(54)
We can similarly obtain
\begin{equation} B_1(\bar{G}_i(u_i(B_1))-\bar{G}_i(u_i(B_2))) \leqslant u_i(B_1)\bar{G}_i(u_i(B_1)) - u_i(B_2)\bar{G}_i(u_i(B_2)). \end{equation}
(55)
Summing (54) and (55), we get \((B_2-B_1)(\bar{G}_i(u_i(B_2))-\bar{G}_i(u_i(B_1))) \leqslant 0.\) Hence, if \(B_1 \lt B_2\), \(\bar{G}_i(u_i(B_2)) \leqslant \bar{G}_i(u_i(B_1))\), implying that \(u_i(B_2) \geqslant u_i(B_1)\).□
Lemma 28.
The limiting value \(m(B)\) defined in (20) and limiting price \(u(B)\) defined in (21) have the following properties.
(a)
\(m(B)\) is non-negative and decreasing in B.
(b)
\(m(B)\) is a Lipschitz-1 continuous, convex function of B.
(c)
\(u(B) \geqslant B\) and is right-continuous and increasing in B.
Proof.
It is identical to the proofs of Lemmas 26 and 27.□

7.2.1 Proof of Proposition 6.

Let \(u = (u_0,u_1,\dots)\) be an arbitrary ergodic policy. For any \(i \geqslant 1\), from the definition of \(m_i(\cdot)\), we have \((u_i - \Delta (i))\bar{G}_i(u_i) \leqslant m_i(\Delta (i)),\) and using (19), \(\lambda \bar{G}_i(u_i)(u_i - \Delta (i)) \leqslant \theta - \mu \Delta (i-1).\) Multiplying both the sides by \(\pi _i\) and using \(\lambda \pi _i\bar{G}_i(u_i) = \pi _{i+1} \mu\) which follows from ergodicity of u, \(\pi _{i+1}\mu (u_i - \Delta (i)) \leqslant \pi _i \theta - \pi _i \mu \Delta (i-1),\) or
\begin{equation} \pi _{i+1}\mu u_i - \pi _i \theta \leqslant \pi _{i+1}\mu \Delta (i) - \pi _i \mu \Delta (i-1). \end{equation}
(56)
A similar argument for \(i = 0\) implies
\begin{equation} \pi _{1}\mu u_0 - \pi _0 \theta \leqslant \pi _{1}\mu \Delta (0). \end{equation}
(57)
Summing (56) and (57) for \(i \geqslant 1\) (summability is ensured by boundedness of \(\Delta (i), i \geqslant 0\)), we obtain \(\mu \sum _{i = 0}^\infty \pi _{i+1} u_i \leqslant \theta .\) Once again using ergodicity of u, we obtain that \(\theta (u) = \lambda \sum _{i = 0}^\infty \pi _i \bar{G}_i(u_i) u_i \leqslant \theta .\) Since the above holds for any arbitrary ergodic policy u, \(\theta ^\ast \leqslant \theta\) as desired. Moreover, replacing the arbitrary policy u with \(u^\ast\), the corresponding inequalities (56) and (57) hold with equality, implying that \(\theta ^\ast = \theta\).

7.3 Proofs of Section 3.2

We begin with proving certain properties of \(\phi _k(\cdot)\) in the following lemma.
Lemma 29.
For each \(i \in \mathcal {Q}\), \(\phi _i(\cdot)\) is continuous, strictly decreasing and unbounded both above and below.
Proof.
Recall that \(m_i(\cdot)\) are continuous, decreasing functions for all \(i \in \mathcal {Q}\), so are \(m_i^{-1}(\cdot)\). In particular, \(\phi _0(\theta)\) is continuous and decreasing. It is also unbounded. Assuming \(\phi _i(\theta)\) is continuous, decreasing, and unbounded, \((\theta -\mu \phi _i(\theta))/\lambda\) is continuous, increasing, and unbounded. Combining it with the properties of \(m_{i+1}^{-1}(\cdot)\), we can infer that \(\phi _{i+1}(\theta)\) is also continuous, decreasing, and unbounded. Therefore, the claim holds via induction.□

7.3.1 Proof of Proposition 9.

Setting \(\delta _0(\theta) = m_0^{-1}(\theta /\lambda) = 0\), we obtain \(\theta _0 = \lambda m_0(0)\). Also observe that \(\delta _0(\cdot)\) is continuous and decreasing on \([0, \lambda m_0(0)]\).
Let us now fix a \(k \in \mathcal {Q}\). Assume that there exist unique \(\theta _0 \geqslant \theta _1 \cdots \geqslant \theta _k\) satisfying \(\delta _i(\theta _i) = 0\) for all \(i \leqslant k\). Further assume that, for all \(i \leqslant k\), \(\delta _i(\cdot)\) are continuous and decreasing on \([0, \theta _i]\). Observe that
\begin{align*} \delta _k(\theta) &= \phi _k(\theta) - \phi _{k-1}(\theta) = \frac{\lambda }{\mu }\left(m_k(\phi _k(\theta))-m_{k+1}(\phi _{k+1}(\theta))\right) \\ &= \frac{\lambda }{\mu }\left(m_k(\phi _k(\theta))-m_{k+1}(\phi _k(\theta))+m_{k+1}(\phi _k(\theta))-m_{k+1}(\phi _{k+1}(\theta))\right) \\ &= \frac{\lambda }{\mu }\left(m_k(\phi _k(\theta))-m_{k+1}(\phi _k(\theta)) + \int _{\phi _k(\theta)}^{\phi _{k+1}(\theta)}\bar{G}_{k+1}(u_{k+1}(B)){\rm d} B\right). \end{align*}
Rearranging the terms and using the definition of \(\delta _{n+1}(\theta)\),
\begin{equation} \int _{\phi _k(\theta)}^{\phi _k(\theta)+\delta _{k+1}(\theta)}\bar{G}_{k+1}(u_{k+1}(B)){\rm d} B = \frac{\mu }{\lambda }\delta _k(\theta) + m_{k+1}(\phi _k(\theta))-m_k(\phi _k(\theta)). \end{equation}
(58)
The first term on the right-hand side is decreasing in \(\theta\) from the induction hypothesis. We argue that the second term is also decreasing in \(\theta\). In view of \(\phi _k(\theta)\) being decreasing in \(\theta\), it is enough to show that \(m_{k+1}(B)-m_k(B)\) in increasing in B for \(B \geqslant 0\). For \(0 \leqslant B_1 \lt B_2\),
\begin{align*} m_{k+1}(B_2) - m_{k+1}(B_1) &= -\int _{B_1}^{B_2} \bar{G}_{k+1}(u_{k+1}(B)) {\rm d} B \geqslant -\int _{B_1}^{B_2} \bar{G}_k(u_k(B)) {\rm d} B = m_k(B_2) - m_k(B_1), \end{align*}
where the inequality follows from Assumption 3. This implies that
\begin{equation*} m_{k+1}(B_2)-m_k(B_2) \geqslant m_{k+1}(B_1)-m_k(B_1) \end{equation*}
as desired. This implies that the left-hand side is also decreasing in \(\theta\). Combining this with the facts that \(\phi _k(\theta)\) is decreasing in \(\theta\), \(u_{k+1}(B)\) is increasing in B and \(\bar{G}_k(u)\) is non-increasing in u, we infer that \(\delta _{k+1}\) is also decreasing in \(\theta\).
Furthermore, for \(\theta = \theta _k\), the first term on the right-hand side is zero whereas the second term is negative. So the left-hand side is also negative for \(\theta = \theta _k\), implying that \(\delta _{k+1}(\theta _k) \lt 0\). This also implies that there exists a unique \(\theta _{k+1} \lt \theta _k\) such that \(\delta _{k+1}(\theta _{k+1}) = 0\). This completes the induction step, and so, the proof of the first statement. Monotonicity of \(\delta _i(\cdot)\) along with \(\theta _i \geqslant \theta _k\) for all \(i \leqslant k\) also implies that \(\delta _i(\theta _k) = \phi _i(\theta _k) - \phi _{i-1}(\theta _k) \geqslant 0\) for all \(i \leqslant k\). This proves the second statement.

7.3.2 Proof of Theorem 10.

Observe that \((\theta _k,\phi _0(\theta _k),\phi _1(\theta _k),\dots)\) satisfy (18) and (19), and \((\phi _i(\theta _k), i \in \mathcal {Q})\) are bounded. It remains to show that \(u^k\) as defined above is ergodic if \(\theta _k \gt \lambda m_k(0)\). Then, the desired claim follows from Proposition 6.
For ergodicity of \(u^k\), we now show that \(\lambda \bar{G}_n(u^k_k) \lt \mu\) if \(\theta _k \gt \lambda m_k(0)\). Recall that \(\delta _k(\theta _k) = 0\) implies
\begin{align*} \theta _k &= \lambda m_k(\phi _k(\theta _k)) + \mu \phi _k(\theta _k) = \lambda (u_k(\phi _k(\theta _k))-\phi _k(\theta _k))\bar{G}_k(u_k(\phi _k(\theta _k))) + \mu \phi _k(\theta _k)\\ &= \lambda u^k_k \bar{G}_k(u^k_k) + (\mu - \lambda \bar{G}_k(u^k_k)) \phi _k(\theta _k). \end{align*}
Furthermore, \(\theta _k \gt \lambda m_k(0)\) implies that \((\mu - \lambda \bar{G}_k(u^k_k)) \phi _k(\theta _k) \gt \lambda (m_k(0) - u^k_k \bar{G}_k(u^k_k)) \geqslant 0\) where the last inequality follows from the definition of \(m_k(\cdot)\). This inequality implies that \(\lambda \bar{G}_k(u^k_k) \lt \mu\) as desired.

7.4 Proofs of Section 3.3

We first prove the following lemma for the case \(\theta _\infty \gt \lambda m(0)\). It will subsequently be used in our main result.
Lemma 30.
If Assumption 2 and Assumption 3 hold and \(\theta _\infty \gt \lambda m(0)\), then there exists an integer K and \(\alpha \in (0,1)\) such that \(\rho \bar{G}_j(u^k_j) \leqslant \alpha\) for all \(k \geqslant j \geqslant K\).
Proof.
We prove the result separately for the cases \(j = k \geqslant K\) and \(K \leqslant j \lt k\).
(a)
If \(\theta _\infty \gt \lambda m(0)\), then using convergence of \(m_i(0)\) as in Assumption 2, there exists an integer K such that for all \(k \geqslant K\), we have \(\frac{\theta _k}{m_k(0)} \geqslant \lambda + \epsilon ,\) where \(\epsilon = \frac{1}{2}(\frac{\theta _\infty }{m(0)} - \lambda)\). Equivalently, \(\theta _k \geqslant (\lambda + \epsilon)m_k(0)\) for all \(k \geqslant K\). As in the proof of Proposition 10, this implies that
\begin{equation*} (\mu - \lambda \bar{G}_k(u^k_k)) \phi _k(\theta _k) \geqslant \lambda (m_k(0) - u^k_k \bar{G}_k(u^k_k)) + \epsilon m_k(0) \geqslant \epsilon m_k(0) \end{equation*}
for all \(k \geqslant K\). This further implies that for all \(k \geqslant K\), we have
\begin{align*} \rho \bar{G}_k(u^k_k) &\leqslant 1 - \frac{\epsilon m_k(0)}{\mu \phi _k(\theta _k)} \leqslant 1 - \frac{\epsilon m(0)}{\mu \phi _\infty } \triangleq \alpha \lt 1. \end{align*}
(b)
For all \(K \leqslant j \leqslant k\), we have \(\phi _j(\theta _k) \geqslant \phi _K(\theta _k) \geqslant \phi _K(\theta _K)\), where the first inequality follows Proposition 9 and the second one from the facts that \(\theta _k \leqslant \theta _K\) and \(\phi _k(\theta)\) is decreasing in \(\theta\) (see Lemma 29). The above inequalities imply that \(u_K(\phi _j(\theta _k)) \geqslant u_K(\phi _K(\theta _K))\), and so \(\bar{G}_K(u_K(\phi _K(\theta _K))) \geqslant \bar{G}_K(u_K(\phi _j(\theta _k)))\). Also, from Assumption 3, \(\bar{G}_K(u_K(\phi _j(\theta _k))) \geqslant \bar{G}_j(u_j(\phi _j(\theta _k)))\). Combining these inequalities, \(\bar{G}_j(u_j(\phi _j(\theta _k))) \leqslant \bar{G}_K(u_K(\phi _K(\theta _K)))\). We get the desired inequality by realizing that \(u^k_j = u_j(\phi _j(\theta _k))\) and using the result of Part (a).

7.4.1 Proof of Theorem 11.

From Lemma 30, if \(\theta _\infty \gt \lambda m(0),\) then \(\rho \bar{G}_k(u^k_k) \leqslant \alpha\) for all \(k \geqslant K\) and so \(\rho \bar{G}_i(u^k_k) \leqslant \alpha\) for all \(i \geqslant k\) and \(k \geqslant K\). Hence, for all \(k \geqslant K\), the policies \(\mathbf {u}^k\) are ergodic for the modified systems as well as the original system. For such k, the stationary distribution of the original system under the policy \(\mathbf {u}^k\) is
\begin{equation} \pi _i(\mathbf {u}^k) \triangleq {\left\lbrace \begin{array}{ll} \sigma (\mathbf {u}^k)^{-1} \rho ^i \prod _{j=0}^{i-1}\bar{G}_j(u^k_j), & i \leqslant k, \\ \sigma (\mathbf {u}^k)^{-1}\rho ^i \prod _{j=0}^{k-1}\bar{G}_j(u^k_j)\prod _{j=k}^{i-1}\bar{G}_j(u^k_k), & i \gt k, \end{array}\right.} \end{equation}
(59)
where \(\sigma (\mathbf {u}^k)\) is the normalizing factor;
\begin{equation*} \sigma (\mathbf {u}^k)=\sum _{i=0}^k \rho ^i \prod _{j=0}^{i-1}\bar{G}_j(u^k_j) + \sum _{i=k+1}^{\infty }\rho ^i \prod _{j=0}^{k-1}\bar{G}_j(u^k_j) \prod _{j=k}^{i-1}\bar{G}_j(u^k_k). \end{equation*}
The revenue rate of the policy \(u^k\) is
\begin{equation} R(\mathbf {u}^k) = \lambda \sum _{i=0}^k \pi _i(\mathbf {u}^k) u^k_i \bar{G}_i(u^k_i) +\lambda u^k_k \sum _{i=k+1}^{\infty } \pi _i(\mathbf {u}^k) \bar{G}_i(u^k_k). \end{equation}
(60)
Similarly, the stationary distribution of the k-truncated system under the policy \(u^k\) is
\begin{equation} \bar{\pi }_i(\mathbf {u}^k) \triangleq {\left\lbrace \begin{array}{ll} \bar{\sigma }(\mathbf {u}^k)^{-1} \rho ^i \prod _{j=0}^{i-1}\bar{G}_j(u^k_j), & i \leqslant k, \\ \bar{\sigma }(\mathbf {u}^k)^{-1}\rho ^i \prod _{j=0}^{k-1}\bar{G}_j(u^k_j)\left(\bar{G}_k(u^k_k)\right)^{i-k}, & i \gt k, \end{array}\right.} \end{equation}
(61)
where
\begin{equation*} \bar{\sigma }(\mathbf {u}^k)=\sum _{i=0}^k \rho ^i \prod _{j=0}^{i-1}\bar{G}_j(u^k_j) + \sum _{i=k+1}^{\infty }\rho ^i \prod _{j=0}^{k-1}\bar{G}_j(u^k_j) \left(\bar{G}_k(u^k_k)\right)^{i-k}. \end{equation*}
The revenue rate of the policy \(\mathbf {u}^k\) for the k-truncated system is
\begin{equation} \theta _k = \lambda \sum _{i=0}^k \bar{\pi }_i(\mathbf {u}^k) u^k_i \bar{G}_i(u^k_i) +\lambda u^k_k \bar{G}_k(u^k_k) \sum _{i=k+1}^{\infty } \bar{\pi }_i(\mathbf {u}^k). \end{equation}
(62)
Combining (60) and (62), we see that \(\theta _k = R(\mathbf {u}^k) + \sum _{i= 0}^{\infty }e_i(k)\) where
\begin{equation*} e_i(k) \triangleq {\left\lbrace \begin{array}{ll} \lambda u^k_i \bar{G}_i(u^k_i) (\bar{\pi }_i(\mathbf {u}^k) - \pi _i(\mathbf {u}^k)), & i \leqslant k, \\ \lambda u^k_k(\bar{\pi }_i(\mathbf {u}^k)\bar{G}_k(u^k_k)- \pi _i(\mathbf {u}^k)\bar{G}_i(u^k_k)), & i \gt k. \end{array}\right.} \end{equation*}
Using Lemma 30 it can be shown that, for all \(k \geqslant K\), \(\bar{\sigma }(\mathbf {u}^k) - \sigma (\mathbf {u}^k) \leqslant c \alpha ^k\), where \(c \gt 0\) is a constant. Moreover, it can also be shown that
\begin{equation*} e_i(k) \leqslant {\left\lbrace \begin{array}{ll}0, & i \leqslant k, \\ c\beta ^k_i \alpha ^{i}, & i \gt k \\ \end{array}\right.} \end{equation*}
for bounded constants \(\beta ^k_i, i \geqslant 0\). Suppose \(\beta ^k_i \leqslant A\). Then, we easily obtain the following bound \(\theta _k \leqslant R(\mathbf {u}^k) + cA\alpha ^k(\frac{\alpha }{1- \alpha })\), for all \(k \geqslant K\). Combining it with the earlier observation \(R(u^k) \leqslant \theta ^\ast \leqslant \theta _k\) yields
\begin{equation} \theta _k - \frac{cA\alpha ^{k+1}}{1- \alpha } \leqslant R(\mathbf {u}^k) \leqslant \theta ^\ast \leqslant \theta _k \leqslant \theta (\mathbf {u}^k) + \frac{cA\alpha ^{k+1}}{1- \alpha }. \end{equation}
(63)
Furthermore, taking \(k \rightarrow \infty\), we conclude that \(\lim _{k \rightarrow \infty } \theta (\mathbf {u}^k) = \theta _\infty = \theta ^\ast\).
Remark 12.
One can use (63) to bound the difference between the revenue rate corresponding to \(u^k\) and the optimal revenue rate \(\theta ^\ast\), as
\begin{equation} \theta ^\ast - R(\mathbf {u}^k) \leqslant \frac{cA\alpha ^{k+1}}{1- \alpha }. \end{equation}
(64)

7.4.2 Proof of Corollary 12.

From (59) and (60), the revenue rate under policy \(\mathbf {u}^k\), \(R(\mathbf {u}^k)\), is a continuous function of \((u^k_i, i \in \mathcal {Q})\) and \((\bar{G}_i(u^k_i), i \in \mathcal {Q})\). From Remark 7, \((u^k_i, k \in \mathcal {Q})\) converges to \(u^\ast _i\) for all \(i \in \mathcal {Q}\). Moreover, from the hypothesis of the corollary, \((\bar{G}_i(u^k_i), k \in \mathcal {Q})\) converges to \(\bar{G}_i(u^\ast _i)\) for all \(i \in \mathcal {Q}\). Consequently, \(\lim _{k \rightarrow \infty } R(\mathbf {u}^k) = R(\mathbf {u}^\ast)\). Also, from Theorem 11, \(R(\mathbf {u}^\ast) = \theta ^\ast\), implying that \(\mathbf {u}^\ast\) is an optimal policy.

7.5 Proofs for Section 3.4

7.5.1 Proof of Lemma 13.

From (19), we have \(\Delta (i-1) = \frac{\theta - \lambda m_i(\Delta (i))}{\mu }\) for \(i = 1,\ldots ,k\). In particular,
\begin{equation*} \Delta (k-1) = \frac{\theta - \lambda m_k(\Delta (k))}{\mu } = \Delta (k). \end{equation*}
Furthermore, from Lemma 26, \(m_{k-1}(\Delta (k-1)) = m_{k-1}(\Delta (k)) \geqslant m_k(\Delta (k))\). Hence, continuing iteratively for \(i = K-2,\dots ,0\), we can infer that \(\Delta (0) \leqslant \Delta (1) \leqslant \cdots \leqslant \Delta (k).\) It remains to show that \(\Delta (0) \geqslant 0\). Notice that \(\Delta (1) \geqslant \Delta (0)\) implies (see Lemma 26)
\begin{equation*} m_1(\Delta (1)) \leqslant m_0(\Delta (1)) \leqslant m_0(\Delta (0)). \end{equation*}
Hence, using (18) and (19) for \(i=1\), we get \(\frac{\theta - \mu \Delta (0)}{\lambda } \leqslant \frac{\theta }{\lambda }\) or \(\Delta (0) \geqslant 0\) as desired.

7.5.2 Properties of the Lower Bound.

This lower bound \({\underline{\theta }}\) has the following properties.
Lemma 31.
(a)
If \(\rho \bar{G}_k(0) \lt 1\), then \({\underline{\theta }} = -\infty\),
(b)
If \(\rho \bar{G}_k(0) \geqslant 1\), then \({\underline{\theta }} \geqslant 0\).
Proof.
(a)
We will show that \(\mu \Delta + \lambda m_k(\Delta)\) has slope at least \(\mu -\lambda \bar{G}_k(0)\) everywhere, i.e., it is strictly increasing in \(\Delta\). This implies that \({\underline{\theta }} = \lim _{\Delta \rightarrow -\infty }(\mu \Delta + \lambda m_k(\Delta)) = -\infty\). Let us consider \(\Delta _1\) and \(\Delta _2 \gt \Delta _1\). Then
\begin{align*} \lambda m_k(\Delta _1) + \mu \Delta _1 &= \lambda (m_k(\Delta _1) + \Delta _1 \bar{G}_k(0)) + (\mu - \lambda \bar{G}_k(0))\Delta _1 \\ &= \lambda \max _{u \geqslant 0}\lbrace u \bar{G}_k(u) + \Delta _1(\bar{G}_k(0) - \bar{G}_k(u))\rbrace +(\mu - \lambda \bar{G}_k(0))\Delta _1\\ &\lt \lambda \max _{u \geqslant 0}\lbrace u \bar{G}_k(u) + \Delta _2 (\bar{G}_k(0) - \bar{G}_k(u))\rbrace \! +\! (\mu - \lambda \bar{G}_k(0))\Delta _2 + (\mu - \lambda \bar{G}_k(0))(\Delta _1-\Delta _2)\\ &= \lambda m_k(\Delta _2) + \mu \Delta _2 + (\mu - \lambda \bar{G}_k(0))(\Delta _1-\Delta _2), \end{align*}
which gives the desired lower bound on the slope, \(\frac{(\lambda m_k(\Delta _2) + \mu \Delta _2) - (\lambda m_k(\Delta _1) + \mu \Delta _1)}{\Delta _2-\Delta _1} \gt (\mu - \lambda \bar{G}_k(0)).\)
(b)
Clearly, \(\mu \Delta + \lambda m_k(\Delta) \geqslant 0\) for all \(\Delta \geqslant 0\). For any \(\Delta \lt 0\), \(m_k(\Delta) \geqslant - \Delta \bar{G}_k(0)\) implying \(\mu \Delta +\lambda m_k(\Delta) \geqslant (\mu - \lambda \bar{G}_k(0)) \Delta\), which also is non-negative since \(\mu \leqslant \lambda \bar{G}_k(0)\). These facts together prove the claim.
Lemma 32.
Suppose \(\rho \bar{G}_k(0) = 1\).
(a)
If \(G_k(u) \gt G_k(0)\) for all \(u \gt 0\) then \({\underline{\theta }} = 0\),
(b)
If there exists a \(u^0 \gt 0\) such that \(G_k(u^0) = G_k(0)\) then \({\underline{\theta }} \geqslant \lambda \bar{G}_k(0) u^0\).
Proof.
(a)
We have already seen in Lemma 31(a) that \({\underline{\theta }} \geqslant 0\). To prove our claim we will show that \({\underline{\theta }} \leqslant \epsilon\) for all \(\epsilon \gt 0\). Recall the definitions of \(m_k(\Delta)\) and \(u_k(\Delta)\) in (16) and (17), respectively. Also, fix an \(\epsilon \in (0,\lambda m_k(0))\) and define
\begin{equation*} \Delta _\epsilon \triangleq {\left\lbrace \begin{array}{ll} \frac{\epsilon -\lambda m_k(0)}{\lambda (G_k(\epsilon /\lambda)-G_k(0))}~\text{if } u_k({\Delta }) \gt \epsilon /\lambda \text{ for all } \Delta \leqslant 0,\\ \max \left\lbrace \Delta \leqslant 0: u_k({\Delta }) \leqslant \epsilon /\lambda \right\rbrace \text{ otherwise}. \end{array}\right.} \end{equation*}
Observe that \(\Delta _\epsilon\) is well defined because \(G_k(u) \gt G_k(0)\) for all \(u \gt 0\). We will prove that \({\underline{\theta }} \leqslant \epsilon\) for each of the two cases in the right-hand side separately. First, let \(u_k({\Delta }) \gt \epsilon /\lambda\) for all \(\Delta \leqslant 0\). Then
\begin{align*} {\underline{\theta }} &\leqslant \mu \Delta _\epsilon + \lambda \max _{u \geqslant 0} \lbrace (u - \Delta _\epsilon)\bar{G}_k(u)\rbrace = \lambda u_k(\Delta _\epsilon) \bar{G}_k(u_k(\Delta _\epsilon)) + (\mu - \lambda \bar{G}_k(u_k(\Delta _\epsilon)))\Delta _\epsilon \\ &\leqslant \lambda m_k(0) + (\mu - \lambda \bar{G}_k(\epsilon /\lambda))\Delta _\epsilon = \lambda m_k(0) + \lambda (\bar{G}_k(0) - \bar{G}_k(\epsilon /\lambda))\Delta _\epsilon = \epsilon , \end{align*}
where the second last inequality follows since \(\rho \bar{G}_k(0) = 1\). In the second case also, we have
\begin{align*} {\underline{\theta }} &\leqslant \lambda u_k(\Delta _\epsilon) \bar{G}_k(u_k(\Delta _\epsilon)) + (\mu - \lambda \bar{G}_k(u_k(\Delta _\epsilon)))\Delta _\epsilon \\ &\leqslant \epsilon + (\mu - \lambda \bar{G}_k(u_k(\Delta _\epsilon)))\Delta _\epsilon = \epsilon + \lambda (\bar{G}_k(0)- \bar{G}_k(u_k(\Delta _\epsilon)))\Delta _\epsilon \leqslant \epsilon , \end{align*}
where the last inequality uses the assumption that \(\rho \bar{G}_k(0) = 1\).
(b)
For all \(\Delta\), \(\mu \Delta + \lambda m_k(\Delta) \geqslant \lambda u^0 \bar{G}_k(u^0) + (\mu - \lambda \bar{G}_k(u^0))\Delta = \lambda u^0\bar{G}_k(0)).\) The last equality holds because \(\bar{G}_k(u^0) = \bar{G}_k(0)\) and \(\mu = \lambda \bar{G}_k(0) = \lambda \bar{G}_k(u^0)\). Hence, by definition, \({\underline{\theta }} \geqslant \lambda \bar{G}_k(u^0)\).

7.5.3 Proof of Proposition 14.

(a)
Notice that existence of a fixed point follows from Lemma 31. So only uniqueness has to be established. However, the proof of Lemma 31(a) also shows that \(\mu \Delta + \lambda m_k(\Delta)\) is strictly increasing for all \(\lambda \leqslant \mu /\bar{G}_k(0)\). So, for any \(\theta \geqslant {\underline{\theta }}\), there cannot be two or more solutions to (32). This also implies monotonicity of the solution, \(D(\theta)\), with \(\theta\).
(b)
Recall that \(f_k(\Delta)\) is increasing and concave and upper bounded. Also, \(f_k(0) = (\theta - \lambda m_k(0))/\mu\). Hence, it is enough to show that \(f_k(\Delta) = \Delta\) has two solutions, \(D_1(\theta)\) and \(D_2(\theta)\), \(D_1(\theta) \leqslant D_2(\theta)\), for all \(\theta \gt \lambda m_k(0)\). Monotonicity of \(D_1(\theta)\) and \(D_2(\theta)\) as stated also immediately follows. So, let us consider any \(\theta \gt \lambda m_k(0)\). Since \(f_k(0) \gt 0\) and \(f_k\Delta)\) is upper bounded, \(f_k(\Delta) = \Delta\) assumes a positive solution. We call it \(D_2(\theta)\). To show existence of another solution, let us define \(\bar{\Delta } \triangleq \theta /(\mu - \lambda \bar{G}_k(0)) \lt 0\). Clearly, \(\theta /\mu + \lambda \bar{G}_k(0)\bar{\Delta }/\mu = \bar{\Delta }\), implying \(f_k(\bar{\Delta }) \leqslant \bar{\Delta }\). The last assertion holds since \(-m_k(\Delta) \leqslant \bar{G}_k(0)\Delta\) for all \(\Delta \lt 0\). So we can conclude that there exists a \(D_1(\theta) \in [\bar{\Delta }, 0)\) that also solves \(g(\Delta) = \Delta\).

7.5.4 Proof of Theorem 15.

(a) We consider the following three cases separately.
(1)
\(\rho \bar{G}_k(0) \lt 1\): We first argue that \(\lambda m_0(\psi _0(\lambda m_k(0))) \geqslant \lambda m_k(0)\). Observe that \(\psi _k(\lambda m_k(0)) = \Delta (\lambda m_k(0)) = 0\). Hence
\begin{equation*} \psi _{k-1}(\lambda m_k(0)) = \frac{\lambda m_k(0)-\lambda m_k(0)}{\mu } = \psi _k(\lambda m_k(0)) = 0 \end{equation*}
and, from Lemma 26, we have \(m_{k-1}(\psi _{k-1}(\lambda m_k(0)) \geqslant m_k(\psi _{k-1}(\lambda m_k(0)) = m_k(0).\) Continuing iteratively for \(i = k-2,\dots ,0\), we can infer that
\begin{equation*} \psi _0(\lambda m_k(0)) \leqslant \psi _1(\lambda m_k(0)) \leqslant \cdots \leqslant \psi _k(\lambda m_k(0)) = 0, \end{equation*}
and that
\begin{equation*} m_0(\psi _0(\lambda m_k(0)) \geqslant m_1(\psi _1(\lambda m_k(0))) \geqslant \cdots \geqslant m_k(0). \end{equation*}
In particular, \(\lambda m_0(\psi _0(\lambda m_k(0))) \geqslant \lambda m_k(0)\).
We next argue that \(\lambda m_0(\psi _0(\theta))\) is decreasing in \(\theta\). Recall that \(\psi _k(\theta) = D(\theta)\) is increasing in \(\theta\). Suppose that, for some i, \(\psi _i(\theta)\) is increasing in \(\theta\). Then, \(m_i(\psi _i(\theta))\) will be decreasing in \(\theta\) and so \(\psi _{i-1}(\theta)\) will also be increasing in \(\theta\). Arguing inductively, we see that \(\psi _0(\theta)\) is increasing in \(\theta\), and finally, that \(\lambda m_0(\psi _0(\theta))\) is decreasing in \(\theta\).
The above two facts, \(\lambda m_0(\psi _0(\lambda m_k(0))) \geqslant \lambda m_k(0)\) and \(\lambda m_0(\psi _0(\theta))\) is decreasing in \(\theta\), together establish both existence and uniqueness of the fixed point of \(\theta = \lambda m_0(\psi _0(\theta))\). In particular, the unique fixed point \(\theta _k \in [\lambda m_k(0), \lambda m_0(\psi _0(\lambda m_k(0)))]\).
(2)
\(\rho \bar{G}_k(0) \geqslant 1\) and \(\Delta ({\underline{\theta }}) \leqslant 0\): Considering \(\psi _k(\theta) = D_2(\theta)\), \(\psi _k(\lambda m_k(0)) = 0\) and \(\psi _k(\theta)\) is increasing in \(\theta\). Hence, as in Case \((1)\), \(\lambda m_0(\psi _0(\lambda m_k(0))) \geqslant \lambda m_k(0)\) and \(\lambda m_0(\psi _0(\theta))\) is decreasing in \(\theta\), establishing existence and uniqueness of the fixed point. Again, the unique fixed point \(\theta _k \in [\lambda m_k(0), \lambda m_0(\psi _0(\lambda m_k(0)))]\).
(3)
\(\rho \bar{G}_k(0) \geqslant 1\) and \(\Delta ({\underline{\theta }}) \gt 0\): Now we consider the following two subcases separately.
(i)
\(\lambda m_0(\psi _0({\underline{\theta }})) \gt {\underline{\theta }}\): Considering \(\psi _k(\theta) = D_2(\theta)\), \(\psi _k(\theta)\) is increasing in \(\theta\). Hence, as in Case \((1)\), \(\lambda m_0(\psi _0(\theta))\) is decreasing in \(\theta\), establishing existence and uniqueness of the fixed point. The unique fixed point \(\theta _k \in [{\underline{\theta }}, \lambda m_0(\psi _0({\underline{\theta }}))]\).
(ii)
\(\lambda m_0(\psi _0({\underline{\theta }})) \leqslant {\underline{\theta }}\): Considering \(\psi _k(\theta) = D_1(\theta)\), we get \(\psi _k(\lambda m_k(0)) = 0\). Again, as in Case \((1)\), \(\lambda m_0(\psi _0(\lambda m_k(0))) \geqslant \lambda m_k(0)\). So, \(\lambda m_0(\psi _0(\theta)) = \theta\) has a solution in \([{\underline{\theta }}, \lambda m_k(0)]\). In the following, we argue via contradiction that \(\lambda m_0(\psi _0(\theta)) = \theta\) has unique solution. Suppose it has two solutions, \(\theta _k\) and \(\theta ^{\prime }_k\) where \(\theta ^{\prime }_k \ge \theta _k\). Then, \(\psi _{k-1}(\theta ^{\prime }_k) = \psi _k(\theta ^{\prime }_k) = D_1(\theta ^{\prime }_k) \leqslant D_1(\theta _k) = \psi _k(\theta _k) = \psi _{k-1}(\theta _k)\). Moreover,
\begin{align*} \lambda \int _{\psi _{k-1}(\theta _k)}^{\psi _{k-1}(\theta ^{\prime }_k)}\bar{G}_{k-1}(u_{k-1}(B)){\rm d} B &\le \lambda \int _{\psi _{k-1}(\theta _k)}^{\psi _{k-1}(\theta ^{\prime }_k)}\bar{G}_k(u_k(B)){\rm d} B \\ & = \lambda \int _{\psi _k(\theta _k)}^{\psi _k(\theta ^{\prime }_k)}\bar{G}_k(u_k(B)){\rm d} B \leqslant - (\theta ^{\prime }_k - \theta _k) \end{align*}
where the inequalities follow from Assumption 3 and the fact that \(\psi _{k-1}(\theta ^{\prime }_k) \le \psi _{k-1}(\theta _k)\). We argue via induction that \(\psi _i(\theta ^{\prime }_k) - \psi _i(\theta _k) \le \psi _{i+1}(\theta ^{\prime }_k) - \psi _{i+1}(\theta _k) \le 0\) and
\begin{equation*} \lambda \int _{\psi _i(\theta _k)}^{\psi _i(\theta ^{\prime }_k)}\bar{G}_i(u_i(B)){\rm d} B \le \lambda \int _{\psi _{i+1}(\theta _k)}^{\psi _{i+1}(\theta ^{\prime }_k)}\bar{G}_{i+1}(u_{i+1}(B)){\rm d} B \le - (\theta ^{\prime }_k - \theta _k) \end{equation*}
for all \(0 \le i \le k-1\). We have readily seen these for \(i = k-1\). Suppose these hold for some \(i \le k-1\). This implies that
\begin{align*} \psi _{i-1}(\theta ^{\prime }_k) - \psi _{i-1}(\theta _k) &= \frac{(\theta ^{\prime }_k-\theta _k)+\lambda \int _{\psi _i(\theta _k)}^{\psi _i(\theta ^{\prime }_k)}\bar{G}_i(u_i(B)){\rm d} B}{\mu }\\ &\leqslant \frac{(\theta ^{\prime }_k-\theta _k)+\lambda \int _{\psi _{i+1}(\theta _k)}^{\psi _{i+1}(\theta ^{\prime }_k)}\bar{G}_{i+1}(u_{i+1}(B)){\rm d} B}{\mu } = \psi _i(\theta ^{\prime }_k) - \psi _i(\theta _k). \end{align*}
Furthermore,
\begin{align*} \lambda \int _{\psi _{i-1}(\theta _k)}^{\psi _{i-1}(\theta ^{\prime }_k)}\bar{G}_{i-1}(u_{i-1}(B)){\rm d} B & \leqslant \lambda \int _{\psi _i(\theta _k)}^{\psi _{i-1}(\theta ^{\prime }_k)+\psi _i(\theta _k)-\psi _{i-1}(\theta _k)}\bar{G}_{i-1}(u_{i-1}(B)){\rm d} B \\ & \leqslant \lambda \int _{\psi _i(\theta _k)}^{\psi _i(\theta ^{\prime }_k)}\bar{G}_{i-1}(u_{i-1}(B)){\rm d} B \leqslant \lambda \int _{\psi _i(\theta _k)}^{\psi _i(\theta ^{\prime }_k)}\bar{G}_i(u_i(B)){\rm d} B \end{align*}
where the first inequality follows because \(\psi _{i-1}(\theta ^{\prime }_k) \le \psi _{i-1}(\theta _k)\), \(\psi _i(\theta _k) \ge \psi _{i-1}(\theta _k)\), \(u_{i-1}(B)\) is non-decreasing in B and \(\bar{G}_{i-1}\) is a non-increasing function, the second because \(\psi _i(\theta ^{\prime }_k) \ge \psi _{i-1}(\theta ^{\prime }_k)+\psi _i(\theta _k)-\psi _{i-1}(\theta _k)\) and the third from Assumption 3 and the fact that \(\psi _i(\theta ^{\prime }_k) \le \psi _i(\theta _k)\). This completes the induction step. In particular, we observe that
\begin{equation} \lambda \int _{\psi _0(\theta _k)}^{\psi _0(\theta ^{\prime }_k)}\bar{G}_0(u_0(B)){\rm d} B \leqslant \lambda \int _{\psi _1(\theta _k)}^{\psi _1(\theta ^{\prime }_k)}\bar{G}_1(u_1(B)){\rm d} B \leqslant - (\theta ^{\prime }_k - \theta _k). \end{equation}
(65)
On the other hand, \(\theta _k = \lambda m_0(\psi _0(\theta _k))\) and \(\theta ^{\prime }_k = \lambda m_0(\psi _0(\theta ^{\prime }_k))\), implying that
\begin{equation} \lambda \int _{\psi _0(\theta _k)}^{\psi _0(\theta ^{\prime }_k)}\bar{G}_0(u_0(B)){\rm d} B = - (\theta ^{\prime }_k - \theta _k). \end{equation}
(66)
From (65) and (66),
\begin{equation*} \lambda \int _{\psi _1(\theta _k)}^{\psi _1(\theta ^{\prime }_k)}\bar{G}_1(u_1(B)){\rm d} B = - (\theta ^{\prime }_k - \theta _k). \end{equation*}
Hence
\begin{equation*} \psi _0(\theta ^{\prime }_k) - \psi _0(\theta _k) = \frac{(\theta ^{\prime }_k-\theta _k)+\lambda \int _{\psi _1(\theta _k)}^{\psi _1(\theta ^{\prime }_k)}\bar{G}_1(u_1(B)){\rm d} B}{\mu } = 0 \end{equation*}
and so, \(\theta ^{\prime }_k - \theta _k = \lambda (m_0(\psi _0(\theta ^{\prime }_k)) - m_0(\psi _0(\theta _k))) = 0.\) This establishes that \(\lambda m_0(\psi _0(\theta)) = \theta\) has unique solution.
Finally we show that, in all the above cases, \(\theta = \lambda m_0(\psi _0(\theta))\) has no solution for other potential choices of \(\psi _k(\theta)\).
(1)
\(\rho \bar{G}_k(0) \geqslant 1\) and \(\Delta ({\underline{\theta }}) \leqslant 0\): Taking \(\psi _k(\theta) = D_1(\theta)\), \(\psi _k(\theta) \lt 0\) for all \(\theta \gt {\underline{\theta }}\). But, from Lemma 13, for \((\theta _k, \Delta (i), 0 \leqslant i \leqslant k)\) to be a solution to (18)–(19) and (32), \(\Delta (k) = \psi _K(\theta _k) \geqslant 0\). Hence we can conclude that, for \(\psi _k(\theta) = D_1(\theta)\), \(\lambda m_0(\psi _0(\theta)) = \theta\) does not have any solution.
(2)
\(\rho \bar{G}_k(0) \geqslant 1\) and \(\Delta ({\underline{\theta }}) \gt 0\): We again consider the following two subcases separately.
(a)
\(\lambda m_0(\psi _0({\underline{\theta }})) \gt {\underline{\theta }}\): If we take \(\psi _k(\theta) = D_1(\theta)\), then as shown in (65),
\begin{equation*} \lambda (m_0(\psi _0({\underline{\theta }})) - m_0(\psi _0(\theta))) = \lambda \int _{\psi _0({\underline{\theta }})}^{\psi _0(\theta)}\bar{G}_0(u_0(B)){\rm d} B \le - (\theta - {\underline{\theta }}) \end{equation*}
for all \(\theta \ge {\underline{\theta }}\). Equivalently, \(\lambda m_0(\psi _0(\theta)) - \theta \geqslant \lambda m_0(\psi _0({\underline{\theta }})) - {\underline{\theta }} \gt 0\) for all \(\theta \geqslant {\underline{\theta }}\). Hence, \(\lambda m_0(\psi _0(\theta)) = \theta\) cannot have a solution.
(b)
\(\lambda m_0(\psi _0({\underline{\theta }})) \leqslant {\underline{\theta }}\): If we take \(\psi _k(\theta) = D_2(\theta)\), as argued in Case \((1)\) of Part (a), \(\lambda m_0(\psi _0(\theta))\) is decreasing in \(\theta\), and so \(\lambda m_0(\psi _0(\theta)) = \theta\) cannot hold for any \(\theta \gt {\underline{\theta }}\).
(b) When \(\psi _k(\theta) = D(\theta)\) or \(\psi _k(\theta) = D_2(\theta)\), as shown in Case (1) of Part (a), \(\lambda m_0(\psi _0(\theta))\) is decreasing in \(\theta\). Hence, the claim follows from [9, Theorem 2.1].

7.6 Proofs of Section 5

7.6.1 Proof of Proposition 24.

Observe that
\begin{align*} {\underline{\theta }} = & \min \left\lbrace \min _{\Delta \leqslant v} \lbrace \mu \Delta +\lambda (v-\Delta)^+\rbrace , \min _{\Delta \gt v} \lbrace \mu \Delta +\lambda (v-\Delta)^+\rbrace \right\rbrace \\ = & \min \left\lbrace \min _{\Delta \leqslant v} \lbrace \lambda v + (\mu -\lambda)\Delta \rbrace , \min _{\Delta \gt v} \mu \Delta \right\rbrace = {\left\lbrace \begin{array}{ll} -\infty & \text{ if } \lambda \lt \mu , \\ \mu v & \text{ if } \lambda \geqslant \mu . \end{array}\right.} \end{align*}
Now we obtain \(D(\theta)\) under the different conditions stated.
(a)
\(\lambda \lt \mu\): Observe that (48) has a unique solution \((\theta -\lambda v)/(\mu - \lambda)\) if the latter is less than or equal to v, or equivalently, if \(\theta \leqslant \mu v\). Also, (48) has a unique solution \(\theta /\mu\) if \(\theta \gt \mu v\). Combining both these cases yields the desired expression.
(b)
\(\lambda = \mu\): The claim for \(\theta = \mu v\) follows from inspection. If \(\theta \gt \mu v\), from (48), \(\mu (v - \Delta) \leqslant \mu (v - \Delta)^+\), implying that \(\Delta \geqslant v\). In this case, \(\theta = \mu \Delta\) which yields the claimed solution.
(c)
\(\lambda \gt \mu\): First let \(\Delta \leqslant v\). Then, (48) becomes \(\Delta = \frac{\theta - \lambda (v-\Delta)}{\mu }\), yielding \(D(\theta) = \frac{\theta }{\mu }(\frac{1- \lambda v/\theta }{1 - \lambda /\mu })\). Similarly, assuming \(\Delta \gt v\) gives \(D(\theta) = \frac{\theta }{\mu }.\)

7.6.2 Proof of Theorem 25.

Clearly,
\begin{align*} \theta ^\infty _1 \geqslant \mu v \lim _{k \rightarrow \infty } \frac{\sum _{i=0}^k (\mu /\lambda)^i}{\sum _{i=0}^{k+1} (\mu /\lambda)^i} = {\left\lbrace \begin{array}{ll}\lambda v & \text{ if } \lambda \lt \mu , \\ \mu v & \text{ if } \lambda \geqslant \mu .\end{array}\right.} \end{align*}
Similarly, we can show that
\begin{equation*} \theta ^\infty _1 \leqslant {\left\lbrace \begin{array}{ll} \lambda v_0 & \text{ if } \lambda \lt \mu , \\ \mu v_0 & \text{ if } \lambda \geqslant \mu . \end{array}\right.} \end{equation*}
Now we consider the following three cases. \(\theta ^\infty _1 \gt \mu v\):. In this case \(\lim _{k \rightarrow \infty } \Delta ^k(k) \gt v\). So, there exist \(k \lt \infty\) such that \(\Delta ^k(k) \gt v_k\) and \(K = \max \lbrace i: \Delta ^i(i) \leqslant v_i\rbrace\) is finite. In this case (47a)–(47b) have a unique solution \((\theta ^K_1, \Delta ^K(i), i \geqslant 0)\). Furthermore, the mean revenue rate \(\theta ^K_1 \leqslant \mu v_K\).
\(\theta ^\infty _1 \leqslant \mu v\):. In this case \(\lim _{k \rightarrow \infty } \Delta ^k(k) \leqslant v\). So, \(\Delta ^i(i) \leqslant v_i\) for all i. Furthermore, the mean revenue rate is \(\theta ^\infty _2\).
\(\lambda \geqslant \mu\) and \(\theta ^K \leqslant \lambda v\):. In this case, following similar arguments as in Section 2.2, the policy \(u = (v_0,v_1,\dots)\) provides a revenue rate \(\lambda v\) which is at least as much as the revenue rate provided by any ergodic policy.

Footnote

1
The subsequent analysis can also be carried out with \(\lim _{k \rightarrow \infty } \theta ^k_i\) replaced by \(\lim \inf _{k \rightarrow \infty } \theta ^k_i\) or \(\lim \sup _{k \rightarrow \infty } \theta ^k_i\) as appropriate if the limit does not exist.

References

[1]
Barış Ata and Tava Lennon Olsen. 2013. Congestion-based leadtime quotation and pricing for revenue maximization with heterogeneous customers. Queueing Systems 73, 1 (2013), 35–78.
[2]
Krishna B. Athreya and Soumendra N. Lahiri. 2006. Measure Theory and Probability Theory (1st ed.). Springer Texts in Statistics, Vol. 19. Springer, New York, NY.
[3]
Dimitri P. Bertsekas. 2007. Dynamic Programming and Optimal Control, Vol. 2. Athena Scientific, Belmont.
[4]
Omar Besbes, Adam N. Elmachtoub, and Yunjie Sun. 2022. Static pricing: Universal guarantees for reusable resources. Operations Research 70, 2 (March2022), 1143–1152.
[5]
Christian Borgs, Jennifer T. Chayes, Sherwin Doroudi, Mor Harchol-Balter, and Kuang Xu. 2014. The optimal admission threshold in observable queues with state dependent pricing. Probability in the Engineering and Informational Sciences 28, 1 (2014), 101–119.
[6]
Sabri Çelik and Costis Maglaras. 2008. Dynamic pricing and lead-time quotation for a multiclass make-to-order queue. Management Science 54, 6 (2008), 1132–1146.
[7]
Hong Chen and Murray Z. Frank. 2001. State dependent pricing with a queue. IIE Transactions 33, 10 (2001), 847–860.
[8]
M. Keith Chen and Michael Sheldon. 2016. Dynamic pricing in a labor market: Surge pricing and flexible work on the uber platform. In Proceedings of the 2016 ACM Conference on Economics and Computation. ACM, New York, NY, 455–455.
[9]
R. Divya, Amar Prakash Azad, and Chandramani Singh. 2017. Fair and optimal mobile assisted offloading. In Proceedings of the 2017 15th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks. IEEE, Paris, 1–8.
[10]
Eugene A. Feinberg and Fenghsu Yang. 2016. Optimal pricing for a GI/M/k/N queue with several customer types and holding costs. Queueing Systems 82, 1 (Feb.2016), 103–120.
[11]
Jennifer M. George and J. Michael Harrison. 2001. Dynamic control of a queue with adjustable service rate. Operations Research 49, 5 (Sept.2001), 720–731.
[12]
Geoffrey Grimmett and David Stirzaker. 2001. Probability and Random Processes (3rd ed.). Oxford University Press, Oxford, New York, NY.
[13]
Refael Hassin. 2016. Rational Queueing (1st ed.). CRC Press, New York, NY.
[14]
Refael Hassin and Moshe Haviv. 2003. To Queue or Not to Queue: Equilibrium Behavior in Queueing Systems (1st ed.). International Series in Operations Research & Management Science, Vol. 59. Springer, New York, NY.
[15]
Donna L. Hoffman and Thomas P. Novak. 2000. Advertising pricing models for the world wide web. In Internet Publishing and Beyond: The Economics of Digital Information and Intellectual Property. Brian Kahin and Hal R. Varian (Eds.), MIT Press, Cambridge, MA, Chapter 2, 45–61.
[16]
Jeunghyun Kim and Ramandeep S. Randhawa. 2018. The value of dynamic pricing in large queueing systems. Operations Research 66, 2 (March2018), 409–425.
[17]
Ashok Krishnan K. S., Chandramani Singh, Siva Theja Maguluri, and Parimal Parag. 2022. Optimal pricing in multi server systems. Performance Evaluation 154 (April2022), 102282.
[18]
Lewis Lehe. 2019. Downtown congestion pricing in practice. Transportation Research Part C: Emerging Technologies 100 (March2019), 200–223.
[19]
Chengcheng Liu and John J. Hasenbein. 2019. Naor’s model with heterogeneous customers and arrival rate uncertainty. Operations Research Letters 47, 6 (Nov.2019), 594–600.
[20]
David W. Low. 1974. Optimal dynamic pricing policies for an \(M/M/s\) queue. Operations Research 22, 3 (June1974), 545–561.
[21]
Idriss Maoui, Hayriye Ayhan, and Robert D. Foley. 2007. Congestion-dependent pricing in a stochastic service system. Advances in Applied Probability 39, 4 (Dec.2007), 898–921.
[22]
Pinhas Naor. 1969. The regulation of queue size by levying tolls. Econometrica: Journal of the Econometric Society 37, 1 (Jan.1969), 15–24.
[23]
Ioannis Ch. Paschalidis and John N. Tsitsiklis. 2000. Congestion-dependent pricing of network services. IEEE/ACM Transactions on Networking 8, 2 (April2000), 171–184.
[24]
Walter Rudin. 1976. Principles of Mathematical Analysis (3rd ed.). McGraw-Hill, New York, NY.
[25]
Ashok Krishnan K. S., Chandramani Singh, Siva Theja Maguluri, and Parimal Parag. 2020. Optimal pricing in finite server systems. In Proceedings of the 2020 18th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks. IEEE, Volos, 1–8.
[26]
Richard Serfozo. 2009. Basics of Applied Stochastic Processes (1st ed.). Springer-Verlag, Berlin.
[27]
Xinchang Wang, Sigrún Andradóttir, and Hayriye Ayhan. 2019. Optimal pricing for tandem queues with finite buffers. Queueing Systems 92, 3 (Aug.2019), 323–396.
[28]
Caesar Wu, Rajkumar Buyya, and Kotagiri Ramamohanarao. 2020. Cloud pricing models: Taxonomy, survey, and interdisciplinary challenges. ACM Computing Surveys 52, 6 (Nov.2020), 1–36.
[29]
Hong Xu and Baochun Li. 2013. Dynamic cloud pricing for revenue maximization. IEEE Transactions on Cloud Computing 1, 2 (Dec.2013), 158–171.
[30]
Serhan Ziya, Hayriye Ayhan, and Robert D. Foley. 2006. Optimal prices for finite capacity queueing systems. Operations Research Letters 34, 2 (March2006), 214–218.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Performance Evaluation of Computing Systems
ACM Transactions on Modeling and Performance Evaluation of Computing Systems  Volume 8, Issue 4
December 2023
119 pages
ISSN:2376-3639
EISSN:2376-3647
DOI:10.1145/3609794
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 August 2023
Online AM: 05 July 2023
Accepted: 16 June 2023
Revised: 28 March 2023
Received: 16 November 2022
Published in TOMPECS Volume 8, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Single server queue
  2. service valuation
  3. pricing
  4. Markov decision process

Qualifiers

  • Research-article

Funding Sources

  • CNI
  • Aircel TCoE
  • Indian Institute of Science, Bengaluru
  • NSF
  • Qualcomm 6G University Relations India
  • Qualcomm Inc.
  • IBM India Research Lab
  • Centre for Networked Intelligence

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 385
    Total Downloads
  • Downloads (Last 12 months)257
  • Downloads (Last 6 weeks)63
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media