Optimal strategies for managing complex authentication systems

Abstract

We study an authentication system that receives requests from different types of users. A centralized controller must assign an authentication method to each request, considering the type, the state of the system and the characteristics of several available methods. Each authentication method has different capacity, service rate, level of security, level of usability and operating cost. We seek to optimize security, usability and operating cost, simultaneously by assigning authentication methods dynamically, in real time. To do this, we model the system as a network of parallel multi-server queues, where each queue represents an authentication method and each customer represents a request. We use two different approaches to handle the multiple objectives: a weighted total cost function, and treating security and latency as constraints while minimizing operating cost. We employ constrained and unconstrained Markov decision processes to determine the structure of policies that effectively balance these three objectives. We conclude that if there are infinitely many servers for each authentication method, then the optimal policy is static. We also show that if one method has finite capacity, then the optimal policy is of trunk reservation form. Our results regarding the structure of the optimal policy are consistent for both modeling approaches. Our work shows that optimal policies have intuitive, easy-to-implement structures that are useful in practice. Under certain assumptions, we provide a straightforward way to obtain an optimal policy. We also offer strategies to use our models to explore non-dominated solutions over the three objective functions.

Appendices

Appendix

Proof of Theorem 1

For each request class there are two possibilities for the costs: either $c_{i1}\ge c_{i2}$, or $c_{i1}<c_{i2}$. The following Lemma deals with the first of these.

Lemma 1

For any customer class l such that $c_{l1} \ge c_{l2}$, we will have $\pi ^*(k,l)=2$ for all k under any optimal policy $\pi ^*$.

Proof

Define $\gamma _\pi (k)$ as the long run average fraction of time that there are k customers at Method 1, under policy $\pi$. Note $\gamma _\pi$ is a probability distribution, with support in $k =0,1,\ldots ,M$. Then, without loss of generality, suppose that for all other customer classes $c_{i1} < c_{i2}$. Consider two policies, $\pi$ and $\pi '$, which are identical except that for a certain fixed k, we have $\pi (k,l)=2$ and $\pi '(k,l)=1$. Then each time the system is in state (k, l), policy $\pi '$ incurs an additional cost of $c_{l1} - c_{l2}$, compared to policy $\pi$. Furthermore, if the gatekeeper sends a request from class l to Method 2 the length of the queue will remain unchanged; the next arrival will observe the exact same state as the previous one and the gatekeeper will have the same set of alternatives. However, if the incoming request is sent to Method 1, then the length of the queue increases by 1. It follows, that for any $k'>k$, we will have $\gamma _{\pi '} (k') \ge \gamma _\pi (k')$. Hence, $\gamma _{\pi '} (M) \ge \gamma _\pi (M)$, so under $\pi$, customers of types different from l will be sent to Method 2 more frequently (as this is the only option when Method 1 is full) and this will result in additional costs of $c_{i2} - c_{i1}$ each time it happens. So $\pi$ incurs additional costs when assigning type-l customers as well as non-type-l customers, and never achieves savings, compared to $\pi '$. Therefore, we conclude $\pi ^*(k,l)=2$ under any optimal policy. Because k was arbitrary, it follows this is true for all k. $\square$

In light of Lemma 1, from now on we assume $c_{i1}<c_{i2}$ for all i. If this is not the case, we can adopt a policy that always assigns requests from classes where this does not hold to Method 2, and solve the problem for the remaining request classes.

For any unichain MDP there exists an optimal stationary deterministic policy $\pi ^*$ that satisfies the following optimality equations (see Theorem 8.4.4 of Puterman 1994):

$$\begin{aligned} w_{\pi ^*}(s)+g_{\pi ^*} = \max _{a\in \mathcal {A}_s}\left\{ r(s,a)+\sum _{j\in \mathcal {S}}p(j|s,a)w_{\pi ^*}(s)\right\} \ \ \forall s\in \mathcal {S} \end{aligned}$$

where $w_{\pi ^*}$ is called the bias vector under policy $\pi ^*$. A stationary and deterministic policy $\pi$ consists of a single decision rule, so here we use $\pi (s)$ to mean the decision rule for state s under policy $\pi$. The bias vector $w_{\pi }$, for a given stationary and deterministic policy $\pi$, is defined component-wise as:

$$\begin{aligned} w_{\pi }(s) =\mathbb {E}_{\pi }\left[ \sum _{n=0}^\infty \left( r(s_n,\pi (s_n))- g_{\pi }\right) |s_0=s\right] . \end{aligned}$$

We are only concerned with the states where more than a single action is available. Let us re-write the optimality equations for states (k, i) such that $k \in \{0,1,2,\ldots ,M-1\},i\in \{1,2\}$) as:

$$\begin{aligned} w_{\pi ^*}(k,i)+g_{\pi ^*} = \max \left\{ -c_{i1} + U_{\pi ^*}(k+1), -c_{i2} + U_{\pi ^*}(k) \right\} \end{aligned}$$

(18)

where the function $U_{\pi }(k)$, for a given policy $\pi$, is defined as:

$$\begin{aligned} U_{\pi }(k) = \mu _{1,k} w_{\pi }((k-1),0) + \sum _{i=1}^I \lambda _i w_{\pi }(k,i)+(m-k)^+\mu _1 w_{\pi }(k,0), \end{aligned}$$

with $(m-k)^+=\max (m-k,0)$. We observe that this model is analogous to an admission control problem presented in Lewis et al. (1999). The following two lemmas follow directly from the proofs of Lemmas 3 and 4 in Lewis et al. (1999), so we state them without proof and refer the reader to that work.

Lemma 2

Under any optimal policy $\pi ^*$, $U_{\pi ^*}(k)$ is strictly decreasing, that is

$$\begin{aligned} U_{\pi ^*}(k+1)-U_{\pi ^*}(k) <0 \ \ \forall k\in (0,1,2,\ldots ,M-1). \end{aligned}$$

Lemma 3

Under any optimal policy $\pi ^*$, $U_{\pi ^*}(k)$ is strictly concave, that is

$$\begin{aligned} (U_{\pi ^*}(k+1)-U_{\pi ^*}(k))-(U_{\pi ^*}(k)-U_{\pi ^*}(k-1)) <0 \ \ \forall k\in (1,2,\ldots ,M-1). \end{aligned}$$

Finally, suppose that $\pi ^*$ is an optimal policy for which the optimality equations hold. Because this is a unichain model with finite state and action spaces such a policy is known to exist. Choose an arbitrary customer class i, and let $k_i^*$ be the smallest k such that $\pi ^*(k,i)=2$. The optimality equations hold, so

$$\begin{aligned} \left( c_{i2}-c_{i1}\right) +\left( U_{\pi ^*}(k_i^*+1)-U_{\pi ^*}(k_i^*)\right) \le 0. \end{aligned}$$

By the concavity of $U_{\pi ^*}$ it follows that

$$\begin{aligned} \left( c_{i2}-c_{i1}\right) +\left( U_{\pi ^*}(k+1)-U_{\pi ^*}(k)\right) \le 0 \end{aligned}$$

for all $k>k_i^*$. Therefore $\pi ^*(k,i)=2$ for all $k>k_i^*$. Since we chose i arbitrarily, the result holds for each class i.

This shows that the optimal policy conforms to part 2 of Definition 1. Now, consider a class i and a k such that $\pi ^*(k,i)=2$. Take another class l, such that

$$\begin{aligned} c_{i2}-c_{i1}>c_{l2}-c_{l1} \end{aligned}$$

The optimality equations hold, so

$$\begin{aligned} \left( c_{i2}-c_{i1}\right) +\left( U_{\pi ^*}(k+1)-U_{\pi ^*}(k)\right) \le 0. \end{aligned}$$

This implies that

$$\begin{aligned} \left( c_{l2}-c_{l1}\right) +\left( U_{\pi ^*}(k+1)-U_{\pi ^*}(k)\right) \le 0. \end{aligned}$$

Hence the thresholds $k_i^*$ are ordered by $(c_{i2}-c_{i1})$.

Finally, we need to show that there exists a request class that is always assigned to Method 1. Let class i be the class with highest value of $(c_{i2}-c_{i1})$. Consider two systems, I and II under policies $\pi _I$ and $\pi _{II}$, which are identical, except that $k_i^*=M$ for policy $\pi _I$ and $k_i^*=M-1$ for policy $\pi _{II}$. By the arguments above, we have $k_j^*<k_i^*$ for every $j \ne i$ in both policies. Then, starting from the same state and observing the same events, the two systems will be coupled until they reach state $(M-1,i)$, here System II incurs an additional cost of $c_{i2} - c_{i1}$, compared to System I. From this point on, the two systems will continue to observe the same events, take the same actions and incur the same costs (with System I having 1 additional customer in the queue) until one of the two following happens:

• 1. System I has 1 customer, System II has 0 and the next event is a service completion, then the systems become coupled again with System II having incurred an additional $c_{i2} - c_{i1}>0$.

• 2. System I enters state $(k_j^*,j)$ (for some arbitrary $j \ne i$) and System II enters state $(k_j^*-1,j)$. Then System I sends the request to Method 2, while System II sends it to Method 1 and the two systems become coupled again, with System II having incurred an additional cost equal to $(c_{i2} - c_{i1})-(c_{j2} - c_{j1})$, which is positive, by the assumption that class i has the highest value of $(c_{i2}-c_{i1})$.

After either of these events the two systems become coupled again and evolve identically until they reach state $(M-1,i)$ again. Clearly, System II incurs more cost than System I each time the sample paths are de-coupled. Hence, policy $\pi _I$ outperforms $\pi _{II}$. By the same argument, it follows that setting $k_i^*=M-1$ outperforms setting $k_i^*=M-2$, and so on until we hit the second highest threshold. Therefore, we conclude that any optimal policy of the form described above must have $k_i^*=M$.

Proof of Proposition 1

Consider the arrivals of all requests, which by assumption follow a Poisson process with rate $\varLambda$. Each arrival collects a reward which is i.i.d. We consider costs as negative rewards, as in previous sections. Therefore, we have a Poisson process, where each arrival has bounded i.i.d. rewards. This is a renewal-reward process. For this type of process it is known (see, for example, Proposition 3.4.1 in Resnick 2013) that the long run average reward is equal to the expectation of a single reward, that is:

$$\begin{aligned} \mathbb {E}^\pi \left[ -\lim _{t \rightarrow \infty }\frac{1}{N(t)}\sum _{n=0}^{N(t)} \sum _{j=1}^{J} c_j \sum _{i=1}^{I} \mathbb {I}\{I_n=i,A_n=j\}\right] =-\sum _{j=1}^{J} c_{j} \sum _{i=1}^{I}\lambda _i q_{ij}. \end{aligned}$$

(19)

By the same argument the long run average fraction Type-I errors is equal to the probability that a single arrival will result in a Type-I error; and similarly for Type-II errors as well as the average latency cost.

The problem of determining an optimal policy for (SP1) consists of determining probabilities $q_{ij},\forall i\in \{1,\ldots ,I\},j\in \{1,\ldots ,J\}$, where $q_{ij}$ is a decision variable, representing the probability of assigning each incoming request from class i to authentication method j.

Since the objective (the reward) and the constraints (fraction of errors, latency cost) of the optimization problem are expressed in terms of expectations, and the state and action spaces are finite, those expectations can be expressed as finite sums. For example, the objective is given by (19). Since the objective and the constraints are all finite linear sums, by adding the normalization and non-negativity we can express the stochastic optimization problem (SP1) as (LP1). So a solution to (LP1) determines an optimal policy for (SP1).

Proof of Proposition 2

We proceed by contradiction. Suppose there exists another policy $\pi '$ that is feasible for (CMDP) and performs better than $\pi ^*$. Then there exists a pair $(y_{\pi '},\gamma _{\pi '})$ such that (6), (7), (8), (10), (11) and (12) hold. Set $x'(k,i)=y_{\pi '}(k,i)\gamma _{\pi '}(k,i)$ for each (k, i). Then the pair $(x',\gamma _{\pi '})$ is feasible for (LP2) and performs better than $(x^*,\gamma ^*)$. This contradicts the optimality of $(x^*,\gamma ^*)$. So $\pi ^*$ is optimal for (CMDP).

Proof of Theorem 2

Let us assume that (LP3) is feasible for the given parameters. Note that (LP3) is bounded, so if it is feasible it has an optimum. Consider an arbitrary optimal solution to the dual of (LP3) and let $\nu _1, \nu _2$ be the dual variables associated with the two error constraints (13) and (14). We can write a Lagrangian relaxation of (LP3) by dualising those two constraints with their corresponding Lagrange multipliers. The objective function of the relaxation would be:

$$\begin{aligned}&r \sum _{i=1}^{I} \lambda _{i} \sum _{k=0}^{M-1}x(k,i) -\nu _1\left( \sum _{i=1}^{I} \lambda _i\left[ \alpha _{i2} \left( 1-\sum _{k=0}^{M-1}x(k,i)\right) +\alpha _{i1}\sum _{k=0}^{M-1}x(k,i)\right] - \bar{\alpha }\right) \\&\quad -\nu _2\left( \sum _{i=1}^{I} \lambda _i \left[ \beta _{i2} \left( 1-\sum _{k=0}^{M-1}x(k,i)\right) +\beta _{i1}\sum _{k=0}^{M-1}x(k,i)\right] -\bar{\beta }\right) . \end{aligned}$$

Define an adjusted reward for each customer class as:

$$\begin{aligned} r_i':=(c_2-c_1)+\nu _1(\alpha _{i2}-\alpha _{i1})+\nu _2(\beta _{i2}-\beta _{i1}). \end{aligned}$$

Note that, under our assumptions, each term in this expression is non-negative, so the adjusted rewards are non-negative. Furthermore, each $r_i'$ is different, because by assumption the impostor probabilities of each class are different. Reordering some terms in the objective function we get the following Lagrangean relaxation of (LP3):

$$\begin{aligned} (LR1): \quad&\max \left\{ \sum _{i=1}^{I} \lambda _{i} r_i' \sum _{k=0}^{M-1}x(k,i) -\nu _1\left( \sum _{i=1}^{I}\lambda _i \alpha _{i2}- \bar{\alpha }\right) -\nu _2\left( \sum _{i=1}^{I} \lambda _i \beta _{i2} - \bar{\beta }\right) \right\} \\&\text {subject to}\\&\sum _{i=1}^I \lambda _i x(k,i) = \mu _{1,k+1}\gamma (k+1) \quad \forall k=0,1,\ldots ,M-1\\&\sum _{k=0}^{M} \gamma (k) =1 \\&0 \le x(k,i) \le \gamma (k) \quad \forall i=1,\ldots ,I, \, k=0,1,\ldots ,M-1. \end{aligned}$$

Note that (LR1) is also a linear program. The following lemma follows immediately from the theory of Linear Programming, specifically that any basic optimal solution must meet the KKT conditions for the corresponding problem, and the KKT conditions of (LR1) are a subset of those for (LP3) (see for example Chapter 12 in Nocedal and Wright 2006).

Lemma 4

Any optimal basic solution for (LP3) is optimal for (LR1) and has the same objective function value.

Note that the solution $x_0=\{x(i,k)=0, \forall i,k, \, \gamma (0)=1 \; \gamma (k)=0,\forall k\ge 1\}$ is always feasible for (LR1). Assuming that (LP3) is feasible then $x_0$ is a basic feasible solution for (LP3). It is obvious that if $r_i'\le 0$$\forall i\in \{1,..,I\}$, then $x_0$ is optimal for (LR1). Therefore, Lemma 4 implies that $x_0$ is an optimal solution to (LP3). So, from now on we assume that there exists at least one $r_i'>0$.

In order to arrive at the main conclusion for this section we need some intermediate results. The following lemmas follow directly from previous work, so we present them without proof and give only the corresponding reference.

Lemma 5

If $(x^*,\gamma ^*)$ is optimal for (LR1) then $\gamma ^*(k)>0, \, \forall k=0,1,\ldots ,M-1$.

For a proof we refer the reader to the proof of Lemma 3.4 in Fan-Orzechowski and Feinberg (2006).

Lemma 6

Taking any basic optimal solution to (LP3), $(x^*,\gamma ^*)$ and setting

$$\begin{aligned} y_{\pi ^*}(k,i)=\frac{x^*(k,i)}{\gamma ^*(k)} \end{aligned}$$

results in a 2-randomized stationary optimal policy for $(CMDP')$.

The proof of Lemma 6 follows from the proof of part (iii) of Theorem 2.1 in Fan-Orzechowski and Feinberg (2007).

Lemma 7

Consider any optimal solution $(x^*,\gamma ^*)$ to (LR1), and define a randomized stationary optimal policy $\pi$ as $y_{\pi }(k,i)=\frac{x^*(k,i)}{\gamma ^*(k)}$. Then for (UMDP) with modified rewards $r((k,i),1)=r_i' \; \forall k=0,1,\ldots ,M-1, \; i=1,\ldots ,I$ and 0 otherwise, we have the following:

• 1. For any i, l, such that $r_i' > r_l'$ ,

  $$\begin{aligned} y_{\pi }(k, i) \ge y_{\pi }(k,l) \quad \forall k = 0,\ldots ,M-1, \quad i,l=1,2,\ldots ,I. \end{aligned}$$

• 2. For each $k = 0,\ldots ,M-1$, all the probabilities $y_{\pi }(k, i), \; i=1,\ldots ,I$ except at most one, are equal to either 0 or 1.

• 3. For a request type l such that $r_l'=\max _{i}\{r_i'\}$ we have $y_\pi (k,l)=1, \; \forall k=0,1,\ldots ,M-1$.

• 4.
  $$\begin{aligned} y_{\pi }(k, i) \ge y_{\pi }(k+1,i) \quad \forall k = 0,\ldots ,M-1, \quad i=1,2,\ldots ,I. \end{aligned}$$

  and for each $i=1,\ldots ,I$ all the probabilities $y_{\pi }(k, i), \; k=0,1,\ldots ,M-1$ except at most one, are equal to either 0 or 1.

This implies that $\pi$ is a randomized trunk reservation policy, where the decision will be randomized in at most $(I-1)$ states.

The previous lemma follows directly from the proof of Theorem 3.1 in Feinberg and Reiman (1994). We refer the reader to that reference for a proof. A variation on this result is presented in both Fan-Orzechowski and Feinberg (2006) and Fan-Orzechowski and Feinberg (2007).

Now, we can proceed to prove Theorem 2. We can see that $\pi ^*$ is properly defined, because of Lemma 5. Then, from Proposition 2 and Lemma 6 we have that any optimal basic solution of (LP3) produces a 2-randomized policy for the $(CMDP')$. By Lemma 4, $(x^*,\gamma ^*)$ is also optimal for (LR1). By Lemma 7 any optimal solution to (LR1) defines an $(I-1)$-randomized trunk reservation policy ordered by its rewards $r_i'$ for the (UMDP) with the adjusted rewards. Then, putting it all together we get that $\pi ^*$ is an optimal 2-randomized trunk reservation policy for $(CMDP')$, which is ordered by the adjusted rewards $r_i'$. Finally, an argument identical to the one presented at the end of A shows that there is a request class for which any optimal policy must always admit.