Efficient Numerical Frameworks for Multi-objective Cyber Security Planning

Abstract

We consider the problem of optimal investment in cyber-security by an enterprise. Optimality is measured with respect to the overall (1) monetary cost of implementation, (2) negative side-effects of cyber-security controls (indirect costs), and (3) mitigation of the cyber-security risk. We consider “passive” and “reactive” threats, the former representing the case where attack attempts are independent of the defender’s plan, the latter, where attackers can adapt and react to an implemented cyber-security defense. Moreover, we model in three different ways the combined effect of multiple cyber-security controls, depending on their degree of complementarity and correlation. We also consider multi-stage attacks and the potential correlations in the success of different stages. First, we formalize the problem as a non-linear multi-objective integer programming. We then convert them into Mixed Integer Linear Programs (MILP) that very efficiently solve for the exact Pareto-optimal solutions even when the number of available controls is large. In our case study, we consider 27 of the most typical security controls, each with multiple intensity levels of implementation, and 37 common vulnerabilities facing a typical SME. We compare our findings against expert-recommended critical controls. We then investigate the effect of the security models on the resulting optimal plan and contrast the merits of different security metrics. In particular, we show the superior robustness of the security measures based on the “reactive” threat model, and the significance of the hitherto overlooked role of correlations.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

A cyber-security plan is a set of defensive measures (a.k.a., controls) that are applied across an enterprise to improve its overall state of security. There are many cyber-security measures to choose from, and each measure can be implemented at multiple levels of intensity. Examples of these security controls (taken from the list of top-20 critical measures by the UK’s Centre for the Protection of National Infrastructure [22]) include: “Inventory of Authorized and Unauthorized Devices”, “Inventory of Authorized and Unauthorized Software”, “Secure Configurations for Hardware and Software on Mobile Devices, Laptops, Workstations, and Servers”, “Malware Defenses”, “Wireless Access Control”, and so on. Each cyber-security measure addresses a specific set of vulnerabilities. For instance, while “Access Control” can mitigate “OS Command Injection”, it has no effect on “DDoS attacks”. Hence a cyber-security plan should be composed of a combination of the measures to provide a well-rounded defense against the range of vulnerabilities that the enterprise faces.

Implementation of each cyber-security measure is not cost-free: it requires monetary investment (direct costs) and can also negatively affect the performance of an enterprise (indirect costs). Therefore, an exhaustive implementation of controls at maximum intensity is likely neither economically feasible nor managerially desirable. In reality, organizations have to deal with cyber-security risk within a limited budget and must be wary of the potential side-effects of the security measures on their existing business processes. Therefore, the mitigation in the security risks has to be judiciously balanced with the direct and indirect costs. A selection analysis should consider the set of controls as well as vulnerabilities jointly. This is because an approach that takes investment decisions for each vulnerability or control separately, ignores the relative importance of the vulnerabilities, and does not optimally use the complementary effects of the controls, and hence, may fail to reach a best overall trade-off. Choosing a desirability metric for a plan is itself a challenging task:

• 1. The three sources of costs (security, direct and indirect) are not easily combinable. For instance, the investment costs are incurred deterministically and at the present, while the security losses are probabilistic in nature and, if at all, will occur at an unknown future time. Also the monetary conversion is not as clear for indirect costs as for the other two, for instance, it is hard to put a monetary value on the annoyance felt by the staff as a result of a more restrictive access control or a stricter password policy.

• 2. The trade-off preferences cannot be exactly arbitrated “a priori”. For instance, even a “security-concerned” enterprise may choose a different plan if “almost” the same security risk mitigation can be achieved at a much lower direct or indirect cost. Likewise, an enterprise that is very sensitive to indirect costs or extra investment may reconsider if a slight increase in these costs can abate the security risk by a relatively significant amount.

To address these issues we adopt a multi-objective optimization framework. Specifically, we simultaneously minimize the security risk, indirect, and direct costs of the enterprise (the latter within the budget). The “solution” of this three-objective optimization is the set of Pareto-optimal (or non-inferior, or simply, Pareto) plans, that are the solutions with the guarantee that no other plan can simultaneously improve all of these three costs (at least one of them strictly).

Of these three costs, the security risk is the most challenging to model. The effect of an individual security measure (at each implementation intensity) can be represented by its “effectiveness” against different vulnerabilities. That is, the reduction in the success probability of exploitation attempts of each vulnerability when only that control is implemented (stand-alone). Complicating the matter is the fact that, often, the same vulnerability can be (partially) mitigated by more than one security measure. Then a modeling question is how to capture the combined efficacy of controls on their overlapping vulnerabilities.

The simplest approach is an “additive” model, where it is assumed that, per each vulnerability, the (blocking) efficacies of controls are added up, heeding that, logically, none of the overall blocking probabilities should exceed 100 %. This capping of the combined efficacies introduces a degree of non-linearity in the model, but one that can be easily dealt with, as we will show later.

Although computationally the simplest, this model bears the underlying assumption that defensive mechanisms have positive externalities on each others’ efficacies. In particular, it potentially allows 100 % efficacy when multiple controls are combined, which is rather unrealistic. A more relaxed modeling assumption is that each control affects the common vulnerabilities independently. Hence, when a vulnerability is attempted, the success chance is the product of successfully bypassing each of its pertaining controls. We will thus refer to this model as “multiplicative”. This model is ostensibly nonlinear in the decision variables, and hence, solving the resulting nonlinear integer program accurately can be very inefficient. However, as we will see later, it can be converted into a Mixed Integer Linear Program (MILP) which is much more efficient to solve accurately.

A problem with the previous two models is that they ignore the possible correlations in the defensive mechanisms of security measures. Due to such correlations, it can be argued that if an attempted exploitation bypasses one of the controls, it will be a strong indication for bypassing the other affecting measures as well. The “independent” blocking probabilities in the heart of multiplicative model, although better than the additive model, can still be a significant over-estimation of the overall effectiveness of a security plan. In this paper, we introduce a novel non-linear model, which we call “best-of”, that captures such correlations. In particular, the combined effectiveness of implemented controls on a common vulnerability is taken to be (only) the highest effectiveness among them. We then develop a technique to convert the resulting nonlinear integer program into a MILP that is surprisingly quite efficient to solve.

Another challenge in modeling the security losses is anticipating the distribution of exploitation attempts across vulnerabilities. One approach is to use the histogram of the past attempts (retrieved from the logs of the enterprise itself or of any similars), or the publicly available statistics of attacks (e.g. [14]). We will refer to this model as the “passive” threat. In reality, the distribution of the attempts may adapt to the implementation of security controls: if a vulnerability is now well-mitigated, then the attempts may shift to other less protected ones. We will refer to this case as “reactive” threat, and establish a connection with a sequential game between the enterprise and attackers. For both passive and reactive cases, we provide methods to solve for the exact Pareto-optimal plans efficiently by converting the nonlinear optimizations into appropriate MILPs.

Finally, we will present a case study and numerical evaluations using our frameworks and a database of major security controls and vulnerabilities. We first compare the derived optimal plans of each model against the expert recommended list of critical controls, which reveals a general consistency, with the best match observed for the “best-of”–“reactive” model. Subsequently, we compare the optimal plans as well as the achieved utilities across our different security risk models. In particular, we observe that the “reactive” threat provides a more robust (and hence more favorable) notion of security risk in the sense that, optimization with respect to reactive threat does not lead to a terribly sub-optimal performance with respect to passive threat, however, the opposite is not true: an optimal plan with respect to passive threat can lead to terrible performance with respect to reactive threat, even for relatively high values of investment budget.

Contributions and Related Works. The main contributions of this work are:

• By reducing the model to MILP we make it possible to compute optimal solutions for cyber-security: the state space we consider in our case study is enormous, of the order of $10^{14}$ possible plans, and our MILP finds the optimal solution in seconds. The closest work in the cyber-security literature [24] takes instead days to converge and crucially lacks a guarantee of optimality.

• Our case study represents the largest cyber-security modeling to date. The data used in the experiments has been extracted from official government organizations’ publications like [4, 5, 17] as well as the publicly available databases of CVE, CWE and CWSS.

Quantitative risk assessment and mitigation in cyber-security has been a thematic topic of research in security, that has in part lead to established methodologies such as Magerit and NIST800-30 among others [21]. Works that explicitly investigate the problem of investment portfolios in cyber-security include [1–3, 7, 8, 11–13, 15, 16, 18–20, 24]. Compared with these references, our work presents a wider modeling framework both in terms of the way controls can be combined (additive, multiplicative, best-of) and in terms of the attacker capabilities and threat types (passive, reactive). Also of the above works only [13, 16, 24] are based on real world data and only [13, 24] model indirect costs. Compared with these last two works their solutions are based on Tabu Search (TS) and genetic algorithms (GA) respectively, and are inherently more inefficient than the solutions here presented and they do not provide any guarantee of optimality within their framework. Also issues like robustness are largely neglected.

2 Modeling and Notations

Let $\mathcal {C}$ represent the set of (cyber-security) controls, each with potentially multiple intensity levels of implementation. We will use $\mathcal {L}_c=\{1,\ldots ,L_c\}$ to denote the set of available implementation levels of control c. A cyber-security plan or a cyber-security investment portfolio $\varvec{x}=(x_c)$ is a vector in $\mathcal {X}:=\times _{c\in \mathcal {C}}(\{0\}\cup \mathcal {L}_c)$, where $x_c=l\in \{0\}\cup \mathcal {L}_c$ represents the decision to implement control c at level l, with zero representing the lack of implementation of that control.

Let $B\in \mathbb {R}^+$ be the (hard) constraint on the total cyber-security budget of the enterprise. Let $D,I,R:\mathcal {X}\rightarrow \mathbb {R}^+$ respectively denote the (total) direct cost, (total) indirect cost, and the (aggregate) “security risk” of the enterprise given a security plan. As we proceed, we explicitly describe each of these functions. But first, we give a high-level description of the problem of cyber-security investment as a (constrained) multi-objective integer programming:

$$\begin{aligned} \min _{\varvec{x}\in \mathcal {X}}\left( D(\varvec{x}), I(\varvec{x}), R(\varvec{x})\right)&&\text {s.t.:}\ D(\varvec{x})\le B \end{aligned}$$

(1)

Let $d_{c}(l)\in \mathbb {R}^+$ be the direct cost of implementing control $c\in \mathcal {C}$ at level $l\in \{0\}\cup \mathcal {L}_c$, with the obvious convention that $d_{c}(0)=0$. The direct cost is a combination of the (one-time) investment (for obtaining the required hardware, software or staff), and the recurrent monetary expenses associated with the implementation. For controls that are already in place, i.e., existing controls, only the recurrent expenses must be considered. Similarly, let $i_{c}(l)\in \mathbb {R}^+$ be the indirect cost of implementing control $c\in \mathcal {C}$ at level $l\in \{0\}\cup \mathcal {L}_c$, where $i_{c}(0)=0$. The indirect costs are those related to reduced performance (due to introduced overhead on resources), lowered morale (e.g. due to restricting access, false positives, stricter password policies), etc., that are not easily convertible to monetary losses. Using these notations, we simply have:

$$\begin{aligned} D(\varvec{x})=\sum _{c\in \mathcal {C}}d_{c}(x_c),&&I(\varvec{x})=\sum _{c\in \mathcal {C}}i_{c}(x_c) \end{aligned}$$

(2)

We will denote the set of vulnerabilities of the enterprise by $\mathcal {V}$. Let $e_{cv}(l)$ be the stand-alone effectiveness of control c at implementation level $l\in \{0\}\cup \mathcal {L}_c$ on vulnerability v, that is, $e_{cv}(l)$ is the probability that an exploitation attempt on vulnerability v is blocked when “only” control c at implementation level l is present. Then $s_{cv}(l):=1-e_{cv}(l)$ will represent the success probability of an attempt at exploitation of vulnerability v when no other control than c at level l is implemented. Trivially, $e_{cv}(0)=0$ $\forall c\in \mathcal {C}$ and $\forall v\in \mathcal {V}$.

Let $\mathcal {C}_v$ be the set of controls that can affect vulnerability v, i.e., $\mathcal {C}_v:=\{c\in \mathcal {C}: e_{cv}(l)>0 \text { for some}\ l\in \mathcal {L}_c\}$. If for a vulnerability v, we have $\Vert \mathcal {C}_v\Vert >1$, then the combined effectiveness of the controls on v needs to be modeled. In particular, let $S_v:\mathcal {X}\rightarrow [0, 1]$ represent the success probability of an exploitation attempt on vulnerability $v\in \mathcal {V}$ given a cyber-security plan. We provide three different candidates for $S_v(\varvec{x})$, in decreasing order of “complementary” effects among the defensive mechanisms of the controls (using the convention: $a^+:=\max \{a,0\}$):

$$\begin{aligned} \text {Additive:}&&S_v(\varvec{x})=\big (1-\sum _{c\in \mathcal {C}_v}e_{cv}(x_c)\big )^+ \end{aligned}$$

(3)

$$\begin{aligned} \text {Multiplicative:}&&S_v(\varvec{x})=\prod _{c\in \mathcal {C}_v}s_{cv}(x_c) \end{aligned}$$

(4)

$$\begin{aligned} \text {Best-of:}&&S_v(\varvec{x})=\min _{c\in \mathcal {C}_v}s_{cv}(x_c) \end{aligned}$$

(5)

Let $\varLambda _{v}$ be the random variable representing the losses to the enterprise when vulnerability $v\in \mathcal {V}$ is “successfully” exploited, and let ${\lambda }_{v}$ be its expected value. These losses are due to the interruption in availability, integrity and/or confidentiality of data assets or services of the enterprise (e.g. tampering or theft of intellectual property, financial or client data, disruption of operations, etc.) as well as the secondary causes of losses such as reputation damage, loss of clients, legal fees, and so on.¹ We assume a “risk-neutral” decision-maker, and hence take the expected value of losses due to successful exploitations to be the measure of the security risk. In order to represent the expected losses, we need to anticipate the rate with which different vulnerabilities will be target of exploitation. This rate may depend on the profile of the enterprise and may also change in the face of the implemented security plan. Let $\pi :\mathcal {X}\rightarrow \varDelta (\mathcal {V})$ represent this relation, where $\varDelta (\mathcal {V})$ represents the set of all probability distributions over the set of vulnerabilities $\mathcal {V}$. In particular, let $\pi (v;\varvec{x})$ be the rate at which vulnerability $v\in \mathcal {V}$ is attempted, given that the implemented plan is $\varvec{x}$. Then the security risk of the (risk-neutral) enterprise in (1) can be written as:

$$\begin{aligned} R(\varvec{x}) = \sum _{v\in \mathcal {V}} \pi (v;\varvec{x}) S_{v}(\varvec{x}) \lambda _v \end{aligned}$$

(6)

Modeling $\pi$ requires anticipating the behavior of the attackers. In what follows, we consider two models for this behavior: “passive” and “reactive” threats.

Passive Threat. In this model, the probability distribution of the attacks is assumed given and that it “stays unchanged” irrespective of the implemented plan. In particular, let $\mathbf {P}\in \varDelta \mathcal {V}$ be the distribution of attempts across vulnerabilities, and we have $\pi (v;\varvec{x})=\mathbf {P}(v)$, $\forall \varvec{x}\in \mathcal {X}$. Then the expected loss (as the risk-neutral measure of security risk) is:

$$\begin{aligned} R(\varvec{x})=\sum _{v\in \mathcal {V}}\mathbf {P}(v) S_v(\varvec{x}) \lambda _v \end{aligned}$$

(7)

where $S_v(\varvec{x})$ comes from (3), (4) or (5), depending on the combination model.

Reactive Threat. As we mentioned, the distribution of exploitation attempts on vulnerabilities may evolve in the face of the new implemented security plan. In particular, the attempts on well-protected vulnerabilities may shift to less protected vulnerabilities. The most pessimistic scenario is the assumption that the attempts will shift to a vulnerability that has the most “effective impact”, i.e., in (6): $\sum _{v\in \arg \max (S_v(\varvec{x}) \lambda _v)}\pi (v;\varvec{x})=1$. Then, the corresponding expected loss (as the risk-neutral measure of security risk) is:

$$\begin{aligned} R(\varvec{x})=\max _{v\in \mathcal {V}} \left( S_v(\varvec{x}) \lambda _v\right) \end{aligned}$$

(8)

Next, we show that this notion of security is closely related to a sequential game.

Connection to Game Theory. Consider the following non-zero-sum sequential two-player game of “perfect information”:

Players: The enterprise ‘e’ (the leader), and the attacker ‘a’ (the follower).

Action spaces: The action of the enterprise is its cyber-security investment plan, $\varvec{x}$. The attacker decides on which one of the vulnerabilities to try to exploit (if any). This can be represented by an indicator $\varvec{y}$. Hence, the action spaces are respectively $\mathcal {X}$ and $\mathcal {Y}:=\{\varvec{y}\in \{0,1\}^\mathcal {V}:\sum _{v\in \mathcal {V}}\varvec{y}(v)\le 1\}$. The enterprise also has a constraint, defining its set of feasible actions: the total direct cost of its action has to be within its budget, which the attacker may not know the value of.

Information structure & strategies: The enterprise (the leader) makes the first “move”, and its action and strategy spaces coincide. The attacker (the follower) observes the “move” of the enterprise $\varvec{x}$ (hence the label: “perfect information”) and, after re-assessing the effectiveness of attempts on each of the vulnerabilities, makes its decision of which one to attempt. Hence, a strategy of the attacker, denoted by $\sigma$, is a function $\sigma :\mathcal {X}\rightarrow \mathcal {Y}$. We will use the notation: $\sigma _v(\varvec{x})$ to represent $\sigma (v;\varvec{x})$, that is, $\sigma _v(\varvec{x})\in \{0,1\}$ $\forall v\in \mathcal {V}$, and $\sum _{v\in \mathcal {V}}\sigma _v(\varvec{x}) \le 1$.

Payoffs. The negative payoff of the enterprise (which it wants to minimize) is a weighted sum of the three costs. Specifically, let $w_d$, $w_i$, and $w_r$ be the weights of the (total) direct and indirect costs and the security damage to the enterprise, respectively, where $w_d,w_i\ge 0$, and $w_r>0$. Referring to (2) and (6), the expected cost of the enterprise $u_e:\mathcal {X}\times \mathcal {Y}\rightarrow \mathbb {R}^+$ is therefore: $u_e(\varvec{x},\sigma (\varvec{x}))= w_d\sum _{c\in \mathcal {C}}d_c(x_c)+w_i\sum _{c\in \mathcal {C}} i_c(x_c)+w_r\sum _{v\in \mathcal {V}}\sigma _v(\varvec{x})S_v(\varvec{x})\lambda _v$. The payoff of the attacker (which it wants to maximize) is (linearly) proportional to the expected security losses of the enterprise due to successful exploitations. In particular, letting $u_a:\mathcal {X}\times \mathcal {Y}\rightarrow \mathbb {R}^+$ represent the expected payoff of the attacker, we can write: $u_a(\varvec{x},\sigma (\varvec{x}))=w'_r\sum _{v\in \mathcal {V}}\sigma _v(\varvec{x})S_v(\varvec{x})\lambda _v$, for some $w'_r>0$, whose exact value may not be known to the enterprise. Note that we assumed exploitation attempts are costless for the attacker. We have the following result:

Proposition 1

Any strategy of the enterprise in a Subgame Perfect Nash Equilibrium (SPNE) of the above non-zero-sum sequential two player game with “perfect information” is a Pareto-optimal solution to the multi-objective problem of (1) where the security cost is according to the “reactive threat” model in (8).

Proof

Denoting the attacker’s best response correspondence by $\sigma ^*$, we have:

$$\begin{aligned} \sigma ^*(\varvec{x})\in \arg \max _{\sigma } w'_r\sum _{v\in \mathcal {V}}\sigma _v(\varvec{x})S_v(\varvec{x})\lambda _v, \end{aligned}$$

which implies $\sum _{v\in \mathcal {V}}\sigma ^*_v(\varvec{x})S_v(\varvec{x})\lambda _v=\max _{v\in \mathcal {V}}\left( S_v(\varvec{x})\lambda _v\right)$. Now, using backward induction (for subgame perfection), the problem of the enterprise becomes:

$$\displaystyle \min _{\varvec{x}\in \mathcal {X}} \left[ w_d\sum _{c\in \mathcal {C}}d_c(x_c)+w_i\sum _{c\in \mathcal {C}} i_c(x_c) +w_r\max _{v\in \mathcal {V}}\left( S_v(\varvec{x})\lambda _v\right) \right] ,\quad \mathbf{s.t. } \sum _{c\in \mathcal {C}}d_c(x_c)\le B.$$

Finally, any solution of the above single optimization is also a Pareto-optimal solution of the multi-objective problem in (8).   $\square$

It is worthwhile to note that the set of SPNE stays the same even if the game is converted to a zero-sum game in which the payoff of the attacker (to be maximized) is exactly the same as the total cost of the defender, i.e., if $u_a(\varvec{x},\sigma (\varvec{x}))=u_e(\varvec{x},\sigma (\varvec{x}))= w_d\sum _{c\in \mathcal {C}}d_c(x_c)+w_i\sum _{c\in \mathcal {C}} i_c(x_c) +w_r\sum _{v\in \mathcal {V}}\sigma _v(\varvec{x})S_v(\varvec{x})\lambda _v$. That is, if the attacker wanted to also maximize the investment and indirect costs of the defender, the optimization problem of the enterprise would not change at all. To see this, note that once the enterprise makes its implementation decision, the attacker cannot affect either the direct or indirect costs of the enterprise. Interestingly, this still holds even if the attacker has its own weights on different components of its overall payoff, i.e., if $u_a(\varvec{x},\sigma (\varvec{x}))= w'_d\sum _{c\in \mathcal {C}}d_c(x_c)+w'_i\sum _{c\in \mathcal {C}} i_c(x_c) +w'_r\sum _{v\in \mathcal {V}}\sigma _v(\varvec{x})S_v(\varvec{x})\lambda _v$, for instance, if the attacker emphatically cares about the investment and indirect costs of the enterprise.²

Justifiability of Perfect Information Assumption. The full observability of the action of the enterprise may be unjustifiable in its literal interpretation. However, the critical point here is the much slower variability of security plans and much faster adaptability of attacks. Specifically, once the security plan is implemented, it will not be modified over a relatively long horizon. Hence, the enterprise can be thought of as having committed to its investment decision. In contrast, the exploitation attempts on different vulnerabilities can explore and “learn” the most effective vulnerability. If the transitory learning phase of the attacker is negligible, then the formalism of perfect information is applicable.

3 Solving the Multi-Objective Optimization

An approach to find the Pareto solutions of multi-objective-optimizations (MOO), including multi-objective integer programs (MOIP) and multi-objective combinatorial optimizations (MOCO) as its sub-branches, is through scalarization. Here, we provide a brief overview. The reader may consult the survey papers and textbooks on MOO for more detailed treatment, e.g. [6, 10, 23].

In scalarization methods, the MOO is transformed into (parametric) instances of single-objective optimization problems, the optimal solution of each is also a Pareto-optimal solution of the original MOO problem. The most widely known method is the “linear scalarization”, where a weighted sum of the individual objectives constitutes the new objective function to be optimized. Specifically, consider a general n-objective optimization problem of $\min _{\varvec{x}\in \mathcal {X}}(F_i(\varvec{x}))$, $i=1,\ldots ,n$. Then a series of single-objective optimizations parametrized by the weight coefficients is constructed as follows: $\min _{\varvec{x}\in \mathcal {X}}\sum _{i=1}^n w_i\tilde{F}_i(\varvec{x})$, where $w_i>0$ and $\sum _{i=1}^nw_i=1$, where $\tilde{F}_i$ is a carefully chosen affine transformation (i.e., normalization) of $F_i$.³ Clearly, any solution of the weighted optimization is on the Pareto-front of the original multi-objective problem (because otherwise, there is an alternative solution that simultaneously improves all of the objective functions and at least one of them strictly, which contradicts the optimality in the scalarized problem).⁴ The Pareto-optimal solutions are found by “sweeping” the weights over the entire simplex with some granularity, solving each of the single objective optimizations, and storing any “new” solution found.

In our problem, if the weights of the direct, indirect and security costs are respectively $w_d, w_i, w_r\ge 0$, such that $w_d+w_d+w_i=1$, then, ignoring normalization for brevity, the resulting single objective optimizations (SOO) is:

$$\begin{aligned} \min _{\varvec{x}\in \mathcal {X}} \left[ w_d{D}(\varvec{x})+w_i{I} (\varvec{x})+w_r{R}(\varvec{x})\right]&&\text {s.t.:}\ D(\varvec{x})\le B. \end{aligned}$$

(9)

The form of $R(\varvec{x})$ in part comes from (7) or (8) depending on the threat model, in which the success rates of each attempted vulnerability comes from (3), (4) or (5) depending on the model for combining efficacies of the controls. Each of these optimizations is an instance of a non-linear integer program, which is NP-hard to solve in general. Exploring the entire set of possible plans can be computationally infeasible since the number of plans is $\prod _{c \in \mathcal {C}}(L_c+1)$, which grows exponentially in the number of controls (this is, for instance, over $10^{14}$ for our case study in Sect. 4). In what follows, we describe a series of tricks that help convert each of these nonlinear integer programs into mixed integer linear programs (MILPs) by introducing carefully designed auxiliary variables.⁵

3.1 Conversions to (binary) MILP

Common to all of our models is the introduction of binary decision variables as follows: $x_{cl}\in \{0,1\}$ for each $c\in \mathcal {C}$ and $l\in \mathcal {L}_c$, which represents whether control c is implemented at level $l\in \mathcal {L}_c$. Using this notation, we first enforce that logically at most only one of the implementation levels per each control is selected:

$$\begin{aligned} \Big (x_{cl}\in \{0,1\}\ \forall l\in \mathcal {L}_c,\forall c\in \mathcal {C}\Big ),\quad \Big (\!\sum _{l\in \mathcal {L}_c} x_{cl}\le 1,\ \forall c\in \mathcal {C}\Big ). \end{aligned}$$

(10)

Recall that $\mathcal {L}_c:=\{1,\ldots ,L_c\}$, and in particular, it did not include level 0. Then the direct and indirect costs can be represented in linear form as follows:

$$\begin{aligned} D(\varvec{x})=\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}d_c(l)x_{cl},&&I(\varvec{x})=\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}i_c(l)x_{cl}. \end{aligned}$$

(11)

Note that $d_c(l)$ and $i_c(l)$ are now just coefficients of the $x_{cl}$ variables.

3.2 Additive Model in (3)

For the passive threat, the expected security damage in the additive model is:

$$\begin{aligned} R(\varvec{x})=\sum _{v\in \mathcal {V}}P_v\Big (1-\sum _{c\in \mathcal {C}_v}e_{cv}(x_c)\Big )^+\lambda _v. \end{aligned}$$

(12)

In order to get rid of the non-linearity introduced by the “positive part” relation, we introduce auxiliary real-valued⁶ variables $y_v$’s for each $v\in \mathcal {V}$ such that: $y_v\ge 0$ and $y_v\ge 1-\sum _{c\in \mathcal {C}}\sum _{l\in \mathcal {L}_c}e_{cv}(l)x_{cl}$. Note that these two inequalities and the goal of the minimization guarantee that at the solution, we have: $y_v=(1-\sum _{c\in \mathcal {C}}\sum _{l\in \mathcal {L}_c}e_{cv}(l)x_{cl})^+$, as desired. Therefore, we can replace the security cost with $\sum _{v\in \mathcal {V}}P_vy_v\lambda _v$. Hence, we have the following simple proposition:

Proposition 2

Each of the scalarized single-objective optimizations in (9) for the additive–passive risk model is equivalent to the following MILP:

$$\begin{aligned} \begin{aligned}&\min _{(x_{cl},y_v)} \Bigg [{w_d}\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}d_c(l)x_{cl} +{w_i}\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}i_c(l)x_{cl} +{w_r}\sum _{v\in \mathcal {V}}(P_v\lambda _vy_v)\Bigg ]\\&{s.t.:}\ (10),\ \sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}\!d_c(l)x_{cl}\le \! B,\ \Big (y_v\ge 0,\ y_v\ge 1-\!\sum _{c\in \mathcal {C}}\sum _{l\in \mathcal {L}_c}\!e_{cv}(l)x_{cl}: \forall v\!\in \!\mathcal {V}\Big ). \end{aligned} \end{aligned}$$

For the reactive threat, the expected security damage as the security risk is: $R(\varvec{x})=\max _{v\in \mathcal {V}}\Big \{\big (1-\sum _{c\in \mathcal {C}_v}e_{cv}(x_c)\big )^+\lambda _v\Big \}$. This can be made linear by simply introducing (only) one auxiliary variable z and imposing $z\ge 0$ and $z\ge (1-\sum _{c\in \mathcal {C}}\sum _{l\in \mathcal {L}_c}e_{cv}(l)x_{cl})\lambda _v$ for “all” $v\in \mathcal {V}$. This yields:

Proposition 3

Each of the scalarized single objective optimizations in (9) for the additive–reactive risk model is equivalent to the following MILP:

$$\begin{aligned} \begin{aligned}&\min _{(x_{cl},z)}\ \Big [{w_d}\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}d_c(l)x_{cl} +{w_i}\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}i_c(l)x_{cl}+{w_r}z\Big ]\\&{s.t.:}\ (10),\ \sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}\!d_c(l)x_{cl}\le \! B,\ z\ge 0,\ \Big (z\ge \big (1-\sum _{c\in \mathcal {C}}\sum _{l\in \mathcal {L}_c}e_{cv}(l)x_{cl}\big ) \lambda _v\ \forall v\in \mathcal {V}\Big )\!. \end{aligned} \end{aligned}$$

3.3 Multiplicative Model in (4)

For the multiplicative model, we provide a modification of the method proposed in [19] and modify it for reactive threats too. First, we extend the optimization variables $x_{cl}$ to explicitly include level zero for each control as well. Hence the “logical” choice constraint, as opposed to (10), becomes:

$$\begin{aligned} \Big (x_{cl}\in \{0,1\}\ \forall l\in \mathcal {L}_c\cup \{0\},\forall c\in \mathcal {C}\Big ),\ \Big (\sum _{l\in \mathcal {L}_c\cup \{0\}}\!\!x_{cl}=1,\ \forall c\in \mathcal {C}\Big ). \end{aligned}$$

(13)

Now, for each vulnerability $v\in \mathcal {V}$, we introduce $\sum _{c\in \mathcal {C}_v}(1+L_c)$ positive real-valued auxiliary (“flow”) variables $y_{vcl}\ge 0$, one for each $l\in \mathcal {L}_c\cup \{0\}$ per each control $c\in \mathcal {C}_v$, with the following interpretation: $y_{vcl}$ is the fraction (“flow”) of the exploitation attempts on vulnerability v that is “handled” by control c at level l. Let $\mathcal {C}_v$, the set of controls that can affect vulnerability v, be enumerated as follows: $\mathcal {C}_v=\{c^v_{1},\ldots , c^v_{|\mathcal {C}_v|}\}$ (the order is immaterial). The total fraction of the exploitation attempts on vulnerability v that is to be handled by the first control in $\mathcal {C}_v$ is 1. That is, for each $v\in \mathcal {V}$, we impose: $\sum _{l\in \mathcal {L}_{c}\cup \{0\}}y_{vcl}=1$ where $c=c^v_1$. A portion of these exploitation attempts gets blocked by controls $c^v_1$, depending on which level it is implemented at, and the “surviving” fraction has to be handled by the next control in $\mathcal {C}_v$. Hence, for each $v\in \mathcal {V}$, we have the following flow-like constraint: $\sum _{l\in \mathcal {L}_{c}\cup \{0\}}y_{vcl}s_{cv}(l)=\sum _{l\in \mathcal {L}_{c'}\cup \{0\}}y_{vc'l}$, where $c'=c^v_i$ and $c=c^{v}_{i-1}$ for all $i=2,\ldots ,|\mathcal {C}_v|$. Note that $s_{cv}(l)$ is just a coefficient in this linear equality constraint, and recall the convention that $s_{cv}(0)=1$ for all $v\in \mathcal {V}$, $c\in \mathcal {C}_v$. The overall probability of success of exploitation attempts of vulnerability v is the fraction that survives the last control in $\mathcal {C}_v$, that is, $\sum _{l\in \mathcal {L}_{c}\cup \{0\}}y_{vcl}s_{cv}(l)$ where $c=c^v_{|\mathcal {C}_v|}$. Enforcing that only the implemented controls have their blocking effect on the vulnerabilities translates to the following constraint: $y_{vcl}\le x_{cl}$ $\forall v\in \mathcal {V}$, $\forall c\in \mathcal {C}_v$, $\forall l\in \mathcal {L}_c\cup \{0\}$. This constraint along with (13) ensures that only one level per control is implemented (including level zero) and only the flow-variable corresponding to the implemented level can be nonzero. Now, recursively putting the equalities together will recover the multiplicative form of the overall success probability of exploitation of v. Putting all ingredients together, we have:

Proposition 4

Each of the scalarized single objective optimizations in (9) for the multiplicative–passive risk model is equivalent to the following MILP:

$$\begin{aligned}&\min _{(x_{cl},y_{cvl})} \Big [{w_d}\!\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}d_c(l)x_{cl}+w_i\!\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}i_c(l)x_{cl}+{w_r}\!\sum _{v\in \mathcal {V}}\!P_v\lambda _v\!\!\!\!\!\sum _{\begin{array}{c} l\in \mathcal {L}_c\cup \{0\}\\ c=c^v_{|\mathcal {C}_v|} \end{array}}\!\!\!y_{vcl}s_{cv}(l)\Big ] \nonumber \\&{s.t.:}\ (13),\ \sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}\!d_c(l)x_{cl}\le B,\ \Big (\!0\le y_{vcl}\le x_{cl}: \forall v\in \mathcal {V},\forall c\in \mathcal {C}_v,\forall l\in \mathcal {L}_c\cup \{0\}\!\Big ),\nonumber \\&\big (\sum _{l\in \mathcal {L}_{c}\cup \{0\}}\!\!\!y_{vcl}=1:\ c=c^{v}_{1},\ \forall v\in \mathcal {V}\Big ),\\&\sum _{l\in \mathcal {L}_{c'}\cup \{0\}}\!\!\!y_{vc'l}=\!\!\!\!\sum _{l\in \mathcal {L}_{c}\cup \{0\}}\!\!\!y_{vcl}s_{cv}(l): c'=c^v_i,c=c^{v}_{i-1}, \forall i=2,\ldots ,|\mathcal {C}_v|,\forall v\in \mathcal {V}.\nonumber \end{aligned}$$

(14)

For the reactive threat model, we can introduce an extra variable z and enforce: $z\ge \lambda _v\sum _{l\in \mathcal {L}_c\cup \{0\}}y_{vcl}s_{cv}(l)$ where $c=c^v_{|\mathcal {C}_v|}$ for all $v\in \mathcal {V}$, along with the rest of the constraints in (14), and change the objective function to the following:

$$\begin{aligned} \min _{(x_{cl},y_{cvl},z)} \Big [{w_d}\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}d_c(l)x_{cl}+{w_i}\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}i_c(l)x_{cl}+{w_r}z\Big ] \end{aligned}$$

(15)

3.4 “Best-of” Model in (5)

For each vulnerability $v\in \mathcal {V}$ define the set of (flow-based) positive auxiliary variables $y_{v,c,l}\ge 0$ for each $c\in \{0\}\cup \mathcal {C}_v$ and $l\in \mathcal {L}_c$, that is, a flow is considered for each control that affects vulnerability v, along with a “no-control” flow $y_{v,0,0}$. For each $v\in \mathcal {V}$, we impose the total “in-flow” corresponding to vulnerability v to be one, i.e., $\sum _{c\in \{0\}\cup \mathcal {C}_v,l\in \mathcal {L}_c}y_{v,c,l}=1$. We will also impose the logical “selection” constraints: $y_{v,c,l}\le x_{cl}$ such that, if a control is not implemented, the corresponding flows will be zero. Then, in (5), we can simply replace $S_v(\varvec{x})=\min _{c\in \mathcal {C}_v}s_{cv}(x_c)$ with $\sum _{c\in \{0\}\cup \mathcal {C}_v,l\in \mathcal {L}_c}y_{v,c,l}s_{cv}(l)$, where we also define $s_{0v}(0)=1$ as coefficients of $y_{v,0,0}$. To see that this conversion indeed works, note that when the total sum of the positive flow variables is constant, the minimization problem, trying to minimize the “out-flow” per each vulnerability, chooses the “pathway” with the highest available reduction, i.e. lowest flow coefficient, exactly as the “best-of” model intends. Putting together:

Proposition 5

Each of the scalarized single objective optimizations in (9) for the best-of–passive risk model is equivalent to the following MILP:

$$\begin{aligned} \min _{(x_{cl},y_{cvl})} \Big [{w_d}\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}d_c(l)x_{cl}+w_i\sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}i_c(l)x_{cl}+{w_r}\sum _{v\in \mathcal {V}}P_v\lambda _v\!\!\!\!\sum _{\begin{array}{c} c\in \mathcal {C}_v\cup \{0\}\\ l\in \mathcal {L}_c \end{array}}\!\!y_{vcl}s_{cv}(l)\Big ]\\ {s. t.:}\qquad \sum _{c\in \mathcal {C}} \sum _{l\in \mathcal {L}_c}d_c(l)x_{cl}\le B,\quad \Big (0\le y_{vcl}\le x_{cl},\ \forall v\in \mathcal {V},\forall c\in \mathcal {C}_v,\forall l\in \mathcal {L}_c\Big ),\\ \Big (\!\!\!\!\!\sum _{\begin{array}{c} c\in \mathcal {C}_v\cup \{0\}\\ l\in \mathcal {L}_c \end{array}}\!\!\!\!y_{vcl}=1,\ \forall v\!\in \!\mathcal {V}\!\Big ),\ \ \Big (\!\!\sum _{l\in \mathcal {L}_c}\!\!x_{cl}\le 1,\ \forall c\!\in \!\mathcal {C}\!\Big ),\ \big (x_{cl}\in \{0,1\},\ \forall l\in \mathcal {L}_c,\forall c\!\in \!\mathcal {C}\big ). \end{aligned}$$

For the “reactive” threat model, the only difference is that the security risk (the third summation in the objective function) is replaced with the extra auxiliary (real-valued) variable z that needs to satisfy the following (linear) constraints: $z \ge \lambda _v\sum _{{c\in \mathcal {C}_v\cup \{0\}, l\in \mathcal {L}_c}}\!\!y_{vcl}s_{cv}(l),\ \forall v\in \mathcal {V}.$

3.5 From Vulnerabilities to Attacks

The expected losses ($\lambda$’s) are more accurately related to attacks as opposed to vulnerabilities. For instance, consider an attack A whose success requires successful exploitation of two vulnerabilities $v_1$ and $v_2$, as part of the stages of the attack, and if successful inflicts an expected damage of $\lambda _A$. Since $\lambda _A$ is only inflicted when both vulnerabilities are successfully exploited, it is not possible to separate the expected loss among $v_1$ and $v_2$ separately. We provide two different models for considering attacks that involve exploiting multiple vulnerabilities and describe how our developed MILPs can be extended to them.

3.6 Independence Across Vulnerabilities

Let $\mathcal {A}$ represent the set of attacks, where the expected inflicted loss if attack $A\in \mathcal {A}$ is successful is $\lambda _A$. Consider the multiplicative model in which the effect of controls on a vulnerability was assumed to be independent. Now assume further that the successful exploitation of different vulnerabilities comprising an attack are also independent events. Then, the expected security damages will be:

$$\begin{aligned}\begin{gathered} R(\varvec{x})=\sum _{A\in \mathcal {A}}P_A \lambda _A\prod _{v\in A}\prod _{c\in \mathcal {C}_v}s_{cv}(x_c) = \sum _{A\in \mathcal {A}}P_A \lambda _A\prod _{c\in \mathcal {C}_v}\prod _{v\in A}s_{cv}(x_c) \end{gathered}\end{aligned}$$

This shows that, by introducing flow variables $y_{Acl}$ for each attack, and performing a pre-processing by computing $s_{cA}(x_c):=\prod _{v\in A}s_{cv}(x_c)$, the same formulation as in Proposition 4 can be applied with $s_{cv}(l)$ replaced by $s_{cA}(l)$.

3.7 Correlations Across Vulnerabilities

The success of exploitation attempts across different vulnerabilities comprising an attack may have positive correlations. These correlations arise due to skills or resources of the attackers: a successful exploitation of a stage of an attack can be a signal about the higher abilities/resources of the attacker. A model that reflects these correlations is the following: the success chance of carrying out an attack is determined by the lowest probability of success across the vulnerabilities that comprise that attack. Now, combining this model with the “best-of” model that takes the correlations across defensive mechanism of controls, we get:

$$\begin{aligned}\begin{gathered} R(\varvec{x})=\sum _{A\in \mathcal {A}}P_A \lambda _A\min _{v\in A}\min _{c\in \mathcal {C}_v}s_{cv}(x_c) = \sum _{A\in \mathcal {A}}P_A \lambda _A\min _{c\in \mathcal {C}_v}\min _{v\in A}s_{cv}(x_c) \end{gathered}\end{aligned}$$

Therefore, by introducing auxiliary variables $y_{Acl}$ per attacks $A\in \mathcal {A}$ as opposed to per vulnerabilities, and performing a pre-processing $s_{cA}(l):=\min _{v\in A}s_{cv}(l)$, we can apply the same formulation as in Proposition 5 with $s_{cv}(l)$ replaced by $s_{cA}(l)$.

3.8 Parameter Uncertainties

The most likely source of uncertainty in the parameters of our models is arguably the effectiveness of the controls against each of the vulnerabilities at different implementation levels, i.e., $e_{cv}(l)$’s. Suppose that each of these parameters are given as an uncertainty interval $[\underline{e}_{cv}(l),\overline{e}_{cv}(l)]$ a subset of [0, 1], with the interpretation that the true (realized) value of the parameter can be anywhere in that interval with an unknown distribution. Collating all the efficacy parameters as $[\varvec{e}_{cv}]$, we can show the uncertainty intervals by their lower and upper end in a concise way as: $[\varvec{\underline{e}}_{cv}]\preceq [\varvec{e}_{cv}]\preceq [\varvec{\overline{e}}_{cv}]$, where $\preceq$ denotes element-wise inequalities.

One way to deal with the uncertainty is to optimize for the “worst” combined realization of the uncertain parameters. Consider the optimizations in (9), with the uncertain parameters $[\varvec{e}_{cv}]$ also as variables. Then finding optimal plans with respect to worst case of the uncertainties in efficacies can be expressed as follows:

$$\begin{aligned}&\min _{\varvec{x}\in \mathcal {X}} \left[ \max _{[\underline{\varvec{e}}_{cv}]\preceq [{\varvec{e}}_{cv}]\preceq [\overline{\varvec{e}}_{cv}]}\left\{ w_d\tilde{D}(\varvec{x})+w_i\tilde{I} (\varvec{x})+w_r\tilde{R}(\varvec{x},[\varvec{e}_{cv}])\right\} \right] \nonumber \\&\text {s.t.:} \max _{[\underline{\varvec{e}}_{cv}]\preceq [{e}_{cv}]\preceq [\overline{\varvec{e}}_{cv}]}\{D(\varvec{x})-B\}\le 0 \end{aligned}$$

(16)

We have the following observation, which we skip the proof of for brevity: For all of the security risk models in this paper, (16) is equivalent to:

$$\begin{aligned} \min _{\varvec{x}\in \mathcal {X}} \left[ w_d\tilde{D}(\varvec{x})+w_i\tilde{I} (\varvec{x})+w_r\tilde{R}(\varvec{x},[\underline{\varvec{e}}_{cv}])\right]&&\text {s.t.:}\ D(\varvec{x})\le B \end{aligned}$$

4 Numerical Evaluations

In this section, we first use our frameworks to investigate a list of the most important security controls for a typical SME (Small and Medium Enterprise) given a realistic set of parameters. As a soft method of validation, we compare the controls that most consistently appear in the Pareto-optimal plans against the top critical cyber-security controls as recommended by experts and policy organizations, specifically, SANS [17] and GCHQ [4, 5]. Subsequently, we provide some comparisons among the different security models.⁷

Parameters for our Case Study: The vulnerabilities that a typical SME faces can be generally categorized into three groups: I-“Software Vulnerabilities”, II-“Social Engineering” (e.g. phishing, pretexting, baiting), and III-“Network Vulnerabilities”. We incorporated a wide range of vulnerabilities from each of these categories. In total, we consider 37 most common vulnerabilities (Table 1 in the Appendix of our tech. report [9]) which we collected from a combination of the publicly available databases such as the Critical Weakness Enumeration (CWE) and the Common Attack Pattern Enumeration and Classification (CAPEC).

Recall that the “Impact” score for each vulnerability in our models, i.e., $I_v$, designated the expected damage inflicted on the SME in case of a successful exploitation of that vulnerability. To obtain relative values for $I_v$, from the vulnerability descriptions in the “Common Weakness Scoring System (CWSS)”, we derived a score for the impact of each vulnerability on three sources of damage: (1) “Data Losses”, damages as a result of a compromise in the confidentiality or integrity of data; (2) “Business Disruption”, losses due to compromise in the availability of services, and (3) “Reputation Damage”. For each vulnerability, we considered a weighted average of these three damages as its overall impact. We also estimated the passive probability of exploitation by combining some relevant features from the Common Weakness Scoring System (CWSS) database. Specifically, features regarding their “System Requirement Score” (e.g. “required privilege”), “Technical Requirement Score” (e.g. “likelihood of discovery” and “ease of execution”), and “Environmental Factor Score” (e.g. “exploitability” and “accessibility of information”), were combined to give a measure of the “relative ease” to exploit each vulnerability and hence get a measure of the overall rate of attempts on each vulnerability. The general trend was similar to the measurement reports of [14].

For cyber-security controls, we need each control to be an actionable process as a single independent measure that can be used to help mitigate vulnerabilities in the system. We derived our controls from the “SANS Top 20 Critical Security controls”, but we separated some of the controls that were in fact represented by a composition of multiple investment decisions. Therefore, overall, we take into account 27 distinct controls, each with multiple levels of implementation, leading to 75 distinct controls. We estimated and normalized costs parameters (both direct and indirect costs) reported in Table 2 in our tech. report [9]. We also gathered estimates of the efficacy parameters based on the defensive mechanism of each measure in the face of the exploitation requirements of each vulnerability (Table 3 in our tech. report [9]).

Validation. Our overall objective is to provide a cyber-security investment framework which is accurate, credible and relevant to the real world. A rigorous validation should take the form of a field validation in the style of clinical trials. However, at this stage, for both economical and security reasons, this approach is not feasible. In reflecting about what can constitute a reasonable validation of our framework we have decided to concentrate on expert advice, in particular the available recommendations from government agencies. These agencies have studied thousands of cyber-security incidents over many years and as such we consider their advice credible and relevant. In particular, we consult with the SANS institute “The Critical Security Controls for Effective Cyber Defense” [17], and the “10 Steps to Cyber Security” [5] and “Common Cyber Attacks: Reducing The Impact” by GCHQ [4].

A subset of the critical controls is common among all of these documents. For instance from the SANS institute the core of recommended controls are the “5 quick wins” [17]: I- Application whitelisting (found in CSC-2); II- Use of standard, secure system configurations (found in CSC-3); III- Patching application software within 48 hours (found in CSC-4); IV- Patching system software within 48 hours (found in CSC 4); and V- Reducing the number of users with administrative privileges (found in CSC 3 and CSC 12)”. A similar set of critical controls is recommended by the latest GCHQ advice [4]: I- Boundary firewalls and Internet gateways; II- Malware protection; III- Patch management; IV- Whitelisting and execution control; V- Secure configuration; VI- Password policy; VII- User access control; It is hence interesting to compare our results with these sets of recommendations and in particular their intersection: I- Patch management; II- Application whitelisting; III- Secure configuration; IV- User access control.

To make a meaningful comparison we have organized the controls appearing in our solutions in a “prevalence ordering”. The “most prevalent” controls are the ones that appear across the most number of Pareto-optimal plans for a large range of parameters: we take this as a measure of the relative importance of each cyber-security control. In particular, For each of our models, we computed the number of times each cyber-security control (at any of its implementation levels) appears in the plan across all Pareto-optimal solutions. We then “ranked” the controls based on this measure of prevalence in decreasing order. The resulting ranks are provided in Table 1 in the Appendix. We observed that overall, “patching”, “firewalls” and “whitelisting” appear among the top controls for all cases and there is a general consistency with the official recommendations. The best match with the official recommendations pertains to the “Best-of – Reactive” model. This reinforces the intuition that the “Best-of” combination of controls concentrates on the contributions of the most effective controls, and the “Reactive” threat concentrates on the most critical vulnerabilities. This observation also underlines the importance of taking into account the hitherto ignored correlations in the defensive mechanisms of the security controls.

The consistency of our results and the official advice is an encouraging first step. In the longer term we expect our mathematical framework to guide and eventually possibly replace expert advice. Another advantage is that we can customize our data to specific organizations and particular threats and so provide better “individualized” investment portfolios than a generic one-size-fit-all recommendation. We can also extend and edit the data with new controls and attacks as the threat scenarios evolve. Our solutions can be efficiently computed for large sets of controls and attacks, way beyond human manual capabilities. Our framework and the resulting tools hence open the door for customizable and accurate quantitative cyber-security advice.

A note on the computational efficiency of our frameworks. It is worth noting that, with their distinct implementation levels, we are considering 75 distinct security controls, which lead to an order of $10^{14}$ distinct cyber-security plans. With this size of the problem, an exhaustive search for finding Pareto-optimal plans is outright impractical. Generic heuristic methods such as “Genetic Algorithms” and “Tabu Search” as used in works like [13, 24] will also take “days” to converge, and even after convergence, there is no guarantee of optimality. In contrast, our MILP-based frameworks, using a generic MILP solver (Matlab’s intlinprog in our case on a typical laptop) solve for an “exact” optimal solution over the following time scales: “additive” (both passive and reactive): fraction of a second; “Multiplicative” (both “passive” and “reactive”): less than a minute; and surprisingly, for the “Best-of” model, about a second for the “passive” case, and less than 10 seconds for the reactive case.

Conclusions and future works. Decision support for cyber-security is a complex multi-objective problem. We modeled a large set of possible vulnerabilities and mitigations, and demonstrated how to efficiently compute Pareto-optimal solutions using Mixed Integer Linear Programming conversions. Many challenges remain, e.g. taking into account the costs of attacks, custom combined efficacies of controls, better approaches to deal with parameter uncertainties, combining learning and optimization, and stronger model validation. Some of these problems are within the realm of optimization engineering, others require more real-world data, which will be direction of our future work.

Appendix

Table 1. Order of prevalence of controls among Pareto-optimal plans, for different security models. In the column-headers, the initials “A.”, “M.” and “B.” stand for “Additive”, “Multiplicative” and “Best-of”, also, “P.” and “R.” designate “Reactive” and “Passive”, respectively. The table is ordered with respect to the Best-of prevalence rank, as it shows the best match with expert recommendation.