Bayesian Optimization for Reverse Stress Testing

Abstract

Bayesian Optimization with an underlying Gaussian Process is used as an optimization solution to a black-box optimization problem in which the function to be optimized has particular properties that result in difficulties. It can only be expressed in terms of a complicated and lengthy stochastic algorithm, with the added complication that the value returned is only required to be sufficiently near to a pre-determined ‘target’. We consider the context of financial stress testing, for which the data used has a significant noise component. Commonly-used Bayesian Optimization acquisition functions cannot analyze the ‘target’ condition in a satisfactory way, but a simple modification of the ‘Lower Confidence Bound’ acquisition function improves results markedly. A proof that the modified acquisition function is superior to the unmodified version is given.

Appendix A

This proof shows that the expected number of ‘expensive’ function evaluations of g (Eq. 2) is greater for LCB acquisition than for ZERO acquisition.

We first define Local Regret, which is the difference between a proposed function estimate and the actual function value. That is followed by Cumulative Regret, which a sum of local regrets. All other notation is defined in the main body of this paper. The following proof depends on a probability bound on the error estimate for g(x_n) at an evaluation point x_n due to Srinivas et al. [8]. With that bound, the general strategy is to calculate an upper bound for local regret, use that bound to determine the expected value of local regret, and then to calculate the expected number of ‘expensive’ function evaluations using the ZERO and LCB acquisition functions.

Definitions: Local regret $r_{n}$ and Cumulative regret $R_{N}$

$$ r_{n} = g\left( {\hat{x}} \right) - g\left( {x_{n} } \right); 1 \le n \le N $$

(A1)

$$ R_{N} = \mathop \sum \nolimits_{n = 1}^{N} r_{n} $$

(A2)

Definition: $\beta_{n}$ Srinivas [8], Appendix A.1, Lemma 5.1, for constants C_n > 0 that satisfy $\mathop \sum \nolimits_{n \ge 1} C_{n}^{ - 1} = 1$, and small $\delta$:

$$ \beta_{n} = 2 \log \left( {\frac{{\left| I \right|C_{n} }}{\delta }} \right). $$

(A3)

Using the definition of local regret (Srinivas [8] Appendix A.1, Lemma 5.1), Equation (A4) provides upper and lower confidence bound for the GP “mu” term.

$$ P\left( {\left| {g\left( x \right) - \mu_{n - 1} \left( x \right)} \right| \le \sqrt {\beta_{n} } \sigma_{n - 1} \left( x \right)} \right) \ge 1 - \delta ; \forall x \in I; 1 \le n \le N $$

(A4)

Then since $\hat{x}$ is an optimal solution (so that $g\left( {\hat{x}} \right) < g\left( {x_{n} } \right)$), these upper and lower bounds apply respectively:

$$ \begin{gathered} \mu_{n - 1} \left( {\hat{x}} \right) + \sqrt {\beta_{n} } \sigma_{n - 1} \left( {\hat{x}} \right) \le \mu_{n - 1} \left( {x_{n} } \right) + \sqrt {\beta_{n} } \sigma_{n - 1} \left( {x_{n} } \right) \hfill \\ \mu_{n - 1} \left( {x_{n} } \right) - \sqrt {\beta_{n} } \sigma_{n - 1} \left( {x_{n} } \right) \le \mu_{n - 1} \left( {\hat{x}} \right) + \sqrt {\beta_{n} } \sigma_{n - 1} \left( {\hat{x}} \right). \hfill \\ \end{gathered} $$

(A5)

The general strategy in this proof is to estimate the maximum cumulative regret in each of these cases for LCB and ZERO acquisitions, and then calculate the expected difference between the two.

Proposition: ZERO acquisition converges faster than LCB acquisition in the case $P\left( * \right) > 1 - \delta$ (Eq. A4).

Proof: First, from Eq. A4 with probability greater than $1 - \delta$, the Local Regret calculation proceeds as in Eq. A6. The first line uses Eq. A4 with $x = \hat{x}$, the second line uses Eq. A5 and the third line uses Eq. A4 with $x = x_{n}$. The fourth line uses an upper bound: $\beta_{n} = \mathop {\max }\limits_{n} \left( {\beta_{n} } \right)$.

$$ \begin{array}{*{20}c} {r_{n} \le \mu _{{n - 1}} (\hat{x}) + \sqrt {\beta _{n} } \sigma _{{n - 1}} (\hat{x}) - g(x_{n} )} \\ { \le \mu _{{n - 1}} (x_{n} ) + \sqrt {\beta _{n} } \sigma _{{n - 1}} (x_{n} ) - g(x_{n} )} \\ { \le 2\sqrt {\beta _{n} } \sigma _{{n - 1}} (x_{n} )} \\ { \le 2\sqrt \beta \sigma _{{n - 1}} (x_{n} )} \\ \end{array} $$

(A6)

Now consider the Local Regret in the cases of LCB and ZERO acquisition. ZERO acquisition is always non-negative but LCB acquisition can be negative. So we partition the values of n into those which result in zero or positive LCB acquisition (set S) and those which result in negative LCB acquisition (the complement, set $S^{\prime}$). These sets are shown in Eq. A7.

$$ \begin{gathered} S = \left\{ {n: \mu_{n - 1} \left( {x_{n} } \right) - \kappa \sigma_{n - 1} \left( {x_{n} } \right) \ge 0; 1 \le n \le N} \right\} \hfill \\ S^{\prime} = \left\{ {n: \mu_{n - 1} \left( {x_{n} } \right) - \kappa \sigma_{n - 1} \left( {x_{n} } \right) < 0; 1 \le n \le N} \right\} \hfill \\ \end{gathered} $$

(A7)

For S, the evaluation points proposed are identical for the two acquisition functions, since they both correspond to the same minimum. Therefore, using superscripts to denote the regrets for the two acquisition functions, the following equality applies.

$$ r_{n}^{{\left( {LCB} \right)}} = r_{n}^{{\left( {ZERO} \right)}} ; n \in S $$

(A8)

For $S^{\prime}$, ZERO acquisition returns a proposal that corresponds to a zero of the acquisition function, whereas the equivalent for LCB acquisition is negative, and we introduce a term $\phi_{n}$ to account for the difference from zero (Eq. A9).

$$ \begin{gathered} \mu_{n - 1} \left( {x_{n} } \right) = \kappa \sigma_{n - 1} \left( {x_{n} } \right) \,\left( {ZERO} \right) \hfill \\ \mu_{n - 1} \left( {x_{n} } \right) = \kappa \sigma_{n - 1} \left( {x_{n} } \right) - \phi_{n} \,\left( {LCB} \right) \hfill \\ \end{gathered} $$

(A9)

This leads to the following expressions for the two regrets, using Eq. A6.

$$ \left\{ {\begin{array}{*{20}l} {\mathop {\max }\limits_{n} \left( {r_{n}^{{\left( {ZERO} \right)}} } \right) = 2\sqrt \beta \sigma _{{n - 1}} \left( {x_{n} } \right) = \frac{{2\sqrt \beta }}{\kappa }\mu _{{n - 1}} \left( {x_{n} } \right)} \hfill \\ {\mathop {\max }\limits_{n} \left( {r_{n}^{{\left( {LCB} \right)}} } \right) = 2\sqrt \beta \sigma _{{n - 1}} \left( {x_{n} } \right) = \frac{{2\sqrt \beta }}{\kappa }\left( {\mu _{{n - 1}} \left( {x_{n} } \right) + \phi _{n} } \right)} \hfill \\ \end{array} } \right. $$

(A10)

Equation (A11) shows the partitioning the Cumulative Regret between sets S and $S^{\prime}$.

$$ R_{N} = \mathop \sum \nolimits_{n \in S} r_{n} + \mathop \sum \nolimits_{{n \in S^{\prime}}} r_{n} $$

(A11)

Then, Eq. A12 show the maximum Cumulative Regret for ZERO and LCB acquisitions.

$$ \left\{ {\begin{array}{*{20}l} {\max \left( {R_{N}^{{\left( {ZERO} \right)}} } \right) = \sum\nolimits_{{n \in S}} {r_{n} } + \frac{{2\sqrt \beta }}{\kappa }\sum\nolimits_{{n \in S^{\prime } }} {\mu _{{n - 1}} } \left( {x_{n} } \right)} \hfill \\ {\max \left( {R_{N}^{{\left( {LCB} \right)}} } \right) = \sum\nolimits_{{n \in S}} {r_{n} } + \frac{{2\sqrt \beta }}{\kappa }\left( {\sum\nolimits_{{n \in S^{\prime } }} {\mu _{{n - 1}} } \left( {x_{n} } \right) + \phi _{n} } \right)} \hfill \\ \end{array} } \right. $$

(A12)

Equations A12 then imply that the inequality in Eq. A13.

$$ \max \left( {R_{N}^{{\left( {LCB} \right)}} } \right) - \max \left( {R_{N}^{{\left( {ZERO} \right)}} } \right) = \frac{2\sqrt \beta }{\kappa }\phi_{n} > 0;1 \le n \le N $$

(A13)

Equation (A13) is a strong indication that ZERO acquisition leads to faster convergence than LCB acquisition, since it applies with high probability $1 - \delta$. This completes the proof.