Scenario Approximation of Robust and Chance-Constrained Programs

Abstract

We consider scenario approximation of problems given by the optimization of a function over a constraint that is too difficult to be handled but can be efficiently approximated by a finite collection of constraints corresponding to alternative scenarios. The covered programs include min-max games, and semi-infinite, robust and chance-constrained programming problems. We prove convergence of the solutions of the approximated programs to the given ones, using mainly epigraphical convergence, a kind of variational convergence that has demonstrated to be a valuable tool in optimization problems.

Appendices

Appendix A: Proofs

Proof of Theorem 5.1

In order to show convergence of the solution of (9), we write in a different way the program. Using the indicator function χ, (9) becomes:

$$ \min_{x\in\mathbf{X}\subseteq\mathbb{R}^{p}}c^{\mathrm{T}}x+\chi \Biggl(x,\bigcap _{i=1}^{n} \bigl\{ f \bigl(x,y^{ (i )} \bigr) \le0 \bigr\} \Biggr). $$

(14)

We set:

To ease notation, we define A(y):={x∈X:f(x,y)≤0}. Now, in order to show that the solution of (9) converges to the solution of (11), we use Theorem 7.33 of [33, pp. 266–267], reproduced in the Appendix as Theorem B.2. We have to verify the following hypotheses (see Definition 4.1):

• 1. (F _n (x)+c ^T x) _n and F(x)+c ^T x are lower semi-continuous and proper;

• 2. (F _n (x)+c ^T x) _n is eventually level-bounded;

• 3. (F _n (x)+c ^T x) _n epi-converges to F(x)+c ^T x.

(F _n (x)+c ^T x) _n and F(x)+c ^T x are lower semi-continuous: c ^T x is continuous; F _n (x) is lower semi-continuous iff the set $\bigcap_{i=1}^{n}A (y^{ (i )} )$ is closed (Example 1.6 in [30], p. 10) and this is guaranteed by f being lower semi-continuous in x for any y∈Y (Proposition 1.7 in [30, p. 11]). Moreover, these functions are proper as the set ⋂ _y∈Y A(y) is nonempty, and therefore:

As concerns eventual level-boundedness of the sequence (F _n (x)+c ^T x) _n , since the function c ^T x is not level-bounded, we need the sequence of indicator functions (F _n (x)) _n to be eventually level-bounded: this is guaranteed by the assumption that there exists an index n ₀ such that the set $\bigcap_{i=1}^{n}A (y^{ (i )} )$ is compact for any n≥n ₀.

As concerns epi-convergence, using Example 6.24(b) in [30, p. 64], we see that if (F _n (x)) _n epi-converges to F(x) and it is an increasing sequence, and c ^T x is continuous and therefore lower semi-continuous, (F _n (x)+c ^T x) _n epi-converges to F(x)+c ^T x: all these conditions are verified apart from the epi-convergence of (F _n (x)) _n to F(x) that has still to be proved.

According to Proposition 4.15 in [30, p. 43], epi-convergence of the indicator functions is equivalent to Painlevé–Kuratowski convergence of the sets. Since the sequence $\bigcap_{i=1}^{n}A (y^{ (i )} )$ is decreasing, according to Exercise 4.3(b) in [33, p. 111], we have:

 □

Proof of Corollary 5.1

In the proof, we will use the following characterizations of lower semi-continuity of f and density of Y ^⋆ in Y. The function f is lower semi-continuous with respect to y, iff, for x fixed, we have that for every λ∈ℝ, the set G _λ (x):={y∈Y:f(x,y)>λ} is open (see [47, Chap. 2, p. 42]); Y ^⋆ is dense in Y iff, for all open subset G of Y, we have G∩Y ^⋆≠∅ (see [47, Chap. 2, p. 26]).

As before, define A(y):={x∈X:f(x,y)≤0} and

$$I : = \bigcap_{y\in\mathbf{Y}}A (y )\quad\textrm{and}\quad I^{\star} : = \bigcap_{y^{\star}\in\mathbf{Y}^{\star}}A \bigl(y^{\star} \bigr). $$

To prove I=I ^⋆, we prove first I⊆I ^⋆ and then I ^⋆⊆I. The first inclusion is trivially verified since Y ^⋆⊆Y. Let us now prove that I ^⋆⊆I by counterposition. Suppose that there exists an x ^⋆ such that x ^⋆∈I ^⋆ and x ^⋆∉I. It respectively means that

$$ \forall y^{\star}\in\mathbf{Y}^{\star},\quad f \bigl(x^{\star},y^{\star} \bigr)\leq0, $$

(15)

and ∃y ₀∈Y such that f(x ^⋆,y ₀)>0. By combination of these two conditions, we get: ∃y ₀∈Y∖Y ^⋆ such that f(x ^⋆,y ₀)>0. Since f is lsc with respect to y, the set G ₀(x) is open: thus, there exists η=η(y ₀)>0 such that, for every y∈B(y ₀,η) (the ball of radius η centered in y ₀), y∈G ₀ i.e. f(x,y)>0. But, since Y ^⋆ is dense in Y, there exists some $y_{0}^{\star}\in\mathsf{B} (y_{0},\eta )\cap\mathbf{Y}^{\star}$. So $f (x^{\star},y_{0}^{\star} )>0$, which contradicts (15). Therefore:

$$\forall y\in\mathbf{Y}\setminus\mathbf{Y}^{\star},\quad f \bigl(x^{\star},y \bigr)\leq0, $$

which, together with (15), gives

$$\forall y\in\mathbf{Y},\quad f \bigl(x^{\star},y \bigr)\leq0. $$

Thus, we have x ^⋆∈I and I ^⋆⊆I. As a consequence, I=I ^⋆. □

Proof of Corollary 5.2

As in the previous theorem, we just need to show that (F _n (x)) _n epi-converges to a certain limit function F(x), i.e. that

$$\bigcap_{i=1}^{n} \bigl\{ f \bigl(x,y^{ (i )} \bigr)\le0 \bigr\} $$

converges in the sense of Painlevé–Kuratowski to a limit set (Proposition 4.15 in [30, p. 43]). Using Theorem 2.7 in [28], we see that $\bigcap_{i=1}^{n} \{ f (x,y^{ (i )} )\le0 \}$ converges in the sense of Painlevé–Kuratowski to Int({x∈X:f(x,Y)≤0}) under the following conditions:

(i) The set {f(x,y ⁽ⁱ⁾)≤0} has to be nonempty and closed: nonemptiness is guaranteed by the statement of the corollary, and closedness by the fact that f is lower semi-continuous in x for ℙ-almost surely any y∈Y (Proposition 1.7 in [30, p. 17]).

(ii) The random variable D(Y):=d(0,Int({x∈X:f(x,Y)≤0})) is integrable.

Therefore, (F _n (x)) _n epi-converges ℙ-as to

$$F (x )=\chi \bigl(x,\mathrm{Int} \bigl( \bigl\{ x\in\mathbf{X}:f (x,Y )\le0 \bigr\} \bigr) \bigr). $$

Note that we can use a result of [18, Proposition 21, p. IV-34] to write

and to express (8) as

$$\min_{x\in\mathbf{X}\subseteq\mathbb{R}^{n}}c^{\mathrm{T}}x+\int_{\mathbf{Y}}\chi \bigl(x, \bigl\{ x\in\mathbf{X}:f (x,y )\le0 \bigr\} \bigr)\mathbb{P} (dy ). $$

Convergence of the solution can be proved using the same conditions as before (see the proof of Theorem 5.1): in particular, we have just to check for eventual level-boundedness of (F _n (x)) _n , but this is guaranteed by integrability of d(0,{f(x,Y)≤0}). □

Proof of Theorem 5.2

The idea is to write this program, using the properties of the indicator function χ, as $\min_{x\in\mathbf{X}\subseteq\mathbb{R}^{p}}a (x )$, where:

The approximate solution is given by min _x∈X a _n (x), where:

$$a_{n} (x )=h (x )+\chi \Biggl(x,\mathrm{lev}_{\le-\alpha} \Biggl\{ \frac{1}{n}\sum_{i=1}^{n} \bigl[- \mathbf{1} \bigl\{ \bigl(x,y^{ (i )} \bigr)\in A \bigr\} \bigr] \Biggr\} \Biggr). $$

If we define the empirical probability based on the sequence (y ⁽ⁱ⁾) _i=1,…,n as $\mathbb{P}_{n} (B )=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1} (y^{ (i )}\in B )$, we have

$$\mathbb{P}_{n} \bigl\{ (x,Y )\in A \bigr\} =\frac{1}{n}\sum_{i=1}^{n}\mathbf{1} \bigl\{ \bigl(x,y^{ (i )} \bigr)\in A \bigr\}. $$

(a) Under hypothesis (i), the function −1{(x,Y)∈A} is lower semi-continuous in x for ℙ-almost any Y∈Y and measurable with respect to $\mathcal{B} (\mathbf{X} )\otimes\mathcal{Y}$. We can then apply Corollary 2.4 in [28, p. 70], to prove that, for almost any independent and identically distributed sequence (y ⁽ⁱ⁾) _i=1,…,n ,

(16)

Now, define

for α∈ℝ. Using the characterization of epi-convergence through level sets (see [48, result (b) on p. 755]),

$$ \mathrm{PK}-\limsup_{n\rightarrow\infty}S_{n} \bigl(\alpha_{n}, \bigl(y^{ (i )} \bigr)_{i} \bigr)\subset S (\alpha )\quad \mathbb{P}\mbox{-}\mathsf{as}, $$

(17)

for any (α _n ) _n such that α _n →α. Recall that, for a sequence of sets (C _n ) _n , PK−lim sup _n→∞ C _n is the set of all cluster points extracted from the sequence (C _n ) _n : this means that if we define a sequence $(\overline{x}_{n} )_{n}$ through $\overline{x}_{n}\in\arg\min_{x\in\mathbf{X}}a_{n} (x )$, the set of cluster points of $(\overline{x}_{n} )_{n}$ is included in S(α).

From Proposition 4.15 in [30, p. 43], Eq. (17) means that

From Proposition 6.21 in [30, p. 63], since h(x) is continuous by (ii),

$$\mathrm{epi}-\liminf_{n\rightarrow\infty}a_{n} (x ) = h (x )+\mathrm{epi}- \liminf_{n\rightarrow\infty}\chi \bigl(x,S_{n} \bigl(\alpha_{n}, \bigl(y^{ (i )} \bigr)_{i} \bigr) \bigr)\ge a (x )\;\mathbb{P}\mbox{-}\mathsf{as}. $$

Proposition 7.29 (a) in [33] yields $\liminf_{n}a_{n} (\overline{x}_{n} )\ge a (\overline{x} )$ provided (iii) holds, and from the obvious relations $a_{n} (\overline{x}_{n} )=h (\overline{x}_{n} )$ and $a (\overline{x} )=h (\overline{x} )$, we get the desired result.

(b) From (16), using Proposition 7.7 (b) in [33], we obtain the existence of a sequence $(\alpha_{n}^{\star} )_{n}$ with $\alpha_{n}^{\star}\uparrow\alpha$ such that

$$S (\alpha )\subseteq\liminf_{n\rightarrow\infty}S_{n} \bigl( \alpha_{n}^{\star}, \bigl(y^{ (i )} \bigr)_{i} \bigr)\quad\mathbb{P}\mbox{-}\mathsf{as}. $$

Therefore we get:

where the first equality derives from Proposition 6.21 in [30, p. 63], since h(x) is continuous by (ii), and the second one from the property of the particular sequence $(\alpha_{n}^{\star} )_{n}$.

Now, we show convergence of the minimizers. The functions a(x) and a _n (x) are lower semi-continuous since, under hypothesis (i), the function 1{(x,Y)∈A} is lower semi-continuous in x for ℙ-almost any Y∈Y. Under hypotheses (ii), (iii) and (iv), a _n (x) and a(x) are proper. The sequence (a _n (x)) _n is eventually level-bounded, from (ii) and (iii). Therefore, for the sequence $(\alpha_{n}^{\star} )_{n}$, Theorem B.2 applies.

(c) Then, we pass to the last part of the theorem. We start from the first statement. In particular, the fact that $\liminf_{n}h (\overline{x}_{n} )\ge h (\overline{x} )$ ℙ-almost surely is a consequence of (a) under hypotheses (i)–(ii)–(iii)–(iv) with α _n =α. As concerns the fact that $\limsup_{n}h (\overline{x}_{n} )\le h (\overline{x} )$ ℙ-almost surely, it can be shown to hold following the proof of Proposition 2.2 in [37] and replacing the fact that G is Carathéodory with (i), continuity of f with (ii), compactness of X with (iii), the existence of an optimal solution stated in (A) with (iv)–(ii) and the remaining part of (A) with (vi) (see Remark 5.3 for a comparison of the hypotheses). As concerns the second statement, we first show that S _n (α,(y ⁽ⁱ⁾) _i ) converges ℙ-almost surely in the Hausdorff metric to S(α), then we show that this implies epi-convergence of the objective functions and we close the proof proving convergence of the solutions. Let A(x) be the set defined in the statement of the theorem. Then we have 1{(x,Y)∈A}=1{Y∈A(x)} and:

This holds because of hypothesis (v). On the other hand, Eq. (2.3) in [49] becomes

$$\bigl\{ x\in\mathbf{X}:\mathbb{P} \bigl\{ Y\in A (x ) \bigr\} \ge\alpha \bigr\} \subseteq\mathrm{cl} \bigl[ \bigl\{ x\in\mathbf{X}:\mathbb{P} \bigl\{ Y\in A (x ) \bigr\} >\alpha \bigr\} \bigr], $$

and this is equivalent to hypothesis (vi). From hypothesis (iii), S _n (α,(y ⁽ⁱ⁾) _i ) ℙ-almost surely converges in the Hausdorff metric to S(α) by Theorem 2.1 in [49].

On the space of nonempty compact subsets of an Euclidean space, convergence in the Hausdorff metric and Painlevé–Kuratowski convergence of sequences of sets are equivalent, and both are equivalent to epi-convergence of the indicator functions of the sets (Proposition 4.15 in [30, p. 43]). This means that PK−lim _n→∞ S _n (α,(y ⁽ⁱ⁾) _i )=S(α) ℙ-as is equivalent to:

From Proposition 6.21 in [30], since h(x) is continuous by (ii),

$$\mathrm{epi}-\lim_{n\rightarrow\infty}h (x )+\chi \bigl(x,S_{n} \bigl( \alpha, \bigl(y^{ (i )} \bigr)_{i} \bigr) \bigr) = h (x )+\chi \bigl(x,S (\alpha ) \bigr)=a (x )\quad\mathbb{P}\mbox{-}\mathsf{as}. $$

Therefore, we can apply Theorem B.2 that holds since the objective functions (a _n (x)) _n and a(x) are lower semi-continuous, proper and eventually level-bounded (from (ii) and (iii)), (a _n (x)) _n is epi-convergent to a(x) and the space X is compact. □

Appendix B: Some Mathematical Concepts

2.1 B.1 Epi-convergence

Since our main result is based on epi-convergence, we provide a short presentation. Let $h:E\rightarrow\overline{\mathbb{R}}$ be a function from the metric space E into the extended reals. Its epigraph is defined by

$$\mathrm{Epi} (h ):= \bigl\{ (x,\lambda )\in E\times\mathbb{R}:h (x )\leq\lambda \bigr\} . $$

The hypograph of h, denoted by Hypo(h), is defined by reversing the inequality. Let (h _n ) _n≥1 (or (h _n ) _n for short) be a sequence of functions from E into $\overline{\mathbb{R}}$. For any x∈E, we introduce the quantities

$$ \left .\begin{array}{c} \displaystyle\mathrm{epi}-\liminf_{n\rightarrow\infty}h_{n} (x ) : = \sup_{k\geq1}\liminf\limits_{n\rightarrow\infty}\inf_{y\in\mathsf{B} (x,1/k )}h_{n} (y ),\\[12pt] \displaystyle\mathrm{epi}-\limsup_{n\rightarrow\infty}h_{n} (x ) : = \sup_{k\geq1}\limsup_{n\rightarrow\infty}\inf_{y\in\mathsf{B} (x,1/k )}h_{n} (y ),\end{array} \right . $$

(18)

where B(x,1/k) denotes the open ball of radius 1/k centered at x. The function x↦epi−lim inf _n→∞ h _n (x) (resp. x↦epi−lim sup _n→∞ h _n (x)) is called the lower (resp. upper) epi-limit of the sequence (h _n ) _n . These functions are lsc. If epi−lim inf _n→∞ h _n (x)=epi−lim sup _n→∞ h _n (x), then (h _n ) _n is said to be epi-convergent at x. If this is true for all x∈E, then the sequence (h _n ) _n epi-converges. Its epi-limit is denoted by epi−lim _n→∞ h _n .

Equalities (18) have a geometric counterpart involving the Painlevé–Kuratowski convergence of epigraphs on the space of closed sets of E×ℝ (see, e.g., [50] or [30]). The Painlevé–Kuratowski convergence is defined as follows. Given a sequence (C _n ) _n≥1 of sets in E, we define

where (C _n(i)) _i≥1 is a subsequence of (C _n ) _n≥1. The subsets PK−lim inf _n→∞ C _n and PK−lim sup _n→∞ C _n are the lower limit and the upper limit of (C _n ) _n≥1. It is not difficult to check that they are both closed and that they satisfy PK−lim inf _n→∞ C _n ⊂PK−lim sup _n→∞ C _n . A sequence (C _n ) _n≥1 is said to converge to C, in the sense of Painlevé–Kuratowski, if

$$C=\mathrm{PK}-\liminf_{n\rightarrow\infty}C_{n}=\mathrm{PK}-\limsup_{n\rightarrow\infty}C_{n}.$$

This is denoted by $C=\mathrm{PK-}\lim_{n\rightarrow\infty}C_{n}$. As mentioned above, this notion is strongly connected with epi-convergence: a sequence of functions $h_{n}:E\rightarrow\overline{\mathbb{R}}$ epi-converges to h iff the sequence (Epi(h _n )) _n≥1 PK-converges to Epi(h), in E×ℝ.

A characterization of epi-convergence can be given using level sets (see [33, p. 246]).

Theorem B.1

Let h:ℝ^d→ℝ and (h _n ) _n be such that h _n :ℝ^d→ℝ. Then:

• (i) epi−lim inf _n→∞ h _n ≥h iff

  $$\limsup_{n} (\mathrm{lev}_{\le\alpha_{n}}h_{n} )\subseteq \mathrm{lev}_{\le\alpha}h, $$

  for all sequences α _n →α;

• (ii) h≥epi−lim sup _n→∞ h _n iff

  $$\liminf_{n} (\mathrm{lev}_{\le\alpha_{n}}h_{n} )\supseteq \mathrm{lev}_{\le\alpha}h, $$

  for some sequence α _n →α, in which case this sequence can be chosen with α _n ↓α;

• (iii) epi−lim _n→∞ h _n =h if and only if both conditions hold.

2.2 B.2 Convergence of Minima

The following result (Theorem 7.33 in [33, pp. 266–267]) plays a fundamental role in our proofs.

Theorem B.2

Suppose that the sequence (h _n ) _n is eventually level-bounded, and epi−lim _n→∞ h _n =h with h _n and h lower semi-continuous and proper. Then:

$$\inf h_{n}\rightarrow\inf h $$

and infh is finite; moreover, there exists n ₀ such that, for any n≥n ₀, the sets argminh _n are nonempty and form a bounded sequence with

$$\limsup_{n} (\arg\min h_{n} )\subseteq\arg\min h. $$

Indeed, for any choice of ε _n ↓0 and x _n ∈ε _n −argminh _n , the sequence (x _n ) _n∈ℕ is bounded and such that all its cluster points belong to argminh. If argminh consists of a unique point $\overline{x}$, one must actually have $x_{n}\rightarrow\overline{x}$.

2.3 B.3 Stationarity and Ergodicity

A sequence of random variables (X _i ) _i=1,… is said to be stationary if the random vectors (X ₁,…,X _n ) and (X _k+1,…,X _n+k ) have the same distribution for all integers n,k≥1. A measurable set B is said to be invariant if

$$\bigl\{ (X_{i} )_{i=1,\dots}\in B \bigr\} = \bigl\{ (X_{i} )_{i=k,k+1,\dots}\in B \bigr\} $$

for every k≥1. A sequence (X _i ) _i=1,… is said to be ergodic if, for every invariant set B,

$$\mathbb{P} \bigl\{ (X_{i} )_{i=1,\dots}\in B \bigr\} \in \{ 0,1 \} . $$

These properties can be introduced also in a more abstract setting. Given a probability space $(\varOmega,\mathcal{A},\mathbb{P} )$, an $\mathcal{A}$-measurable transformation T:Ω→Ω is said to be measure-preserving if ℙ(T ⁻¹ A)=ℙ(A) for all $A\in\mathcal{A}$. Equivalently, ℙ is said to be stationary with respect to T. The sets $A\in\mathcal{A}$ that satisfy T ⁻¹ A=A are called invariant sets and constitute a sub-σ-field $\mathcal{I}$ of $\mathcal{A}$. A measurable and measure-preserving transformation T is said to be ergodic if ℙ(A)=0 or 1 for all invariant sets A. Equivalently, the sub-σ-field $\mathcal{I}$ reduces to the trivial σ-field {Ω,∅} (up to the ℙ-null sets). The previous definitions can be recovered remarking that any stationary sequence (X _i ) _i=1,… can almost surely be rewritten using a measurable and measure-preserving transformation T as X _t (ω)=X ₀(T ^t ω) (see, e.g., [51, Proposition 6.11]).

2.4 B.4 Random Sets

Given a Polish space E, the set of all subsets of E is denoted by 2^E. A random set is a set-valued map Γ:Ω→2^E having some sort of measurability property. Here, we shall use graph measurability. The graph of Γ is denoted by Gr(Γ) and defined by

$$Gr(\varGamma)=\bigl\{(\omega,x)\in\varOmega\times E:x\in\varGamma(\omega)\bigr\}. $$

In this framework, Γ is said to be a random set if Gr(Γ) is a member of the product σ-field $\mathcal{A}\otimes\mathcal{B}(E)$. Then, Γ is said to be graph-measurable.