Abstract
We study a version of stochastic dual dynamic programming (SDDP) with a distributionally robust objective. The classical SDDP algorithm uses a finite (nominal) probability distribution for the random outcomes at each stage. We modify this by defining a distributional uncertainty set in each stage to be a Euclidean neighbourhood of the nominal probability distribution. We derive a formula for the worst-case expectation of future costs over this set that can be applied in the backward pass of SDDP. We verify the correctness of this algorithm, show its almost sure convergence under standard assumptions, and illustrate it by applying it to a model of the New Zealand hydrothermal electricity system.
Similar content being viewed by others
References
Artzner P, Delbaen F, Eber J-M, Heath D (1999) Coherent measures of risk. Math Finance 9(3):203–228
Bayraksan G, Love DK (2015) Data-driven stochastic programming using phi-divergences. In: the operations research revolution. Tutorials in operations research, INFORMS, pp 1–19
Bertsimas D, Brown DB, Caramanis C (2011) Theory and applications of robust optimization. SIAM Rev 53(3):464–501
Bertsimas D, Gupta V, Kallus N (2013) Data-driven robust optimization. ArXiv preprint arXiv:1401.0212
Bertsimas D, Gupta V, Kallus N (2014) Robust SAA. ArXiv preprint arXiv:1408.4445
Bezanson J, Edelman A, Karpinski S, Shah VB (2017) Julia: A fresh approach to numerical computing. SIAM Rev 59(1):65–98
de Matos VL, Philpott AB, Finardi EC (2015) Improving the performance of stochastic dual dynamic programming. J Comput Appl Math 290:196–208
Dowson O (2017) SDDP in Julia. Technical report, University of Auckland
Dunning I, Huchette J, Lubin M (2017) JuMP: A modeling language for mathematical optimization. SIAM Rev 59(2):295–320
Esfahani PM, Kuhn D (2015) Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. ArXiv preprint arXiv:1505.05116
Gao R, Kleywegt AJ (2016) Distributionally robust stochastic optimization with Wasserstein distance. ArXiv preprint arXiv:1604.02199
Girardeau P, Leclere V, Philpott AB (2014) On the convergence of decomposition methods for multistage stochastic convex programs. Math Oper Res 40(1):130–145
Gurobi Optimization Inc. (2016) Gurobi optimizer reference manual
Huang J, Zhou K, Guan Y (2017) A study of distributionally robust multistage stochastic optimization. ArXiv preprint arXiv:1708.07930
Infanger G, Morton DP (1996) Cut sharing for multistage stochastic linear programs with interstage dependency. Math Program 75(2):241–256
Klabjan D, Simchi-Levi D, Song M (2013) Robust stochastic lot-sizing by means of histograms. Prod Oper Manag 22(3):691–710
Kozmík V, Morton DP (2015) Evaluating policies in risk-averse multi-stage stochastic programming. Math Program 152(1):275–300
Pereira MVF, Pinto LMVG (1991) Multi-stage stochastic optimization applied to energy planning. Math Program 52(1–3):359–375
Philpott AB, de Matos VL (2012) Dynamic sampling algorithms for multi-stage stochastic programs with risk aversion. Eur J Oper Res 218(2):470–483
Philpott AB, Pritchard G (2018) EMI-DOASA. http://www.epoc.org.nz/papers/EMI-DOASA.pdf. Accessed 15 May 2018
Philpott AB, de Matos VL, Finardi E (2013) On solving multistage stochastic programs with coherent risk measures. Oper Res 61(4):957–970
Rockafellar RT (1972) Convex analysis. Princeton University Press, Princeton
Shapiro A (2011) Analysis of stochastic dual dynamic programming method. Eur J Oper Res 209(1):63–72
Shapiro A, Dentcheva D, Ruszczyński A (2009) Lectures on stochastic programming: modeling and theory. SIAM
Acknowledgements
We are very grateful for discussions with Oscar Dowson, Eddie Anderson, Karen Willcox, and Michael Kapteyn, and for comments from participants at the Workshop on Stochastic Programming Honoring Maarten van der Vlerk held at the University of Groningen on August 10–11, 2017. Andy Philpott acknowledges the financial support of the New Zealand Marsden Fund under contract UOA1520.
Author information
Authors and Affiliations
Corresponding author
Additional information
This paper is dedicated to the memory of our friend and colleague Maarten van der Vlerk.
Appendices
Appendix A
This appendix contains proofs of all the results in the paper. We begin by proving some technical lemmas. Recall the problem
Lemma 1
Given \(z_{i}\), \(i=1,2,\ldots ,m\), \(\rho _{r}(z)\) is a continuous and nondecreasing function of \(r\ge 0\).
Proof
Observe that given \(r\ge 0\) and \(z_{i}\), \(i=1,2,\ldots ,m\), P is a convex optimization problem with a compact feasible region, so it has an optimal solution, with optimal value denoted \(\rho _{r}(z)\). Moreover for fixed z, the optimal value function is a concave increasing function of r. By Rockafellar (1972, Theorem 10.1) \(\rho _{r}(z)\) is therefore continuous on \( \{r:r>0\}\). It is also easy to show that \(\lim _{_{r\rightarrow 0}}\rho _{r}(z)=\sum _{i=1}^{m}q_{i}z_{i}\), so \(\rho _{r}(z)\) is also continuous at \( r=0\). \(\square \)
Consider the problem
Lemma 2
P(n) has an optimal solution if and only if \(a^{2}\le nb^{2}\).
Proof
The objective function is continuous and the feasible region is compact, and so P(n) has an optimal solution if and only if the feasible region is nonempty. If \(a^{2}\le nb^{2}\) then \(y_{i}=\frac{a}{n}\) is feasible as it satisfies
If \(a^{2}>nb^{2}\) then the feasible region is empty, for any feasible y satisfies \(\sum _{i=1}^{n}y_{i}=a\) so
But this contradicts
which is true because
Lemma 3
Suppose \(a^{2}\le nb^{2}\). The optimal solution to P(n) is
Proof
Since P(n) is a convex program we can solve it by minimizing the Lagrangian, giving
Differentiating gives
so
Since
we have
If \(a=0\) then
so
This gives
so
and
as required.
If \(a\ne 0\) then substituting for \(\lambda \) gives
Now
so
Let \(z=\sum _{i=1}^{n}z_{i}\), and \(d=\sum _{i=1}^{n}z_{i}^{2}\), so
Equation (10) is a quadratic in \(\mu \),
that has roots
The first root is
This gives
with objective
The other root of (10) gives
which has value \(-a\bar{z}+s\sqrt{nb^{2}-a^{2}}\), a local maximum of P(n).) \(\square \)
Lemma 6
Let \(\bar{z}=\frac{\sum _{i=1}^{n}z_{i}}{n}\). For all i,
Proof
Without loss of generality choose \(i=1\). Then
This expression is \((n-1)^{2}\) times the variance of the quantities \( z_{2},z_{3},\ldots ,z_{n}\) which is nonnegative, so it follows that
from which the result follows. \(\square \)
Lemma 4
If \(r\le \sqrt{\frac{m}{m-1}}\min _{i}\{q_{i}\}\) then P has optimal solution
with optimal value \(\sum _{i=1}^{m}q_{i}z_{i}+\left( \sqrt{m}s\right) r\).
Proof
We consider a solution in which we drop the constraint \(p\ge 0\). This gives \(p_{i}=q_{i}+y_{i}\) where y solves
Thus Lemma 3 with \(a=0\) gives
whence
Now by Lemma 6
so
as \(r\le \sqrt{\frac{m}{m-1}}\min _{i}\{q_{i}\}\) by assumption.
We also have
Lemma 5
If \(r_{1}<r_{2}\) then \(k(r_{1})\le k(r_{2}).\)
Proof
Recall for any fixed k and r that if \(r\le \sqrt{\frac{1}{m(m-1)}}\),
and otherwise
If we assume that k(r) is constant at value k as r varies then
Since \(z_{i}\le z_{i+1}\), for a fixed k and r we have
Also \(\frac{dp_{k+1}(r)}{dr}\le \frac{dp_{i}(r)}{dr}\) for \(i=k+2,\ldots ,m\) and \(\frac{dp_{k+1}(r)}{dr}\le 0\). This means that as r increases \( p_{k+1}(r)\) becomes negative before (or at that same r as) any \(p_{i}(r)\), \(i=k+2,\ldots ,m\). In other words as r increases k(r) remains constant and then increases at some value of r. If the \(z_{i}\) are distinct then k(r) increases by 1 at each step. We therefore have \(k(0)=0\), and k(r) is piecewise constant and nondecreasing with r. \(\square \)
Proposition 1
Suppose that \(Z(x,\omega )\) is a convex function of x for each \(\omega \in \varOmega \), and that \(g(\tilde{x},\omega )\) is a subgradient of \(Z(x,\omega )\) at \(\tilde{x}\). Then \(\mathbb {E}_{\mathbb {P}^{*}}[g(\tilde{x},\omega )]\) is a subgradient of \(\max _{\mathbb {P}\in \mathcal {P}}\mathbb {E}_{\mathbb {P} }[Z(x,\omega )]\) at \(\tilde{x}\), where \(\mathbb {P}^{*}\in \arg \max _{ \mathbb {P}\in \mathcal {P}}\mathbb {E}_{\mathbb {P}}[Z(\tilde{x},\omega _{m})]\).
Proof
For any x,
which demonstrates that \(\mathbb {E}_{\mathbb {P}^{*}}[g(\tilde{x},\omega )]\) is a subgradient at \(\tilde{x}\). \(\square \)
Proposition 2
If for any \(x_{t}\in \mathcal {X}_{t}(\omega _{t})\), \(h_{t+1,k}-\bar{\pi } _{t+1,k}^{\top }H_{t+1}x_{t}\le \mathbb {E}_{\mathbb {P}_{t}^{*}}[Q_{t+1}(x_{t},\omega _{t+1})]\) for every \(k=1,2,\ldots ,\nu \), then
Proof
For any \(x_{t}\in \mathcal {X}_{t}(\omega _{t})\) the optimal choice of \( \theta _{t+1}\) satisfies
by hypothesis. It follows that
giving the desired result. \(\square \)
Appendix B
This appendix gives a formulation of a risk-neutral version of the hydrothermal scheduling model that we solve in Sect. 5. More details of this model can be found in Philpott and Pritchard (2013). The risk-neutral hydrothermal scheduling model we solve seeks a policy of electricity generation over T weeks that meets demand and minimizes the expected fuel cost of thermal generation. All data are deterministic except for weekly inflows that are assumed to be stagewise independent. This results in a large-scale stochastic programming model which is defined as follows. Let \( \bar{x}_{j}\) denote the known storage in reservoir j at the start of the first week, and \(x_{j}\left( t\right) \) denote the storage in reservoir j at the end of week t, and let \(C_{T}(x)\) be the known expected future cost of the system at the end of week T when reservoir levels are x. The risk-neutral hydrothermal scheduling model can then be formulated as follows
This description uses the following indices:
Index Refers to
- t :
-
Index of week
- i :
-
Node in transmission network
- b :
-
Index of load block
- m :
-
Index of plant
- j :
-
Index of reservoir
- \(\mathcal {N}\) :
-
Set of nodes in transmission network
- \(\mathcal {F}(i)\) :
-
Set of thermal plants at node i
- \(\mathcal {H}(i)\) :
-
Set of hydro plants at node i
- \(\mathcal {J}\) :
-
Set of reservoirs
and the following notation for parameters and decision variables:
Symbol | Meaning | Units |
---|---|---|
\(\phi _{m}\) | Short-run marginal cost of thermal plant m | $/MWh |
\(f_{m}(b,t)\) | Generation of thermal plant m in load block b in week t | MW |
\(x_{j}(t)\) | Storage in reservoir j at end of week t | \(\text {m}^{3}\) |
\(\bar{x}_{j}\) | Known storage in reservoir j at start of week t | \(\text {m}^{3}\) |
h(b, t) | Vector of hydro releases in block b, week t | \(\text {m}^{3}/\text {s}\) |
s(b, t) | Vector of hydro spills in block b, week t | \(\text {m}^{3}/\text {s}\) |
\(\omega (t)\) | Inflow (assumed constant over the week) | \(\text {m}^{3}/\text {s}\) |
\(\gamma _{m}\) | Conversion factor for water flow into energy | \(\text {MWs}/\text {m}^{3}\) |
\(D_{i}(b,t)\) | Electricity demand in node i in block b, week t | MW |
T(b, t) | Number of hours in load block b in week t | h |
S | Number of seconds per hour (3600) | |
y(b, t) | Flow in transmission lines in load block b in week t | MW |
\(g_{i}(y)\) | Sum of flow into node i when transmission flows are y | MW |
\(a_{m}\) | Thermal plant capacity | MW |
\(b_{m}\) | Hydro plant capacity | \(\text {m}^{3}/\text {s}\) |
\(c_{m}\) | Spillway capacity | \(\text {m}^{3}/\text {s}\) |
\(r_{j}\) | Reservoir capacity | \(\text {m}^{3}\) |
Y | Feasible set of transmission flows | |
A | Node-arc incidence matrix of water network |
Rights and permissions
About this article
Cite this article
Philpott, A.B., de Matos, V.L. & Kapelevich, L. Distributionally robust SDDP. Comput Manag Sci 15, 431–454 (2018). https://doi.org/10.1007/s10287-018-0314-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10287-018-0314-0