Abstract
We present a clustering-based preconditioning strategy for KKT systems arising in stochastic programming within an interior-point framework. The key idea is to perform adaptive clustering of scenarios (inside-the-solver) based on their influence on the problem at hand. This approach thus contrasts with existing (outside-the-solver) approaches that cluster scenarios based on problem data alone. We derive spectral and error properties for the preconditioner and demonstrate that scenario compression rates of up to 94 % can be obtained, leading to dramatic computational savings. In addition, we demonstrate that the proposed preconditioner can avoid scalability issues of Schur decomposition in problems with large first-stage dimensionality.
Similar content being viewed by others
Notes
We make a slight remark regarding notation: \(K_\mathcal {S}\) is a block diagonal matrix while \(K_S\) in an entry of such block matrix. A similar observation applies to matrices \(B_\mathcal {S}\) and vectors \(q_\mathcal {S},t_\mathcal {S}\) with corresponding entries \(B_S,q_S,t_S\).
References
Birge, J.: Aggregation bounds in stochastic linear programming. Math. Progr. 31, 25–41 (1985)
Bishop, C.M., et al.: Pattern recognition and machine learning, vol. 4. Springer, New York (2006)
Byrd, R.H., Chin, G.M., Neveitt, W., Nocedal, J.: On the use of stochastic Hessian information in optimization methods for machine learning. SIAM J. Optim. 21(3), 977–995 (2011)
Calafiore, G.C., Campi, M.C.: The scenario approach to robust control design. IEEE Trans. Autom. Control 51(5), 742–753 (2006)
Casey, M.S., Sen, S.: The scenario generation algorithm for multistage stochastic linear programming. Math. Oper. Res. 30(3), 615–631 (2005)
Chiang, N., Grothey, A.: Solving security constrained optimal power flow problems by a structure exploiting interior point method. Optim. Eng. pp. 1–23 (2012)
Colombo, M., Gondzio, J., Grothey, A.: A warm-start approach for large-scale stochastic linear programs. Math. Progr. 127(2), 371–397 (2011)
de Oliveira, W.L., Sagastizábal, C., Penna, D., Maceira, M., Damázio, J.M.: Optimal scenario tree reduction for stochastic streamflows in power generation planning problems. Optim. Methods Softw. 25(6), 917–936 (2010)
Dollar, H.S.: Constraint-style preconditioners for regularized saddle point problems. SIAM J. Matrix Anal. Appl. 29(2), 672–684 (2007)
Dupačová, J., Gröwe-Kuska, N., Römisch, W.: Scenario reduction in stochastic programming. Math. Progr. 95(3), 493–511 (2003)
Ferris, M.C., Munson, T.S.: Interior-point methods for massive support vector machines. SIAM J. Optim. 13(3), 783–804 (2002)
Gondzio, J., Grothey, A.: Reoptimization with the primal-dual interior point method. SIAM J. Optim. 13, 842–864 (2003)
Heitsch, H., Römisch, W.: Scenario tree reduction for multistage stochastic programs. Comput. Manag. Sci. 6, 117–133 (2009)
Jung, J., Oleary, D.P., Tits, A.L.: Adaptive constraint reduction for training support vector machines. Electron. Trans. Numer. Anal. 31, 156–177 (2008)
Kang, J., Cao, Y., Word, D.P., Laird, C.D.: An interior-point method for efficient solution of block-structured NLP problems using an implicit Schur-complement decomposition. Comput. Chem. Eng. (2014, in press)
Latorre, J.M., Cerisola, S., Ramos, A.: Clustering algorithms for scenario tree generation: application to natural hydro inflows. Eur. J. Oper. Res. 181(3), 1339–1353 (2007)
Linderoth, J., Shapiro, A., Wright, S.: The empirical behavior of sampling methods for stochastic programming. Ann. Oper. Res. 142(1), 215–241 (2006)
Lubin, M., Petra, C.G., Anitescu, M., Zavala, V.M.: Scalable stochastic optimization of complex energy systems. In: IEEE international conference for high performance computing, networking, storage and analysis (SC). pp. 1–10 (2011)
Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2, 575–601 (1992)
Petra, C., Anitescu, M.: A preconditioning technique for Schur complement systems arising in stochastic optimization. Comput. Optim. Appl. 52, 315–344 (2012)
Pritchard, G., Zakeri, G., Philpott, A.: A single-settlement, energy-only electric power market for unpredictable and intermittent participants. Oper. Res. 58(4–part–2), 1210–1219 (2010)
Shetty, C.M., Taylor, R.W.: Solving large-scale linear programs by aggregation. Comput. Oper. Res. 14(5), 385–393 (1987)
Szyld, D.B., Vogel, J.A.: Fqmr: a flexible quasi-minimal residual method with inexact preconditioning. SIAM J. Sci. Comput. 23(2), 363–380 (2001)
Tits, A., Absil, P., Woessner, W.: Constraint reduction for linear programs with many inequality constraints. SIAM J. Optim. 17(1), 119–146 (2006)
Zavala, V.M., Botterud, A., Constantinescu, E.M., Wang, J.: Computational and economic limitations of dispatch operations in the next-generation power grid. In: IEEE conference on innovative technologies for and efficient and reliable power supply (2010)
Zavala, V.M., Constantinescu, E.M., Krause, T., Anitescu, M.: On-line economic optimization of energy systems using weather forecast information. J. Process Control 19(10), 1725–1736 (2009)
Zavala, V.M., Kim, K., Anitescu, M., Birge, J.: A stochastic market clearing formulation with consistent pricing properties. Technical Report ANL/MCS-P5110-0314, Argonne National Laboratory (2015)
Zipkin, P.H.: Bounds for row-aggregation in linear programming. Oper. Res. 28(4), 903–916 (1980)
Acknowledgments
Victor M. Zavala acknowledges funding from the DOE Office of Science under the Early Career program. Carl Laird and Yankai Cao acknowledge support by the National Science Foundation CAREER Grant CBET #0955205. The authors thank Jacek Gondzio for providing feedback on a previous version of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cao, Y., Laird, C.D. & Zavala, V.M. Clustering-based preconditioning for stochastic programs. Comput Optim Appl 64, 379–406 (2016). https://doi.org/10.1007/s10589-015-9813-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-015-9813-x