Skip to main content
SpringerLink
Log in

Clustering-based preconditioning for stochastic programs

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We present a clustering-based preconditioning strategy for KKT systems arising in stochastic programming within an interior-point framework. The key idea is to perform adaptive clustering of scenarios (inside-the-solver) based on their influence on the problem at hand. This approach thus contrasts with existing (outside-the-solver) approaches that cluster scenarios based on problem data alone. We derive spectral and error properties for the preconditioner and demonstrate that scenario compression rates of up to 94 % can be obtained, leading to dramatic computational savings. In addition, we demonstrate that the proposed preconditioner can avoid scalability issues of Schur decomposition in problems with large first-stage dimensionality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. We make a slight remark regarding notation: \(K_\mathcal {S}\) is a block diagonal matrix while \(K_S\) in an entry of such block matrix. A similar observation applies to matrices \(B_\mathcal {S}\) and vectors \(q_\mathcal {S},t_\mathcal {S}\) with corresponding entries \(B_S,q_S,t_S\).

References

  1. Birge, J.: Aggregation bounds in stochastic linear programming. Math. Progr. 31, 25–41 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bishop, C.M., et al.: Pattern recognition and machine learning, vol. 4. Springer, New York (2006)

    MATH  Google Scholar 

  3. Byrd, R.H., Chin, G.M., Neveitt, W., Nocedal, J.: On the use of stochastic Hessian information in optimization methods for machine learning. SIAM J. Optim. 21(3), 977–995 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Calafiore, G.C., Campi, M.C.: The scenario approach to robust control design. IEEE Trans. Autom. Control 51(5), 742–753 (2006)

    Article  MathSciNet  Google Scholar 

  5. Casey, M.S., Sen, S.: The scenario generation algorithm for multistage stochastic linear programming. Math. Oper. Res. 30(3), 615–631 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chiang, N., Grothey, A.: Solving security constrained optimal power flow problems by a structure exploiting interior point method. Optim. Eng. pp. 1–23 (2012)

  7. Colombo, M., Gondzio, J., Grothey, A.: A warm-start approach for large-scale stochastic linear programs. Math. Progr. 127(2), 371–397 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. de Oliveira, W.L., Sagastizábal, C., Penna, D., Maceira, M., Damázio, J.M.: Optimal scenario tree reduction for stochastic streamflows in power generation planning problems. Optim. Methods Softw. 25(6), 917–936 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  9. Dollar, H.S.: Constraint-style preconditioners for regularized saddle point problems. SIAM J. Matrix Anal. Appl. 29(2), 672–684 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  10. Dupačová, J., Gröwe-Kuska, N., Römisch, W.: Scenario reduction in stochastic programming. Math. Progr. 95(3), 493–511 (2003)

    Article  MATH  Google Scholar 

  11. Ferris, M.C., Munson, T.S.: Interior-point methods for massive support vector machines. SIAM J. Optim. 13(3), 783–804 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  12. Gondzio, J., Grothey, A.: Reoptimization with the primal-dual interior point method. SIAM J. Optim. 13, 842–864 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  13. Heitsch, H., Römisch, W.: Scenario tree reduction for multistage stochastic programs. Comput. Manag. Sci. 6, 117–133 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Jung, J., Oleary, D.P., Tits, A.L.: Adaptive constraint reduction for training support vector machines. Electron. Trans. Numer. Anal. 31, 156–177 (2008)

    MathSciNet  MATH  Google Scholar 

  15. Kang, J., Cao, Y., Word, D.P., Laird, C.D.: An interior-point method for efficient solution of block-structured NLP problems using an implicit Schur-complement decomposition. Comput. Chem. Eng. (2014, in press)

  16. Latorre, J.M., Cerisola, S., Ramos, A.: Clustering algorithms for scenario tree generation: application to natural hydro inflows. Eur. J. Oper. Res. 181(3), 1339–1353 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  17. Linderoth, J., Shapiro, A., Wright, S.: The empirical behavior of sampling methods for stochastic programming. Ann. Oper. Res. 142(1), 215–241 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  18. Lubin, M., Petra, C.G., Anitescu, M., Zavala, V.M.: Scalable stochastic optimization of complex energy systems. In: IEEE international conference for high performance computing, networking, storage and analysis (SC). pp. 1–10 (2011)

  19. Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2, 575–601 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  20. Petra, C., Anitescu, M.: A preconditioning technique for Schur complement systems arising in stochastic optimization. Comput. Optim. Appl. 52, 315–344 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  21. Pritchard, G., Zakeri, G., Philpott, A.: A single-settlement, energy-only electric power market for unpredictable and intermittent participants. Oper. Res. 58(4–part–2), 1210–1219 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  22. Shetty, C.M., Taylor, R.W.: Solving large-scale linear programs by aggregation. Comput. Oper. Res. 14(5), 385–393 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  23. Szyld, D.B., Vogel, J.A.: Fqmr: a flexible quasi-minimal residual method with inexact preconditioning. SIAM J. Sci. Comput. 23(2), 363–380 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  24. Tits, A., Absil, P., Woessner, W.: Constraint reduction for linear programs with many inequality constraints. SIAM J. Optim. 17(1), 119–146 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  25. Zavala, V.M., Botterud, A., Constantinescu, E.M., Wang, J.: Computational and economic limitations of dispatch operations in the next-generation power grid. In: IEEE conference on innovative technologies for and efficient and reliable power supply (2010)

  26. Zavala, V.M., Constantinescu, E.M., Krause, T., Anitescu, M.: On-line economic optimization of energy systems using weather forecast information. J. Process Control 19(10), 1725–1736 (2009)

    Article  Google Scholar 

  27. Zavala, V.M., Kim, K., Anitescu, M., Birge, J.: A stochastic market clearing formulation with consistent pricing properties. Technical Report ANL/MCS-P5110-0314, Argonne National Laboratory (2015)

  28. Zipkin, P.H.: Bounds for row-aggregation in linear programming. Oper. Res. 28(4), 903–916 (1980)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

Victor M. Zavala acknowledges funding from the DOE Office of Science under the Early Career program. Carl Laird and Yankai Cao acknowledge support by the National Science Foundation CAREER Grant CBET #0955205. The authors thank Jacek Gondzio for providing feedback on a previous version of the manuscript.

Author information

Authors and Affiliations

  1. School of Chemical Engineering, Purdue University, 480 Stadium Mall Drive, West Lafayette, IN, 47907, USA

    Yankai Cao & Carl D. Laird

  2. Department of Chemical and Biological Engineering, University of Wisconsin-Madison, 1415 Engineering Hall, Madison, WI, 53706-1691, USA

    Victor M. Zavala

Authors
  1. Yankai Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carl D. Laird
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Victor M. Zavala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Victor M. Zavala.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, Y., Laird, C.D. & Zavala, V.M. Clustering-based preconditioning for stochastic programs. Comput Optim Appl 64, 379–406 (2016). https://doi.org/10.1007/s10589-015-9813-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-015-9813-x

Keywords

Search

Navigation