Skip to main content
SpringerLink
Log in

Scalable parallel computations forlarge-scale stochastic programming

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Stochastic programming provides an effective framework for addressing decision problemsunder uncertainty in diverse fields. Stochastic programs incorporate many possiblecontingencies so as to proactively account for randomness in their input data; thus, theyinevitably lead to very large optimization programs. Consequently, efficient algorithms thatcan exploit the capabilities of advanced computing technologies ‐ including multiprocessorcomputers ‐ become imperative to solve large‐scale stochastic programs. This paper surveysthe state‐of‐the‐art in parallel algorithms for stochastic programming. Algorithms are reviewed,classified and compared. Qualitative comparisons are based on the applicability, scope, easeof implementation, robustness and reliability of each algorithm, while quantitative comparisonsare based on the computational performance of algorithmic implementations onmultiprocessor systems. Emphasis is placed on the potential of parallel algorithms to solvelarge‐scale stochastic programs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. K.A. Ariyawansa and D.D. Hudson, Performance of a benchmark parallel implementation of the Van Slyke and Wets algorithm for two-stage stochastic programs on the Sequent/Balance, Concurrency: Practice and Experience 3(1991)109-128.

    Google Scholar 

  2. E.M.L. Beale, On minimizing a convex function subject to linear inequalities, J. Royal Stat. Soc. 17B(1955)173-184.

    Google Scholar 

  3. A.J. Berger, J.M. Mulvey, E. Rothberg and R.J. Vanderbei, Solving multistage stochastic programs using tree dissection, SIAM J. Optim., to appear.

  4. A.J. Berger, J.M. Mulvey and A. Ruszczyński, An extension of the DQA algorithm to convex stochastic programs, SIAM J. Optim. 4(1994)735-753.

    Google Scholar 

  5. N.J. Berland, Load balancing in hypercube solution of stochastic optimization problems, ORSA J. Comp. 7(1995)358-364.

    Google Scholar 

  6. D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Englewood Cliffs, 1989.

    Google Scholar 

  7. J.R. Birge, Decomposition and partitioning methods for multistage stochastic linear programs, Oper. Res. 33(1985)989-1007.

    Google Scholar 

  8. J.R. Birge, C.J. Donohue, D.F. Holmes and O.G. Svintsitski, A parallel implementation of the nested decomposition algorithm for multistage stochastic linear programs, Math. Prog. 75(1996)327-352.

    Google Scholar 

  9. J.R. Birge and D.F. Holmes, Efficient solution of two-stage stochastic linear programs using interior point methods, Comp. Optim. Appl. 1(1992)245-276.

    Google Scholar 

  10. J.R. Birge and F.V. Louveaux, A multicut algorithm for two-stage stochastic linear programs, Eur. J. Oper. Res. 34(1988)384-392.

    Google Scholar 

  11. J.R. Birge and L. Qi, Computing block-angular Karmarkar projections with applications to stochastic programming, Manag. Sci. 34(1988)1472-1479.

    Google Scholar 

  12. J.R. Birge and R.J-B Wets, Designing approximation schemes for stochastic optimization problems, in particular for stochastic programs with recourse, Math. Prog. Study 27(1986)54-102.

    Google Scholar 

  13. Y. Censor and S.A. Zenios, Parallel Optimization: Theory, Algorithms and Applications, Oxford Series in Numerical Analysis, Oxford University Press, Oxford, 1997.

    Google Scholar 

  14. G.B. Dantzig, Linear programming under uncertainty, Manag. Sci. 1(1955)197-206.

    Google Scholar 

  15. G.B. Dantzig, Planning under uncertainty using parallel computing, Ann. Oper. Res. 14(1988)1-16.

    Google Scholar 

  16. G.B. Dantzig and P.W. Glynn, Parallel processors for planning under uncertainty, Ann. Oper. Res. 22(1990)1-21.

    Google Scholar 

  17. G.B. Dantzig, J.K. Ho and G. Infanger, Solving stochastic linear programs on a hypercube multicomputer, Report SOL 91-10, Operations Research Department, Stanford University, Stanford, 1991.

    Google Scholar 

  18. G.B. Dantzig and G. Infanger, Large-scale stochastic linear programs: Importance sampling and Benders decomposition, Report SOL 91-4, Operations Research Department, Stanford University, Stanford, 1991.

    Google Scholar 

  19. J. Dupačová, Stochastic programming with incomplete information: A survey of results on post-optimization and sensitivity analysis, Optim. 18(1987)507-532.

    Google Scholar 

  20. J. Dupačová, Stability and sensitivity analysis for stochastic programming, Ann. Oper. Res. 27(1990)115-142.

    Google Scholar 

  21. J. Dupačová, Scenario-based stochastic programs: Resistance with respect to sample, Ann. Oper. Res. 64(1996)21-38.

    Google Scholar 

  22. J. Dupačová, M. Bertocchi and V. Moriggia, Postoptimality for scenario-based financial planning models with an application to bond portfolio management, in: World-wide Asset and Liability Management, eds. W.T. Ziemba and J.M. Mulvey, Cambridge University Press, Cambridge, 1997.

    Google Scholar 

  23. J. Eckstein, R. Qi, V.I. Ragulin and S.A. Zenios, Data-parallel implementations of dense linear programming algorithms, Report 92-05-06, Decision Sciences Department, The Wharton School, University of Pennsylvania, Philadelphia, 1992.

    Google Scholar 

  24. Y. Ermoliev and R.J-B Wets, Stochastic programming: An introduction, in: Numerical Techniques for Stochastic Optimization, eds. Y. Ermoliev and R.J-B Wets, Springer, Berlin, 1988, pp. 1-32.

    Google Scholar 

  25. H.I. Gassmann, MSLiP: A computer code for the multistage stochastic linear programming problem, Math. Prog. 47(1990)407-423.

    Google Scholar 

  26. H.I. Gassmann and S.W. Wallace, Solving linear programs with multiple right-hand sides: Pricing and ordering schemes, Ann. Oper. Res. 64(1996)237-259.

    Google Scholar 

  27. B. Golub, M. Holmer, R. McKendall, L. Pohlman and S.A. Zenios, Stochastic programming models for money management, Eur. J. Oper. Res. 85(1995)282-296.

    Google Scholar 

  28. J. Higle and S. Sen, Stochastic decomposition: An algorithm for two-stage linear programs with recourse, Math. Oper. Res. 16(1991)650-669.

    Google Scholar 

  29. M. Holmer, R. McKendall, C. Vassiadou-Zeniou and S.A. Zenios, Dynamic models for fixed-income portfolio management under uncertainty, J. Econ. Dynamics and Control 22(1998)1517-1541.

    Google Scholar 

  30. D. Holmes, A collection of stochastic programming problems, Technical Report 94-11, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, 1994.

    Google Scholar 

  31. G. Infanger, Monte Carlo (importance) sampling within a Benders decomposition algorithm for stochastic linear programs, Ann. Oper. Res. 39(1992)69-95.

    Google Scholar 

  32. E.R. Jessup, D. Yang and S.A. Zenios, Parallel factorization of structured matrices arising in stochastic programming, SIAM J. Optim. 4(1994)833-846.

    Google Scholar 

  33. P. Kall, A. Ruszczyński and K. Frauendorfer, Approximation techniques in stochastic programming, in: Numerical Techniques for Stochastic Optimization, eds. Y. Ermoliev and R.J-B Wets, Springer, Berlin, 1988, pp. 33-64.

    Google Scholar 

  34. P. Kall and S.W. Wallace, Stochastic Programming, Wiley, Chichester, 1994.

    Google Scholar 

  35. F.V. Louveaux, A solution method for multistage stochastic programs with application to an energy investment problem, Oper. Res. 28(1980)889-902.

    Google Scholar 

  36. I. Lustig, J.M. Mulvey and T.J. Carpenter, Formulating stochastic programs for interior point methods, Oper. Res. 39(1991)757-770.

    Google Scholar 

  37. J.M. Mulvey and A. Ruszczyński, A new scenario decomposition method for large-scale stochastic optimization, Oper. Res. 43(1995)477-490.

    Google Scholar 

  38. J.M. Mulvey, R.J. Vanderbei and S.A. Zenios, Robust optimization of large-scale systems, Oper. Res. 43(1995)264-281.

    Google Scholar 

  39. J.M. Mulvey and H. Vladimirou, Evaluation of a parallel hedging algorithm for stochastic network programming, in: Impact of Recent Computer Advances on Operations Research, eds. R. Sharda, B.L. Golden, E. Wasil, O. Balci and W. Stewart, North-Holland, New York, 1989, pp. 106-119.

    Google Scholar 

  40. J.M. Mulvey and H. Vladimirou, Solving multistage stochastic networks: An application of scenario aggregation, Networks 21(1991)619-643.

    Google Scholar 

  41. J.M. Mulvey and H. Vladimirou, Applying the progressive hedging algorithm to stochastic generalized networks, Ann. Oper. Res. 31(1991)399-424.

    Google Scholar 

  42. S.S. Nielsen and S.A. Zenios, A massively parallel algorithm for nonlinear stochastic network problems, Oper. Res. 41(1993)319-337.

    Google Scholar 

  43. S.S. Nielsen and S.A. Zenios, Proximal minimizations with D-functions and the massively parallel solution of linear stochastic network programs, Int. J. Supercomp. Appl. 7(1994)349-364.

    Google Scholar 

  44. S.S. Nielsen and S.A. Zenios, Scalable parallel Benders decomposition for stochastic linear programming, Parallel Comp. 23(1997)1069-1088.

    Google Scholar 

  45. S.S. Nielsen and S.A. Zenios, Solving multistage stochastic network programs on massively parallel computers, Math. Prog. 73(1996)227-250.

    Google Scholar 

  46. Optimization Subroutine Library: Guide and Reference, Release 2, IBM, Kingston, New York, July 1991, SC23-0519-02.

  47. R-J. Qi and S.A. Zenios, On the scalability of data-parallel decomposition algorithms for stochastic programs, J. Parallel Distrib. Comp. 22(1994)565-570.

    Google Scholar 

  48. R.T. Rockafellar and R.J-B Wets, Scenarios and policy aggregation in optimization under uncertainty, Math. Oper. Res. 16(1991)119-147.

    Google Scholar 

  49. A. Ruszczyński, A regularized decomposition method for minimizing a sum of polyhedral functions, Math. Prog. 35(1986)309-333.

    Google Scholar 

  50. A. Ruszczyński, Interior point methods in stochastic programming, Report WP-93-8, International Institute for Applied Systems Analysis, Laxenburg, Austria, 1993.

    Google Scholar 

  51. A. Ruszczyński, Regularized decomposition of stochastic programs: Algorithmic techniques and numerical results, Report WP-93-21, International Institute for Applied Systems Analysis, Laxenburg, Austria, 1993.

    Google Scholar 

  52. R.M. Van Slyke and R.J-B Wets, L-shaped linear programs with applications to optimal control and stochastic programming, SIAM J. Applied Math. 17(1969)638-663.

    Google Scholar 

  53. R.J. Vanderbei, LOQO user's manual, Technical Report SOR-92-5, Department of Civil Engineering and Operations Research, Princeton University, Princeton, 1992.

    Google Scholar 

  54. R.J. Vanderbei and T.J. Carpenter, Symmetric indefinite systems for interior point methods, Math. Prog. 58(1993)1-32.

    Google Scholar 

  55. H. Vladimirou, Stochastic networks: Solution methods and applications in financial planning, Ph.D. Thesis, Department of Civil Engineering and Operations Research, Princeton University, Princeton, 1991.

    Google Scholar 

  56. H. Vladimirou, Computational assessment of distributed decomposition methods for stochastic linear programs, Eur. J. Oper. Res. 108(1998)653-670.

    Google Scholar 

  57. H. Vladimirou and J.M. Mulvey, Parallel and distributed computing for stochastic network programming, Technical Report RC 17051, IBM Thomas J. Watson Research Center, Yorktown Heights, 1992.

    Google Scholar 

  58. H. Vladimirou and S.A. Zenios, Stochastic linear programs with restricted recourse, Eur. J. Oper. Res. 101(1997)177-192.

    Google Scholar 

  59. R.J-B Wets, Large-scale linear programming techniques in stochastic programming, in: Numerical Techniques for Stochastic Optimization, eds. Y. Ermoliev and R.J-B Wets, Springer, Berlin, 1988, pp. 65-94.

    Google Scholar 

  60. R.J-B Wets, Stochastic programming, in: Handbooks in Operations Research and Management Science: Optimization, eds. G.L. Nemhauser, A.H.G. Rinnooy Kan and M.J. Todd, North-Holland, Amsterdam, 1989, pp. 573-629.

    Google Scholar 

  61. D. Yang and S.A Zenios, A scalable parallel interior point algorithm for stochastic linear programming and robust optimization, Comp. Optim. Appl. 7(1997)143-158.

    Google Scholar 

  62. S.A. Zenios and R. A. Lasken, Nonlinear network optimization on a massively parallel Connection Machine, Ann. Oper. Res. 14(1988)147-165.

    Google Scholar 

  63. J.R. Birge and F.V. Louveaux, Introduction to Stochastic Programming, Springer, New York, 1997.

    Google Scholar 

Download references

Authors
  1. H. Vladimirou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S.A. Zenios
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vladimirou, H., Zenios, S. Scalable parallel computations forlarge-scale stochastic programming. Annals of Operations Research 90, 87–129 (1999). https://doi.org/10.1023/A:1018977102079

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018977102079

Keywords

Search

Navigation