Skip to main content
Springer Nature Link
Log in

Algorithmic innovations and software for the dual decomposition method applied to stochastic mixed-integer programs

  • Full Length Paper
  • Published:
Mathematical Programming Computation Aims and scope Submit manuscript

Abstract

We present algorithmic innovations for the dual decomposition method to address two-stage stochastic programs with mixed-integer recourse and provide an open-source software implementation that we call DSP. Our innovations include the incorporation of Benders-like cuts in a dual decomposition framework to tighten Lagrangian subproblems and aid the exclusion of infeasible first-stage solutions for problems without (relative) complete recourse. We also use an interior-point cutting-plane method with new termination criteria for solving the Lagrangian master problem. We prove that the algorithm converges to an optimal solution of the Lagrangian dual problem in a finite number of iterations, and we also prove that convergence can be achieved even if the master problem is solved suboptimally. DSP can solve instances specified in C code, SMPS files, and Julia script. DSP also implements a standard Benders decomposition method and a dual decomposition method based on subgradient dual updates that we use to perform benchmarks. We present extensive numerical results using SIPLIB instances and a large unit commitment problem to demonstrate that the proposed innovations provide significant improvements in the number of iterations and solution times. The software reviewed as part of this submission has been given the Digital Object Identifier (DOI) https://doi.org/10.5281/zenodo.998971.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Achterberg, T.: SCIP: solving constraint integer programs. Math. Program. Comput. 1(1), 1–41 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Ahmed, S.: A scenario decomposition algorithm for 0–1 stochastic programs. Oper. Res. Lett. 41(6), 565–569 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  3. Ahmed, S., Tawarmalani, M., Sahinidis, N.V.: A finite branch-and-bound algorithm for two-stage stochastic integer programs. Math. Program. 100(2), 355–377 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bezanson, J., Karpinski, S., Shah, V.B., Edelman, A.: Julia: a fast dynamic language for technical computing. arXiv preprint arXiv:1209.5145 (2012)

  5. Birge, J.R., Dempster, M.A., Gassmann, H.I., Gunn, E.A., King, A.J., Wallace, S.W.: A standard input format for multiperiod stochastic linear programs. IIASA Laxenburg Austria (1987)

  6. Birge, J.R., Louveaux, F.: Introduction to Stochastic Programming. Springer, Berlin (2011)

    Book  MATH  Google Scholar 

  7. Bixby, R.E.: Solving real-world linear programs: a decade and more of progress. Oper. Res. 50(1), 3–15 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  8. Carøe, C.C., Schultz, R.: Dual decomposition in stochastic integer programming. Oper. Res. Lett. 24(1–2), 37–45 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  9. Crainic, T.G., Fu, X., Gendreau, M., Rei, W., Wallace, S.W.: Progressive hedging-based metaheuristics for stochastic network design. Networks 58(2), 114–124 (2011)

    MathSciNet  MATH  Google Scholar 

  10. Dawande, M., Hooker, J.N.: Inference-based sensitivity analysis for mixed integer/linear programming. Oper. Res. 48(4), 623–634 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  11. Fisher, M.L.: An applications oriented guide to lagrangian relaxation. Interfaces 15(2), 10–21 (1985)

    Article  Google Scholar 

  12. Fisher, M.L.: The Lagrangian relaxation method for solving integer programming problems. Manag. Sci. 50(12–supplement), 1861–1871 (2004)

  13. Forrest, J.: Cbc. https://projects.coin-or.org/Cbc

  14. Forrest, J.: Clp. https://projects.coin-or.org/Clp

  15. Frangioni, A.: About lagrangian methods in integer optimization. Ann. Oper. Res. 139(1), 163–193 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  16. Gade, D., Küçükyavuz, S., Sen, S.: Decomposition algorithms with parametric gomory cuts for two-stage stochastic integer programs. Math. Program. 144(1–2), 39–64 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  17. Gamrath, G., Lübbecke, M.E.: Experiments with a generic Dantzig–Wolfe decomposition for integer programs. In: International Symposium on Experimental Algorithms, pp. 239–252. Springer (2010)

  18. Gassmann, H.I., Schweitzer, E.: A comprehensive input format for stochastic linear programs. Ann. Oper. Res. 104(1–4), 89–125 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  19. Geoffrion, A.M.: Lagrangean Relaxation for Integer Programming. Springer, Berlin (1974)

    Book  MATH  Google Scholar 

  20. Gertz, E.M., Wright, S.J.: Object-oriented software for quadratic programming. ACM Trans. Math. Softw. (TOMS) 29(1), 58–81 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Goffin, J.L., Vial, J.P.: Cutting planes and column generation techniques with the projective algorithm. J. Optim. Theory Appl. 65(3), 409–429 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  22. Gondzio, J.: Warm start of the primal-dual method applied in the cutting-plane scheme. Math. Program. 83(1–3), 125–143 (1998)

    MathSciNet  MATH  Google Scholar 

  23. Gondzio, J., Gonzalez-Brevis, P., Munari, P.: New developments in the primal-dual column generation technique. Eur. J. Oper. Res. 224(1), 41–51 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  24. Gondzio, J., Grothey, A.: A new unblocking technique to warmstart interior point methods based on sensitivity analysis. SIAM J. Optim. 19(3), 1184–1210 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  25. Gondzio, J., Sarkissian, R.: Column generation with a primal-dual method. Relatorio tecnico, University of Geneva 102 (1996)

  26. Guo, G., Hackebeil, G., Ryan, S.M., Watson, J.P., Woodruff, D.L.: Integration of progressive hedging and dual decomposition in stochastic integer programs. Oper. Res. Lett. 43(3), 311–316 (2015)

    Article  MathSciNet  Google Scholar 

  27. Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2015). http://www.gurobi.com

  28. Helmberg, C.: ConicBundle. https://www-user.tu-chemnitz.de/~helmberg/ (2004)

  29. IBM Corp.: IBM ILOG CPLEX Optimization Studio 12.6.1 (2014). http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/index.html

  30. Kim, K., Mehrotra, S.: A two-stage stochastic integer programming approach to integrated staffing and scheduling with application to nurse management. Oper. Res. 63(6), 1431–1451 (2015). https://doi.org/10.1287/opre.2015.1421

    Article  MathSciNet  MATH  Google Scholar 

  31. King, A.: Stochastic Modeling Interface (2007). https://projects.coin-or.org/Smi

  32. Kleywegt, A.J., Shapiro, A., Homem-de Mello, T.: The sample average approximation method for stochastic discrete optimization. SIAM J. Optim. 12(2), 479–502 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  33. Laporte, G., Louveaux, F.V.: The integer L-shaped method for stochastic integer programs with complete recourse. Oper. Res. Lett. 13(3), 133–142 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  34. Lee, C., Liu, C., Mehrotra, S., Shahidehpour, M.: Modeling transmission line constraints in two-stage robust unit commitment problem. IEEE Trans. Power Syst. 29(3), 1221–1231 (2014)

    Article  Google Scholar 

  35. Linderoth, J., Wright, S.: Decomposition algorithms for stochastic programming on a computational grid. Comput. Optim. Appl. 24(2–3), 207–250 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  36. Løkketangen, A., Woodruff, D.L.: Progressive hedging and tabu search applied to mixed integer (0, 1) multistage stochastic programming. J. Heuristics 2(2), 111–128 (1996)

    Article  MATH  Google Scholar 

  37. Lubin, M., Dunning, I.: Computing in operations research using Julia. arXiv preprint arXiv:1312.1431 (2013)

  38. Lubin, M., Martin, K., Petra, C.G., Sandıkçı, B.: On parallelizing dual decomposition in stochastic integer programming. Oper. Res. Lett. 41(3), 252–258 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  39. Lubin, M., Petra, C.G., Anitescu, M., Zavala, V.: Scalable stochastic optimization of complex energy systems. In: 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pp. 1–10. IEEE (2011)

  40. Lulli, G., Sen, S.: A branch-and-price algorithm for multistage stochastic integer programming with application to stochastic batch-sizing problems. Manag. Sci. 50(6), 786–796 (2004)

    Article  MATH  Google Scholar 

  41. Märkert, A., Gollmer, R.: Users Guide to ddsip–a C package for the dual decomposition of two-stage stochastic programs with mixed-integer recourse (2014)

  42. Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  43. Mitchell, J.E.: Computational experience with an interior point cutting plane algorithm. SIAM J. Optim. 10(4), 1212–1227 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  44. OptiRisk Systems: FortSP: a stochastic programming solver, version 1.2 (2014). http://www.optirisk-systems.com/manuals/FortspManual.pdf

  45. Papavasiliou, A., Oren, S.S.: Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network. Oper. Res. 61(3), 578–592 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  46. Papavasiliou, A., Oren, S.S., O’Neill, R.P.: Reserve requirements for wind power integration: a scenario-based stochastic programming framework. IEEE Trans. Power Syst. 26(4), 2197–2206 (2011)

    Article  Google Scholar 

  47. Ralphs, T.K., Galati, M.V.: Decomposition in integer linear programming. Integer Program. Theory Pract. 3, 57–110 (2005)

  48. Ralphs, T.K., Hassanzadeh, A.: A generalization of Benders algorithm for two-stage stochastic optimization problems with mixed integer recourse. Technical Report 14T-005, Department of Industrial and Systems Engineering, Lehigh University (2014)

  49. Saltzman, M., Ladányi, L., Ralphs, T.: The COIN-OR open solver interface: technology overview. In: CORS/INFORMS Conference. Banff (2004)

  50. Santoso, T., Ahmed, S., Goetschalckx, M., Shapiro, A.: A stochastic programming approach for supply chain network design under uncertainty. Eur. J. Oper. Res. 167(1), 96–115 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  51. Sen, S., Higle, J.L.: The C\(^3\) theorem and a D\(^2\) algorithm for large scale stochastic mixed-integer programming: set convexification. Math. Program. 104(1), 1–20 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  52. Sherali, H.D., Fraticelli, B.M.: A modification of Benders’ decomposition algorithm for discrete subproblems: an approach for stochastic programs with integer recourse. J. Global Optim. 22(1–4), 319–342 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  53. Shinano, Y., Achterberg, T., Berthold, T., Heinz, S., Koch, T.: ParaSCIP: a parallel extension of SCIP. In: Competence in High Performance Computing 2010, pp. 135–148. Springer (2011)

  54. Shinano, Y., Heinz, S., Vigerske, S., Winkler, M.: Fiberscip-a shared memory parallelization of scip, pp. 13–55. Zuse Institute Berlin, Technical Report ZR (2013)

    Google Scholar 

  55. Tarhan, B., Grossmann, I.E.: Improving dual bound for stochastic MILP models using sensitivity analysis. Working paper (2015)

  56. Watson, J.P., Woodruff, D.L.: Progressive hedging innovations for a class of stochastic mixed-integer resource allocation problems. CMS 8(4), 355–370 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  57. Watson, J.P., Woodruff, D.L., Hart, W.E.: PySP: modeling and solving stochastic programs in Python. Math. Program. Comput. 4(2), 109–149 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  58. Wunderling, R.: Paralleler und objektorientierter Simplex-Algorithmus. Ph.D. Thesis, Technische Universität Berlin (1996). http://www.zib.de/Publications/abstracts/TR-96-09/

  59. Zhang, M., Kucukyavuz, S.: Finitely convergent decomposition algorithms for two-stage stochastic pure integer programs. SIAM J. Optim. 24(4), 1933–1951 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  60. Zverovich, V., Fábián, C.I., Ellison, E.F., Mitra, G.: A computational study of a solver system for processing two-stage stochastic LPs with enhanced Benders’ decomposition. Math. Program. Comput. 4(3), 211–238 (2012)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics and Computer Science Division, Argonne National Laboratory, 9700 South Cass Avenue, Lemont, IL, 60439, USA

    Kibaek Kim

  2. Department of Chemical and Biological Engineering, University of Wisconsin-Madison, 1415 Engineering Drive, Madison, WI, 53706, USA

    Victor M. Zavala

Authors
  1. Kibaek Kim
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Victor M. Zavala
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Kibaek Kim.

Additional information

This material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract number DE-AC02-06CH11357. V. M. Zavala acknowledges funding from the early career program of the U.S. Department of Energy under grant DE-SC0014114. We gratefully acknowledge the computing resources provided on Blues, a high-performance computing cluster operated by the Laboratory Computing Resource Center at Argonne National Laboratory. We also thank Julie Bessac for providing wind speed prediction data, Sven Leyffer for providing feedback on an earlier version of the manuscript, the SCIP team at the Zuse Institute Berlin for valuable comments and support, and the anonymous reviewers for constructive suggestions that considerably improved the original version of this article.

Appendices

A Computational results for SIPLIB test instances using FiberSCIP

We present the computational results from FiberSCIP (compiled with ug-0.7.5) [54] for solving SIPLIB test instances using 16 computing cores in extensive form. FiberSCIP terminates after 6 hours of solution. Results are reported in Table 8.

Table 8 Computational results for SIPLIB instances using FiberSCIP for solving the extensive forms
Full size table

For all the dcap instances, FiberSCIP found upper and lower bounds with small duality gap after 6 hours of solution. An optimal solution is found for dcap243_200 instance. Optimal solutions were also found for the sslp_5_25 and sslp_15_45 instances within the 6-hour time limit. However, most sslp_10_50 instances were not able to be solved by FiberSCIP, because of insufficient memory. Note that memory was not sufficient for solving the stochastic unit commitment problems by using ParaSCIP.

B Notations: two-stage stochastic unit commitment

We present notations for the two-stage stochastic unit commitment considered in Sect. 5.2.

Table 9 Notations for the stochastic unit commitment model
Full size table

C Formulation: two-stage stochastic unit commitment

We present a two-stage stochastic unit commitment model formulation, where the commitment decisions for slow generators are made in the first stage and the commitment decisions for fast generators and the power dispatch decision are made in the second stage. Notations are given in Table 9. In the model, we consider ramping constraints, reserve constraints and transmission line capacity constraints. We assume that the power generation cost is piecewise linear convex.

$$\begin{aligned} \min \;&\sum _{j\in \mathcal {S}} \sum _{t\in \mathcal {T}} \sum _{g\in \mathcal {G}} \pi _j \left( C^\text {fx}_g x_{jgt} + C^\text {up}_g u_{jgt} + C^\text {dn}_g v_{jgt} + \sum _{k\in K} C^\text {mar}_{gk} q_{jgkt}\right) \end{aligned}$$
(22a)
$$\begin{aligned} \text {s.t.} \;&1 - x_{jg(t-1)} \ge u_{jgt}, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22b)
$$\begin{aligned}&x_{jg(t-1)} \ge v_{jgt}, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22c)
$$\begin{aligned}&x_{jgt} - x_{jg(t-1)} = u_{jgt} - v_{jgt}, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22d)
$$\begin{aligned}&x_{jgt} \ge \sum _{\tau =\max \{1,t-UT_g+1\}}^t u_{jg\tau }, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22e)
$$\begin{aligned}&1 - x_{jgt} \ge \sum _{\tau =\max \{1,t-DT_g+1\}}^t u_{jg\tau }, \nonumber \\&\quad \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22f)
$$\begin{aligned}&-RD_g \le p_{jgt} - p_{jg(t-1)} \le RU_g - s_{jgt}, \nonumber \\&\;\forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22g)
$$\begin{aligned}&s_{jgt} \le RC_g x_{jgt}, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22h)
$$\begin{aligned}&\sum _{g\in \mathcal {G}} s_{jgt} \ge SR_t, \; \forall j\in \mathcal {S}, t\in \mathcal {T}, \end{aligned}$$
(22i)
$$\begin{aligned}&p_{jgt} = P^\text {min}_g x_{jgt} + \sum _{k\in \mathcal {K}} q_{jgkt}, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22j)
$$\begin{aligned}&p_{jgt} + s_{jgt} \le P^\text {max}_g x_{jgt}, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T}, \end{aligned}$$
(22k)
$$\begin{aligned}&q_{jgkt} \le Q^\text {max}_{gk} x_{jgt}, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, k\in \mathcal {K}, t\in \mathcal {T}, \end{aligned}$$
(22l)
$$\begin{aligned}&\sum _{g\in \mathcal {G}} p_{jgt} = \sum _{n\in \mathcal {N}} D_{jnt} - \sum _{w\in \mathcal {W}} W_{jwt}, \; \forall j\in \mathcal {S}, t\in \mathcal {T}, \end{aligned}$$
(22m)
$$\begin{aligned}&-F^\text {max}_l \le \sum _{g\in \mathcal {G}} LSF_{lg} p_{jgt} - \sum _{n\in \mathcal {N}} LSF_{ln} D_{jnt} \nonumber \\&\quad + \sum _{w\in \mathcal {W}} LSF_{lw} W_{jwt} \le F^\text {max}_l, \; \forall j\in \mathcal {S}, l\in \mathcal {L}, t\in \mathcal {T}, \end{aligned}$$
(22n)
$$\begin{aligned}&x_{igt} = x_{jgt}, \; u_{igt} = u_{jgt}, \; v_{igt} = v_{jgt}, \nonumber \\&\quad \; \forall i,j\in \mathcal {S}, g\in \mathcal {G}_s, t\in \mathcal {T} \end{aligned}$$
(22o)
$$\begin{aligned}&x_{jg0} = X^\text {init}_g, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, \end{aligned}$$
(22p)
$$\begin{aligned}&x_{jgt} = 1, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \{ 1, \dots , UT^\text {init}_g \}, \end{aligned}$$
(22q)
$$\begin{aligned}&x_{jgt} = 0, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \{1, \dots , DT^\text {init}_g\}, \end{aligned}$$
(22r)
$$\begin{aligned}&p_{jg0} = P^\text {init}_g, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, \end{aligned}$$
(22s)
$$\begin{aligned}&x_{jgt} \in \{0,1\}, \; 0\le u_{jgt}, v_{jgt} \le 1, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T} \end{aligned}$$
(22t)
$$\begin{aligned}&p_{jgt}, q_{jgkt}, s_{jgt} \ge 0, \; \forall j\in \mathcal {S}, g\in \mathcal {G}, t\in \mathcal {T} \end{aligned}$$
(22u)

The objective (22a) is to minimize the expected value of the sum of operating, start-up, shut-down, and production cost. Equations (22b)–(22d) ensure the logical relation of the commitment, start-up and shut-down decisions. Equations (22e) and (22f) respectively represent the minimum downtime and uptime of generators in each time period. Equations (22g) and (22h) are ramping constraints, and equation (22i) is a spinning reserve constraint. Equations (22j) and (22k) are the constraints for minimum power generation and maximum power generation, respectively. Equation (22l) represents the piecewise linearized power generation cost. Equations (22m) and (22n) are the flow balance constraint and the transmission line flow constraint, respectively. Equation (22o) ensures that the decisions for slow generators does not change for scenarios. This is also called a nonanticipativity constraint. Equations (22p)–(22s) represent the initial conditions of generators and production level.

D Julia model script: two-stage stochastic unit commitment

We provide the Julia model script for the stochastic unit commitment problem.

figure i
figure j

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, K., Zavala, V.M. Algorithmic innovations and software for the dual decomposition method applied to stochastic mixed-integer programs. Math. Prog. Comp. 10, 225–266 (2018). https://doi.org/10.1007/s12532-017-0128-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12532-017-0128-z

Keywords

Mathematics Subject Classification