Skip to main content
Springer Nature Link
Log in

A regularized smoothing method for fully parameterized convex problems with applications to convex and nonconvex two-stage stochastic programming

  • Full Length Paper
  • Series B
  • Published:
Mathematical Programming Submit manuscript

Abstract

We present an approach to regularize and approximate solution mappings of parametric convex optimization problems that combines interior penalty (log-barrier) solutions with Tikhonov regularization. Because the regularized mappings are single-valued and smooth under reasonable conditions, they can be used to build a computationally practical smoothing for the associated optimal value function. The value function in question, while resulting from parameterized convex problems, need not be convex. One motivating application of interest is two-stage (possibly nonconvex) stochastic programming. We show that our approach, being computationally implementable, provides locally bounded upper bounds for the subdifferential of the value function of qualified convex problems. As a by-product of our development, we also recover that in the given setting the value function is locally Lipschitz continuous. Numerical experiments are presented for two-stage convex stochastic programming problems, comparing the approach with the bundle method for nonsmooth optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Ahmed, S.: Convexity and decomposition of mean-risk stochastic programs. Math. Program. 106(3), 433–446 (2006)

    Article  MathSciNet  Google Scholar 

  2. Attouch, H.: Convergence de fonctions convexes, de sous-differentiels et semi-groupes. CR Acad. Sci. Paris 284(1), 539–542 (1977)

    MathSciNet  MATH  Google Scholar 

  3. Bank, B., Guddat, J., Klatte, D., Kummer, B., Tammer, K.: Non-linear Parametric Optimization. Springer, Berlin (1983)

    MATH  Google Scholar 

  4. Bonnans, J., Gilbert, J., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization. Springer-Verlag, Theoretical and Practical Aspects. Universitext, Berlin (2006)

    MATH  Google Scholar 

  5. Burke, J., Hoheisel, T.: Epi-convergent smoothing with applications to convex composite functions. SIAM J. Optim. 23(3), 1457–1479 (2013)

    Article  MathSciNet  Google Scholar 

  6. Burke, J., Hoheisel, T.: Epi-convergence properties of smoothing by infimal convolution. Set-Valued Var. Anal. 25(1), 1–23 (2017)

    Article  MathSciNet  Google Scholar 

  7. Burke, J., Hoheisel, T., Kanzow, C.: Gradient consistency for integral-convolution smoothing functions. Set-Valued Var. Anal. 21(2), 359–376 (2013)

    Article  MathSciNet  Google Scholar 

  8. Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, Berlin (2000)

    Book  Google Scholar 

  9. Beck, A., Teboulle, M.: Smoothing and first order methods: a unified framework. SIAM J. Optim. 22(2), 557–580 (2012)

    Article  MathSciNet  Google Scholar 

  10. Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. 134(1), 71–99 (2012)

    Article  MathSciNet  Google Scholar 

  11. Chen, X., Qi, L., Sun, D.: Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities. Math. Comput. 67(222), 519–540 (1998)

    Article  MathSciNet  Google Scholar 

  12. Deák, I.: Two-stage stochastic problems with correlated normal variables: computational experiences. Ann. Oper. Res. 142(1), 79–97 (2006)

    Article  MathSciNet  Google Scholar 

  13. Dinh, N., Goberna, M., López, M.: On the stability of the optimal value and the optimal set in optimization problems. J. Convex Anal. 19, 927–953 (2012)

    MathSciNet  MATH  Google Scholar 

  14. Dempe, S., Mehlitz, P.: Lipschitz continuity of the optimal value function in parametric optimization. J. Global Optim. 61(2), 363–377 (2015)

    Article  MathSciNet  Google Scholar 

  15. Dentcheva, D., Ruszczyński, A., Shapiro, A.: Lectures on Stochastic Programming. SIAM, Philadelphia (2009)

    MATH  Google Scholar 

  16. Drummond, L., Svaiter, B.: On well definedness of the Central Path. J. Optim. Theory Appl. 102(2), 223–237 (1999)

    Article  MathSciNet  Google Scholar 

  17. Fiacco, A.V., Ishizuka, Y.: Sensitivity and stability analysis for nonlinear programming. Ann. Oper. Res. 27(1), 215–235 (1990)

    Article  MathSciNet  Google Scholar 

  18. Fiacco, A.V.: Introduction to Sensitivity and Stability Analysis in Nonlinear Programming (1983)

  19. Fiacco, A.V., McCormick, G.P.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, New York (1968)

    MATH  Google Scholar 

  20. Frangioni, A.: Generalized bundle methods. SIAM J. Optim. 13(1), 117–156 (2002)

    Article  MathSciNet  Google Scholar 

  21. Guo, L., Lin, G.-H., Ye, J., Zhang, J.: Sensitivity analysis of the value function for parametric mathematical programs with equilibrium constraints. SIAM J. Optim. 24(3), 1206–1237 (2014)

    Article  MathSciNet  Google Scholar 

  22. Hellemo, L., Barton, P., Tomasgard, A.: Decision-dependent probabilities in stochastic programs with recourse. CMS 15(3), 369–395 (2018)

    Article  MathSciNet  Google Scholar 

  23. Izmailov, A., Solodov, M.: A Note on Error Estimates for some Interior Penalty Methods. Recent Advances in Optimization. Lecture Notes in Economics and Mathematical Systems 563 (2006)

  24. Lang, S.: Real and Functional Analysis. Graduate Texts in Mathematics. Springer, Berlin (1993)

    Book  Google Scholar 

  25. Liu, J., Cui, Y., Pang, J.-S., Sen, S.: Two-stage Stochastic Programming with Linearly Biparameterized Quadratic Recourse, p. 2018. Tech. rep. The University of Southern California, December (2018)

  26. Mordukhovich, B.S., Nam, N.M., Yen, N.D.: Subgradients of marginal functions in parametric mathematical programming. Math. Program. 116(1–2), 369–396 (2009)

    Article  MathSciNet  Google Scholar 

  27. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I. Springer Berlin Heidelberg, Berlin (2006)

    Book  Google Scholar 

  28. Mordukhovich, B.S.: Variational Analysis and Applications. Springer, Berlin (2018)

    Book  Google Scholar 

  29. Moré, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)

    Article  MathSciNet  Google Scholar 

  30. Monteiro, R.D., Zhou, F.: On the existence and convergence of the central path for convex programming and some duality results. Comput. Optim. Appl. 10(1), 51–77 (1998)

    Article  MathSciNet  Google Scholar 

  31. Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)

    Article  MathSciNet  Google Scholar 

  32. Oliveira, W., Sagastizábal, C.: Level bundle methods for oracles with on-demand accuracy. Optim. Methods Softw. 29(6), 1180–1209 (2014)

    Article  MathSciNet  Google Scholar 

  33. Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex proximal bundle methods in depth: a unified analysis for inexact oracles. Math. Program. 148(1-2), 241–277 (2014)

    Article  MathSciNet  Google Scholar 

  34. Pérez-Aros, P.: Ergodic Approach to Robust Optimization and Infinite Programming Problems. preprint (2020)

  35. Qi, L., Sun, D., Zhou, G.: A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities. Math. Program. 87(1), 1–35 (2000)

    Article  MathSciNet  Google Scholar 

  36. Rockafellar, R., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Finance 26(7), 1443–1471 (2002)

    Article  Google Scholar 

  37. Rockafellar, T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (2009)

    MATH  Google Scholar 

  38. Ralph, D., Xu, H.: Implicit smoothing and its application to optimization with piecewise smooth equality constraints. J. Optim. Theory Appl. 124(3), 673–699 (2005)

    Article  MathSciNet  Google Scholar 

  39. Sagastizábal, C.: Divide to conquer: decomposition methods for energy optimization. Math. Program. 134(1), 187–222 (2012)

    Article  MathSciNet  Google Scholar 

  40. Van Slyke, R., Wets, R.: L-shaped linear programs with applications to control and stochastic programming. SIAM J. Appl. Math. 17, 638–663 (1969)

    Article  MathSciNet  Google Scholar 

  41. Wachter, A., Biegler, T.L.: On the implementation of a primal–dual interior point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)

    Article  MathSciNet  Google Scholar 

  42. Wright, S.J.: Primal–Dual Interior-Point Methods. Society for Industrial and Applied Mathematics, Philadelphia (1997)

    Book  Google Scholar 

  43. Xu, H., Ye, J.J.: Approximating stationary points of stochastic mathematical programs with equilibrium constraints via sample averaging. Set-Valued Var. Anal. 19(2), 283–309 (2010)

    Article  MathSciNet  Google Scholar 

  44. Xu, H., Ye, J.J.: Necessary optimality conditions for two-stage stochastic programs with equilibrium constraints. SIAM J. Optim. 20(4), 1685–1715 (2010)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors thank the referees and Editor for beneficial comments. The first and second authors are grateful to Ecole Polytechnique, France, for the support through the 2018–2019 Gaspard Monge Visiting Professor Program. Research of the second author is partly funded by CNPq Grant 306089/2019-0, CEPID CeMEAI, and FAPERJ in Brazil. The third author is supported by CNPq Grant 303913/2019-3, by FAPERJ Grant E-26/202.540/2019, by PRONEX–Optimization, and by the Russian Foundation for Basic Research Grant 19-51-12003 NNIOa.

Author information

Authors and Affiliations

  1. Instituto de Matemática Pura e Aplicada, Rio de Janeiro, RJ, Brazil

    Pedro Borges & Mikhail Solodov

  2. IMECC/UNICAMP, Campinas, São Paulo, Brazil

    Claudia Sagastizábal

Authors
  1. Pedro Borges
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Claudia Sagastizábal
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mikhail Solodov
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Claudia Sagastizábal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Borges, P., Sagastizábal, C. & Solodov, M. A regularized smoothing method for fully parameterized convex problems with applications to convex and nonconvex two-stage stochastic programming. Math. Program. 189, 117–149 (2021). https://doi.org/10.1007/s10107-020-01582-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-020-01582-2

Keywords

Mathematics Subject Classification