Skip to main content
SpringerLink
Log in

Asymptotic behavior of solutions: An application to stochastic NLP

  • Full Length Paper
  • Series B
  • Published:
Mathematical Programming Submit manuscript

Abstract

In this article we study the consistency of optimal and stationary (KKT) points of a stochastic non-linear optimization problem involving expectation functionals, when the underlying probability distribution associated with the random variable is weakly approximated by a sequence of random probability measures. The optimization model includes constraints with expectation functionals those are not captured in direct application of the previous results on optimality conditions exist in the literature. We first study the consistency of stationary points of a general NLP problem with convex and locally Lipschitz data and then apply those results to the stochastic NLP problem and stochastic minimax problem. Moreover, we derive an exponential bound for such approximations using a large deviation principle.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andrews, D.W.K.: Consistency in nonlinear econometric models: a generic uniform law of large numbers. Econometrica 55, 1465–1471 (1987)

    MathSciNet  MATH  Google Scholar 

  2. Artstein, Z., Wets, R.J.-B.: Consistency of minimizers and the SLLN for stochastic programs. J. Convex Anal. 2, 1–17 (1995)

    MathSciNet  MATH  Google Scholar 

  3. Attouch, H., Wets, R.J.-B.: Approximation and convergence in nonlinear optimization. In: Nonlinear Programming (Madison, Wis., 1980), vol. 4, pp. 367–394. Academic Press, New York (1981)

  4. Aumann, R.J.: Integrals of set-valued functions. J. Math. Anal. Appl. 12, 1–12 (1965)

    MathSciNet  MATH  Google Scholar 

  5. Bastin, F., Cirillo, C., Toint, P.: Convergence theory for nonconvex stochastic programming with an application to mixed logit. Math. Program., Ser. B 108, 207–234 (2006)

    MathSciNet  MATH  Google Scholar 

  6. Bayraksan, G., Morton, D.: Assessing solution quality in stochastic programs. Math. Program. 108, 495–514 (2006)

    MathSciNet  MATH  Google Scholar 

  7. Billingsley, P.: Convergence of Probability Measures. Wiley Series in Probability and Statistics: Probability and Statistics, 2nd edn. Wiley, New York (1999)

  8. Birge, J.R., Qi, L.Q.: Subdifferential convergence in stochastic programs. SIAM J. Optim. 2, 436–453 (1995)

    MathSciNet  MATH  Google Scholar 

  9. Birge, J.R., Wets, R.J.-B.: Designing approximation schemes for stochastic problems, in particular for stochastic programs with recourse. Math. Program. Stud. 27, 54–102 (1986)

    MathSciNet  MATH  Google Scholar 

  10. Borwein, J.M., Vanderwerff, J.D.: Epigraphical and uniform convergence of convex functions. Trans. AMS 348, 1617–1631 (1996)

    MathSciNet  MATH  Google Scholar 

  11. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

    MATH  Google Scholar 

  12. Czarnecki, M.-O., Rifford, L.: Approximation and regularization of Lipschitz functions: convergence of the gradients. Trans. AMS 358, 4467–4520 (2006)

    MathSciNet  MATH  Google Scholar 

  13. Dai, L., Chen, C.H., Birge, J.R.: Convergence properties of two-stage stochastic programming. J. Optim. Theory Appl. 106, 489–509 (2000)

    MathSciNet  MATH  Google Scholar 

  14. Dupačová, J.: Minimax approach to stochastic linear programming and the moment problem (in Czech). Ekon.-Matemat. Obzor. 3, 279–307 (1977)

    MATH  Google Scholar 

  15. Dupačová, J.: Stability and sensitivity analysis for stochastic programming. Ann. Oper. Res. 27, 115–142 (1990)

    MathSciNet  MATH  Google Scholar 

  16. Dupačová, J.: Uncertainties in minimax stochastic programs. Optimization 60, 1235–1250 (2011)

    MathSciNet  MATH  Google Scholar 

  17. Dupac̆ová, J., Wets, R.J.-B.: Asymptotic behaviour of statistical estimators and of optimal solutions of stochastic optimization problems. Ann. Stat. 16, 1517–1549 (1988)

    MATH  Google Scholar 

  18. Hess, C.: Epi-convergence of sequences of normal integrands and strong consistency of the maximum likelihood estimator. Ann. Stat. 24, 1298–1315 (1996)

    MathSciNet  MATH  Google Scholar 

  19. Hess, C., Seri, R.: Generic consistency for approximate stochastic programming and statistical problems. SIAM J. Optim. 29, 290–317 (2019)

    MathSciNet  MATH  Google Scholar 

  20. Higle, J.L., Sen, S.: On the convergence of algorithms with implications for stochastic and nondifferentiable optimization. Math. Oper. Res. 17, 112–131 (1992)

    MathSciNet  MATH  Google Scholar 

  21. Homem-de-Mello, T.: On rates of convergence for stochastic optimization problems under non-independent and identically distributed sampling. SIAM J. Optim. 19, 524–551 (2008)

    MathSciNet  MATH  Google Scholar 

  22. Homem-de-Mello, T., Bayraksan, G.: Monte Carlo sampling-based methods for stochastic optimization. Surv. Oper. Res. Manag. Sci. 19, 56–85 (2014)

    MathSciNet  Google Scholar 

  23. Huber, P.J.: The behaviour of maximum likelihood estimates under non-constant conditions. In: Proc. 5th Berkeley Symp. Math. Statist. Probab., vol. 1, pp. 221–233. University of California Press (1949)

  24. Kall, P.: Approximation to optimization problems: an elementary review. Math. Oper. Res. 11, 9–18 (1986)

    MathSciNet  MATH  Google Scholar 

  25. Kall, P.: Stochastic programming with recourse: upper bounds and moment problems–a review. In: Guddat, J., et al. (eds.) Advances in Mathematical Optimization, pp. 86–103. Akademie, Berlin (1988)

    Google Scholar 

  26. Kaniovski, Y.M., King, A.J., Wets, R.J.-B.: Probabilistic bounds (via large deviations) for the solutions of stochastic programming problems. Ann. Oper. Res. 56, 189–208 (1995)

    MathSciNet  MATH  Google Scholar 

  27. Kaňková, V.: Stability in the stochastic programming. Kybernetika 14, 339–349 (1978)

    MathSciNet  MATH  Google Scholar 

  28. King, A.J., Rockafellar, R.T.: Asymptotic theory for solutions in statistical estimation and stochastic programming. Math. Oper. Res. 18, 148–162 (1993)

    MathSciNet  MATH  Google Scholar 

  29. Merkovsky, R.R., Ward, D.E.: General constraint qualifications in nondifferentiable programming. Math. Program. 47, 389–405 (1990)

    MathSciNet  MATH  Google Scholar 

  30. Newey, W.K.: Semiparamertic efficiency bounds. J. Appl. Econom. 5, 99–135 (1990)

    MATH  Google Scholar 

  31. Newey, W.K.: Uniform convergence in probability and stochastic equicontinuity. Econometrica 59, 1161–1167 (1991)

    MathSciNet  MATH  Google Scholar 

  32. Pflug, G.: Stochastic optimization and statistical inference. In: Ruszczyński, A., Shapiro, A. (eds.) Stochastic Programming. Handbooks in Operations Research and Management Science, vol. 10, pp. 427–482, Chapter 7. Elsevier, Amsterdam (2003)

  33. Pflug, G., Ruszczyński, A., Schultz, R.: On the Glivenko-Cantelli problem in stochastic programming: linear recourse and extensions. Math. Oper. Res. 23, 204–220 (1998)

    MathSciNet  MATH  Google Scholar 

  34. Pflug, G., Wozabal, D.: Ambiguity in portfolio selection. Quant. Finance 7, 435–442 (2007)

    MathSciNet  MATH  Google Scholar 

  35. Polak, E.: Optimization, Algorithms and Consistent Approximations. Applied Mathematical Sciences, vol. 124. Springer, New York (1997)

  36. Rassoul-Agha, F., Seppäläinen, T.: A course on large deviations with an introduction to Gibbs measures. Graduate Studies in Mathematics, vol. 162. American Mathematical Society, Providence (2015)

  37. Riis, M., Andersen, K.A.: Applying the minimax criterion in stochastic recourse programs. Eur. J. Oper. Res. 165, 569–584 (2005)

    MathSciNet  MATH  Google Scholar 

  38. Robinson, S.M.: Analysis of sample-path optimization. Math. Oper. Res. 21, 513–528 (1996)

    MathSciNet  MATH  Google Scholar 

  39. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317. Springer, Berlin (1998)

  40. Römisch, W.: Stability of stochastic programming problems. In: Ruszczyński, A., Shapiro, A. (eds.) Stochastic Programming. Handbooks in Operations Research and Management Science, vol. 10, pp. 483–554, Chapter 8. Elsevier, Amsterdam (2003)

  41. Römisch, W., Schultz, R.: Distribution sensitivity in stochastic programming. Math. Program. 50, 197–226 (1991)

    MathSciNet  MATH  Google Scholar 

  42. Shapiro, A.: Asymptotic properties of statistical estimators in stochastic programming. Ann. Stat. 17, 841–858 (1989)

    MathSciNet  MATH  Google Scholar 

  43. Shapiro, A.: Asymptotic analysis of stochastic programs. Ann. Oper. Res. 30, 169–186 (1991)

    MathSciNet  MATH  Google Scholar 

  44. Shapiro, A.: Asymptotic behavior of optimal solutions in stochastic programming. Math. Oper. Res. 18, 829–845 (1993)

    MathSciNet  MATH  Google Scholar 

  45. Shapiro, A.: Simulation-based optimization–convergence analysis and statistical inference. Stoch. Models 12, 425–454 (1996)

    MathSciNet  MATH  Google Scholar 

  46. Shapiro, A., Homem-de-Mello, T.: On the rate of convergence of optimal solutions of Monte Carlo approximations of stochastic programs. SIAM J. Optim. 11, 70–86 (2000)

    MathSciNet  MATH  Google Scholar 

  47. Shapiro, A.: Monte Carlo sampling methods. In: Ruszczyński, A., Shapiro, A. (eds.) Stochastic Programming. Handbooks in Operations Research and Management Science, vol. 10, pp. 353–425, Chapter 6. Elsevier, Amsterdam (2003)

  48. Shapiro, A., Kleywegt, A.: Minimax analysis of stochastic programs. Optim. Methods Softw. 17, 532–542 (2002)

    MATH  Google Scholar 

  49. Vogel, S.: Stability results for stochastic programming problems. Optimization 19, 269–288 (1988)

    MathSciNet  MATH  Google Scholar 

  50. Vogel, S.: A stochastic approach to stability in stochastic programming. J. Comput. Appl. Math. 56, 65–96 (1994)

    MathSciNet  MATH  Google Scholar 

  51. Vogel, S., Lachout, P.: On continuous convergence and epi-convergence of random functions, part I: theory an relations, part II: sufficient conditions and applications. Kybernetika 39, 75–98 and 99-118 (2003)

  52. Wald, A.: Note on the consistency of the maximum likelihood estimate. Ann. Math. Stat. 20, 595–601 (1949)

    MathSciNet  MATH  Google Scholar 

  53. Wang, W., Ahmed, S.: Sample average approximation of expected value constrained stochastic programs. Oper. Res. Lett. 36, 515–519 (2008)

    MathSciNet  MATH  Google Scholar 

  54. Wets, R.J.-B.: Modelling and solution strategies for unconstrained stochastic optimization problems. Ann. Oper. Res. 1, 3–22 (1984)

    Google Scholar 

  55. Wolfowitz, J.: On Wald’s proof of the consistency of the maximum likelihood estimate. Ann. Math. Stat. 20, 601–602 (1949)

    MathSciNet  MATH  Google Scholar 

  56. Zervos, M.: On the epiconvergence of stochastic optimization problems. Math. Oper. Res. 24, 495–508 (1999)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Chicago Booth School of Business, Chicago, IL, 60637, USA

    Arnab Sur & John R. Birge

Authors
  1. Arnab Sur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John R. Birge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arnab Sur.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research is supported by the University of Chicago Booth School of Business.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sur, A., Birge, J.R. Asymptotic behavior of solutions: An application to stochastic NLP. Math. Program. 191, 281–306 (2022). https://doi.org/10.1007/s10107-020-01554-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-020-01554-6

Keywords

Mathematics Subject Classification

Search

Navigation