Skip to main content
SpringerLink
Log in

A new hybrid stochastic approximation algorithm

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

In this paper, we consider optimizing the performance of a stochastic system that is too complex for theoretical analysis to be possible, but can be evaluated by using simulation or direct experimentation. To optimize the expected performance of such system as a function of several input parameters, we propose a hybrid stochastic approximation algorithm for finding the root of the gradient of the response function. At each iteration of the hybrid algorithm, alternatively, either an average of two independent noisy negative gradient directions or a scaled noisy negative gradient direction is selected. The almost sure convergence of the hybrid algorithm is established. Numerical comparisons of the hybrid algorithm with two other existing algorithms in a simple queueing system and five nonlinear unconstrained stochastic optimization problems show the advantage of the hybrid algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Andradóttir, S.: Stochastic optimization with applications to discrete event systems. Ph.D. dissertation, Department of Operations Research, Stanford University, Stanford, CA (1990)

  2. Andradóttir S.: A scaled stochastic application algorithm. Manag. Sci. 42, 475–498 (1996)

    Article  MATH  Google Scholar 

  3. Azadivar F., Talavage J.: Optimization of stochastic simulation models. Math. Comput. Simul. XXII, 231–241 (1980)

    Article  MathSciNet  Google Scholar 

  4. Bertsekas D.P., Tsitsiklis J.N.: Gradient convergence in gradient methods with errors. SIAM J. Optim. 10, 627–642 (2003)

    Article  MathSciNet  Google Scholar 

  5. Blum J.R.: Multidimensional stochastic approximation methods. Ann. Math. Stat. 25, 737–744 (1954)

    Article  MATH  Google Scholar 

  6. Broadie, M., Cicek, D., Zeevi, A.: General bounds and finite-time improvement for the kiefer-wolfowitz stochastic approximation algorithm. Oper. Res. (working paper) (2011)

  7. Fabian V.: On asymptotic normality in stochastic approximation. Ann. Math. Stat. 39, 1327–1332 (1968)

    Article  MathSciNet  MATH  Google Scholar 

  8. Fang H.-T., Chen H.-F.: Blind channel identification based on noisy observation by stochastic approximation method. J. Glob. Optim. 27, 249–271 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  9. Moré J.J., Garbow B.S., Hillstrom K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    Article  MATH  Google Scholar 

  10. Nemirovski A., Juditsky A., Lan G., Shapiro A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19, 1574–1609 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  11. Nevel’son M.B., Khas’minskij R.Z.: Stochastic Approximation and Recursive Estimation. American Mathematical Society, Providence, RI (1973)

    MATH  Google Scholar 

  12. Kesten H.: Accelerated stochastic approximation. Ann. Math. Stat. 29, 41–59 (1958)

    Article  MathSciNet  MATH  Google Scholar 

  13. Kiefer J., Wolfowitz J.: Stochastic estimation of the modulus of a regression function. Ann. Math. Stat. 23, 462–466 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  14. Kushner H.J., Clark D.S.: Stochastic approximation for constrained and unconstrained systems. Springer-Verleg, Berlin (1978)

    Book  Google Scholar 

  15. Pardalos P.M., Resende M.: Handbook of applied optimization. Oxford University Press, Oxford (2002)

    MATH  Google Scholar 

  16. Polyak B., Juditsky A.: Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30, 838–855 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  17. Robbins H., Monro S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  18. Schraudolph, N.N., Graepel, T.: Combining conjugate direction methods with stochastic approximation of gradients. In: Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics, Key West, FL, pp. 7–13 (2002)

  19. Robbins, H., Siegmund, D.: A convergence theorem for nonnegative almost supermartingales and some applications. In: Optimizing Methods in Statistics, pp. 233–257 (1971)

  20. Spall J.C.: Multivariate stochastic approximation using a simultanoues perturbation gradient approximation. IEEE Trans. Autom. Control 37, 332–341 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  21. Spall J.C.: Introduction to stochastic search and optimization. Wiley, London (2003)

    Book  MATH  Google Scholar 

  22. Xu Z., Dai Y.H.: A stochastic approximation frame algorithm with adaptive directions. Numer. Math. Theory Methods Appl. 1, 460–474 (2008)

    MathSciNet  MATH  Google Scholar 

  23. Xu Z.: A combined direction stochastic approximation algorithm. Optim. Lett. 4, 117–129 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  24. Xu, Z.: New combinatorial direction stochastic approximation algorithms. Optim. Methods Softw. (2011). doi:10.1080/10556788.2011.645542

  25. Xu, Z., Dai, Y.H.: New stochastic approximation algorithms with adaptive step sizes. Optim. Lett. (2011). doi:10.1007/s11590-011-0380-5

  26. Törn A., Ali M.M., Viitanen S.: Stochastic global optimization: problem classes and solution techniques. J. Glob. Optim. 14, 437–447 (1999)

    Article  MATH  Google Scholar 

  27. Uryasev S., Pardalos P.M.: Stochastic optimization: algorithms and applications. Kluwer Academic Publishers, Dordrecht (2001)

    MATH  Google Scholar 

  28. Wasan M.: Stochastic approximation. Cambridge University Press, Cambridge (1970)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai University, Shanghai, 200444, People’s Republic of China

    Zi Xu

  2. School of Mathematical Sciences, Fudan University, Shanghai, 200433, People’s Republic of China

    Xinming Wu

Authors
  1. Zi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinming Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinming Wu.

Additional information

Z. Xu was supported by China NSF under the Grant 11101261 and Key Disciplines of Shanghai Municipality (S30104).

X. Wu was supported by China NSF under the Grant 11026036.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xu, Z., Wu, X. A new hybrid stochastic approximation algorithm. Optim Lett 7, 593–606 (2013). https://doi.org/10.1007/s11590-012-0443-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-012-0443-2

Keywords

Search

Navigation