Skip to main content
SpringerLink
Log in

Stochastic approximation search algorithms with randomization at the input

  • Topical Issue
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

This work presents a comprehensive survey of the development of pseudogradient stochastic approximation algorithms with randomized input disturbance, considers the problems of their applicability in optimization problems with linear constraints, and discusses new possibilities to use them for multiagent control for load balancing of nodes in computational networks. Justifications of the algorithms’ correctness and their optimal convergence rate are based on the foundational works of B.T. Polyak.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Polyak, B.T., Vvedenie v optimizatsiyu, Moscow: Nauka, 1983. Translated into English under the title Introduction to Optimization, New York: Optimization Software, 1987.

    MATH  Google Scholar 

  2. Schweppe, F.C., Uncertain Dynamic Systems, New York: Prentice Hall, 1973.

    Google Scholar 

  3. Fel’dbaum, A.A., On Dual Control Problems, in Metody optimizatsii avtomaticheskikh sistem (Optimization Methods for Automated Systems), Moscow: Nauka, 1972, pp. 89–108.

    Google Scholar 

  4. Ljung, L., System Identification: Theory for the User, Englewood Cliffs: Prentice Hall, 1987. Translated under the title Identifikatsiya sistem. Teoriya dlya pol’zovatelya, Moscow: Nauka, 1991.

    MATH  Google Scholar 

  5. Tsypkin, Ya.Z., Informatsionnaya teoriya identifikatsii (Information Theory of Identification), Moscow: Nauka, 1995.

    MATH  Google Scholar 

  6. Bai, E.W., Nagpal, K.M., and Tempo, R., Bounded-Error Parameter Estimation: Noise Models and Recursive Algorithms, Automatica, 1996, vol. 32. pp. 985–999.

    Article  MATH  MathSciNet  Google Scholar 

  7. Polyak, B.T. and Shcherbakov, P.S., Robastnaya ustoichivost’ i upravlenie (Robust Stability and Control), Moscow: Nauka, 2002.

    Google Scholar 

  8. Polyak, B.T., Khlebnikov, M.V., and Shcherbakov, P.S., Upravlenie lineinymi sistemami pri vneshnikh vozmushcheniyakh. Tekhnika lineinykh matrichnykh neravenstv (Control over Linear Systems under External Interference. Linear Matrix Inequalities), Moscow: LENAND, 2014.

    Google Scholar 

  9. Sokolov, V.F., Estimating Performance of the Robust Control System under Unknown Upper Disturbance Boundaries and Measurement Noise, Autom. Remote Control, 2010, vol. 71, no. 9, pp. 1741–1756.

    Article  MATH  MathSciNet  Google Scholar 

  10. Calafiore, G. and Polyak, B.T., Stochastic Algorithms for Exact and Approximate Feasibility of Robust LMIs, IEEE Trans. Automat. Control, 2001, vol. 46, pp. 1755–1759.

    Article  MATH  MathSciNet  Google Scholar 

  11. Granichin, O.N. and Polyak, B.T., Randomizirovannye algoritmy otsenivaniya i optimizatsii pri pochti proizvol’nykh pomekhakh (Randomized Estimation and Optimization Algorithms for Almost Arbitrary Noise), Moscow: Nauka, 2003.

    Google Scholar 

  12. Tempo, R., Calafiore, G., and Dabbene, F., Randomized Algorithms for Analysis and Control of Uncertain Systems: With Applications, New York: Springer-Verlag, 2013.

    Book  Google Scholar 

  13. Granichin, O., Volkovich, V., and Toledano-Kitai, D., Randomized Algorithms in Automatic Control and Data Mining, Springer, 2014.

    Google Scholar 

  14. Robbins, H. and Monro, S., A Stochastic Approximation Method, Ann. Math. Statist., 1951, vol. 22, pp. 400–407.

    Article  MATH  MathSciNet  Google Scholar 

  15. Kiefer, J. and Wolfowitz, J., Statistical Estimation on the Maximum of a Regression Function, Ann. Math. Statist., 1952, vol. 23, pp. 462–466.

    Article  MATH  MathSciNet  Google Scholar 

  16. Blum, J.R., Multidimensional Stochastic Appoximation, Ann. Math. Statist., 1954, vol. 9, pp. 737–744.

    Article  Google Scholar 

  17. Granichin, O.N., A Stochastic Recursive Procedure with Correlated Noises in the Observation that Employs Trial Perturbations at the Input, Vestn. Leningr. Univ., Mat., 1989, vol. 22, no. 1, pp. 27–31.

    MATH  MathSciNet  Google Scholar 

  18. Granichin, O.N., Procedure of Stochastic Approximation with Disturbances at the Input, Autom. Remote Control, 1992, vol. 53, no. 2, part 1, pp. 232–237.

    MathSciNet  Google Scholar 

  19. Polyak, B.T. and Tsybakov, A.B., Optimal Orders of Accuracy for Search Algorithms of Stochastic Optimization, Probl. Peredachi Inf., 1990, vol. 26, no. 2, pp. 126–133.

    MATH  MathSciNet  Google Scholar 

  20. Polyak, V.T. and Tsybakov, A.V., On Stochastic Approximation with Arbitrary Noise (the KW Case), in Topics in Nonparametric Estimation, Khasminskii, R.Z., Ed., Adv. Sov. Math., Providence: Am. Math. Soc., 1992, no. 12, pp. 107–113.

    Google Scholar 

  21. Rastrigin, L.A., Statisticheskie metody poiska (Statistical Search Methods), Moscow: Nauka, 1968.

    Google Scholar 

  22. Spall, J.C., Multivariate Stochastic Approximation Using a Simultaneous Perturbation Gradient Approximation, IEEE Trans. Automat. Control, 1992, vol. 37, no. 3, pp. 332–341.

    Article  MATH  MathSciNet  Google Scholar 

  23. Spall, J.C., A One—Measurement Form of Simultaneous Perturbation Stochastic Approximation, Automatica, 1997, vol. 33, pp. 109–112.

    Article  MATH  MathSciNet  Google Scholar 

  24. Polyak, B.T., Convergence and Rate of Convergence of Recursive Stochastic Algorithms. I, Autom. Remote Control, 1976, vol. 37, no. 12, part 1, pp. 1858–1868.

    MATH  MathSciNet  Google Scholar 

  25. Polyak, B.T., Convergence and Convergence Rate of Iterative Stochastic Algorithms. II. The Linear Case, Autom. Remote Control, 1977, vol. 38, no. 4, part 2, pp. 537–542.

    MATH  Google Scholar 

  26. Tsypkin, Ya.Z., Adaptatsiya i obuchenie v avtomaticheskikh sistemakh (Adaptation and Learning in Automated Systems), Moscow: Nauka, 1968.

    Google Scholar 

  27. Polyak, B.T. and Tsypkin, Ya.Z., Pseudogradient Algorithms of Adaptation and Learning, Autom. Remote Control, 1973, vol. 34, no. 3, part 1, pp. 377–398.

    MathSciNet  Google Scholar 

  28. Polyak, B.T. and Tsypkin, Ya.Z., Adaptive Estimation Algorithms (Convergence, Optimality, Stability), Autom. Remote Control, 1979, vol. 40, no. 3, part 1, pp. 378–389.

    MATH  Google Scholar 

  29. Polyak, B.T. and Tsypkin, Ya.Z., Optimal Pseudogradient Adaptation Procedure, Autom. Remote Control, 1980, vol. 41, no. 8, part 1, pp. 1101–1110.

    MATH  Google Scholar 

  30. Polyak, B.T. and Tsypkin, Ya.Z., Robust Pseudogradient Adaptation Procedure, Autom. Remote Control, 1980, vol. 41, no. 10, part 2, pp. 1404–1409.

    MATH  MathSciNet  Google Scholar 

  31. Tsypkin, Ya.Z. and Poznyak, A.S., Optimal Stochastic Optimization Search Algorithms, Dokl. Akad. Nauk USSR, 1981, vol. 260, no. 3, pp. 550–553.

    MathSciNet  Google Scholar 

  32. Polyak, B.T. and Tsypkin, Ya.Z., Gradient Methods of Stochastic Optimization, Izmer., Kontrol’, Avtomatiz., 1989, no. 3, pp. 50–54.

    Google Scholar 

  33. Nazin, A.V., Polyak, B.T., and Tsybakov, A.B., Passive Stochastic Approximation, Autom. Remote Control, 1989, vol. 50, no. 11, part 2, pp. 1563–1569.

    MATH  MathSciNet  Google Scholar 

  34. Polyak, B.T., New Method of Stochastic Approximation Type, Autom. Remote Control, 1990, vol. 51, no. 7, part 2, pp. 937–946.

    MATH  MathSciNet  Google Scholar 

  35. Polyak, V.T. and Yuditskij, A.V., Acceleration of Stochastic Approximation Procedures by Averaging, SIAM J. Control Optim., 1992, vol. 30, no. 4, pp. 838–855.

    Article  MATH  MathSciNet  Google Scholar 

  36. Kushner, H.J. and Yin, G.G., Stochastic Approximation Algorithms and Applications, New York: Springer-Verlag, 2002.

    Google Scholar 

  37. Borkar, V.S., Stochastic Approximation. A Dynamical Systems Viewpoint, Cambridge: Cambridge Univ. Press, 2008.

    Google Scholar 

  38. Granichin, O., Gurevich, L., and Vakhitov, A., Discrete-Time Minimum Tracking Based on Stochastic Approximation Algorithm with Randomized Differences, 48th Conf. Decision Control, Shanghai, China, 2009, pp. 5763–5767.

    Google Scholar 

  39. Granichin, O.N., and Amelina, N.O., Simultaneous Perturbation Stochastic Approximation for Tracking under Unknown but Bounded Interference, IEEE Trans. Automat. Control, 2015, vol. 60, no. 5.

    Google Scholar 

  40. Tsitsiklis, J., Bertsekas, D., and Athans, M., Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms, IEEE Trans. Automat. Control, 1986, vol. 31, no. 9, pp. 803–812.

    Article  MATH  MathSciNet  Google Scholar 

  41. Amelina, N.O. and Fradkov, A.L., Approximate Consensus in the Dynamic Stochastic Network with Incomplete Information and Measurement Delays, Autom. Remote Control, 2012, vol. 73, no. 11, pp. 1765–1783.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saint Petersburg State University, St. Petersburg, Russia

    O. N. Granichin

Authors
  1. O. N. Granichin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to O. N. Granichin.

Additional information

Original Russian Text © O.N. Granichin, 2015, published in Avtomatika i Telemekhanika, 2015, No. 5, pp. 43–59.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Granichin, O.N. Stochastic approximation search algorithms with randomization at the input. Autom Remote Control 76, 762–775 (2015). https://doi.org/10.1134/S0005117915050033

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117915050033

Keywords

Search

Navigation