Abstract
Dynamical processes in nature and technology are usually described by continuous-or discrete-time dynamical models, which have the form of nonlinear stochastic differential or difference equations. Hence, a topical problem is to develop effective methods for a simpler description of dynamical systems. The main requirement to simplification methods is preserving certain properties of a process under study. One group of such methods is represented by the methods of continuous- or discrete-time averagedmodels, which are surveyed in this paper. New results for stochastic networked systems are also introduced. As is shown below, the method of averaged models can be used to reduce the analytical complexity of a closed loop stochastic system. The corresponding upper bounds on the mean square distance between the states of an original stochastic system and its approximate averaged model are obtained.
Similar content being viewed by others
References
Bogolyubov, N.N. and Mitropol’skii, Yu.A., Asimptoticheskie metody v teorii nelineinykh kolebanii (Asymptotical Methods in Theory of Nonlinear Oscillations), Moscow: Gostekhizdat, 1955.
Volosov, V.M., Averaging in Systems of Ordinary Differential Equations, Russ. Math. Surveys, 1962, vol. 17, no. 6, pp. 1–126.
Mishchenko, E.F. and Rozov, N.Kh., Differentsial’nye uravneniya s malym parametrom i relaksatsionye kolebaniya (Differential Equations with Small Parameter and Relaxation Oscillations, Moscow: Nauka, 1975, vol. 1.
Gerashchenko, E.I. and Gerashchenko, S.M., Metod razdeleniya dvizhenii i optimizatsiya nelineinykh sistem (The Method of Motions Separation and Optimization of Nonlinear Systems), Moscow: Nauka, 1975.
O’Malley, R., Introduction to Singular Perturbations, New York: Academic, 1974.
Kokotovic, P.V., O’Malley, R., and Sannuti, P., Singular Perturbations and Order Reduction in Control Theory—An Overview, Automatica, 1976, vol. 12, no. 2. P. 123–132.
Young, K.-K., Kokotovic, P., and Utkin, V., A Singular Perturbation Analysis of High-Gain Feedback Systems, IEEE Trans. Autom. Control, 1977, vol. 22, no. 6, pp. 931–938.
Khas’minskii, R.Z., On Random Processes Defined by Differential Equations with Small Parameter, Teor. Veroyatn. Primen., 1966, vol. 11, no. 2, pp. 240–259.
Khas’minskii, R.Z., On the Averaging Principle for Itô Stochastic Differential Equations, Kibern., 1968, vol. 4, no. 3, pp. 260–279.
Gikhman, I.I. and Skorokhod, A.V., Stokhasticheskie differentsial’nye uravneniya (Stochastic Differential Equations), Kiev: Naukova Dumka, 1968.
Freidlin, M.I., The Averaging Principle and Theorems on Large Deviations, Russ. Math. Surveys, 1978, vol. 33, no. 5, pp. 117–176.
Wentzell, A.D. and Freidlin, M.I., Fluktuatsii v dinamicheskikh sistemakh pod deistviem malykh sluchainykh vozmushchenii (Fluctuations in Dynamical Systems Effected by Small Random Perturbations), Moscow: Nauka, 1979.
Derevitskii, D.P. and Fradkov, A.L., Two Models Analyzing the Dynamics of Adaptation Algorithms, Autom. Remote Control, 1974, vol. 35, no. 1, pp. 67–75.
Derevitskii, D. and Fradkov, A., Investigation of Discrete-Time Adaptive Control Systems Using Continuous-Time Models, Izv. Akad. Nauk SSSR, Tekhn. Kibern., 1975, vol. 5, pp. 93–99.
Ljung, L., Analysis of Recursive Stochastic Algorithms, IEEE Trans. Autom. Control, 1977, vol. 22, no. 4, pp. 551–575.
Derevitskii, D.P. and Fradkov, A.L., Prikladnaya teoriya diskretnykh adaptivnykh sistem upravleniya (Applied Theory of Discrete-Time Adaptive Control Systems), Moscow: Nauka, 1981.
Fradkov, A., Continuous-Time Averaged Models of Discrete-Time Stochastic Systems: Survey and Open Problems, Proc. 2011 50 IEEE Conf. on Decision and Control and Eur. Control Conf. (CDCECC), Orlando, Florida, USA, 2011, pp. 2076–2081.
Fradkov, A.L., Averaged Continuous-Time Models in Identification and Control, Proc. Eur. Control Conf., Strasbourg, 2014, pp. 2822–2826.
Polyak, B.T., Vvedenie v optimizatsiyu, Moscow: Nauka, 1983. Translated under the title Introduction to Optimization, Translations Series in Mathematics and Engineering, New York: Optimization Software, 1987.
Robbins, H. and Monro, S., A Stochastic Approximation Method, Ann. Math. Stat., 1951, vol. 22, pp. 400–407.
Kushner, H.J. and Clark, D.S., Stochastic Approximation Methods for Constrained and Unconstrained Systems, New York: Springer-Verlag, 2012.
Meerkov, S.M., On Simplification of the Description of Slow Markovian Roaming, Autom. Remote Control, 1972, vol. 33, no. 3, pp. 404–414.
Informatsionnye materialy: Kibernetika, tom 68 (Information Materials: Cybernetics, vol. 68), 1973.
Ljung, L., Convergence of Recursive Stochastic Algorithms, Proc. IFAC Sympos. on Stochastic Control, 1974, pp. 551–575.
Basar, E.T., Control Theory: Twenty-Five Seminal Papers (1932–1981), New York: Wiley–IEEE Press, 2001.
Gerencs´er, L., A Representation Theorem for the Error of Recursive Estimators, SIAM J. Control Optim., 2006, vol. 44, no. 6, pp. 2123–2188.
Polyak, B.T. and Tsypkin, Ya.Z., Pseudogradient Algorithms of Adaptation and Learning, Autom. Remote Control, 1973, vol. 34, no. 3, pp. 45–68.
Nevel’son, M.B. and Khas’minskii, R.Z., Continuous Procedures of Stochastic Approximation, Probl. Inf. Transm., 1971, vol. 7, no. 2, pp. 139–148.
Khas’minskii, R.Z., The Averaging Principle for Stochastic Differential Equations, Probl. Inf. Transm., 1968, vol. 4, no. 2, pp. 68–69.
Ibragimov, I.A. and Linnik, Yu.V., Nezavisimye i statsionarno svyazannye velichiny (Independent and Stationary-Bound Variables), Moscow: Nauka, 1965.
Bernstein, S.N., Stokhasticheskie uravneniya v konechnykh raznostyakh i stokhasticheskie differentsial’-nye uravneniya (Stochastic Finite-Difference Equations and Stochastic Differential Equations), Moscow: Akad. Nauk SSSR, 1964, vol. 4, pp. 484–542.
Krasovskii, N., Stability of Motion, Stanford: Stanford Univ. Press, 1963.
Akhmetkaliev, T., On the Relationship between the Stability of Stochastic Difference and Differential Equations, Differ. Uravn., 1965, no. 8, pp. 1016–1026.
Polyak, B.T., Convergence and Rate of Convergence of Recursive Stochastic Algorithms. I, Autom. Remote Control, 1976, vol. 37, pp. 537–542.
Nevel’son, M.B. and Khas’minskii, R.Z., Stokhasticheskaya approksimatsiya i rekurrentnoe otsenivanie (Stochastic Approximation and Recursive Estimation), Moscow: Nauka, 1972.
Kushner, H., Convergence of Recursive Adaptive and Identification Procedures via Weak Convergence Theory, IEEE Trans. Autom. Control, 1977, vol. 22, no. 6, pp. 921–930.
Derevitskii, D. and Fradkov, A., Application of the Theory of Markov Processes to Analyze the Dynamics of Adaptation Algorithms, Avtom. Vychisl. Tekh., 1974, no. 2, pp. 39–48.
Derevitskii, D. and Fradkov, A., Analysis of the Dynamics of Some Adaptation Algorithms, in Voprosy kibernetiki. Adaptivnye sistemy (Problems of Cybernetics. Adaptive Systems), Moscow: Akad. Nauk SSSR, 1974, pp. 79–84.
Neśić, D., Teel, A.R., and Kokotović, P., Sufficient Conditions for Stabilization of Sampled-Data Nonlinear Systems via Discrete-Time Approximations, Syst. Control Lett., 1999, vol. 38, no. 4, pp. 259–270.
Dragan, V. and Khalanai, A., Preserving Exponential Stability In Discrete-Time Control Systems with Adaptive Stabilization, Sib. Math. J., 1990, vol. 31, no. 6, pp. 1046–1050.
Braverman, E. and Pyatnitskii, Yu.S., Passing of Random Signal Through Absolutely Stable Systems, Autom. Remote Control, 1971, vol. 32, no. 2, pp. 202–206.
Derevitskii, D.P., Ripa, K.K., and Fradkov, A.L., Investigation of the Dynamics of Certain Randomsearch Algorithms, Probl. Sluchainogo Poiska, 1975, no. 4, pp. 32–47.
Andrievskii, B.R., Blajkin, A.T., Derevitskii, D.P., and Fradkov, A.L., A Method to Investigate the Dynamics of Digital Adaptive Control Systems for Aerial Vehicles, in Upravlenie v prostranstve (Spatial Control), Moscow: Nauka, 1976, vol. 1, pp. 149–153.
Derevitskii, D.P., Synthesis of a Discrete Stochastic Adaptive Stabilization System Using a Continuous- Time Model, Avtom. Vychisl. Tekh., 1975, no. 6, pp. 50–52.
Andrievskii, B.R., Blajkin, A.T., Derevitskii, D.P., and Fradkov, A.L., A Synthesis Method of Discrete- Time Adaptive Stabilization Systems for a Stochastic Plant, Referaty dokladov VII Vsesoyuznogo soveshchaniya po problemam upravleniya (Abstracts of papers of the VII All-Union Meeting on Control Problems), 1977, pp. 102–105.
Levitan, V.D., Analysis of the Dynamics of Discrete-Time Adaptation Processes with a Discontinuous Stochastic Model, in Voprosy kibernetiki. Adaptivnye sistemy upravleniya (Problems of Cybernetics. Adaptive Control Systems), Moscow: Akad. Nauk SSSR, 1977, pp. 127–129.
Amelina, N., Fradkov, A., Jiang, Y., and Vergados, D.J., Approximate Consensus in Stochastic Networks with Application to Load Balancing, IEEE Trans. Inform. Theory, 2015, vol. 61, no. 4, pp. 1739–1752.
Pezeshki-Esfahani, H. and Heunis, A.J., Strong Diffusion Approximations for Recursive Stochastic Algorithms, IEEE Trans. Inform. Theory, 1997, vol. 43, no. 2, pp. 512–523.
Weiss, A. and Mitra, D., Digital Adaptive Filters: Conditions for Convergence, Rates of Convergence, Effects of Noise and Errors Arising from the Implementation, IEEE Trans. Inform. Theory, 1979, vol. 25, no. 6, pp. 637–652.
Benveniste, A., Métivier, M., and Priouret, P., Adaptive Algorithms and Stochastic Approximations, Berlin: Springer-Verlag, 2012.
Amelina, N.O. and Fradkov, A.L., Approximate Consensus in the Dynamic Stochastic Network with Incomplete Information and Measurement Delays, Autom. Remote Control, 2012, vol. 73, no. 11, pp. 1765–1783.
Amelina, N. and Fradkov, A., Approximate Consensus in Multi-Agent Nonlinear Stochastic Systems, Proc. Europ. Control Conf. (ECC), 2014, pp. 2833–2838.
Amelina, N., Fradkov, A., and Amelin, K., Approximate Consensus in Multi-Agent Stochastic Systems with Switched Topology and Noise, Proc. IEEE Multiconf. on Systems and Control (MSC’2012), Dubrovnik, Croatia, 2012, pp. 445–450.
Amelin, K., Amelina, N., Granichin, O., and Granichina, O., Multi-Agent Stochastic Systems with Switched Topology and Noise, Proc. 13 ACIS Int. Conf. on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), Kyoto, Japan, 2012, pp. 438–443.
Amelina, N., Granichin, O., and Kornivetc, A., Local Voting Protocol in Decentralized Load Balancing Problem with Switched Topology, Noise, and Delays, Proc. IEEE 52 Ann. Conf. on Decision and Control (CDC), 2013, pp. 4613–4618.
Gorodeisky, Z., Deterministic Approximation of Best-Response Dynamics for the Matching Pennies Game, Game. Econom. Behav., 2009, vol. 66, no. 1, pp. 191–201.
Benveniste, A., Design of Adaptive Algorithms for the Tracking of Time-Varying Systems, Int. J. Adapt. Control Signal Process., 1987, vol. 1, no. 1, pp. 3–29.
Benveniste, A., Priouret, P., and Metivier, M., Algorithmes adaptatifs et approximations stochastiques: théorie et applications à l’identification, au traitement du signal et à la reconnaissance des formes, Paris: Masson, 1987.
Benveniste, A. and Ruget, G., A Measure of the Tracking Capability of Recursive Stochastic Algorithms with Constant Gains, IEEE Trans. Autom. Control, 1982, vol. 27, no. 3, pp. 639–649.
Dugard, L. and Landau, I., Recursive Output Error Identification Algorithms Theory and Evaluation, Automatica, 1980, vol. 16, no. 5, pp. 443–462.
Kulchitskii, O., The New Method of Dynamical Stochastic Uncertain Systems Investigation, Preprints of Int. Conf. “Stochastic Optimization,” Part 1, 1984, pp. 127–129.
Kushner, H.J., Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory, Cambridge: MIT Press, 1984, vol. 6.
Ljung, L., On Positive Real Transfer Functions and the Convergence of Some Recursive Schemes, IEEE Trans. Autom. Control, 1977, vol. 22, no. 4, pp. 539–551.
Ljung, L. and Söderström, T., Theory and Practice of Recursive Identification, Cambridge: MIT Press, 1983.
Solo, V., The Convergence of AML, IEEE Trans. Autom. Control, 1979, vol. 24, no. 6, pp. 958–962.
Bozin, A. and Zarrop, M., Self Tuning Optimizer-Convergence and Robustness Properties, Proc. 1st Eur. Control Conf., 1991, pp. 672–677.
Ermol’ev, Yu.M. and Kaniovskii, Yu.M., Asymptotic Properties of Some Stochastic Programming Methods with Constant Step, Zh. Vych. Mat. Mat. Fiz., 1979, vol. 19, no. 2, pp. 356–366.
Coito, F.J. and Lemos, J.M., Adaptive Optimization with Constraints: Convergence and Oscillatory Behaviour, in Pattern Recogn. Image Anal., New York: Springer, 2005, pp. 19–26.
Kulchitskii, O.Yu., The Method of Convergence Analysis of Adaptive Filtration Algorithms, Probl. Inf. Transm., 1985, vol. 21, no. 4, pp. 285–297.
Kushner, H.J. and Shwartz, A., Weak Convergence and Asymptotic Properties of Adaptive Filters with Constant Gains, IEEE Trans. Inform. Theory, 1984, vol. 30, no. 2, pp. 177–182.
Metivier, M. and Priouret, P., Applications of a Kushner and Clark Lemma to General Classes of Stochastic Algorithms, IEEE Trans. Inform. Theory, 1984, vol. 30, no. 2, pp. 140–151.
Derevitskii, D.P., The Method of Investigation of the Dynamics of Discrete-Time Adaptive Control Systems of Dynamical Objects, in Voprosy kibernetiki. Adaptivnye sistemy (Problems of Cybernetics. Adaptive Systems), Moscow: Akad. Nauk SSSR, 1976, pp. 64–72.
Andrievsky, B., Blazhkin, A., Derevitsky, D., and Fradkov, A., The Method of Investigation of Dynamics of Digital Adaptive Systems of Flight Control, Proc. 6th IFAC Sympos. on Automatic Control in the Space, Yerevan, 1974.
Derevitskii, D.P. and Fradkov, A.L., The Method of Continuous-Time Models in the Theory of Discrete- Time Adaptive Systems, in Voprosy kibernetiki. Zadachi i metody adaptivnogo upravleniya (Problems of Cybernetics. Problems andMethods of Adaptive Control), Moscow: Akad. Nauk SSSR, 1980, pp. 75–98.
Derevitskii, D. and Fradkov, A., Averaging Method for Discrete-Time Stochastic Systems and Its Application in Adaptive Control, Preprints Int. Conf. “Stochastic Optimization,” vol. 1, pp. 74–76
Fradkov, A.L., Adaptivnoe upravlenie slozhnymi sistemami (Adaptive Control of Complex Systems), Moscow: Nauka, 1990.
Kulchitskii, O.Y., Algorithms of Stochastic Approximation in Adaptation Loop of Linear Dynamical System. I, II, Autom. Remote Control, 1983, vol. 44, no. 9, pp. 1189–1203; 1984, vol. 45, no. 3, pp. 104–113.
Mosca, E., Zappa, G., and Lemos, J.M., Robustness of Multipredictor Adaptive Regulators: Musmar, Automatica, 1989, vol. 25, no. 4, pp. 521–529.
Mosca, E., Lemos, J.M., Mendonca, T., and Nistri, P., Input Variance Constrained Adaptive Control and Singularly Perturbed ODE’s, Proc. 1st ECC, Grenoble, 1991, pp. 2176–2180.
Tsykunov, A.M., Adaptivnoe upravlenie ob”ektami s posledeistviem (Adaptive Control of Plants with Aftereffect), Moscow: Nauka, 1984.
Oja, E. and Karhunen, J., On Stochastic Approximation of the Eigenvectors and Eigenvalues of the Expectation of a Random Matrix, J. Math. Anal. Appl., 1985, vol. 106, no. 1, pp. 69–84.
Zhdanov, A.I., Recursive Estimation of Minimum Eigenvalues of Information Matrices, Autom. Remote Control, 1987, vol. 48, no. 4, part 1, pp. 443–451.
Stankovic, M.S., Johansson, K.H., and Stipanović, D.M., Distributed Seeking of Nash Equilibria in Mobile Sensor Networks, Proc. 49th IEEE Conf. on Decision and Control (CDC), 2010, pp. 5598–5603.
Astanovskaya, N.V., Varshavskii, V.I., Revako, V.M., and Fradkov, A.L., The Method of Continuous- Time Models in the Problem of Decentralized Resource Allocation, in Primenenie metodov sluchainogo poiska v SAPR (Application of Random Search in CAD), Tallin: Valgus, 1979.
Borovkov, A.A., Asimptoticheskie metody v teorii massovogo obsluzhivaniya (Asymptotic Methods in Queueing Theory), Moscow: Nauka, 1979.
Kuan, C.-M. and Hornik, K., Convergence of Learning Algorithms with Constant Learning Rates, IEEE Trans. Neural Networks, 1991, vol. 2, no. 5, pp. 484–489.
Choi, J. and Horowitz, R., Learning Coverage Control of Mobile Sensing Agents in One-Dimensional Stochastic Environments, IEEE Trans. Autom. Control, 2010, vol. 55, no. 3, pp. 804–809.
Choi, J., Oh, S., and Horowitz, R., Distributed Learning and Cooperative Control for Multi-Agent Systems, Automatica, 2009, vol. 45, no. 12, pp. 2802–2814.
Huang, M. and Manton, J., Coordination and Consensus of Networked Agents with Noisy Measurements: Stochastic Algorithms and Asymptotic Behavior, SIAM J. Control Optim., 2009, vol. 48, no. 1, pp. 134–161.
Borkar, V.S. and Manjunath, D., Distributed Topology Control of Wireless Networks, Wireless Networks, 2008, vol. 14, no. 5, pp. 671–682.
Granichin, O. and Fomin, V., Adaptive Control Using Test Signals in the Feedback Channel, Autom. Remote Control, 1986, vol. 47, no. 2, part 2, pp. 238–248.
Granichin, O., Volkovich, V., and Toledano-Kitai, D., Randomized Algorithms in Automatic Control and Data Mining, Berlin: Springer-Verlag, 2015.
Amelin, K. and Granichin, O., Randomized Control Strategies under Arbitrary External Noise, IEEE Trans. Autom. Control, 2016, vol. 61, no. 5, pp. 1328–1333.
Spall, J.C., Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control, vol. 64, Hoboken: Wiley-Interscience, 2003.
Tsypkin, Ya.Z., Adaptatsiya i obuchenie v avtomaticheskikh sistemakh (Adaptation and Learning in Automatic Systems), Moscow: Nauka, 1968.
Yakubovich, V.A., The Method of Recurrent Objective Inequalities in the Theory of Adaptive Systems, in Voprosy kibernetiki. Adaptivnye sistemy upravleniya (Problems of Cybernetics. Adaptive Control Systems), Moscow: Akad. Nauk SSSR, 1976, pp. 32–63.
Andrievskii, B.R. and Fradkov, A.L., Analysis of the Dynamics of an Adaptive Control Algorithm for a Linear Dynamical Plant, in Voprosy kibernetiki. Adaptivnye sistemy upravleniya (Problems of Cybernetics. Adaptive Control Systems), Moscow: Akad. Nauk SSSR, 1976, pp. 99–103.
Fradkov, A.L., Speed-Gradient Scheme and Its Application in Adaptive Control Problems, Autom. Remote Control, 1980, vol. 40, no. 9, pp. 1333–1342.
Yoshihara, K., Moment Inequalities for Mixing Sequences, Kodai Math. J., 1978, vol. 1, no. 2, pp. 316–328.
Acknowledgements
This work was supported in part by the Russian Foundation for Basic Research, projects nos. 17-08-01728, 19-03-00375. The results on the analysis of continuous-discrete and networked systems in Sections 8-11 were obtained at the Institute for Problems in Mechanical Engineering, Russian Academy of Sciences, under the support of the Russian Science Foundation, project no. 16-19-00057-P.
Author information
Authors and Affiliations
Corresponding author
Additional information
Russian Text © The Author(s), 2019, published in Avtomatika i Telemekhanika, 2019, No. 10, pp. 3–36.
Rights and permissions
About this article
Cite this article
Amelina, N.O., Granichin, O.N. & Fradkov, A.L. The Method of Averaged Models for Discrete-Time Adaptive Systems. Autom Remote Control 80, 1755–1782 (2019). https://doi.org/10.1134/S0005117919100011
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S0005117919100011