Abstract
Two methods of determining the lower bounds of the rate of convergence of finite stochastic automata are presented. The rate of convergence, defined as the percentage decrease in the distance between the transient probability distribution and the equilibrium probability distribution in each step, is determined as a function of the probability transition matrix. Formulas for parameter optimization for a class of stochastic automata for fast convergence and maximum expediency are derived and illustrative examples of fourth-order systems are given.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
S. Lakshimivarahan and M. A. L. Thathachar, “Absolutely expedient learning algorithms for stochastic automata,”IEEE Trans. on Systems, Man and Cybernetics SMC-3 (May):281–285 (1973).
R. Viswanathan and K. S. Narendra, “On variable-structure stochastic automata,” inPattern Recognition and Machine Learning (Plenum Press, New York, 1971), pp. 277–287.
B. Chandrasekaran and D. W. C. Shen, “On expendiency and convergence in variable-structure automata,”IEEE Trans. on Systems Science and Cybernetics SSC-4 (March): 52–60 (1968).
K. S. Fu, “Stochastic automata as models of learning systems,” inAdaptive Learning and Pattern Recognition Systems, Ed. by K. S. Fu and J. M. Mendel (Academic Press, New York, 1970), Chapter 11.
R. W. McLaren, “A stochastic automation model for the synthesis of learning systems,”IEEE Trans. on Systems Sciences and Cybernetics SSC-2 (December): 109–114 (1966).
K. S. Fu and R. W. McLaren, “An application of stochastic automata to the synthesis of learning systems,” School of Electrical Engineering, Purdue University, Lafayette, Indiana, Tech. Rept. TR-EE65-17, 1965.
M. L. Tsetlin, “On the behavior of finite automata in random environments,”Automatika i Telemekhanika 22:1345–1354 (1961).
J. P. Cleave, “The synthesis of finite state homogeneous markov chains,”Cybernetica 5(1) (1962).
R. R. Bush and F. Mosteller,Stochastic Models for Learning (Wiley, New York, 1958).
H. H. Yeh, “Comments on ‘A Stochastic Automaton Model for the Synthesis of Learning Systems’,”IEEE Trans, on Systems Science and Cybernetics SSC-5(4) (1969).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Yeh, HH. Parameter optimization of stochastic automata operating in random environments. International Journal of Computer and Information Sciences 4, 247–263 (1975). https://doi.org/10.1007/BF01007762
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01007762