Abstract
A nonstationary environment is one in which the suitability of the strategies available to a learning element changes with time. Since the optimal action in such a case is not fixed, the learning problem (i.e., the determination of the optimal strategy) becomes considerably difficult. In this paper, a two-level approach is presented for a learning automaton operating in a nonstationary environment. The lower level consists of a standard absolutely expedient learning algorithm for stationary environments. The higher level on the other hand is a tracking algorithm, based on Bayesian decision theory, for detecting changes in the environment and reinitializing the lower level algorithm in a suitable manner. Simulation studies empirically demonstrate the clear superiority of the two-level approach over the single-level learning in nonstationary environments.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
K. S. Narendra and M. A. L. Thathachar. Learning Automata — an Introduction. Prentice Hall, Englewood Cliffs, New Jersey, 1989.
M. L. Tsetlin. Automaton Theory and Modeling of Biological Systems. Academic Press, New York, 1973.
V. I. Varshavskii and I. P. Vorontsova. “On the Behavior of Stochastic Automata with a Variable Structure”. Automation and Remote Control, vol. 24, pp. 327–333, 1963.
B. J. Oommen and H. Masum. “Switching Models for Nonstationary Random Environments”. IEEE Trans. Syst., Man, and Cybern., SMC-25, pp. 1334–1339, 1995.
P. R. Srikantakumar and K. S. Narendra. “A Learning Model for Routing in Telephone Networks”. SIAM Journal of Control and Optimization, vol. 20, pp. 34–57, 1982.
O. V. Nedzelnitsky, Jr. and K. S. Narendra. “Nonstationary Models of Learning Automata Routing in Data Communication Networks”. IEEE Trans. Syst., Man, and Cybern., SMC-17, pp. 1004–1015, 1987.
M. Herbster and M. Warmuth. “Tracking the Best Expert”. Proceedings of the Twelfth International Conference on Machine Learning, ICML95, pp. 286–294, 1995.
N. Baba and Y. Sawaragi. “On the Learning Behavior of Stochastic Automata under a Nonstationary Environment”. IEEE Trans. Syst., Man, and Cybern., SMC-5, pp. 273–275, 1975.
A. G. Barto and P. Anandan. “Pattern-recognizing Stochastic Learning Automata”. IEEE Trans. Syst., Man, and Cybern., SMC-15, pp. 360–375, 1985.
S. Zacks and Z. Barzily. “Bayes Procedures for Detecting a Shift in the Probability of Success in a Series of Bernoulli Trials”. Journal of Statistical Planning and Inference, No. 5, pp. 107–119, 1981.
M. A. L. Thathachar and P. S. Sastry. “A New Approach to the Design of Reinforcement Schemes for Learning Automata”. IEEE Trans. Syst., Man, and Cybern., SMC-15, pp. 168–175, 1985.
S. Mukhopadhyay and M.A.L. Thathachar. “Associative Learning of Boolean Functions”. IEEE Trans. Syst., Man and Cybern., SMC-19, pp. 1008–1015, 1989.
W. Lam and S. Mukhopadhyay. “A Two-Level Approach to Learning in Nonstationary Environments”. Management Sciences Department Technical Report, The University of Iowa, Iowa City, Iowa, 1995.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lam, W., Mukhopadhyay, S. (1996). A two-level approach to learning in nonstationary environments. In: McCalla, G. (eds) Advances in Artifical Intelligence. Canadian AI 1996. Lecture Notes in Computer Science, vol 1081. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61291-2_58
Download citation
DOI: https://doi.org/10.1007/3-540-61291-2_58
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61291-9
Online ISBN: 978-3-540-68450-3
eBook Packages: Springer Book Archive