Skip to main content
SpringerLink
Log in
Book cover

Springer Handbook of Automation pp 221–235Cite as

Cybernetics and Learning Automata

  • Chapter

Part of the book series: Springer Handbooks ((SHB))

Abstract

Stochastic learning automata are probabilistic finite state machines which have been used to model how biological systems can learn. The structure of such a machine can be fixed or can be changing with time. A learning automaton can also be implemented using action (choosing) probability updating rules which may or may not depend on estimates from the environment being investigated. This chapter presents an overview of the field of learning automata, perceived as a completely new paradigm for learning, and explains how it is related to the area of cybernetics.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   309.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Abbreviations

ATM:

air traffic management

ATM:

asynchronous transfer mode

ATM:

automatic teller machine

DEA:

discrete estimator algorithm

DGPA:

discretized generalized pursuit algorithm

DPA:

discrete pursuit algorithm

DTSE:

discrete TSE algorithm

FSSA:

fixed structure stochastic automaton

GPA:

generalized pursuit algorithm

IP:

inaction–penalty

IP:

industrial protocol

IP:

integer programming

IP:

intellectual property

IP:

internet protocol

LA:

learning automata

RE:

random environment

RI:

reward–inaction

RNG:

random-number generator

RP:

reward–penalty

SELA:

stochastic estimator learning algorithm

TSE:

total system error

VSSA:

variable structure stochastic automata

References

  1. M.L. Tsetlin: On the behaviour of finite automata in random media, Autom. Remote Control 22, 1210–1219 (1962), Originally in Avtom. Telemekh. 22, 1345–1354 (1961), in Russian

    MATH  Google Scholar 

  2. M.L. Tsetlin: Automaton Theory and Modeling of Biological Systems (Academic, New York 1973)

    Google Scholar 

  3. K.S. Narendra, M.A.L. Thathachar: Learning Automata (Prentice-Hall, Upper Saddle River 1989)

    Google Scholar 

  4. R.R. Bush, F. Mosteller: Stochastic Models for Learning (Wiley, New York 1958)

    Google Scholar 

  5. C.R. Atkinson, G.H. Bower, E.J. Crowthers: An Introduction to Mathematical Learning Theory (Wiley, New York 1965)

    MATH  Google Scholar 

  6. V.I. Varshavskii, I.P. Vorontsova: On the behavior of stochastic automata with a variable structure, Autom. Remote Control 24, 327–333 (1963)

    MathSciNet  Google Scholar 

  7. M.S. Obaidat, G.I. Papadimitriou, A.S. Pomportsis: Learning automata: theory, paradigms, and applications, IEEE Trans. Syst. Man Cybern. B 32, 706–709 (2002)

    Article  Google Scholar 

  8. S. Lakshmivarahan: Learning Algorithms Theory and Applications (Springer, New York 1981)

    MATH  Google Scholar 

  9. K. Najim, A.S. Poznyak: Learning Automata: Theory and Applications (Pergamon, Oxford 1994)

    Google Scholar 

  10. A.S. Poznyak, K. Najim: Learning Automata and Stochastic Optimization (Springer, Berlin 1997)

    MATH  Google Scholar 

  11. M.A.L.T. Thathachar, P.S. Sastry: Networks of Learning Automata: Techniques for Online Stochastic Optimization (Kluwer, Boston 2003)

    Google Scholar 

  12. S. Misra, B.J. Oommen: GPSPA: a new adaptive algorithm for maintaining shortest path routing trees in stochastic networks, Int. J. Commun. Syst. 17, 963–984 (2004)

    Article  Google Scholar 

  13. M.S. Obaidat, G.I. Papadimitriou, A.S. Pomportsis, H.S. Laskaridis: Learning automata-based bus arbitration for shared-medium ATM switches, IEEE Trans. Syst. Man Cybern. B 32, 815–820 (2002)

    Article  Google Scholar 

  14. B.J. Oommen, T.D. Roberts: Continuous learning automata solutions to the capacity assignment problem, IEEE Trans. Comput. C 49, 608–620 (2000)

    Article  Google Scholar 

  15. G.I. Papadimitriou, A.S. Pomportsis: Learning-automata-based TDMA protocols for broadcast communication systems with bursty traffic, IEEE Commun. Lett. 3(3), 107–109 (2000)

    Article  Google Scholar 

  16. A.F. Atlassis, N.H. Loukas, A.V. Vasilakos: The use of learning algorithms in atm networks call admission control problem: a methodology, Comput. Netw. 34, 341–353 (2000)

    Article  Google Scholar 

  17. A.F. Atlassis, A.V. Vasilakos: The use of reinforcement learning algorithms in traffic control of high speed networks. In: Advances in Computational Intelligence and Learning (Kluwer, Dordrecht 2002) pp. 353–369

    Google Scholar 

  18. A. Vasilakos, M.P. Saltouros, A.F. Atlassis, W. Pedrycz: Optimizing QoS routing in hierarchical ATM networks using computational intelligence techniques, IEEE Trans. Syst. Sci. Cybern. C 33, 297–312 (2003)

    Article  Google Scholar 

  19. F. Seredynski: Distributed scheduling using simple learning machines, Eur. J. Oper. Res. 107, 401–413 (1998)

    Article  MATH  Google Scholar 

  20. J. Kabudian, M.R. Meybodi, M.M. Homayounpour: Applying continuous action reinforcement learning automata (CARLA) to global training of hidden Markov models, Proc. ITCCʼ04 (Las Vegas 2004) pp. 638–642

    Google Scholar 

  21. M.R. Meybodi, H. Beigy: New learning automata based algorithms for adaptation of backpropagation algorithm parameters, Int. J. Neural Syst. 12, 45–67 (2002)

    Google Scholar 

  22. C. Unsal, P. Kachroo, J.S. Bay: Simulation study of multiple intelligent vehicle control using stochastic learning automata, Trans. Soc. Comput. Simul. Int. 14, 193–210 (1997)

    Google Scholar 

  23. B.J. Oommen, E.V. de St. Croix: Graph partitioning using learning automata, IEEE Trans. Comput. C 45, 195–208 (1995)

    Article  Google Scholar 

  24. G. Santharam, P.S. Sastry, M.A.L. Thathachar: Continuous action set learning automata for stochastic optimization, J. Franklin Inst. 331(5), 607–628 (1994)

    Article  MathSciNet  Google Scholar 

  25. B.J. Oommen, G. Raghunath, B. Kuipers: Parameter learning from stochastic teachers and stochastic compulsive liars, IEEE Trans. Syst. Man Cybern. B 36, 820–836 (2006)

    Article  Google Scholar 

  26. V. Krylov: On the stochastic automaton which is asymptotically optimal in random medium, Autom. Remote Control 24, 1114–1116 (1964)

    Google Scholar 

  27. V.I. Krinsky: An asymptotically optimal automaton with exponential convergence, Biofizika 9, 484–487 (1964)

    Google Scholar 

  28. M.F. Norman: On linear models with two absorbing barriers, J. Math. Psychol. 5, 225–241 (1968)

    Article  MATH  MathSciNet  Google Scholar 

  29. I.J. Shapiro, K.S. Narendra: Use of stochastic automata for parameter self-optimization with multi-modal performance criteria, IEEE Trans. Syst. Sci. Cybern. SSC-5, 352–360 (1969)

    Article  Google Scholar 

  30. M.A.L. Thathachar, B.J. Oommen: Discretized reward–inaction learning automata, J. Cybern. Inf. Sci. 2(1), 24–29 (1979)

    Google Scholar 

  31. J.K. Lanctôt, B.J. Oommen: Discretized estimator learning automata, IEEE Trans. Syst. Man Cybern. 22, 1473–1483 (1992)

    Article  Google Scholar 

  32. B.J. Oommen, J.P.R. Christensen: ϵ-optimal discretized linear reward–penalty learning automata, IEEE Trans. Syst. Man Cybern. B 18, 451–457 (1998)

    MathSciNet  Google Scholar 

  33. B.J. Oommen, E.R. Hansen: The asymptotic optimality of discretized linear reward–inaction learning automata, IEEE Trans. Syst. Man Cybern. 14, 542–545 (1984)

    MATH  MathSciNet  Google Scholar 

  34. B.J. Oommen: Absorbing and ergodic discretized two action learning automata, IEEE Trans. Syst. Man Cybern. 16, 282–293 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  35. P.S. Sastry: Systems of Learning Automata: Estimator Algorithms Applications. Ph.D. Thesis (Department of Electrical Engineering, Indian Institute of Science, Bangalore 1985)

    Google Scholar 

  36. M.A.L. Thathachar, P.S. Sastry: A new approach to designing reinforcement schemes for learning automata, Proc. IEEE Int. Conf. Cybern. Soc. (Bombay 1984)

    Google Scholar 

  37. M.A.L. Thathachar, P.S. Sastry: A class of rapidly converging algorithms for learning automata, IEEE Trans. Syst. Man Cybern. 15, 168–175 (1985)

    MATH  MathSciNet  Google Scholar 

  38. M.A.L. Thathachar, P.S. Sastry: Estimator algorithms for learning automata, Proc. Platin. Jubil. Conf. Syst. Signal Process. (Department of Electrical Engineering, Indian Institute of Science, Bangalore 1986)

    Google Scholar 

  39. M. Agache: Estimator Based Learning Algorithms. MSC Thesis (School of Computer Science, Carleton University, Ottawa 2000)

    Google Scholar 

  40. M. Agache, B.J. Oommen: Generalized pursuit learning schemes: new families of continuous and discretized learning automata, IEEE Trans. Syst. Man Cybern. B 32(2), 738–749 (2002)

    Article  Google Scholar 

  41. M.A.L. Thathachar, P.S. Sastry: Pursuit algorithm for learning automata. Unpublished paper that can be available from the authors

    Google Scholar 

  42. A.V. Vasilakos, G. Papadimitriou: Ergodic discretize destimator learning automata with high accuracy and high adaptation rate for nonstationary environments, Neurocomputing 4, 181–196 (1992)

    Article  MATH  Google Scholar 

  43. A.F. Atlasis, M.P. Saltouros, A.V. Vasilakos: On the use of a stochastic estimator learning algorithm to the ATM routing problem: a methodology, Proc. IEEE GLOBECOM (1998)

    Google Scholar 

  44. M.K. Hashem: Learning Automata-Based Intelligent Tutorial-Like Systems. Ph.D. Thesis (School of Computer Science, Carleton University, Ottawa 2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Carleton University, 1125 Colonel Bye Drive, K1S5B6, Ottawa, Canada

    John Oommen Dr

  2. School of Information Technology, Indian Institute of Technology, 721302, Kharagpur, India

    Sudip Misra PhD

Authors
  1. John Oommen Dr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudip Misra PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to John Oommen Dr or Sudip Misra PhD .

Editor information

Editors and Affiliations

  1. PRISM Center, and School of Industrial Engineering, Purdue University, 315 N. Grant Street, 47907, West Lafayette, IN, USA

    Shimon Y. Nof

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Oommen, J., Misra, S. (2009). Cybernetics and Learning Automata. In: Nof, S. (eds) Springer Handbook of Automation. Springer Handbooks. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78831-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78831-7_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78830-0

  • Online ISBN: 978-3-540-78831-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation