Skip to main content
SpringerLink
Log in

A Reinforcement Learning Adaptive Fuzzy Controller for Differential Games

  • Published:
Journal of Intelligent and Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper we develop a reinforcement fuzzy learning scheme for robots playing a differential game. Differential games are games played in continuous time, with continuous states and actions. Fuzzy controllers are used to approximate the calculation of future reinforcements of the game due to actions taken at a specific time. If an immediate reinforcement reward function is defined, we may use a fuzzy system to tell what is the predicted reinforcement in a specified time ahead. This reinforcement is then used to adapt a fuzzy controller that stores the experience accumulated by the player. Simulations of a modified two car game are provided in order to show the potentiality of the technique. Experiments are performed in order to validate the method. Finally, it should be noted that although the game used as an example involves only two players, the technique may also be used in a multi-game environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Andrecut, M., Ali, M.K.: Fuzzy reinforcement learning. Int. J. Mod. Phys. C 13(5), 659–674 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  2. Arslan, G., Shamma, J.S.: Anticipatory learning in general evolutionary games. In: Proceedings of the 45th IEEE Conference on Decision and Control, pp. 6289–6294 (2006)

  3. Bryson, A.E., Ho, Y. (eds.): Applied Optimal Control: Optimization, Estimation, and Control. Taylor & Francis, Levittown (1975) (Rev. printing)

    Google Scholar 

  4. Buijtenen, W.M., Schram, G., an H. B. Verbruggen, R.B.: Adaptive fuzzy control of satellite attitude by reinforcement learning. IEEE Trans. Fuzzy Syst. 6(2), 185–194 (1998)

    Article  Google Scholar 

  5. Conlisk, J.: Adaptation in games: two solutions to the crawford puzzle. J. Econ. Behav. Organ. 22, 25–50 (1993)

    Article  Google Scholar 

  6. Dai, X., Li, C., Rad, A.B.: An approach to tune fuzzy controllers based on reinforcement learning for autonomous vehicle control. IEEE Trans. Intell. Transp. Syst. 6(3), 285–293 (2005)

    Article  Google Scholar 

  7. Fudenberg, D., Levine, D.K.: The Theory of Learning in Games. MIT, Cambridge (1998)

    MATH  Google Scholar 

  8. Ge, J., Tang, L., Reimann, J., Vachtsevanos, G.: Hierarchical decomposition approach for pursuit-evasion differential game with multiple players, p. 7 (2006)

  9. Ge, J., Tang, L., Reimann, J., Vachtsevanos, G.: Suboptimal approaches to multiplayer pursuit-evasion differential games, pp. 5272–5278 (2006)

  10. Givigi, S.N., Schwartz, H.M.: Swarm robot systems based on the evolution of personality traits. Turkish Journal of Electrical Engineering & Computer Sciences (Elektrik): Special Issue on Swarm Robotics 15(2), 257–282 (2007)

    Google Scholar 

  11. Harmon, M.E., III, L.C.B.: Residual advantage learning applied to a differential game. In: Proceedings of the International Conference on Neural Networks, pp. 1–6 (1996)

  12. Harmon, M.E., III, Baird, L.C., Klopf, A.H.: Reinforcement learning applied to a differential game. Adapt. Behav. 4(1), 3–28 (1995)

    Article  Google Scholar 

  13. Hofbauer, J., Sigmund, K.: Evolutionary game dynamics. Bull. Am. Math. Soc. 40(4), 479–519 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  14. Isaacs, R.: Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization. Wiley, New York (1965)

    MATH  Google Scholar 

  15. Ishibuchi, H., Sakamoto, R., Nakashima, T.: Learning fuzzy rules from iterative execution of games. Fuzzy Sets Syst. 135, 213–240 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  16. Mannor, S., Shamma, J.: Multi-agent learning for engineers. Artificial Intelligence, Special issue on “Foundations of Multi-Agent Learning”, pp. 417–422 (2007)

  17. Merz, A.W.: The homicidal chauffeur. AIAA J. 12(3), 259–260 (1974)

    Article  MathSciNet  Google Scholar 

  18. Myerson, R.B.: Game Theory: Analysis of Conflict. Harvard University Press, Cambridge (1991)

    MATH  Google Scholar 

  19. von Neumann, J., Morgenstern, O.: The Theory of Games and Economic Behavior, 2nd edn. Princeton University Press, Princeton (1947)

    Google Scholar 

  20. Sheppard, J.W.: Colearning in differential games. Mach. Learn. 33, 201–233 (1998)

    Article  MATH  Google Scholar 

  21. Starr, A.W., Ho, Y.C.: Nonzero-sum differential games. J. Optim. Theory Appl. 3(3), 184–206 (1969)

    Article  MATH  MathSciNet  Google Scholar 

  22. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its application to modeling and control. IEEE Trans. Syst. Man Cybern. 15, 116–132 (1985)

    MATH  Google Scholar 

  23. Wang, L.X.: A Course in Fuzzy Systems and Control. Prentice Hall, Englewood Cliffs (1997)

    MATH  Google Scholar 

  24. Weibull, J.W.: Evolutionary Game Theory. MIT, Cambridge (1995)

    MATH  Google Scholar 

  25. Yeung, D.W.K., Petrosyan, L.A.: Cooperative Stochastic Differential Games. Springer, New York (2006)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Royal Military College of Canada, P.O. Box 17000 Station Forces, Kingston, Ontario, K7K 7B4, Canada

    Sidney N. Givigi Jr.

  2. Systems and Computer Engineering, Carleton University, 1125 Colonel By Drive, Ottawa, Ontario, K1S 5B6, Canada

    Howard M. Schwartz & Xiaosong Lu

Authors
  1. Sidney N. Givigi Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Howard M. Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaosong Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sidney N. Givigi Jr..

Rights and permissions

Reprints and permissions

About this article

Cite this article

Givigi, S.N., Schwartz, H.M. & Lu, X. A Reinforcement Learning Adaptive Fuzzy Controller for Differential Games. J Intell Robot Syst 59, 3–30 (2010). https://doi.org/10.1007/s10846-009-9380-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-009-9380-4

Keywords

Search

Navigation