Skip to main content
Springer Nature Link
Log in

On the Efficient Implementation Biologic Reinforcement Learning Using Eligibility Traces

  • Conference paper
Advances in Neural Networks - ISNN 2006 (ISNN 2006)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3971))

Included in the following conference series:

  • 80 Accesses

Abstract

The eligibility trace is one of the basic mechanisms in reinforcement learning to handle delayed reward. In this paper, we have used meta-heuristic method to solve hard combinatorial optimization problems. Our proposed solution introduce Ant-Q learning method to solve Traveling Salesman Problem (TSP). The approach is based on population that use positive feedback as well as greedy search and suggest ant reinforcement learning algorithms using eligibility traces which is called replace-trace methods(Ant-TD(λ)). Although replacing traces are only slightly, they can produce a significant improvement in learning rate. We could know through an experiment that proposed reinforcement learning method converges faster to optimal solution than ACS and Ant-Q.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Colorni, A., Dorigo, M., Maniezzo, V.: An Investigation of Some Properties of an Ant Algorithm. In: Manner, R., Manderick, B. (eds.) Proceedings of the Parallel Problem Solving from Nature Conference, pp. 509–520. Elsevier Publishing, Amsterdam (1992)

    Google Scholar 

  2. Colorni, A., Dorigo, M., Maniezzo, V.: Distributed Optimization by Ant Colonies. In: Varela, F., Bourgine, P. (eds.) Proceedings of the First European Conference of Artificial Life, pp. 134–144. Elsevier Publishing, Amsterdam (1991)

    Google Scholar 

  3. Watkins, C.J.C.H.: Learning from Delayed Rewards. Ph.D. Thesis, King’s College, Cambridge, U.K (1989)

    Google Scholar 

  4. Fiecher, C.N.: Efficient Reinforcement Learning. In: Proceedings of the Seventh Annual ACM Conference on Computational Learning Theory, pp. 88–97 (1994)

    Google Scholar 

  5. Barnald, E.: Temporal-Difference Methods and Markov Model. IEEE Trans. Systems, Man and Cybernetics 23, 357–365 (1993)

    Article  Google Scholar 

  6. Gambardella, L.M., Dorigo, M.: Solving Symmetric and Asymmetric TSPs by Ant Colonies. In: Proceedings of IEEE International Conference of Evolutionary Computation, pp. 622–627. IEEE Press, Los Alamitos (1996)

    Chapter  Google Scholar 

  7. Gambardella, L.M., Dorigo, M.: Ant-Q: A Reinforcement Learning Approach to the Traveling Salesman Problem. In: Prieditis, A., Russell, S. (eds.) Proceedings of ML 1995, Twelfth International Conference on Machine Learning, pp. 252–260. Morgan Kaufmann, San Francisco (1995)

    Google Scholar 

  8. Dorigo, M., Gambardella, L.M.: A Study of Some Properties of Ant-Q. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 656–665. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  9. Dorigo, M., Maniezzo, V., Colorni, A.: The Ant System: Optimization by a Colony of Cooperation Agents. IEEE Trans. Systems, Man and Cybernetics-Part B 26(1), 29–41 (1996)

    Article  Google Scholar 

  10. Stutzle, T., Hoos, H.: The Ant System and Local Search for the Traveling Salesman Problem. In: Proceedings of IEEE 4th International Conference of Evolutionary (1997)

    Google Scholar 

  11. Gambardella, L.M., Dorigo, M.: Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem. IEEE Trans. Evolutionary Computation 1(1) (1997)

    Google Scholar 

  12. Stutzle, T., Dorigo, M.: ACO Algorithms for the Traveling Salesman Problem. In: Miettinen, K., Makela, M., Neittaanmaki, P., Periaux, J. (eds.) Evolutionary Algorithms in Engineering and Computer Science. Wiley, Chichester (1999)

    Google Scholar 

  13. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  14. Lee, S.G.: Multiagent Reinforcement Learning Algorithm Using Temporal Difference Error. In: Wang, J., Liao, X.-F., Yi, Z. (eds.) ISNN 2005. LNCS, vol. 3496, pp. 627–633. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Catholic University, 43-1, Yeokgok 2-Dong, Wonmi-Gu, Bucheon-Si, Gyeonggi-Do, 420-743, Korea

    SeungGwan Lee

Authors
  1. SeungGwan Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China

    Jun Wang

  2. Computational Intelligence Laboratory, School of Computer Science and Engineering, University of Electronic Science and Technology of China, 610054, Chengdu, P.R. China

    Zhang Yi

  3. Department of Electrical Engineering, University of Louisville, 40292, Louisville, KY, U.S.A

    Jacek M. Zurada

  4. Laboratory for Computational Biology, Shanghai Center for Systems Biomedicine, 800 Dong Chuan Rd., 200240, Shanghai, China

    Bao-Liang Lu

  5. School of Electrical and Electronic Engineering, University of Manchester, UK

    Hujun Yin

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Lee, S. (2006). On the Efficient Implementation Biologic Reinforcement Learning Using Eligibility Traces. In: Wang, J., Yi, Z., Zurada, J.M., Lu, BL., Yin, H. (eds) Advances in Neural Networks - ISNN 2006. ISNN 2006. Lecture Notes in Computer Science, vol 3971. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11759966_71

Download citation

  • DOI: https://doi.org/10.1007/11759966_71

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-34439-1

  • Online ISBN: 978-3-540-34440-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics