Skip to main content
SpringerLink
Log in

Off-line model-free and on-line model-based evolution for tracking navigation using evolvable hardware

  • Conference paper
  • First Online:
Evolutionary Robotics (EvoRobots 1998)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1468))

Included in the following conference series:

Abstract

Recently there has been great interest in the idea that evolvable systems based on the principles of Artificial Life can be used to continuously and autonomously adapt the behavior of physically embedded systems such as mobile robots, plants and intelligent home devices. At the same time, we have seen the introduction of evolvable hardware(EHW): new integrated circuits that are able to adapt their hardware autonomously and almost continuously to changes in the environment [11]. This paper describes how a navigation system for a physical mobile robot can be evolved using a Boolean function approach implemented on evolvable hardware. The task of the mobile robot is to track a moving target represented by a colored ball, while avoiding obstacles during its motion. Our results show that a dynamic Boolean function approach is sufficient to produce this navigation behavior. Although the classical model-free evolution method is often infeasible in the real world due to the number of possible interactions with the environment, we demonstrate that a model-based evolution method can reduce the interactions with the real world by a factor of 250, thus allowing us to apply the evolution process on-line and to obtain an adaptive tracking-avoiding system, provided the implementation can be accelerated by the utilization of evolvable hardware.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Thomas Back, Ulrich Hammel, and Hans-Paul Schwefel. Evolutionary computation: Comments on the history and current state. IEEE Transactions on Evolutionary Computation, 1(1):3–18, 1997.

    Google Scholar 

  2. Randall D. Beer and J.C. Gallagher. Evolving dynamic neural networks for adaptive behavior. Adaptive Behavior, 1(1):91–122, July 1992.

    Google Scholar 

  3. Lashon B. Booker. Classifier systems that learn internal world models. Machine Learning, 3(2–3):161–192, 1988.

    Google Scholar 

  4. Rodney Brooks. Artificial life and real robots. In F. J. Varela and P. Bourgine, editors, Proceedings of the First European Conference on Artificial Life, pages 3–10, Cambridge, MA, 1992. MIT Press / Bradford Books.

    Google Scholar 

  5. Dave Cliff, Inman Harvey, and Philip Husbands. Explorations in evolutionary robotics. Adaptive Behavior, 2(1):73–110, July 1993.

    Google Scholar 

  6. Marco Dorigo and Marco Colombetti. Robot shaping: developing autonomous agents through learning. Artificial Intelligence, 71:321–370, 1994.

    Google Scholar 

  7. Dario Floreano and Francesco Mondada. Automatic creation of an autonomous agent: Genetic evolution of a neural-network driven robot. In J-A. Meyer Dave Cliff, Philip Husbands and S. Wilson, editors, From Animals to Animats 3: Proceedings of the 3rd International Conference on Simulation of Adaptive Behavior. MIT Press, 1994.

    Google Scholar 

  8. Johan J. Grefenstette and Connie Loggia Ramsey. An approach to anytime learning. In Proceedings of the Ninth International Conference on Machine Learning, pages 189–195, San Mateo, CA., 1992. Morgan Kaufman.

    Google Scholar 

  9. John J. Grefenstette and A. Schultz.An evolutionary approach to learning in robots. In Proceedings of the Machine Learning Workshop on Robot Learning, Eleventh International Conference on Machine Learning. New Brunswick, NJ, 1994.

    Google Scholar 

  10. Inman Harvey, Philip Husbands, and Dave Cliff. Seeing the light: Artificial evolution, real vision. In J-A. Meyer Dave Cliff, Philip Husbands and S. Wilson, editors, From Animals to Animats 3: Proceedings of the 3rd International Conference on Simulation of Adaptive Behavior. MIT Press, 1994.

    Google Scholar 

  11. Tetsuya Higuchi, Tatsuya Niwa, Toshio Tanaka, Hitoshi Iba, Hugo de Garis, and T. Furuya. Evolvable hardware with genetic learning: A first step towards building a darwin machine. In Jean-Arcady Meyer, Herbert L. Roitblat, and Stewart W. Wilson, editors, Proceedings of the 2nd International Conference on the Simulation of Adaptive Behavior, pages 417–424. MIT Press, 1992.

    Google Scholar 

  12. Tsutomu Hoshino, Daisuke Mitsumoto, and Tohru Nagano. Fractal fitness landscape and loss of robustness in evolutionary robot navigation. Autonomous Robots, 5:1–16, 1988.

    Google Scholar 

  13. Philip Husbands, Inman Harvey, Dave Cliff, and Geoffrey Miller. The use of genetic algorithms for the development of sensorimotor control systems. In F. Moran, A. Moreno, J.J. Merelo, and P. Chacon, editors, Proceedings of the third European Conference on Artificial Life, pages 110–121, Granada, Spain, 1995. Springer.

    Google Scholar 

  14. Leslie Pack Kaelbling. Learning in Embedded Systems. Bradford Book, MIT Press, Cambridge, 1993.

    Google Scholar 

  15. Leslie Pack Kaelbling. Associative reinforcement learning: A generate and test algorithm. Machine Learning, 15(3):299–320, 1994.

    Google Scholar 

  16. Leslie Pack Kaelbling and Andrew W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237–277, 1996.

    Google Scholar 

  17. Didier Keymeulen, Marc Durantez, Kenji Konaka, Yasuo Kuniyoshi, and Tetsuya Higuchi. An evolutionary robot navigation system using a gate-level evolvable hardware. In Proceeding of the First International Conference on Evolvable Systems: from Biology to Hardware, pages 195–210. Springer Verlag, 1996.

    Google Scholar 

  18. Didier Keymeulen, Masaya Iwata, Kenji Konaka, Yasuo Kuniyoshi, and Tetsuya Higuchi. Evolvable hardware: a robot navigation system testbed. New Generation Computing, 16(2), 1998.

    Google Scholar 

  19. John Koza. Evolution of subsumption using genetic programming. In F. J. Varela and P. Bourgine, editors, Proceedings of the First European Conference on Artificial Life, pages 3–10, Cambridge, MA, 1992. MIT Press / Bradford Books.

    Google Scholar 

  20. Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8(3–4):297–321, 1992.

    Google Scholar 

  21. Sridhar Mahadevan. Enhancing transfer in reinforcement learning by building stochastic models of robot actions. In Proceedings of the Ninth International Conference on Machine Learning, pages 290–299, 1992.

    Google Scholar 

  22. Maja Mataric and Dave Cliff. Challenges in evolving controllers for physical robots. Robotics and Autonomous Systems, 19(1):67–83, November 1996.

    Google Scholar 

  23. Masahiro Murakawa, Syuji Yoshizawa, Isamu Kajitani, and Tetsuya Higuchi. Evolvable hardware for generalized neural networks. In Martha E. Pollack, editor, Proc. of Fifteenth International Joint Conference on Artificial Intelligence, pages 1146–1151. Morgan Kaufmann Publishers, 1997.

    Google Scholar 

  24. Domenico Parisi, Stefano Nolfi, and F. Cecconi. Learning, behavior and evolution. In Proceedings of the First European Conference on Artificial Life, pages 207–216, Cambridge, MA, 1992. MIT Press / Bradford Books.

    Google Scholar 

  25. Connie Loggia Ramsey and John J. Grefenstette. Case-based initialization of genetic algorithms. In Proceedings of the fifth International Conference on Genetic Algorithms, pages 84–91, San Mateo, CA., 1993. Morgan Kaufmann.

    Google Scholar 

  26. Craig W. Reynolds. An evolved, vision-based model of obstacle avoidance behavior. In Artificial Life III, Sciences of Complexity, Proc. Vol. XVII, pages 327–346. Addison-Wesley, 1994.

    Google Scholar 

  27. Mehrdad Salami, Masaya Iwata, and Tetsuya Higuchi. Lossless image compression by evolvable hardware. In Proceedings of the Fourth European Conference on Artificial Life, Complex Adaptive Systems, pages 407–416, Boston, USA, 1997. MIT Press / Bradford Book.

    Google Scholar 

  28. Luc Steels and Rodney Brooks, editors. The Artificial Life Route to Artificial Intelligence: Building Embodied, Situated Agents. Lawrence Erlbaum Assoc, 1995.

    Google Scholar 

  29. Richard S. Sutton. Learning to predict by the method of temporal differences. Machine Learning, 3(1):9–44, 1988.

    Google Scholar 

  30. Richard S. Sutton. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the Seventh International Conference on Machine Learning, pages 216–224, 1990.

    Google Scholar 

  31. Richard S. Sutton. Planning by incremental dynamic programming. In Proceedings of the Eighth International Workshop on Machine Learning, pages 353–357. Morgan Kaufmann, 1991.

    Google Scholar 

  32. Adrian Thompson. Evolving electronic robot controllers that exploit hardware resources. In F. Moran, A. Moreno, J.J. Merelo, and P. Chacon, editors, Advances in Artificial Life: Proceedings 3rd European Conference on Artificial Life, pages 640–656, Granada, Spain, 1995. Springer-Verlag.

    Google Scholar 

  33. Christopher J.C.H. Watkins and Peter Dayan Q-learning.Machine Learning, 8(3):279–292, 1992.

    Google Scholar 

  34. Steven D. Whitehead and Dana H. Ballard. A role for anticipation in reactive systems that learn. In Proceedings of the Sixth International Conference on Machine Learning, pages 354–357, 1989.

    Google Scholar 

  35. Stewart Wilson. Classifier systems and the animat problem. Machine Learning, 2:199–228, 1987.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrotechnical Laboratory, 305, Tsukuba, Ibaraki, Japan

    Didier Keymeulen, Masaya Iwata, Yasuo Kuniyoshi & Tetsuya Higuchi

  2. Logic Design Corp., 305, Mito, Ibaraki, Japan

    Kenji Konaka & Ryouhei Suzuki

Authors
  1. Didier Keymeulen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masaya Iwata
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kenji Konaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ryouhei Suzuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yasuo Kuniyoshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tetsuya Higuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Philip Husbands Jean-Arcady Meyer

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Keymeulen, D., Iwata, M., Konaka, K., Suzuki, R., Kuniyoshi, Y., Higuchi, T. (1998). Off-line model-free and on-line model-based evolution for tracking navigation using evolvable hardware. In: Husbands, P., Meyer, JA. (eds) Evolutionary Robotics. EvoRobots 1998. Lecture Notes in Computer Science, vol 1468. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-64957-3_74

Download citation

  • DOI: https://doi.org/10.1007/3-540-64957-3_74

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-64957-1

  • Online ISBN: 978-3-540-49902-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation