Skip to main content
SpringerLink
Log in

Combining Simulation and Reality in Evolutionary Robotics

  • Published:
Journal of Intelligent and Robotic Systems Aims and scope Submit manuscript

Abstract

Evolutionary Robotics (ER) is a promising methodology, intended for the autonomous development of robots, in which their behaviors are obtained as a consequence of the structural coupling between robot and environment. It is essential that there be a great amount of interaction to generate complex behaviors. Thus, nowadays, it is common to use simulation to speed up the learning process; however simulations are achieved from arbitrary off-line designs, rather than from the result of embodied cognitive processes. According to the reality gap problem, controllers evolved in simulation usually do not allow the same behavior to arise once transferred to the real robot. Some preliminary approaches for combining simulation and reality exist in the ER literature; nonetheless, there is no satisfactory solution available. In this work we discuss recent advances in neuroscience as a motivation for the use of environmentally adapted simulations, which can be achieved through the co-evolution of robot behavior and simulator. We present an algorithm in which only the differences between the behavior fitness obtained in reality versus that obtained in simulations are used as feedback for adapting a simulation. The proposed algorithm is experimentally validated by showing the successful development and continuous transference to reality of two complex low-level behaviors with Sony AIBO1 robots: gait optimization and ball-kicking behavior.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bedau, M.A.: Can unrealistic computer models illuminate theoretical biology? In: Wu, A.S. (ed.) Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Program, pp. 20–23. Morgan Kaufmann, San Francisco, CA (1999)

    Google Scholar 

  2. Bongard, J.C., Lipson, H.: Once more unto the breach: co-evolving a robot and its simulator. In: Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALIFE9), pp. 57–62 (2004)

  3. Brooks, R.: Artificial life and real robots. In: F.J. Varela and P. Bourgine. (ed.) Toward a Practice in Autonomous Systems. Proceedings of the First European Conference on Artificial Life, pp. 3–10. MIT Press, Cambridge, MA (1992)

  4. Di Paolo, E.A.: Homeostatic adaptation to inversion of the visual field and other sensorimotor disruptions. In: J-A Meyer, A. Berthoz, D. Floreano, H.L. Roitblat, and S.W. Wilson (eds.) From Animals to Animats, Vol. 6. Proceedings of the VI International Conference on Simulation of Adaptive Behavior, pp. 440–449. MIT Press, Cambridge, MA (2000)

    Google Scholar 

  5. Ficici, S.: Solution concepts in coevolutionary algorithms. PhD thesis, Brandeis University (2004)

  6. Franklin, M.S, Zyphur, M.J.: The role of dreams in the evolution of the human mind. Evolutionary Psychology 3, 59–78 (2005)

    Google Scholar 

  7. Golubovic, D., Hu, H.: A hybrid evolutionary algorithm for gait generation of Sony legged robots. In: Proceedings of the 28th Annual Conference of the IEEE Industrial Electronics Society, Seville, Spain, 5–8 November 2002

  8. Grefenstette, J.J., Ramsey C.L.: An approach to anytime learning. In: Proceedings of the Ninth International Machine Learning Workshop, Morgan Kaufmann. San Mateo, CA (1992)

  9. Hardt, M., Stryk, O.: The role of motion dynamics in the design, control and stability of bipedal and quadrupedal robots. In: Kaminka, G.A., Lima, P.U., Rojas, R. (eds.) Lecture Notes in Computer Science, vol. 2752, pp. 206–223. Springer, Berlin Heidelberg New York (2002)

    Google Scholar 

  10. Hobson, J.A., Pace-Schott, E.F.: The cognitive neuroscience of sleep: neuronal systems, consciousness and learning. Nat. Rev., Neurosci. 3(9), 679–93 (2002)

    Article  Google Scholar 

  11. Hornby, G.S., Fujita, M., Takamura, S., Yamamoto, T., Hanagata, O.: Autonomous evolution of gaits with the Sony quadruped robot. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1297–1304. Morgan Kaufmann, San Mateo, CA (1999)

  12. Husbands, P., Harvey, I., Cliff, D.: An evolutionary approach to situated AI. In: A. Sloman et al. (ed.) Proceedings of the 9th Conference of the Society for the Study of Artificial Intelligence and the Simulation of Behavior (AISB), pp. 61–70. IOS Press, Amsterdam, The Netherlands (1993)

    Google Scholar 

  13. Jakobi, N.: Minimal simulations for evolutionary robotics. PhD thesis, University of Sussex (1998)

  14. Kim, M., Uther, W.: Automatic gait optimisation for quadruped robots. In: Roberts, J., Wyeth, G. (eds.) Australasian Conference on Robotics and Automation, Brisbane (December 2003)

  15. Kitano, H., Hamahashi, S., Kitazawa, J., Takao, K., Imai, S.: The virtual biology laboratories: a new approach of computational biology. In: Husbands, P., Harvey, I. (eds.) Proceedings of the Fourth European Conference on Artificial Life, pp. 274–283. MIT Press, Cambridge, MA (1997)

    Google Scholar 

  16. Kohl, N. and Stone, P.: Policy gradient reinforcement learning for fast quadrupedal locomotion. In: Proceedings of IEEE International Conference on Robotics and Automation (2004)

  17. Nolfi, S., Floreano, D., Miglio, O., Mondada, F.: How to evolve autonomous robots: different approaches in evolutionary robotics. In: Brooks, R., Maes, P. (eds.) Artificial Life IV, pp. 190–197. MIT Press, Cambridge, MA (1994)

    Google Scholar 

  18. Nolfi, S., Floreano, D.: Evolutionary robotics – the biology, intelligence, and technology of self-organizing machines. In: Intelligent Robotics and Automation Agents. MIT Press, Cambridge, MA (2000)

  19. Quilan, M.J., Chalup, S.K., Middleton, R.H.: Techniques for improving vision and locomotion on the Sony AIBO robot. In: Australasian Conference on Robotics and Automation (ACRA 2003) (2003)

  20. Revonsuo, A.: The reinterpretation of dreams: an evolutionary hypothesis of the function of dreaming. Behav. Brain Sci. 23(66), 877–903 (2000)

    Article  Google Scholar 

  21. Röfer, T.: Evolutionary gait-optimization using a fitness function based on proprioception. In: Proceedings of RoboCup 2004: Robot Soccer World Cup VIII. Lecture Notes of Artificial Intelligence, pp. 310–322. Springer, Berlin Heidelberg New York (2004)

  22. Ruiz-del-Solar, J., Zagal, J.C., Guerrero, P., Vallejos, P., Middleton, C., Olivares, X.: UChile1 team description. In: Proceedings of RoboCup 2003: Robot Soccer World Cup VII. Lecture Notes of Artificial Intelligence. Springer, Berlin Heidelberg New York (2003)

  23. Stickgold, R, Hobson, J.A., Fosse, R. Fosse, M.: Sleep, learning and dreams: off-line memory reprocessing. Science 294 (5544), 1052–1057 (2001)

    Article  Google Scholar 

  24. Varela, F.J.: Principles of Biological Autonomy, New York: Elsevier (1979)

    Google Scholar 

  25. Zagal J.C., Ruiz-del-Solar J.: UCHILSIM: A dynamically and visually realistic simulator for the RoboCup four legged league. In: Proceedings of RoboCup 2004: Robot Soccer World Cup VIII. Lecture Notes of Artificial Intelligence, pp. 34–45. Springer, Berlin Heidelberg New York (2004)

  26. Zagal, J.C., Ruiz-del-Solar, J.: Learning to kick the ball using back to reality. In: Proceedings of RoboCup 2004: Robot Soccer World Cup VIII. Lecture Notes of Artificial Intelligence, pp. 335–346. Springer, Berlin Heidelberg New York (2004)

  27. Zagal, J.C., Ruiz-del-Solar, J., Vallejos, P.: Back to reality: crossing the reality gap in evolutionary robotics. In: Proceedings of the 5th IFAC Symposium on Intelligent Autonomous Vehicles, IAV 2004. Elsevier, Amsterdam, The Netherlands (2005)

  28. Zagal, J.C., Sarmiento, I., Ruiz-del-Solar, J.: An application interface for UCHILSIM and the arrival of new challenges. In: Proceedings of RoboCup 2005: Robot Soccer World Cup IX. Lecture Notes of Artificial Intelligence, pp. 464–471. Springer, Berlin Heidelberg New York (2005)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Universidad de Chile, Av. Tupper 2007, 6513027, Santiago, Chile

    Juan Cristóbal Zagal & Javier Ruiz-del-Solar

Authors
  1. Juan Cristóbal Zagal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Javier Ruiz-del-Solar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Cristóbal Zagal.

Additional information

1AIBO is a trademark of Sony Corporation

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zagal, J.C., Ruiz-del-Solar, J. Combining Simulation and Reality in Evolutionary Robotics. J Intell Robot Syst 50, 19–39 (2007). https://doi.org/10.1007/s10846-007-9149-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-007-9149-6

Keywords

Search

Navigation