Skip to main content
SpringerLink
Log in

Simulation-only experiments to mimic the effects of the reality gap in the automatic design of robot swarms

  • Published:
Swarm Intelligence Aims and scope Submit manuscript

Abstract

The reality gap—the discrepancy between reality and simulation—is a critical issue in the off-line automatic design of control software for robot swarms, as well as for single robots. It is understood that the reality gap manifests itself as a drop in performance: when control software generated in simulation is ported to physical robots, the performance observed is often disappointing compared with the one obtained in simulation. In this paper, we investigate whether, to observe the effects of the reality gap, it is necessary to assume that the control software is designed in a context that is simpler than the one in which it is evaluated. In the first experiment, we show that a performance drop may be observed also in an artificial, simulation-only reality gap: control software is generated on the basis of a simulation model and assessed on a second one. We will call this second model a pseudo-reality. We selected the simulation model to be used as a pseudo-reality by trial and error, so as to qualitatively replicate previously published observations made in experiments with physical robots. The results show that a performance drop occurs even if we can exclude that pseudo-reality is more complex than the simulation model used for the design. In the second experiment, we eliminate the trial-and-error selection of the first experiment by evaluating control software across multiple pseudo-realities, which are sampled around the original simulation model used for the design. The results of the second experiment confirm those of the first one and show that they do not depend on the specific pseudo-reality we previously selected by trial and error. Moreover, they suggest that one could use multiple pseudo-realities to evaluate automatic design methods and, from this simulation-only evaluation, infer their robustness to the reality gap.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., Schneider, J., Sidor, S., Tobin, J., Welinder, P., Weng, L., & Zaremba, W. (2018). Learning dexterous in-hand manipulation. eprint arXiv:1808.00177.

  • Beni, G. (2004). From swarm intelligence to swarm robotics. In E. Şahin & W. M. Spears (Eds.), Swarm robotics, SAB (Vol. 3342, pp. 1–9). Berlin Heidelberg: Springer.

    Google Scholar 

  • Berman, S., Kumar, V., & Nagpal, R. (2011). Design of control policies for spatially inhomogeneous robot swarms with application to commercial pollination. In L. Zexiang (Ed.), IEEE international conference robotics and automation, ICRA (pp. 378–385). Piscataway: IEEE Press.

  • Birattari, M. (2009). Tuning metaheuristics: A machine learning perspective. Berlin Heidelberg: Springer.

    Book  MATH  Google Scholar 

  • Birattari, M., Delhaisse, B., Francesca, G., & Kerdoncuff, Y. (2016). Observing the effects of overdesign in the automatic design of control software for robot swarms. In M. Dorigo, et al. (Eds.), Swarm intelligence, 10th international conference, ANTS (Vol. 9882, pp. 45–57). Cham: Springer, LNCS.

  • Birattari, M., Ligot, A., Bozhinoski, D., Brambilla, M., Francesca, G., Garattoni, L., et al. (2019). Automatic off-line design of robot swarms: A manifesto. Frontiers in Robotics and AI, 6(59), 1–6.

    Google Scholar 

  • Birattari, M., Stützle, T., Paquete, L., & Varrentrapp, K. (2002). A racing algorithm for configuring metaheuristics. In W. Langdon, et al. (Eds.), Proceedings of the genetic and evolutionary computation conference, GECCO (pp. 11–18). San Francisco, CA: Morgan Kaufmann.

  • Boeing, A., & Braunl, T. (2012). Leveraging multiple simulators for crossing the reality gap. In International Conference on control automation: Robotics and vision, ICARCV (pp. 1113–1119). Piscataway, NJ: IEEE Press.

  • Bongard, J., & Lipson, H. (2004). Once more unto the breach: co-evolving a robot and its simulator. In J. Pollack, et al. (Eds.), Artificial life IX: Proceedings of the conference on the simulation and synthesis of living systems (pp. 57–62).

  • Brambilla, M., Brutschy, A., Dorigo, M., & Birattari, M. (2015). Property-driven design for swarm robotics: A design method based on prescriptive modeling and model checking. ACM Transactions on Autonomous and Adaptive Systems, 9(4), 17.1–28.

    Google Scholar 

  • Brambilla, M., Ferrante, E., Birattari, M., & Dorigo, M. (2013). Swarm robotics: A review from the swarm engineering perspective. Swarm Intelligence, 7(1), 1–41.

    Article  Google Scholar 

  • Bredeche, N., Montanier, J. M., Liu, W., & Winfield, A. F. (2012). Environment-driven distributed evolutionary adaptation in a population of autonomous robotic agents. Mathematical and Computer Modelling of Dynamical Systems, 18(1), 101–129.

    Article  MATH  Google Scholar 

  • Brooks, R. (1992). Artificial life and real robots. In F. J. Varela & P. Bourgine (Eds.), Towards a practice of autonomous systems. Proceedings of the first european conference on artificial life (pp. 3–10). Cambridge, MA: MIT Press.

  • Caruana, R., Lawrence, S., & Lee Giles, C. (2001). Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In T. Leen, T. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems 13, NIPS 2000 (pp. 402–408). MIT Press.

  • Dorigo, M., & Birattari, M. (2007). Swarm intelligence. Scholarpedia, 2(9), 1462.

    Article  Google Scholar 

  • Floreano, D., Husbands, P., & Nolfi, S. (2008). Evolutionary robotics. In Springer Handbook of robotics (pp. 1423–1451). Springer, Berlin, Germany.

  • Floreano, D., & Mondada, F. (1996). Evolution of plastic neurocontrollers for situated agents. In: P. Maes, et al. (Eds.), From animals to animats 4: Proceedings of the international conference on simulation of adaptive behavior. Zurich: ETH Zurich.

  • Floreano, D., & Urzelai, J. (2001). Evolution of plastic control networks. Autonomous Robots, 11(3), 311–317.

    Article  MATH  Google Scholar 

  • Francesca, G., & Birattari, M. (2016). Automatic design of robot swarms: Achievements and challenges. Frontiers in Robotics and AI, 3(29), 1–9.

    Google Scholar 

  • Francesca, G., Brambilla, M., Brutschy, A., Garattoni, L., Miletitch, R., Podevijn, G., et al. (2015). AutoMoDe-Chocolate: Automatic design of control software for robot swarms. Swarm Intelligence, 9(2/3), 125–152.

    Article  Google Scholar 

  • Francesca, G., Brambilla, M., Brutschy, A., Trianni, V., & Birattari, M. (2014). AutoMoDe: A novel approach to the automatic design of control software for robot swarms. Swarm Intelligence, 8(2), 89–112.

    Article  Google Scholar 

  • Garattoni, L., Francesca, G., Brutschy, A., Pinciroli, C., & Birattari, M. (2015). Software infrastructure for e-puck (and TAM). Tech. Rep. TR/IRIDIA/2015-004, IRIDIA, Université libre de Bruxelles, Belgium.

  • Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4(1), 1–58.

    Article  Google Scholar 

  • Gutiérrez, Á., Campo, A., Dorigo, M., Donate, J., Monasterio-Huelin, F., & Magdalena, L. (2009). Open e-puck range and bearing miniaturized board for local communication in swarm robotics. In K. Kosuge (Ed.), IEEE international conference on robotics and automation, ICRA (pp. 3111–3116). Piscataway, NJ: IEEE Press.

  • Haasdijk, E., Bredeche, N., & Eiben, A. (2014). Combining environment-driven adaptation and task-driven optimisation in evolutionary robotics. PLoS ONE, 9(6), e98466.

    Article  Google Scholar 

  • Hamann, H. (2018). Swarm robotics: A formal approach. Berlin: Springer.

    Book  Google Scholar 

  • Hamann, H., & Wörn, H. (2008). A framework of space–time continuous models for algorithm design in swarm robotics. Swarm Intelligence, 2(2–4), 209–239.

    Article  Google Scholar 

  • Hasselmann, K., Ligot, A., Francesca, G., & Birattari, M. (2018a). Reference models for AutoMoDe. Tech. Rep. TR/IRIDIA/2018-002, IRIDIA, Université libre de Bruxelles, Belgium.

  • Hasselmann, K., Robert, F., & Birattari, M. (2018b). Automatic design of communication-based behaviors for robot swarms. In M. Dorigo, et al. (Eds.), Swarm intelligence, ANTS, LNCS (Vol. 11172, pp. 16–29). Springer: Cham.

  • Jakobi, N. (1997). Evolutionary robotics and the radical envelope-of-noise hypothesis. Adaptive Behavior, 6(2), 325–368.

    Article  Google Scholar 

  • Jakobi, N. (1998). Minimal simulations for evolutionary robotics. PhD thesis, University of Sussex, Falmer, UK

  • Jakobi, N., Husbands, P., Harvey, I. (1995). Noise and the reality gap: the use of simulation in evolutionary robotics. In F. Morán, et al. (Eds.), Advances in artificial life (Vol. 929, pp. 704–720). London: Springer, LNCS.

    Chapter  Google Scholar 

  • König, L., & Mostaghim, S. (2009). Decentralized evolution of robotic behavior using finite state machines. International Journal of Intelligent Computing and Cybernetics, 2(4), 695–723.

    Article  MathSciNet  MATH  Google Scholar 

  • Koos, S., Mouret, J. B., & Doncieux, S. (2013). The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation, 17(1), 122–145.

    Article  Google Scholar 

  • Kuckling, J., Ligot, A., Bozhinoski, D., & Birattari, M. (2018). Behavior trees as a control architecture in the automatic modular design of robot swarms. In M. Dorigo, et al. (Eds.), Swarm intelligence, ANTS, LNCS (Vol. 11172, pp. 30–43). Springer: Cham.

  • Lee, J. B., & Arkin, R. C. (2003). Adaptive multi-robot behavior via learning momentum. In C. S. George Lee (Ed.), IEEE/RSJ international conference on intelligent robots and systems, IROS (pp. 2029–2036). Piscataway, NJ: IEEE Press.

  • Ligot, A., & Birattari, M. (2019). Simulation-only experiments to mimic the effects of the reality gap in the automatic design of robot swarms. Supplementary material http://iridia.ulb.ac.be/supp/IridiaSupp2019-002.

  • López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016). The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives, 3, 43–58.

    Article  MathSciNet  Google Scholar 

  • Miglino, O., Lund, H., & Nolfi, S. (1995). Evolving mobile robots in simulated and real environments. Artificial Life, 2(4), 417–434.

    Article  Google Scholar 

  • Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, JC., Floreano, D., & Martinoli, A. (2009). The e-puck, a robot designed for education in engineering. In P. Gonçalves, P. Torres & C. Alves (Eds.), Proceedings of the 9th conference on autonomous robot systems and competitions (pp. 59–65). Instituto Politécnico de Castelo Branco, Portugal.

  • Mondada, F., Franzi, E., & Ienne, P. (1994). Mobile robot miniaturisation: A tool for investigation in control algorithms. In T. Yoshikawa & F. Miyazaki (Eds.), Experimental robotics III (pp. 501–513). Berlin, Heidelberg: Springer.

  • Morgan, N., & Bourlard, H. (1990). Generalization and parameter estimation in feedforward nets: Some experiments. In D. S. Touretzky (Ed.), Advances in neural information processing systems 2, NIPS 1990 (pp. 630–637). San Francisco: Morgan Kaufmann.

  • Nolfi, S., Floreano, D., Miglino, G., & Mondada, F. (1994). How to evolve autonomous robots: Different approaches in evolutionary robotics. In R. A. Brooks & P. Maes (Eds.), Artificial Life IV: Proceedings of the workshop on the synthesis and simulation of living systems (pp. 190–197). Cambridge, MA: MIT Press.

  • Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018). Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 1–8).

  • Pinciroli, C., Trianni, V., O’Grady, R., Pini, G., Brutschy, A., Brambilla, M., et al. (2012). ARGoS: A modular, parallel, multi-engine simulator for multi-robot systems. Swarm Intelligence, 6(4), 271–295.

    Article  Google Scholar 

  • Reina, A., Valentini, G., Fernández-Oto, C., Dorigo, M., & Trianni, V. (2015). A design pattern for decentralised decision making. PLoS ONE, 10(10), e0140950.

    Article  Google Scholar 

  • Şahin, E. (2004). Swarm robotics: From sources of inspiration to domains of application. In E. Şahin & W. M. Spears (Eds.), Swarm robotics, SAB (Vol. 3342, pp. 10–20). Berlin Heidelberg: Springer, LNCS

  • Silva, F., Duarte, M., Correia, L., Oliveira, S., & Christensen, A. (2016). Open issues in evolutionary robotics. Evolutionary Computation, 24(2), 205–236.

    Article  Google Scholar 

  • Silva, F., Urbano, P., Correia, L., & Christensen, A. L. (2015). odNEAT: An algorithm for decentralised online evolution of robotic controllers. Evolutionary Computation, 23(3), 421–449.

    Article  Google Scholar 

  • Urzelai, J., & Floreano, D. (2000). Evolutionary robotics: Coping with environmental change. In: L. D. Whitney, et al (Eds.), Proceedings of conference on the genetic and evolutionary computation conference, GECCO (pp. 941–948). San Francisco, CA: Morgan Kaufmann.

  • Watson, R., Ficici, S., & Pollack, J. (2002). Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robotics and Autonomous Systems, 39(1), 1–18.

    Article  Google Scholar 

  • Zagal, J. C., & Ruiz-Del-Solar, J. (2007). Combining simulation and reality in evolutionary robotics. Journal of Intelligent and Robotic Systems, 50(1), 19–39.

    Article  Google Scholar 

  • Zagal, J. C., Ruiz-Del-Solar, J., & Vallejos, P. (2004). Back to reality: Crossing the reality gap in evolutionary robotics. IFAC/EURON Symposium on Intelligent Autonomous Vehicles, IAV, 37, 834–839.

    Google Scholar 

Download references

Acknowledgements

The project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant Agreement No 681872). Mauro Birattari acknowledges support from the Belgian Fonds de la Recherche Scientifique—FNRS.

Author information

Authors and Affiliations

  1. IRIDIA, Université libre de Bruxelles, Brussels, Belgium

    Antoine Ligot & Mauro Birattari

Authors
  1. Antoine Ligot
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mauro Birattari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Antoine Ligot or Mauro Birattari.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The experiments were conceived by the two authors and performed by Antoine Ligot. The article was drafted by Antoine Ligot and revised by the two authors. The research was directed by Mauro Birattari.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ligot, A., Birattari, M. Simulation-only experiments to mimic the effects of the reality gap in the automatic design of robot swarms. Swarm Intell 14, 1–24 (2020). https://doi.org/10.1007/s11721-019-00175-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11721-019-00175-w

Keywords

Search

Navigation