Skip to main content
SpringerLink
Log in

An insightful comparison between experiments in mobile robotics and in science

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Experiments are essential ingredients of science, both to confirm/refute a theory and to discover new theories. It is a common belief that experimentation in mobile robotics has not yet reached a level of maturity comparable with that reached in science, for example in physics, considered as the paradigm of a mature, stable, and well-founded scientific discipline. In this paper, starting from a representative sample of the current state of the art, we identify some basic issues of experiments in mobile robot localization and mapping. These issues, when viewed in the context of some general principles about experiments in science and engineering, lead us to derive some insightful considerations on the role of experiments in mobile robotics. Reflecting the background of the authors, the paper has an interdisciplinary nature at the meeting point of mobile robotics and philosophy of science.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Amigoni, F. (2008). Experimental evaluation of some exploration strategies for mobile robots. In IEEE int. conf. on robotics and automation (pp. 2818–2823).

  • Amigoni, F., Gasparini, S., & Gini, M. (2007). Good experimental methodologies for robotic mapping: A proposal. In IEEE int. conf. on robotics and automation (pp. 4176–4181).

  • Barr, R. S., Golden, B. L., Kelly, J. P., Resende, M. G. C., & Stewart, W. R. (1995). Designing and reporting on computational experiments with heuristic methods. Journal of Heuristics, 1, 9–32.

    Article  MATH  Google Scholar 

  • Bonsignorio, F., Hallam, J., & del Pobil, A.P. (2007). Good experimental methodology—GEM guidelines. http://www.heronrobots.com/EuronGEMSig/Downloads/GemSigGuidelinesBeta.pdf.

  • Comport, A. I., Malis, E., & Rives, P. (2007). Accurate quadrifocal tracking for robust 3d visual odometry. In IEEE int. conf. on robotics and automation (pp. 40–45).

  • Cromer, A. (1993). Uncommon sense. New York: Oxford University Press.

    Google Scholar 

  • Davison, A. J. (2003). Real-time simultaneous localisation and mapping with a single camera. In IEEE int. conf. on computer vision (pp. 1403–1410).

  • Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007). Monoslam: real-time single camera slam. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), 1052–1067.

    Article  Google Scholar 

  • EURON GEM Sig (2007). http://www.heronrobots.com/EuronGEMSig/.

  • Franklin, A. (1981). What makes a ‘good’ experiment? British Journal for the Philosophy of Science, 32(4), 367–379.

    Article  MathSciNet  Google Scholar 

  • Franklin, A. (2009). Experiment in physics. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy. http://plato.stanford.edu/archives/spr2009/entries/physics-experiment/.

  • Frese, U., Larsson, P., & Duckett, T. (2005). A multilevel relaxation algorithm for simultaneous localization and mapping. IEEE Transactions on Robotics, 21(2), 196–207.

    Article  Google Scholar 

  • Grisetti, G., Stachniss, C., & Burgard, W. (2007). Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE Transactions on Robotics, 23(1), 34–46.

    Article  Google Scholar 

  • Grosser, M. (1962). The discovery of Neptune. Cambridge: Harvard University Press.

    Google Scholar 

  • Guivant, J., Nebot, E., & Baiker, S. (2000). Autonomous navigation and map building using laser range sensors in outdoor applications. Journal of Robotic Systems, 17(10), 565–583.

    Article  MATH  Google Scholar 

  • Gutmann, J. S., & Konolige, K. (1999). Incremental mapping of large cyclic environments. In IEEE int. symp. on computational intelligence in robotics and automation (pp. 318–325).

  • Hacking, I. (1981). Do we see through a microscope? Pacific Philosophical Quarterly, 62, 305–322.

    Google Scholar 

  • Hacking, I. (1983). Representing and intervening. Introductory topics in the philosophy of natural science. Cambridge: Cambridge University Press.

    Google Scholar 

  • Hahnel, D., Burgard, W., Fox, D., & Thrun, S. (2003). An efficient fastslam algorithm for generating maps of large-scale cyclic environments from raw laser range measurements. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 206–211).

  • Hanks, S., Pollack, M. E., & Cohen, P. R. (1993). Benchmarks, test beds, controlled experimentation, and the design of agent architectures. AI Magazine, 14(4), 17–42.

    Google Scholar 

  • Howard, A., & Roy, N. (2003). The robotics data set repository (radish). http://radish.sourceforge.net/.

  • Johnson, D. (2002). A theoretician’s guide to the experimental analysis of algorithms. In M. H. Goldwasser, D. S. Johnson, & C. C. McGeoch (Eds.), Data structures, near neighbor searches, and methodology: fifth and sixth dimacs implementation challenges (pp. 215–250). Providence: American Mathematical Society.

    Google Scholar 

  • Leonard, J. J., & Durrant-Whyte, H. F. (1991). Simultaneous map building and localization for an autonomous mobile robot. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 1442–1447).

  • Liu, Y., & Thrun, S. (2003). Results for outdoor-slam using sparse extended information filters. In IEEE int. conf. on robotics and automation (pp. 1227–1233).

  • Lu, F., & Milios, E. (1997). Globally consistent range scan alignment for environment mapping. Autonomous Robots, 4(4), 333–349.

    Article  Google Scholar 

  • Minguez, J., Montesano, L., & Lamiraux, F. (2006). Metric-based iterative closest point scan matching for sensor displacement estimation. IEEE Transactions on Robotics, 22(5), 1047–1054.

    Article  Google Scholar 

  • Montemerlo, M., Thrun, S., Koller, D., & Wegbreit, B. (2003). Fastslam 2.0: An improved particle filtering algorithm for simultaneous localization and mapping that provably converges. In Int. joint conf. on artificial intelligence (pp. 1151–1156).

  • Montgomery, D. (2005). Design and analysis of experiments. New York: Wiley.

    MATH  Google Scholar 

  • Neira, J., & Tardos, J. D. (2001). Data association in stochastic mapping using the joint compatibility test. IEEE Transactions on Robotics and Automation, 17(6), 890–897.

    Article  Google Scholar 

  • Newman, P., Leonard, J., Tardos, J. D., & Neira, J. (2002). Explore and return: experimental validation of real-time concurrent mapping and localization. In IEEE int. conf. on robotics and automation (pp. 1802–1809).

  • Newman, P., Cole, D., & Ho, K. (2006). Outdoor slam using visual appearance and laser ranging. In IEEE int. conf. on robotics and automation (pp. 1180–1187).

  • Paz, L. M., Pinies, P., Tardos, J. D., & Neira, J. (2008). Large-scale 6-dof slam with stereo-in-hand. IEEE Transactions on Robotics, 24(5), 946–957.

    Article  Google Scholar 

  • Permis (2000). Performance metrics for intelligent systems. http://www.isd.mel.nist.gov/research_areas/research_engineering/Performance_Metrics/index.htm.

  • Rawseeds (2006). http://rawseeds.elet.polimi.it/.

  • RoSta (2007). Robot standards and reference architectures. http://www.robot-standards.eu/.

  • Stachniss, C., Frese, U., & Grisetti, G. (2007). Openslam.org. http://www.openslam.org/.

  • Thrun, S., Burgard, W., & Fox, D. (2005). Probabilistic robotics. Cambridge: The MIT Press.

    MATH  Google Scholar 

  • Westfall, R. (1971). The construction of modern science. Mechanisms and mechanics. New York: Wiley.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Elettronica e Informazione, Politecnico di Milano, Milano, Italy

    Francesco Amigoni & Viola Schiaffonati

  2. Dipartimento di Tecnica e Gestione dei Sistemi Industriali, Università di Padova, Vicenza, Italy

    Monica Reggiani

Authors
  1. Francesco Amigoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monica Reggiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Viola Schiaffonati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesco Amigoni.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Amigoni, F., Reggiani, M. & Schiaffonati, V. An insightful comparison between experiments in mobile robotics and in science. Auton Robot 27, 313–325 (2009). https://doi.org/10.1007/s10514-009-9137-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-009-9137-8

Keywords

Search

Navigation