Skip to main content
Springer Nature Link
Log in

Inference-Enabled Information-Theoretic Exploration of Continuous Action Spaces

  • Chapter
  • First Online:
Robotics Research

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 3))

  • 3996 Accesses

Abstract

We consider an autonomous exploration problem in which a mobile robot is guided through an a priori unknown environment by a controller that chooses the most informative action within a local region. We propose a novel approach to efficiently evaluate information gain over the continuous action space that leverages supervised learning, with the anticipated mutual information achieved by a discrete set of action primitives serving as training data. We describe an autonomous exploration algorithm that uses this approach to cover a priori unknown environments. Computational results demonstrate that the method offers an improved rate of entropy reduction, surpassing a baseline approach that selects from the discrete action set, which in some instances requires more computational effort and yields less information.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Thrun, S., Burgard, W., Fox, D.: Exploration. Probabilistic Robotics, pp. 569–605. MIT Press, Cambridge (2005)

    Google Scholar 

  2. Elfes, A.: Robot Navigation: integrating perception, environmental constraints and task execution within a probabilistic framework. In: Proceedings of the International Workshop on Reasoning with Uncertainty in Robotics, pp. 93–129 (1995)

    Google Scholar 

  3. Elfes, A.: Using occupancy grids for mobile robot perception and navigation. Computer 22(6), 46–57 (1989)

    Article  Google Scholar 

  4. Julian, B.J., Karaman, S., Rus, D.: On mutual information-based control of range sensing robots for mapping applications. Int. J. Robot. Res. 33(10), 1375–1392 (2014)

    Article  Google Scholar 

  5. Rasmussen, C.E., Williams, C.K.I.: Gaussian Process for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  6. Deisenroth, M.P., Fox, D., Rasmussen, C.E.: Gaussian processes for data-efficient learning in robotics and control. IEEE Trans. Pattern Anal. Mach. Intell. 37(2), 408–423 (2015)

    Article  Google Scholar 

  7. O’Callaghan, S.T., Ramos, F.T.: Gaussian process occupancy maps. Int. J. Robot. Res. 31(1), 42–62 (2012)

    Article  MathSciNet  Google Scholar 

  8. Smola, A., Schlkopf, B.: A tutorial on support vector regression. Stat. Comput. 14(3), 199–222 (2004)

    Article  MathSciNet  Google Scholar 

  9. Whaite, P., Ferrie, F.P.: Autonomous exploration: driven by uncertainty. IEEE Trans. Pattern Anal. Mach. Intell. 19(3), 193–205 (1997)

    Article  Google Scholar 

  10. Bourgault, F., Makarenko, A.A., Williams, S.B., Grocholsky, B., Durrant-Whyte, H.F.: Information based adaptive robotic exploration. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 540–545 (2002)

    Google Scholar 

  11. Stachniss, C., Grisetti, G., Burgard, W.: Information gain-based exploration using rao-blackwellized particle filters. In: Proceedings of the Robotics: Science and Systems Conference, pp. 65–72 (2005)

    Google Scholar 

  12. Kim, A., Eustice, R.M.: Perception-driven navigation: active visual SLAM for robotic area coverage. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3196–3203 (2013)

    Google Scholar 

  13. Kollar, T., Roy, N.: Trajectory optimization using reinforcement learning for map exploration. Int. J. Robot. Res. 27(2), 175–196 (2008)

    Article  Google Scholar 

  14. Kollar, T., Roy, N.: Efficient optimization of information-theoretic exploration in SLAM. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1369–1375 (2008)

    Google Scholar 

  15. Yang, K., Gan, S.K., Sukkarieh, S.: A Gaussian process-based RRT planner for the exploration of unknown and cluttered environment with a UAV. Adv. Robot. 27(6), 431–443 (2013)

    Article  Google Scholar 

  16. Charrow, B., Kumar, V., Michael, N.: Approximate representations for multi-robot control policies that maximize mutual information. Auton. Robot. 37(4), 383–400 (2014)

    Article  Google Scholar 

  17. Wurm, K.M., Hennes, D., Holz, D., Rusu, R.B., Stachniss, C., Konolige, K., Burgard, W.: Hierarchies of octrees for efficient 3D mapping. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4249–4255 (2011)

    Google Scholar 

  18. Quin, P., Paul, G., Alempijevic, A., Liu, D., Dissanayake, G.: Efficient neighbourhood-based information gain approach for exploration of complex 3D environments. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1343–1348 (2013)

    Google Scholar 

  19. Deisenroth, M.P., Rasmussen, C.E., Peters, J.: Gaussian process dynamic programming. Neurocomputing 72(7–9), 1508–1524 (2009)

    Article  Google Scholar 

  20. Martinez-Cantin, R., de Freitas, N., Doucet, A., Castellanos, J.A.: Active policy learning for robot planning and exploration under uncertainty. In: Proceedings of the Robotics: Science and Systems Conference (2007)

    Google Scholar 

  21. Velez, J., Hemann, G., Huang, A.S., Posner, I., Roy, N.: Modelling observation correlations for active exploration and robust object detection. J. Artif. Intell. Res. 44, 423–453 (2012)

    MATH  Google Scholar 

  22. Hollinger, G.A., Englot, B., Hover, F.S., Mitra, U., Sukhatme, G.S.: Active planning for underwater inspection and the benefit of adaptivity. Int. J. Robot. Res. 32(1), 3–18 (2013)

    Article  Google Scholar 

  23. Jadidi, M.G., Miro, J.V., Valencia, R., Andrade-Cetto, J.: Exploration on continuous Gaussian process frontier maps. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 6077–6082 (2014)

    Google Scholar 

  24. Shannon, C.E., Weaver, W.: The Mathematical Theory of Communication. University of Illinois Press, Champaign (1949)

    MATH  Google Scholar 

  25. Kim, S., Kim, J.: Continuous occupancy maps using overlapping local Gaussian processes. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4709–4714 (2013)

    Google Scholar 

  26. Sobol, I.M.: On the distribution of points in a cube and the approximate evaluation of integrals. USSR Comput. Math. Math. Phys. 7(4), 86–112 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  27. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  28. Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), (2011)

    Google Scholar 

  29. Rasmussen, C.E., Nickisch, H.: Gaussian processes for machine learning (GPML) toolbox. J. Mach. Learn. Res. 11, 3011–3015 (2010)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Chengjiang Long from the Computer Science Department and Kiril Manchevski from Mechanical Engineering Department of Stevens Institute of Technology for help with computational resources.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Stevens Institute of Technology, Castle Point on Hudson, Hoboken, NJ, 07030, USA

    Shi Bai, Jinkun Wang & Brendan Englot

  2. Department of Electrical and Computer Engineering, Stevens Institute of Technology, Castle Point on Hudson, Hoboken, NJ, 07030, USA

    Kevin Doherty

Authors
  1. Shi Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinkun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kevin Doherty
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Brendan Englot
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shi Bai .

Editor information

Editors and Affiliations

  1. Istituto Italiano di Tecnologia, Genova, Italy, University of Pisa, Pisa, Italy , Pisa, Italy

    Antonio Bicchi

  2. Inst. für Informatik, Albert-Ludwigs-Universität Freiburg Inst. für Informatik, Freiburg, Germany

    Wolfram Burgard

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Cite this chapter

Bai, S., Wang, J., Doherty, K., Englot, B. (2018). Inference-Enabled Information-Theoretic Exploration of Continuous Action Spaces. In: Bicchi, A., Burgard, W. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 3. Springer, Cham. https://doi.org/10.1007/978-3-319-60916-4_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-60916-4_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-60915-7

  • Online ISBN: 978-3-319-60916-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics