Skip to main content
SpringerLink
Log in

Learning environmental fields with micro underwater vehicles: a path integral—Gaussian Markov random field approach

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Autonomous underwater vehicles (AUVs) are advancing the state of the art in numerous scientific and commercial applications. The current surge in micro electronics enables the development of small micro AUVs (\(\mu \)AUVs) which are expected to gain increasing popularity in industrial applications such as monitoring of liquid-based processes. This paper presents an information theoretic approach for exploration and monitoring of liquid containing tanks with \(\mu \)AUVs. The controller is based on ideas from path integral control and inference with Gaussian Markov random fields (GMRFs). Both parts are combined in a receding horizon scheme to the PI-GMRF controller. The control problem is formulated within the stochastic optimal control domain and a solution is stated as a path integral. In order to close the control theoretic loop each \(\mu \)AUV maintains a belief representation of the environment expressed with GMRFs which allows reasoning by computing posterior distributions conditioned on measurements. Each \(\mu \)AUV has its own controller instance and the system is decentral. Only the exchange of measurements and intended control inputs of each \(\mu \)AUV is required through the communication link. The approach is validated in simulations for an advection–diffusion scenario and benchmarked against random walk, which it outperforms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Bourgault, F., Makarenko, A. A., Williams, S. B., Grocholsky, B., & Durrant-Whyte, H. (2002). Information based adaptive robotic exploration. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 540–545). Lausanne.

  • Csató, L., & Opper, M. (2002). Sparse on-line Gaussian processes. Neural Computation, 14(3), 641–668.

    Article  MATH  Google Scholar 

  • Cui, R., Li, Y., & Yan, W. (2016). Mutual information-based multi-AUV path planning for scalar field sampling using multidimensional RRT. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 46(7), 993–1004.

    Article  Google Scholar 

  • Devries, L., Majumdar, S. J., & Paley, D. A. (2013). Observability-based optimization of coordinated sampling trajectories for recursive estimation of a strong, spatially varying flowfield. Journal of Intelligent and Robotic Systems: Theory and Applications, 70(1–4), 527–544.

    Article  Google Scholar 

  • Duecker, D. A., Geist, A., Hengeler, M., Kreuzer, E., Pick, M. A., Rausch, V., et al. (2017). Embedded spherical localization system for micro underwater vehicles based on attenuation of electro-magnetic Carrier signals. Sensors, 17, 959–981.

    Article  Google Scholar 

  • Fiorelli, E., Leonard, N. E., Bhatta, P., Paley, D. A., Bachmayer, R., & Fratantoni, D. M. (2006). Multi-AUV control and adaptive sampling in Monterey Bay. IEEE Journal of Oceanic Engineering, 31(4), 935–948.

    Article  Google Scholar 

  • Geist, R. A., Hackbarth, A., Kreuzer, E., Rausch, V., Sankur, M., & Solowjow, E. (2016). Towards a hyperbolic acoustic one-way localization system for underwater swarm robotics. IEEE international conference on robotics and automation (ICRA) (pp. 4551–4556). Stockholm.

  • Gershman, S. J., Horvitz, E. J., & Tenenbaum, J. B. (2015). Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science, 349(6245), 273–278.

    Article  MathSciNet  MATH  Google Scholar 

  • Gómez, V., Thijssen, S., Symington, A., Hailes, S., & Kappen, H. J. (2015). Real-time stochastic optimal control for multi-agent quadrotor swarms. Preprint. arXiv:1502.04548.

  • Graham, R., & Cortés, J. (2012). Adaptive information collection by robotic sensor networks for spatial estimation. IEEE Transactions on Automatic Control, 57(6), 1404–1419.

    Article  MathSciNet  MATH  Google Scholar 

  • Griffiths, A., Dikarev, A., Green, P. R., Lennox, B., Poteau, X., & Watson, S. (2016). AVEXIS-aqua vehicle explorer for in-situ sensing. IEEE Robotics and Automation Letters, 1(1), 282–287.

    Article  Google Scholar 

  • Hackbarth, A., Gray, A., & Kreuzer, E. (2012). Multi-agent motion control of autonomous vehicles in 3D flow fields. Proceedings in Applied Mathematics and Mechanics, 12(1), 733–734.

    Article  Google Scholar 

  • Hackbarth, A., Kreuzer, E., & Solowjow, E. (2015). Hippocampus: A micro underwater vehicle for swarm applications. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2258–2263). Hamburg.

  • Hollinger, G. A., & Sukhatme, G. S. (2014). Sampling-based robotic information gathering algorithms. The International Journal of Robotics Research, 33(9), 1271–1287.

    Article  Google Scholar 

  • Jonsson, J., Sundqvist, J., Nguyen, H., Kratz, H., Berglund, M., Ogden, S., et al. (2011). Miniaturized submersible for exploration of small aqueous environments. In MTS/IEEE OCEANS (pp. 1–8). Kona, HI.

  • Kappen, H. J. (2007). An introduction to stochastic control theory, path integrals and reinforcement learning. Cooperative Behavior in Neural Systems, 887, 149–181.

    Article  MathSciNet  MATH  Google Scholar 

  • Krause, A. (2007). Nonmyopic observation selection: Theory and applications. PhD thesis. Pittsburgh, PA: Carnegie Mellon University.

  • Krige, D. G. (1951). A statistical approach to some basic mine valuation problems on the Witwatersrand. Journal of Chemical, Metallurgical, and Mining Society of South Africa, 52, 201–215.

    Google Scholar 

  • LaValle, S. M., & Kuffner, J. J. (2001). Randomized kinodynamic planning. The International Journal of Robotics Research, 20(5), 378–400.

    Article  Google Scholar 

  • Le Ny, J., & Pappas, G. J. (2013). Adaptive deployment of mobile robotic networks. IEEE Transactions on Automatic Control, 58(3), 654–666.

    Article  MathSciNet  MATH  Google Scholar 

  • Leonard, N. E., Paley, D. A., Lekien, F., Sepulchre, R., Fratantoni, D. M., & Davis, R. E. (2007). Collective motion, sensor networks, and ocean sampling. Proceedings of the IEEE, 95(1), 48–74.

    Article  Google Scholar 

  • Lindgren, F., Rue, H., & Lindström, J. (2011). An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach. Journal of the Royal Statistical Society: Series B, 73(4), 423–498.

    Article  MathSciNet  MATH  Google Scholar 

  • Ma, K. C., Liu, L., & Sukhatme, G. S. (2016). Informative planning and online learning with sparse Gaussian processes. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2102–2108). Daejeon.

  • Marchant, R., & Ramos, F. (2014). Bayesian optimisation for informative continuous path planning. In IEEE international conference on robotics and automation (ICRA) (pp. 6136–6143). Hong Kong.

  • Marchant, R., Ramos, F., Sanner, S., et al. (2014). Sequential Bayesian optimisation for spatial–temporal monitoring. In Conference on uncertainty in artificial intelligence (UAI) (pp. 553–562). Quebec City, QC.

  • Marino, A., & Antonelli, G. (2015). Experiments on sampling/patrolling with two autonomous underwater vehicles. Robotics and Autonomous Systems, 67, 61–71.

    Article  Google Scholar 

  • Mintchev, S., Donati, E., Marrazza, S., & Stefanini, C. (2014). Mechatronic design of a miniature underwater robot for swarm operations. In IEEE international conference on robotics and automation (ICRA) (pp. 2938–2943). Hong Kong.

  • Myers, V., & Williams, D. P. (2010). A POMDP for multi-view target classification with an autonomous underwater vehicle. In MTS/IEEE OCEANS (pp. 1–5). Seattle, WA.

  • Paley, D. A., & Peterson, C. (2009). Stabilization of collective motion in a time-invariant flowfield. Journal of Guidance, Control, and Dynamics, 32(3), 771–779.

    Article  Google Scholar 

  • Pan, Y., Theodorou, E. A., & Kontitsis, M. (2014). Model-based path integral stochastic control: A Bayesian nonparametric approach. Preprint. arXiv:1412.3038.

  • Rue, H., & Held, L. (2005). Gaussian Markov random fields: Theory and applications. Boca Raton: CRC Press.

    Book  MATH  Google Scholar 

  • Ryan, A., & Hedrick, J. K. (2010). Particle filter based information-theoretic active sensing. Robotics and Autonomous Systems, 58(5), 574–584.

    Article  Google Scholar 

  • Simpson, D., Lindgren, F., & Rue, H. (2012). Think continuous: Markovian Gaussian models in spatial statistics. Spatial Statistics, 1, 16–29.

    Article  Google Scholar 

  • Stanković, M. S., & Stipanović, D. M. (2010). Extremum seeking under stochastic noise and applications to mobile sensors. Automatica, 46(8), 1243–1251.

    Article  MathSciNet  MATH  Google Scholar 

  • Sydney, N., & Paley, D. A. (2014). Multivehicle coverage control for a nonstationary spatiotemporal field. Automatica, 50(5), 1381–1390.

    Article  MathSciNet  MATH  Google Scholar 

  • Theodorou, E., Buchli, J., & Schaal, S. (2010a). A generalized path integral control approach to reinforcement learning. Journal of Machine Learning Research, 11(Nov), 3137–3181.

    MathSciNet  MATH  Google Scholar 

  • Theodorou, E., Buchli, J., & Schaal, S. (2010b). Reinforcement learning of motor skills in high dimensions: A path integral approach. In IEEE international conference on robotics and automation (ICRA) (pp. 2397–2403). Anchorage, AK.

  • Vasudevan, S., Ramos, F., Nettleton, E., & Durrant-Whyte, H. (2009). Gaussian process modeling of large-scale terrain. Journal of Field Robotics, 26(10), 812–840.

    Article  MATH  Google Scholar 

  • Williams, G., Drews, P., Goldfain, B., Rehg, J. M., & Theodorou, E. A. (2016). Aggressive driving with model predictive path integral control. In IEEE international conference on robotics and automation (ICRA) (pp. 1433–1440). Stockholm.

  • Xu, Y., Choi, J., Dass, S., & Maiti, T. (2013). Efficient Bayesian spatial prediction with mobile sensor networks using Gaussian Markov random fields. Automatica, 49(12), 3520–3530.

    Article  MathSciNet  MATH  Google Scholar 

  • Xu, Y., Choi, J., Dass, S., & Maiti, T. (2015). Bayesian prediction and adaptive sampling algorithms for mobile sensor networks: Online environmental field reconstruction in space and time. Berlin: Springer.

    MATH  Google Scholar 

  • Xu, Y., Choi, J., & Oh, S. (2011). Mobile sensor network navigation using Gaussian processes with truncated observations. IEEE Transactions on Robotics, 27(6), 1118–1131.

    Article  Google Scholar 

  • Zhang, B. Z. B., & Sukhatme, G. (2007). Adaptive sampling for estimating a scalar field using a robotic boat and a sensor network. In IEEE international conference on robotics and automation (ICRA). Roma.

Download references

Acknowledgements

This work was funded by the German Research Foundation (DFG) under Grant No. Kr 752/33-1. The support is greatly acknowledged.

Author information

Authors and Affiliations

  1. Institute of Mechanics and Ocean Engineering, Hamburg University of Technology, Eissendorfer Strasse 42, 21073, Hamburg, Germany

    Edwin Kreuzer & Eugen Solowjow

Authors
  1. Edwin Kreuzer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eugen Solowjow
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eugen Solowjow.

Additional information

This is one of the several papers published in Autonomous Robots comprising the Special Issue on Online Decision Making in Multi-Robot Coordination.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kreuzer, E., Solowjow, E. Learning environmental fields with micro underwater vehicles: a path integral—Gaussian Markov random field approach. Auton Robot 42, 761–780 (2018). https://doi.org/10.1007/s10514-017-9685-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-017-9685-2

Keywords

Search

Navigation