Abstract
The paper describes an autonomous water vehicle (ASV) capable of autonomously mapping shallow water environments above and below the water surface. Over the past two years, Fraunhofer IOSB has developed a system that is fully electrified and equipped with extensive sensor technology (multibeam sonar, lidar, cameras, IMU, GNSS). For autonomous navigation, the complete processing pipeline was implemented, from obstacle detection and avoidance to trajectory planning and control to multi-sensor localization and mapping. Above water, both lidar-based mapping and photogrammetric methods are used; underwater, bathymetry data is obtained using sonar. The interface to the operator is realized by an interactive digital map table, which allows intuitive mission specification and evaluation.
Zusammenfassung
Der Beitrag beschreibt ein autonomes Wasserfahrzeug (ASV), welches in der Lage ist, eigenständig Über- und Unterwasserkarten von Flachwasserumgebungen zu erzeugen. Am Fraunhofer IOSB wurde in den vergangenen zwei Jahren ein System entwickelt, welches vollständig elektrifiziert ist und mit umfangreicher Sensorik (Multibeam-Sonar, Lidar, Kameras, IMU, GNSS) ausgestattet wurde. Für die autonome Navigation wurde die komplette Verarbeitungskette umgesetzt von der Hinderniserkennung und -vermeidung über Trajektorienplaner und -regler bis hin zur multisensoriellen Lokalisierung und Kartierung. Über Wasser kommen hierbei sowohl eine lidarbasierte Kartierung als auch photogrammetrische Methoden zum Einsatz; unter Wasser wird eine Bathymetrie mittels Sonar erzeugt. Die Schnittstelle zum Bediener wird durch einen interaktiven Lagetisch realisiert, der eine intuitive Missionsspezifikation und -auswertung ermöglicht.
About the authors

Angelika Zube received her degree in electrical engineering and information technology from the Karlsruhe Institute of Technology in 2011, where she also received her PhD in informatics in 2019. She is now a research associate in the Multi-Sensor Systems research group at the Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation IOSB in Karlsruhe, Germany. Her research focuses on control of autonomous mobile robots and mobile manipulators.

Dominik Kleiser received his master’s degree in Computer Science from the Karlsruhe Institute of Technology in 2019. He is now a research associate in the Multi-Sensor Systems research group at the Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation IOSB in Karlsruhe, Germany. His research focus is on autonomous mobile robots. In the past, he has been involved in several projects on the topic of autonomous underwater and surface vehicles.

Alexander Albrecht received his degree in mechanical engineering from the Karlsruhe Institute of Technology in 2015. He is now a research associate in the Multi-Sensor Systems research group at the Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation IOSB in Karlsruhe, Germany. His research focuses on multi-sensor fusion and SLAM for autonomous mobile robots.

Philipp Woock received his degree in computer science in 2009 from the Karlsruhe Institute of Technology (KIT) where he also received his PhD 2015 in computer science. He is now research associate at the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB in Karlsruhe, Germany. His main research interest is mobile robotics with focus on multi-sensor fusion and processing of sonar sensor data.

Thomas Emter received his degree in electrical engineering and information technology from the Karlsruhe Institute of Technology in 2005, where he also received his PhD in informatics in 2021. He is now a research associate in the Multi-Sensor Systems research group at the Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation IOSB in Karlsruhe, Germany. His research focuses on multi-sensor fusion and SLAM for autonomous mobile robots.

Boitumelo Ruf received his masters degree in computer science from the University of Stuttgart in 2015. In 2022, he received his PhD in aerial photogrammetry from the Karlsruher Institute of Technology. He is now research associate at the Fraunhofer IOSB in Karlsruhe. His resarch focus is on computer vision and photogrammetric 3D reconstruction for aerial and ground-based robots.

Dr.-Ing. Igor Tchouchenkov received the Dipl.-Ing. (M.S.E.E.) and the Dr.-Ing. (Ph.D.E.E.) degrees from the Belarussian State University (Minsk) in 1984 and 1987, respectively. From 1999 to 2011, he was at the Institute for Process Control and Robotics at the University of Karlsruhe (today KIT). Since 2011, he is the head of the research group “Distributed Systems” at Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB in Karlsruhe. He is member of “Lernende Systeme” – Germany’s Platform for Artificial Intelligence and VDI/VDE-GMA Technical Committee 5.15 “Agent Systems”. His research topics are control systems, information acquisition, exploration and surveillance, drone detection, classification and counter-measures.

Aleksej Buller received his degree in electrical engineering with a focus on automation technology from the Technical University of Kaiserslautern in 2012. After his self-employment experience and employment as a test engineer in a company for safety-related automation solutions, he joined the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation in 2017 as a research associate in the department Interoperability and Assistance Systems (IAS). His research focuses on image-based reconnaissance and surveillance with drones.

Boris Wagner received his degree in cartography from the University of Applied Sciences Karlsruhe in 2007 and the master degree in geomatics from the University of Applied Sciences Dessau in 2017. Since 2007 he is a research associate at the Fraunhofer IOSB Karlsruhe. His research focus is mainly on standard-compliant geospatial data representation for situational awareness and crisis management.

Ganzorig Baatar received his degree (M.Sc.) in Technical Computer Science from Techinical University Ilmenau in 2013. Since 2014, has been working as research assistant at the Technical University Ilmenau for 2 years. He has changed to Fraunhofer IOSB-AST in 2015 and working since as research associate. His main research topics cover up sonar data processing, software development and technical development of underwater vehicles. He is also proceeding his PhD Project in Machine Learning and Object classification for underwater sonar data.

Janko Petereit received his degree in electrical engineering and information technology from the Karlsruhe Institute of Technology in 2009, where he also received his PhD in informatics in 2016. He now manages the Multi-Sensor Systems research group at the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB in Karlsruhe, Germany. His research focuses on motion planning and multi-sensor fusion for autonomous mobile robots.
References
1. Barbier, M., E. Bensana and X. Pucel. 2018. A generic and modular architecture for maritime autonomous vehicles. In: 2018 IEEE/OES Autonomous Underwater Vehicle Workshop (AUV). IEEE, pp. 1–6. DOI: 10.1109/AUV.2018.8729765.Search in Google Scholar
2. Campos, D. F., A. Matos and A. M. Pinto. 2019. An Adaptive Velocity Obstacle Avoidance Algorithm for Autonomous Surface Vehicles. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 8089–8096. DOI: 10.1109/IROS40897.2019.8968156.Search in Google Scholar
3. Chiang, H.-T. L. and L. Tapia. 2018. COLREG-RRT: An RRT-Based COLREGS-Compliant Motion Planner for Surface Vehicle Navigation. IEEE Robotics and Automation Letters 3(3): 2024–2031. DOI: 10.1109/LRA.2018.2801881.Search in Google Scholar
4. Claro, R., R. Silva and A. Pinto. 2020. Detection and Mapping of Monopiles in Offshore Wind Farms using Autonomous Surface Vehicles. In: Global Oceans 2020: Singapore – U.S. Gulf Coast. IEEE, pp. 1–4. DOI: 10.1109/IEEECONF38699.2020.9389348.Search in Google Scholar
5. Clunie, T., M. DeFilippo, M. Sacarny and P. Robinette. 2021. Development of a Perception System for an Autonomous Surface Vehicle using Monocular Camera, LIDAR, and Marine RADAR. In: 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 14 112–14 119. DOI: 10.1109/ICRA48506.2021.9561275.Search in Google Scholar
6. Dalpe, A. J., A E. Cook, M.-W. Thein and M. Renken. 2018. A Multi-Layered Approach to Autonomous Surface Vehicle Map-Based Autonomy. In: OCEANS 2018 MTS/IEEE Charleston. IEEE, pp. 1–8. DOI: 10.1109/OCEANS.2018.8604602.Search in Google Scholar
7. Du, Z., V. Reppa and R. R. Negenborn. 2021. MPC-based COLREGS Compliant Collision Avoidance for a Multi-Vessel Ship-Towing System. In: 2021 European Control Conference (ECC). IEEE, pp. 1857–1862. DOI: 10.23919/ECC54610.2021.9655091.Search in Google Scholar
8. Emter, T. and J. Petereit. 2018. 3D SLAM With Scan Matching and Factor Graph Optimization. In: ISR 2018; 50th International Symposium on Robotics. VDE.Search in Google Scholar
9. Ferri, G., A. Manzi, F. Fornai, F. Ciuchi and C. Laschi. 2015. The HydroNet ASV, a Small-Sized Autonomous Catamaran for Real-Time Monitoring of Water Quality: From Design to Missions at Sea. IEEE Journal of Oceanic Engineering 40(3): 710–726. DOI: 10.1109/JOE.2014.2359361.Search in Google Scholar
10. Fossen, T. I., K. Y. Pettersen and R. Galeazzi. 2015. Line-of-sight path following for dubins paths with adaptive sideslip compensation of drift forces. In: IEEE Transactions on Control Systems Technology. IEEE, pp. 820–827.10.1109/TCST.2014.2338354Search in Google Scholar
11. Frese, C., A. Fetzner and C. Frey. 2014. Multi-Sensor Obstacle Tracking for Safe Human-Robot Interaction. In: ISR/Robotik 2014; 41st International Symposium on Robotics. VDE, pp. 1–8.Search in Google Scholar
12. Guindel, C., J. Beltrán, D. Martín and F. García. 2017. Automatic extrinsic calibration for lidar-stereo vehicle sensor setups. In: IEEE International Conference on Intelligent Transportation Systems. IEEE, pp. 674–679.10.1109/ITSC.2017.8317829Search in Google Scholar
13. Guoqing, Z., Z. Chenliang, Z. Xianku and Z. Weidong. 2019. Robust Finite Time Path Following Control for Underactuated Ships with Ship-to-Ship Avoidance Strategy. In: Chinese Control Conference (CCC). IEEE, pp. 2334–2339.10.23919/ChiCC.2019.8865501Search in Google Scholar
14. Heide, N. F., A. Albrecht, T. Emter and J. Petereit. 2019. Performance Optimization of Autonomous Platforms in Unstructured Outdoor Environments Using a Novel Constrained Planning Approach. In: Proceedings of the IEEE Intelligent Vehicles Symposium (IV) 2019. IEEE, pp. 2359–2364. DOI: 10.1109/IVS.2019.8813805.Search in Google Scholar
15. Iwen, D. and M. WĄż. 2019. Benefits of using ASV MBES surveys in shallow waters and restriced areas. In: 2019 European Navigation Conference (ENC). IEEE, pp. 1–3. DOI: 10.1109/EURONAV.2019.8714128.Search in Google Scholar
16. Jung, J., J. Park, J. Choi and H.-T. Choi. 2019. Terrain Based Navigation for an Autonomous Surface Vehicle with a Multibeam Sonar. In: OCEANS 2019 – Marseille. IEEE, pp. 1–4. DOI: 10.1109/OCEANSE.2019.8867221.Search in Google Scholar
17. Kleiser, D., A. Albrecht, T. Emter, A. Zube, J. Petereit and P. Woock. 2020. Mapping Shallow Water Environments using a Semi-Autonomous Multi-Sensor Surface Vehicle. In: Proceedings of the IEEE Oceans Conference 2020. Fraunhofer Center for Machine Learning. DOI: 10.1109/IEEECONF38699.2020.9389482.Search in Google Scholar
18. Knapitsch, A., J. Park, Q.-Y. Zhou and V. Koltun. 2017. Tanks and Temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics 36(4): 78.10.1145/3072959.3073599Search in Google Scholar
19. Lepetit, V., F. Moreno-Noguer and P. Fua. 2009. EPnP: An accurate O(n) solution to the PnP problem. International Journal of Computer Vision 81(2): 155.10.1007/s11263-008-0152-6Search in Google Scholar
20. Odetti, A., M. Altosole, G. Bruzzone, M. Caccia and M. Viviani. 2019. Design and Construction of a Modular Pump-Jet Thruster for Autonomous Surface Vehicle Operations in Extremely Shallow Water. Journal of Marine Science and Engineering 7(7): 222. ISSN: 2077-1312. DOI: 10.3390/jmse7070222.Search in Google Scholar
21. Odetti, A., G. Bruzzone and M. Caccia. 2021. Introducing a Portable ASV for Extremely Shallow Waters. Hydro International.Search in Google Scholar
22. Osten, J., C. Weyers, K. Bregler, T. Emter and J. Petereit. 2021. Modular and scalable automation for field robots. at – Automatisierungstechnik 69(4): 307–315. DOI: 10.1515/auto-2020-0039.Search in Google Scholar
23. Pereira, M. I., R. M. Claro, P. N. Leite and A. M. Pinto. 2021. Advancing Autonomous Surface Vehicles: A 3D Perception System for the Recognition and Assessment of Docking-Based Structures. IEEE Access 9: 53 030–53 045. DOI: [10.1109/ACCESS.2021.3070694].Search in Google Scholar
24. Petereit, J. 2017. Adaptive State × Time Lattices: A Contribution to Mobile Robot Motion Planning in Unstructured Dynamic Environments. KIT Scientific Publishing. DOI: 10.5445/KSP/1000058693.Search in Google Scholar
25. Petereit, J., T. Emter and C. W. Frey. 2013. Safe mobile robot motion planning for waypoint sequences in a dynamic environment. In: 2013 IEEE International Conference on Industrial Technology (ICIT). IEEE, pp. 181–186. DOI: 10.1109/ICIT.2013.6505669.Search in Google Scholar
26. Santamaria, E., F. Segor, I. Tchouchenkov and R. Schönbein. 2013. Rapid Aerial Mapping with Multiple Heterogeneous Unmanned Vehicles. International Journal on Advances in Systems and Measurements 6(3-4):384–393.Search in Google Scholar
27. Schiaretti, M., L. Chen and R. R. Negenborn. 2017. Survey on Autonomous Surface Vessels: Part II – Categorization of 60 Prototypes and Future Applications. In: (T. Bektaş, S. Coniglio, A. Martinez-Sykora and S. Voß, eds) Proceedings of Computational Logistics. ICCL 2017, Lecture Notes in Computer Science, vol. 10572. Springer, Cham. DOI: 10.1007/978-3-319-68496-3_16.Search in Google Scholar
28. Schönberger, J. L., E. Zheng, J.-M. Frahm and M. Pollefeys. 2016. Pixelwise view selection for unstructured multi-view stereo. In: Proc. Europ. Conf. on Computer Vision. Springer, pp. 501–518.10.1007/978-3-319-46487-9_31Search in Google Scholar
29. Schönberger, J. L. and J.-M. Frahm. 2016. Structure-from-Motion Revisited. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. IEEE, pp. 4104–4113.10.1109/CVPR.2016.445Search in Google Scholar
30. Schöps, T., J. Schönberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys and A. Geiger, 2017. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In: IEEE Conference on Computer Vision and Pattern. IEEE, pp. 3260–3269.10.1109/CVPR.2017.272Search in Google Scholar
31. Tsai, C.-M., Y.-H. Lai, J.-W. Perng, I.-F. Tsui and Y.-J. Chung. 2019. Design and Application of an Autonomous Surface Vehicle with an AI-Based Sensing Capability. In: 2019 IEEE Underwater Technology (UT). IEEE, pp. 1–4. DOI: 10.1109/UT.2019.8734350.Search in Google Scholar
32. van Cappelle, L. E., L. Chen and R. R. Negenborn. 2018. Survey on Short-Term Technology Developments and Readiness Levels for Autonomous Shipping. In: (R. Cerulli, A. Raiconi and S. Voß, eds) Computational Logistics. Springer International Publishing, Cham, pp. 106–123. ISBN: 978-3-030-00898-7.10.1007/978-3-030-00898-7_7Search in Google Scholar
33. Xie, S., D. Wang, J. Luo, Y. Peng and H. Li. 2014. Trajectory Tracking Control of Unmanned Ship Based on Back-stepping Algorithm. In: Proceeding of the IEEE International Conference on Information and Automation. IEEE, pp. 990–993.10.1109/ICInfA.2014.6932795Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston