Abstract
In future ocean battlefields, cooperative pursuit and confrontation using Autonomous Underwater Vehicles (AUVs) will emerge as a crucial combat method. This paper presents a cooperative pursuit-evasion game strategy algorithm, utilizing multi-step Q-learning, to address the challenges posed by ocean currents and obstacles in complex underwater environments. The algorithm enables multiple AUVs to perform cooperative operations and jointly hunt down evading AUVs. By integrating optimal control and game theory, we enhance the learning capabilities of reinforcement learning for discrete behaviors. Through numerical example simulations, we validate the effectiveness of the proposed method. This research marks an important step towards enabling AUVs to operate collaboratively and effectively in future ocean battlefields.
Supported in part by the National Natural Science Foundation of China under Grant 52271321, 61873161, Shanghai Rising-Star Program under Grant 20QA1404200 and Natural Science Foundation of Shanghai under Grant 22ZR1426700.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
De Souza, C., Newbury, R., Cosgun, A., Castillo, P., Vidolov, B., Kulić, D.: Decentralized multi-agent pursuit using deep reinforcement learning. IEEE Robot. Autom. Lett. 6(3), 4552–4559 (2021)
Zhu, K., Zhang, T.: Deep reinforcement learning based mobile robot navigation: a review. Tsinghua Sci. Technol. 26(5), 674–691 (2021)
Fang, Y., Huang, Z.W., Pu, J.Y., Zhang, J.S.: AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method. Ocean Eng. 24514, 110452 (2022)
Xi, M., Yang, J., Wen, J., Liu, H., Li, Y., Song, H.H.: Comprehensive ocean information-enabled AUV path planning via reinforcement learning. IEEE Internet Things J. 9(18), 17440–17451 (2022)
Liu, L., Tian, B., Zhao, X., Zong, Q.: UAV autonomous trajectory planning in target tracking tasks via a DQN approach. In: 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 277–282 (2019)
Xu, G., Wang, Y., Liu, H.: UAV multi-target surveillance cruise trajectory planning based on DQN algorithm. In: 2022 China Automation Congress (CAC), pp. 851–856 (2022)
Wang, Z., Sui, Y., Qin, H., Lu, H.: State super sampling soft actor-critic algorithm for multi-AUV hunting in 3D underwater environment. J. Mar. Sci. Eng. 11(7), 1257 (2023)
Setiaji, B., Pujastuti, E., Filza, M.F., Masruro, A., Pradana, Y.A.: Implementation of reinforcement learning in 2d based games using open AI gym. In: 2022 International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS), pp. 293–297 (2022)
Frihauf, P., Krstic, M., Basar, T.: Nash equilibrium seeking in noncooperative games. IEEE Trans. Autom. Control 57(5), 1192–1207 (2012)
Feinberg, E.A.: Continuous time discounted jump markov decision processes: a discrete-event approach. Math. Oper. Res. 29(3), 492–524 (2004)
Feng, Y., Dai, L., Gao, J., Cheng, G.: Uncertain pursuit-evasion game. Soft. Comput. 24(4), 2425–2429 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, X., Sun, B., Su, Z. (2023). Cooperative Pursuit-Evasion Game for Multi-AUVs in the Ocean Current and Obstacle Environment. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14269. Springer, Singapore. https://doi.org/10.1007/978-981-99-6489-5_16
Download citation
DOI: https://doi.org/10.1007/978-981-99-6489-5_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6488-8
Online ISBN: 978-981-99-6489-5
eBook Packages: Computer ScienceComputer Science (R0)