Abstract
In the realm of maritime emergencies, unmanned aerial vehicles (UAVs) play a crucial role in enhancing search and rescue (SAR) operations. They help in efficiently rescuing distressed crews, strengthening maritime surveillance, and maintaining national security due to their cost-effectiveness, versatility, and effectiveness. However, the vast expanse of sea territories and the rapid changes in maritime conditions make a single SAR center insufficient for handling complex emergencies. Thus, it is vital to develop strategies for quickly deploying UAV resources from multiple SAR centers for area reconnaissance and supporting maritime rescue operations. This study introduces a graph-structured planning model for the maritime SAR path planning problem, considering multiple rescue centers (MSARPPP-MRC). It incorporates workload distribution among SAR centers and UAV operational constraints. We propose a reinforcement learning-based genetic algorithm (GA-RL) to tackle the MSARPPP-MRC problem. GA-RL uses heuristic rules to initialize the population and employs the Q-learning method to manage the progeny during each generation, including their retention, storage, or disposal. When the elite repository’s capacity is reached, a decision is made on the utilization of these members to refresh the population. Additionally, adaptive crossover and perturbation strategies are applied to develop a more effective SAR scheme. Extensive testing proves that GA-RL surpasses other algorithms in optimization efficacy and efficiency, highlighting the benefits of reinforcement learning in population management.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availibility
No datasets were generated or analysed during the current study.
References
Agbissoh OTOTED, Li B, Ai B, Gao S, Xu J, Chen X, Lv G (2019) A decision-making algorithm for maritime search and rescue plan. Sustainability 11(7):2084
Zhou X (2022) A comprehensive framework for assessing navigation risk and deploying maritime emergency resources in the south china sea. Ocean Eng 248:110797
Lee S, Morrison JR (2015) Decision support scheduling for maritime search and rescue planning with a system of uavs and fuel service stations. In: 2015 International conference on unmanned aircraft systems (ICUAS). IEEE, pp. 1168–1177
Ai B, Jia M, Xu H, Xu J, Wen Z, Li B, Zhang D (2021) Coverage path planning for maritime search and rescue using reinforcement learning. Ocean Eng 241:110098
Wang Z, Gao W, Li G, Wang Z, Gong M (2024) Path planning for unmanned aerial vehicle via off-policy reinforcement learning with enhanced exploration. IEEE Trans Emerg Topics Comput Intell. https://doi.org/10.1109/TETCI.2024.3369485
Yue Guan Wang (2019) A novel searching method using reinforcement learning scheme for multi-uavs in unknown environments. Appl Sci 9(22):4964
Zhao L, Bai Y, Paik JK (2024) Optimal coverage path planning for usv-assisted coastal bathymetric survey: models, solutions, and lake trials. Ocean Eng 296:116921
Kyriakakis NA, Marinaki M, Matsatsinis N, Marinakis Y (2022) A cumulative unmanned aerial vehicle routing problem approach for humanitarian coverage path planning. Eur J Oper Res 300:992–1004
Ma Y, Li B, Huang W, Fan Q (2023) An improved NSGA-II based on multi-task optimization for multi-uav maritime search and rescue under severe weather. J Marine Sci Eng 11(4):781. https://doi.org/10.3390/jmse11040781
Ma Q, Zhang D, Wan C, Zhang J, Lyu N (2022) Multi-objective emergency resources allocation optimization for maritime search and rescue considering accident black-spots. Ocean Eng 261:112178. https://doi.org/10.1016/j.oceaneng.2022.112178
Wu J, Cheng L, Chu S (2023) Modeling the leeway drift characteristics of persons-in-water at a sea-area scale in the seas of China. Ocean Eng 270:113444
Koopman BO (1957) The theory of search: Iii. the optimum distribution of searching effort. Operations research 5(5), 613–626. INFORMS
Karakaya M (2014) Uav route planning for maximum target coverage. arXiv preprint arXiv:1403.2906
Yang L, Yin R, Xue Y, Tian Y, Liu H (2023) A time-domain planning method for surface rescue process of amphibious aircraft for medium/distant maritime rescue. Appl Sci-basel. https://doi.org/10.3390/app13042169
Theile M, Bayerlein H, Nai R, Gesbert D, Caccamo M (2020) UAV coverage path planning under varying power constraints using deep reinforcement learning. In: 2020 IEEE RSJ International conference on intelligent robots and systems (IROS). IEEE, pp. 1444–1449
Li B, Patankar S, Moridian B, Mahmoudian N (2018) Planning large-scale search and rescue using team of uavs and charging stations. In: 2018 IEEE International symposium on safety, security, and rescue robotics (SSRR). IEEE, pp. 1–8
Li L, Gu Q, Liu L (2020) Research on path planning algorithm for multi-uav maritime targets search based on genetic algorithm. In: 2020 IEEE international conference on information technology, big data and artificial intelligence (ICIBA). IEEE, vol. 1, pp. 840–843
Xi M, Yang J, Wen J, Liu H, Li Y, Song HH (2022) Comprehensive ocean information-enabled AUV path planning via reinforcement learning. IEEE Internet Things J 9(18):17440–17451
Jonnarth A, Zhao J, Felsberg M (2023) End-to-end reinforcement learning for online coverage path planning in unknown environments. arXiv preprint arXiv:2306.16978
Li R, Gong W, Wang L, Lu C, Pan Z, Zhuang X (2023) Double dqn-based coevolution for green distributed heterogeneous hybrid flowshop scheduling with multiple priorities of jobs. IEEE Trans Autom Sci Eng. https://doi.org/10.1109/TASE.2023.3327792
Song Y, Wei L, Yang Q, Wu J, Xing L, Chen Y (2023) Rl-ga: a reinforcement learning-based genetic algorithm for electromagnetic detection satellite scheduling problem. Swarm Evol Comput 77:101236101236
Rani S, Babbar H, Kaur P, Alshehri MD, Shah SH (2022) An optimized approach of dynamic target nodes in wireless sensor network using bio inspired algorithms for maritime rescue. IEEE Trans Intell Transp Syst 24(2):2548–2555
Zhou Y, Kong L, Yan L, Liu Y, Wang H (2024) A memetic algorithm for a real-world dynamic pickup and delivery problem. Memetic Comput 10:1–15
Chen L, Liu H, Liu H-L, Gu F (2022) A bi-level transformation based evolutionary algorithm framework for equality constrained optimization. Memetic Comput 14(4):423–432
Palubeckis G (2022) Metaheuristic approaches for ratio cut and normalized cut graph partitioning. Memetic Comput 14(3):253–285
Wu J, Cheng L, Chu S, Song Y (2024) An autonomous coverage path planning algorithm for maritime search and rescue of persons-in-water based on deep reinforcement learning. Ocean Eng 291:116403. https://doi.org/10.1016/j.oceaneng.2023.116403
Song Y, Wu Y, Guo Y, Yan R, Suganthan PN, Zhang Y, Pedrycz W, Das S, Mallipeddi R, Ajani OS et al (2024) Reinforcement learning-assisted evolutionary algorithm: a survey and research opportunities. Swarm Evol Comput 86:101517
Song Y, Suganthan PN, Pedrycz W, Yan R, Fan D, Zhang Y (2024) Energy-efficient satellite range scheduling using a reinforcement learning-based memetic algorithm. IEEE Trans Aerosp Electr Syst. https://doi.org/10.1109/TAES.2024.3371964
Yao F, Song Y-J, Zhang Z-S, Xing L-N, Ma X, Li X-J (2019) Multi-mobile robots and multi-trips feeding scheduling problem in smart manufacturing system: an improved hybrid genetic algorithm. Int J Adv Rob Syst 16(4):1729881419868126
Hollander M, Wolfe DA, Chicken E (2013) Nonparametric statistical methods. Wiley, Hoboken
Acknowledgements
This work was supported by the National Natural Science Foundation of China (723B2002), the Science and Technology Innovation Team of Shaanxi Province (2023-CX-TD-07), and the Key R &D Program Projects in Shaanxi Province (2024GH-ZDXM-48), and the Natural Science Foundation Project of Hunan Province (2024JJ5109, 2024JJ7098).
Author information
Authors and Affiliations
Contributions
Conceptualization: Haowen Zhan, Yanjie Song; Methodology: Haowen Zhan, Yue Zhang, Yanjie Song; Formal analysis and investigation: Yanjie Song, Zengyun Gao; Data Curation: Haowen Zhan, Yanjie Song, Zengyun Gao; Software: Haowen Zhan, Yanjie Song; Writing—original draft preparation: Haowen Zhan, Jingbo Huang, Yanjie Song; Writing—review and editing: Yue Zhang, Jingbo Huang, Yanjie Song; Visualization: Yue Zhang, Jie Wu; Funding acquisition: Lining Xing; Resources: Jie Wu; Zengyun Gao; Supervision: Lining Xing
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhan, H., Zhang, Y., Huang, J. et al. A reinforcement learning-based evolutionary algorithm for the unmanned aerial vehicles maritime search and rescue path planning problem considering multiple rescue centers. Memetic Comp. 16, 373–386 (2024). https://doi.org/10.1007/s12293-024-00420-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12293-024-00420-8