Skip to main content

Advertisement

A reinforcement learning-based evolutionary algorithm for the unmanned aerial vehicles maritime search and rescue path planning problem considering multiple rescue centers

  • Regular Research Paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

In the realm of maritime emergencies, unmanned aerial vehicles (UAVs) play a crucial role in enhancing search and rescue (SAR) operations. They help in efficiently rescuing distressed crews, strengthening maritime surveillance, and maintaining national security due to their cost-effectiveness, versatility, and effectiveness. However, the vast expanse of sea territories and the rapid changes in maritime conditions make a single SAR center insufficient for handling complex emergencies. Thus, it is vital to develop strategies for quickly deploying UAV resources from multiple SAR centers for area reconnaissance and supporting maritime rescue operations. This study introduces a graph-structured planning model for the maritime SAR path planning problem, considering multiple rescue centers (MSARPPP-MRC). It incorporates workload distribution among SAR centers and UAV operational constraints. We propose a reinforcement learning-based genetic algorithm (GA-RL) to tackle the MSARPPP-MRC problem. GA-RL uses heuristic rules to initialize the population and employs the Q-learning method to manage the progeny during each generation, including their retention, storage, or disposal. When the elite repository’s capacity is reached, a decision is made on the utilization of these members to refresh the population. Additionally, adaptive crossover and perturbation strategies are applied to develop a more effective SAR scheme. Extensive testing proves that GA-RL surpasses other algorithms in optimization efficacy and efficiency, highlighting the benefits of reinforcement learning in population management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Algorithm 1
Fig. 2
Algorithm 2
Algorithm 3
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availibility

No datasets were generated or analysed during the current study.

References

  1. Agbissoh OTOTED, Li B, Ai B, Gao S, Xu J, Chen X, Lv G (2019) A decision-making algorithm for maritime search and rescue plan. Sustainability 11(7):2084

    Article  Google Scholar 

  2. Zhou X (2022) A comprehensive framework for assessing navigation risk and deploying maritime emergency resources in the south china sea. Ocean Eng 248:110797

    Article  Google Scholar 

  3. Lee S, Morrison JR (2015) Decision support scheduling for maritime search and rescue planning with a system of uavs and fuel service stations. In: 2015 International conference on unmanned aircraft systems (ICUAS). IEEE, pp. 1168–1177

  4. Ai B, Jia M, Xu H, Xu J, Wen Z, Li B, Zhang D (2021) Coverage path planning for maritime search and rescue using reinforcement learning. Ocean Eng 241:110098

    Article  Google Scholar 

  5. Wang Z, Gao W, Li G, Wang Z, Gong M (2024) Path planning for unmanned aerial vehicle via off-policy reinforcement learning with enhanced exploration. IEEE Trans Emerg Topics Comput Intell. https://doi.org/10.1109/TETCI.2024.3369485

    Article  Google Scholar 

  6. Yue Guan Wang (2019) A novel searching method using reinforcement learning scheme for multi-uavs in unknown environments. Appl Sci 9(22):4964

    Article  Google Scholar 

  7. Zhao L, Bai Y, Paik JK (2024) Optimal coverage path planning for usv-assisted coastal bathymetric survey: models, solutions, and lake trials. Ocean Eng 296:116921

    Article  Google Scholar 

  8. Kyriakakis NA, Marinaki M, Matsatsinis N, Marinakis Y (2022) A cumulative unmanned aerial vehicle routing problem approach for humanitarian coverage path planning. Eur J Oper Res 300:992–1004

    Article  MathSciNet  Google Scholar 

  9. Ma Y, Li B, Huang W, Fan Q (2023) An improved NSGA-II based on multi-task optimization for multi-uav maritime search and rescue under severe weather. J Marine Sci Eng 11(4):781. https://doi.org/10.3390/jmse11040781

    Article  Google Scholar 

  10. Ma Q, Zhang D, Wan C, Zhang J, Lyu N (2022) Multi-objective emergency resources allocation optimization for maritime search and rescue considering accident black-spots. Ocean Eng 261:112178. https://doi.org/10.1016/j.oceaneng.2022.112178

    Article  Google Scholar 

  11. Wu J, Cheng L, Chu S (2023) Modeling the leeway drift characteristics of persons-in-water at a sea-area scale in the seas of China. Ocean Eng 270:113444

    Article  Google Scholar 

  12. Koopman BO (1957) The theory of search: Iii. the optimum distribution of searching effort. Operations research 5(5), 613–626. INFORMS

  13. Karakaya M (2014) Uav route planning for maximum target coverage. arXiv preprint arXiv:1403.2906

  14. Yang L, Yin R, Xue Y, Tian Y, Liu H (2023) A time-domain planning method for surface rescue process of amphibious aircraft for medium/distant maritime rescue. Appl Sci-basel. https://doi.org/10.3390/app13042169

    Article  Google Scholar 

  15. Theile M, Bayerlein H, Nai R, Gesbert D, Caccamo M (2020) UAV coverage path planning under varying power constraints using deep reinforcement learning. In: 2020 IEEE RSJ International conference on intelligent robots and systems (IROS). IEEE, pp. 1444–1449

  16. Li B, Patankar S, Moridian B, Mahmoudian N (2018) Planning large-scale search and rescue using team of uavs and charging stations. In: 2018 IEEE International symposium on safety, security, and rescue robotics (SSRR). IEEE, pp. 1–8

  17. Li L, Gu Q, Liu L (2020) Research on path planning algorithm for multi-uav maritime targets search based on genetic algorithm. In: 2020 IEEE international conference on information technology, big data and artificial intelligence (ICIBA). IEEE, vol. 1, pp. 840–843

  18. Xi M, Yang J, Wen J, Liu H, Li Y, Song HH (2022) Comprehensive ocean information-enabled AUV path planning via reinforcement learning. IEEE Internet Things J 9(18):17440–17451

    Article  Google Scholar 

  19. Jonnarth A, Zhao J, Felsberg M (2023) End-to-end reinforcement learning for online coverage path planning in unknown environments. arXiv preprint arXiv:2306.16978

  20. Li R, Gong W, Wang L, Lu C, Pan Z, Zhuang X (2023) Double dqn-based coevolution for green distributed heterogeneous hybrid flowshop scheduling with multiple priorities of jobs. IEEE Trans Autom Sci Eng. https://doi.org/10.1109/TASE.2023.3327792

    Article  Google Scholar 

  21. Song Y, Wei L, Yang Q, Wu J, Xing L, Chen Y (2023) Rl-ga: a reinforcement learning-based genetic algorithm for electromagnetic detection satellite scheduling problem. Swarm Evol Comput 77:101236101236

    Article  Google Scholar 

  22. Rani S, Babbar H, Kaur P, Alshehri MD, Shah SH (2022) An optimized approach of dynamic target nodes in wireless sensor network using bio inspired algorithms for maritime rescue. IEEE Trans Intell Transp Syst 24(2):2548–2555

    Google Scholar 

  23. Zhou Y, Kong L, Yan L, Liu Y, Wang H (2024) A memetic algorithm for a real-world dynamic pickup and delivery problem. Memetic Comput 10:1–15

    Google Scholar 

  24. Chen L, Liu H, Liu H-L, Gu F (2022) A bi-level transformation based evolutionary algorithm framework for equality constrained optimization. Memetic Comput 14(4):423–432

    Article  Google Scholar 

  25. Palubeckis G (2022) Metaheuristic approaches for ratio cut and normalized cut graph partitioning. Memetic Comput 14(3):253–285

    Article  Google Scholar 

  26. Wu J, Cheng L, Chu S, Song Y (2024) An autonomous coverage path planning algorithm for maritime search and rescue of persons-in-water based on deep reinforcement learning. Ocean Eng 291:116403. https://doi.org/10.1016/j.oceaneng.2023.116403

    Article  Google Scholar 

  27. Song Y, Wu Y, Guo Y, Yan R, Suganthan PN, Zhang Y, Pedrycz W, Das S, Mallipeddi R, Ajani OS et al (2024) Reinforcement learning-assisted evolutionary algorithm: a survey and research opportunities. Swarm Evol Comput 86:101517

    Article  Google Scholar 

  28. Song Y, Suganthan PN, Pedrycz W, Yan R, Fan D, Zhang Y (2024) Energy-efficient satellite range scheduling using a reinforcement learning-based memetic algorithm. IEEE Trans Aerosp Electr Syst. https://doi.org/10.1109/TAES.2024.3371964

    Article  Google Scholar 

  29. Yao F, Song Y-J, Zhang Z-S, Xing L-N, Ma X, Li X-J (2019) Multi-mobile robots and multi-trips feeding scheduling problem in smart manufacturing system: an improved hybrid genetic algorithm. Int J Adv Rob Syst 16(4):1729881419868126

    Google Scholar 

  30. Hollander M, Wolfe DA, Chicken E (2013) Nonparametric statistical methods. Wiley, Hoboken

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (723B2002), the Science and Technology Innovation Team of Shaanxi Province (2023-CX-TD-07), and the Key R &D Program Projects in Shaanxi Province (2024GH-ZDXM-48), and the Natural Science Foundation Project of Hunan Province (2024JJ5109, 2024JJ7098).

Author information

Authors and Affiliations

Authors

Contributions

Conceptualization: Haowen Zhan, Yanjie Song; Methodology: Haowen Zhan, Yue Zhang, Yanjie Song; Formal analysis and investigation: Yanjie Song, Zengyun Gao; Data Curation: Haowen Zhan, Yanjie Song, Zengyun Gao; Software: Haowen Zhan, Yanjie Song; Writing—original draft preparation: Haowen Zhan, Jingbo Huang, Yanjie Song; Writing—review and editing: Yue Zhang, Jingbo Huang, Yanjie Song; Visualization: Yue Zhang, Jie Wu; Funding acquisition: Lining Xing; Resources: Jie Wu; Zengyun Gao; Supervision: Lining Xing

Corresponding authors

Correspondence to Yue Zhang or Yanjie Song.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhan, H., Zhang, Y., Huang, J. et al. A reinforcement learning-based evolutionary algorithm for the unmanned aerial vehicles maritime search and rescue path planning problem considering multiple rescue centers. Memetic Comp. 16, 373–386 (2024). https://doi.org/10.1007/s12293-024-00420-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-024-00420-8

Keywords