Skip to main content
Springer Nature Link
Log in

A Monte Carlo hyper-heuristic algorithm with low-level heuristics reward prediction for missile path planning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Missile path planning under multiple aircraft relay guidance is important in long-range air-to-ground strikes. Traditional meta-heuristic algorithms applied in path planning problems lack flexibility in the algorithm iteration process, and current hyper-heuristic (HH) algorithms have difficulty estimating the performance of low-level heuristics (LLHs) applied to the population in different states. This study proposes a Monte Carlo hyper-heuristic (MCHH) algorithm, which is adaptive to various path planning scenarios. The LLH set contains 18 LLHs generated from the basic operators in three meta-heuristic algorithms. The high-level strategy (HLS) evaluates the states of individuals and the reward of the LLH applied to each individual. A discrete state-action-reward table is used to predict the effectiveness of different LLHs and thus determine the optimal LLH applied in iterations. The table is trained through the MC method. The results of simulation cases and algorithm comparison demonstrate the efficiency and superiority of the MCHH algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Algorithm 2
Algorithm 3
Algorithm 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The data are available from the corresponding author upon reasonable request.

References

  1. Xiao W, Wang H, Shi Z ( 2014) Research on collaborative decision-making in ship-to-air missile relay guidance. In: 2014 IEEE Workshop on Electronics, Computer and Applications, pp. 681– 684. IEEE, Ottawa. https://doi.org/10.1109/IWECA.2014.6845712

  2. Helgason RV, Kennington JL, Lewis KR (2001) Cruise missile mission planning: a heuristic algorithm for automatic path generation. J Heuristics 7:473–494. https://doi.org/10.1023/A:1011325912346

    Article  MATH  Google Scholar 

  3. Aggarwal S, Kumar N (2020) Path planning techniques for unmanned aerial vehicles: a review, solutions, and challenges. Comput Commun 149:270–299. https://doi.org/10.1016/j.comcom.2019.10.014

    Article  MATH  Google Scholar 

  4. Gan L, Yan Z, Zhang L, Liu K, Zheng Y, Zhou C, Shu Y (2022) Ship path planning based on safety potential field in inland rivers. Ocean Eng 260:111928. https://doi.org/10.1016/j.oceaneng.2022.111928

    Article  Google Scholar 

  5. Li X, Lu Y, Zhao X, Deng X, Xie Z (2024) Path planning for intelligent vehicles based on improved d* lite. J Supercomput 80:1294–1330. https://doi.org/10.1007/s11227-023-05528-1

    Article  MATH  Google Scholar 

  6. Vcs S, Ah S (2022) Nature inspired meta heuristic algorithms for optimization problems. Computing 104:251–269. https://doi.org/10.1007/s00607-021-00955-5

    Article  MathSciNet  MATH  Google Scholar 

  7. Xu S, Bi W, Zhang A, Wang Y (2024) A deep reinforcement learning approach incorporating genetic algorithm for missile path planning. Int J Mach Learn Cybern 15:1795–1814. https://doi.org/10.1007/s13042-023-01998-0

    Article  MATH  Google Scholar 

  8. Sun R, Yang T (2024) Hybrid parameter-based pso flexible needle percutaneous puncture path planning. J Supercomput 80:5408–5427. https://doi.org/10.1007/s11227-023-05661-x

    Article  MATH  Google Scholar 

  9. Yu X, Luo W (2023) Reinforcement learning-based multi-strategy cuckoo search algorithm for 3d uav path planning. Expert Syst Appl 223:119910. https://doi.org/10.1016/j.eswa.2023.119910

    Article  MATH  Google Scholar 

  10. Yu X, Li C, Zhou J (2020) A constrained differential evolution algorithm to solve uav path planning in disaster scenarios. Knowledge-Based Syst 204:106209. https://doi.org/10.1016/j.knosys.2020.106209

    Article  MATH  Google Scholar 

  11. Dokeroglu T, Kucukyilmaz T, Talbi E-G (2024) Hyper-heuristics: a survey and taxonomy. Comput Ind Eng 187:109815. https://doi.org/10.1016/j.cie.2023.109815

    Article  MATH  Google Scholar 

  12. Drake JH, Kheiri A, Özcan E, Burke EK (2020) Recent advances in selection hyper-heuristics. Eur J Oper Res 285(2):405–428. https://doi.org/10.1016/j.ejor.2019.07.073

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhong R, Zhang E, Munetomo M (2024) Evolutionary multi-mode slime mold optimization: a hyper-heuristic algorithm inspired by slime mold foraging behaviors. J Supercomput 80:12186–12217. https://doi.org/10.1007/s11227-024-05909-0

    Article  Google Scholar 

  14. Zhang S, Xu Y, Zhang W (2021) Multitask-oriented manufacturing service composition in an uncertain environment using a hyper-heuristic algorithm. J Manuf Syst 60:138–151. https://doi.org/10.1016/j.jmsy.2021.05.012

    Article  MATH  Google Scholar 

  15. Maashi M, Kendall G, Özcan E (2015) Choice function based hyper-heuristics for multi-objective optimization. Appl Soft Comput 28:312–326. https://doi.org/10.1016/j.asoc.2014.12.012

    Article  MATH  Google Scholar 

  16. Wei D, Wang F, Ma H (2019) Autonomous path planning of auv in large-scale complex marine environment based on swarm hyper-heuristic algorithm. Appl Sci 9(13):2654. https://doi.org/10.3390/app9132654

    Article  MATH  Google Scholar 

  17. Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26–38. https://doi.org/10.1109/MSP.2017.2743240

    Article  Google Scholar 

  18. Kallestad J, Hasibi R, Hemmati A, Sörensen K (2023) A general deep reinforcement learning hyperheuristic framework for solving combinatorial optimization problems. Eur J Oper Res 309(1):446–468. https://doi.org/10.1016/j.ejor.2023.01.017

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhang Y, Bai R, Qu R, Tu C, Jin J (2022) A deep reinforcement learning based hyper-heuristic for combinatorial optimisation with uncertainties. Eur J Oper Res 300(2):418–427. https://doi.org/10.1016/j.ejor.2021.10.032

    Article  MathSciNet  MATH  Google Scholar 

  20. Gölcük Ozsoydan FB (2021) Q-learning and hyper-heuristic based algorithm recommendation for changing environments. Eng Appl Artif Intell 102:104284. https://doi.org/10.1016/j.engappai.2021.104284

    Article  MATH  Google Scholar 

  21. Raychaudhuri S ( 2008) Introduction to monte carlo simulation. In: 2008 Winter Simulation Conference, pp. 91– 100. IEEE, Miami. https://doi.org/10.1109/WSC.2008.4736059

  22. Jones GL, Qin Q (2022) Markov chain monte carlo in practice. Annu Rev Stat Appl 9:557–578. https://doi.org/10.1146/annurev-statistics-040220-090158

    Article  MathSciNet  MATH  Google Scholar 

  23. Qi W, Cheng D, Sun H, Lin Q, Li X, Huang Z, Jiang W, Jiang Y (2022) Airspace allocation method in mission planning of early warning aircraft. Publishing House of Electronics Industry, Beijing, pp 64–82

    MATH  Google Scholar 

  24. Luo W, Chen L, Liu K, Gu H, Lü J (2022) Optimizing constrained guidance policy with minimum overload regularization. IEEE Trans Biomed Circuits Syst 69(7):2994–3005. https://doi.org/10.1109/TCSI.2022.3163463

    Article  MATH  Google Scholar 

  25. Xu S, Bi W, Zhang A, Mao Z (2022) Optimization of flight test tasks allocation and sequencing using genetic algorithm. Appl Soft Comput 115:108241. https://doi.org/10.1016/j.asoc.2021.108241

    Article  MATH  Google Scholar 

  26. Kalita DJ, Singh VP, Kumar V (2023) A lightweight knowledge-based pso for svm hyper-parameters tuning in a dynamic environment. J Supercomput 79:18777–18799. https://doi.org/10.1007/s11227-023-05385-y

    Article  MATH  Google Scholar 

  27. Bakshi M, Chowdhury C, Maulik U (2023) Cuckoo search optimization-based energy efficient job scheduling approach for iot-edge environment. J Supercomput 79:18227–18255. https://doi.org/10.1007/s11227-023-05358-1

    Article  MATH  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62073267, 62473319), the Fundamental Research Funds for the Central Universities (Grant No. G2020KY05110), and the Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University (Grant No. CX2022019).

Author information

Authors and Affiliations

  1. School of Aeronautics, Northwestern Polytechnical University, Xi’an, 710072, Shaanxi, China

    Shuangfei Xu, Zhanjun Huang, Wenhao Bi & An Zhang

  2. National Key Laboratory of Aircraft Configuration Design, Xi’an, 710072, Shaanxi, China

    Zhanjun Huang, Wenhao Bi & An Zhang

Authors
  1. Shuangfei Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhanjun Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wenhao Bi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. An Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Shuangfei Xu was involved in conceptualization, methodology, software, and writing–original draft. Zhanjun Huang helped with formal analysis and writing–review and editing. Wenhao Bi contributed to conceptualization, investigation, and writing–review and editing. An Zhang participated in resources, funding acquisition, and supervision.

Corresponding author

Correspondence to Wenhao Bi.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, S., Huang, Z., Bi, W. et al. A Monte Carlo hyper-heuristic algorithm with low-level heuristics reward prediction for missile path planning. J Supercomput 81, 374 (2025). https://doi.org/10.1007/s11227-024-06771-w

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06771-w

Keywords