Skip to main content

A Reinforcement-Learning-Driven Bees Algorithm for Large-Scale Earth Observation Satellite Scheduling

  • Conference paper
  • First Online:
Bio-Inspired Computing: Theories and Applications (BIC-TA 2022)

Abstract

The Earth Observation Satellite Scheduling Problem (EOSSP) is difficult to solve due to its scale and constraints. Through analysing the problem, we build a mathematical programming model of the EOSSP. After that, we propose a reinforcement-learning-driven bees algorithm (RLBA) to solve a large-scale EOSSP (LSEOSSP). The RLBA adopts a Q-learning method to select search operations from global search and neighbourhood search. We define a new state action combination in the Q-learning method to cooperate with the population search and obtain high-quality solutions. Through experimental verification, the performance of the proposed algorithm is obviously better than that of several comparison algorithms, and the RLBA can solve LSEOSSP well.

This work was supported by the Special Projects in Key Fields of Universities in Guangdong (2021ZDZX1019) and the Hunan Provincial Innovation Foundation For Postgraduate (CX20200585).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Barkaoui, M., Berger, J.: A new hybrid genetic algorithm for the collection scheduling problem for a satellite constellation. J. Oper. Res. Soc. 71(9), 1390–1410 (2020)

    Article  Google Scholar 

  2. Chang, Z., Zhou, Z., Xing, L., Yao, F.: Integrated scheduling problem for earth observation satellites based on three modeling frameworks: an adaptive bi-objective memetic algorithm. Memet. Comput. 13(2), 203–226 (2021)

    Article  Google Scholar 

  3. He, L., de Weerdt, M., Yorke-Smith, N.: Tabu-based large neighbourhood search for time/sequence-dependent scheduling problems with time windows. In: Proceedings of the International Conference on Automated Planning and Scheduling, vol. 29, pp. 186–194 (2019)

    Google Scholar 

  4. Huang, Y., Luo, A., Zhang, M., Zhang, X., Lin, M., Song, Y.: A novel mission planning model and method for combat system-of-systems architecture design. In: 2022 8th International Conference on Big Data and Information Analytics (BigDIA), pp. 239–245. IEEE (2022)

    Google Scholar 

  5. Jiang, X., Song, Y., Xing, L.: Dual-population artificial bee colony algorithm for joint observation satellite mission planning problem. IEEE Access 10, 28911–28921 (2022)

    Article  Google Scholar 

  6. Li, G.: Online scheduling of distributed earth observation satellite system under rigid communication constraints. Adv. Space Res. 65(11), 2475–2496 (2020)

    Article  Google Scholar 

  7. Niu, X., Tang, H., Wu, L.: Satellite scheduling of large areal tasks for rapid response to natural disaster using a multi-objective genetic algorithm. Int. J. Disaster Risk Reduct. 28, 813–825 (2018)

    Article  Google Scholar 

  8. Ou, J., et al.: Deep reinforcement learning method for satellite range scheduling problem. Swarm and Evolutionary Computation, p. 101233 (2023)

    Google Scholar 

  9. Pham, D.T., Ghanbarzadeh, A., Koç, E., Otri, S., Rahim, S., Zaidi, M.: The bees algorithm-a novel tool for complex optimisation problems. In: Intelligent Production Machines and Systems, pp. 454–459. Elsevier (2006)

    Google Scholar 

  10. Song, Y., Wei, L., Yang, Q., Wu, J., Xing, L., Chen, Y.: Rl-ga: a reinforcement learning-based genetic algorithm for electromagnetic detection satellite scheduling problem. Swarm and Evolutionary Computation, p. 101236 (2023)

    Google Scholar 

  11. Vasquez, M., Hao, J.K.: A logic-constrained knapsack formulation and a tabu algorithm for the daily photograph scheduling of an earth observation satellite. Comput. Optim. Appl. 20(2), 137–157 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  12. Waiming, Z., Xiaoxuan, H., Wei, X., Peng, J.: A two-phase genetic annealing method for integrated earth observation satellite scheduling problems. Soft. Comput. 23(1), 181–196 (2019)

    Article  Google Scholar 

  13. Wang, X., Wu, G., Xing, L., Pedrycz, W.: Agile earth observation satellite scheduling over 20 years: formulations, methods, and future directions. IEEE Syst. J. 15(3), 3881–3892 (2020)

    Article  Google Scholar 

  14. Wei, L., Song, Y., Xing, L., Chen, M., Chen, Y.: A hybrid multi-objective coevolutionary approach for the multi-user agile earth observation satellite scheduling problem. In: Pan, L., Cui, Z., Cai, J., Li, L. (eds.) BIC-TA 2021. CCIS, vol. 1565, pp. 247–261. Springer, Singapore (2022). https://doi.org/10.1007/978-981-19-1256-6_18

    Chapter  Google Scholar 

  15. Wei, L., Xing, L., Wan, Q., Song, Y., Chen, Y.: A multi-objective memetic approach for time-dependent agile earth observation satellite scheduling problem. Comput. Ind. Eng. 159, 107530 (2021)

    Google Scholar 

Download references

Acknowledgement

Junwei Ou and Yanjie Song contribute equally to this article. Thanks to Prof. Cham for his valuable comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yan-jie Song .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Song, Yj., Ou, Jw., Pham, D.T., Li, Jt., Huang, Jb., Xing, Ln. (2023). A Reinforcement-Learning-Driven Bees Algorithm for Large-Scale Earth Observation Satellite Scheduling. In: Pan, L., Zhao, D., Li, L., Lin, J. (eds) Bio-Inspired Computing: Theories and Applications. BIC-TA 2022. Communications in Computer and Information Science, vol 1801. Springer, Singapore. https://doi.org/10.1007/978-981-99-1549-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-1549-1_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-1548-4

  • Online ISBN: 978-981-99-1549-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics