Skip to main content
Springer Nature Link
Log in

A Hyper-Heuristic Algorithm with Q-Learning for Distributed Flow Shop-Vehicle Transport-U-Assembly Integrated Scheduling Problem

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14862))

Included in the following conference series:

  • 696 Accesses

Abstract

Distributed Flow Shop-Vehicle Transportation-U-Assembly (DPFS_VT_UA) Integrated Scheduling Problem is a challenging combinatorial optimization problem with many practical applications. In this paper, we propose a Hyper-Heuristic Algorithm with Q-Learning (HHQL) to solve the DPFS_VT_UA integrated scheduling problem. The method combines the advantages of q -learning and hyper-heuristic frameworks. Firstly, according to the characteristics of DPFS_VT_UA, a mixed-integer linear programming (MILP) model of DPFS_VT_UA is established, and an encoding and decoding scheme is designed; secondly, in order to achieve a more in-depth exploration of the solution space for the DPFS_VT_UA problem, the 9 low-level heuristic operators (i.e., 9 effective neighborhood operators) are designed. These low-level heuristics can effectively explore the search space and improve the quality of the solution. The Q-learning mechanism is applied as a high-level strategy to manipulate the Low-Level Heuristics (LLHs), which are then executed in order to search the solution space. Experimental results and statistical analysis show that HHQL significantly outperforms the existing algorithms by a significant margin, demonstrating the effectiveness and efficiency of HHQL in solving DFS_VT_UA integrated scheduling problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Framinan, J.M., Gupta, J.N.D., Leisten, R.: A review and classification of heuristics for permutation flow-shop scheduling with makespan objective. J. Oper. Res. Soc. 55, 1243–1255 (2004)

    Article  Google Scholar 

  2. Qian, B., Wang, L., Hu, R., Wang, W.-L., Huang, D.-X., Wang, X.: A hybrid differential evolution method for permutation flow-shop scheduling. Int. J. Adv. Manufact. Technol. 38, 757–777 (2008)

    Article  Google Scholar 

  3. Gonzalez, T., Sahni, S.: Flowshop and Jobshop Schedules: complexity and Approximation. Oper. Res. 26, 36–52 (1978)

    Article  MathSciNet  Google Scholar 

  4. Naderi, B., Ruiz, R.: The distributed permutation flowshop scheduling problem. Comput. Oper. Res. 37, 754–768 (2010)

    Article  MathSciNet  Google Scholar 

  5. Pan, Q.K., Gao, L., Li, X.Y., Jose, F.M.: Effective constructive heuristics and meta-heuristics for the distributed assembly permutation flowshop scheduling problem. Appl. Soft Comput. 81, 105492 (2019)

    Article  Google Scholar 

  6. Zhang, Z.-Q., Qian, B., Hu, R., Jin, H.-P., Wang, L.: A matrix-cube-based estimation of distribution algorithm for the distributed assembly permutation flow-shop scheduling problem. Swarm Evol. Comput. 60, 100785 (2021)

    Article  Google Scholar 

  7. Marandi, F., Fatemi Ghomi, S.M.T.: Integrated multi-factory production and distribution scheduling applying vehicle routing approach. Int. J. Prod. Res. 57, 722–748 (2019)

    Article  Google Scholar 

  8. Luo, W., Qiao, Y., Lin, X., Xu, P., Preuss, M.: Hybridizing niching, particle swarm optimization, and evolution strategy for multimodal optimization. IEEE Trans. Cybern. 52, 6707–6720 (2022)

    Article  Google Scholar 

  9. Zhang, F., Mei, Y., Nguyen, S., Zhang, M.: Evolving scheduling heuristics via genetic programming with feature selection in dynamic flexible job-shop scheduling. IEEE Trans. Cybern. 51, 1797–1811 (2021)

    Article  Google Scholar 

  10. Gao, K., Yang, F., Zhou, M., Pan, Q., Suganthan, P.N.: Flexible job-shop rescheduling for new job insertion by using discrete Jaya algorithm. IEEE Trans. Cybern. 49, 1944–1955 (2019)

    Article  Google Scholar 

  11. Dong, B., Jiao, L., Wu, J.: A two-phase knowledge based hyper-heuristic scheduling algorithm in cellular system. Knowl.-Based Syst. 88, 244–252 (2015)

    Article  Google Scholar 

  12. Ashraf, A., Qingjie, Z., Bangyal, W.H.K., Iqbal, M.: Analysis of brain imaging data for the detection of early age autism spectrum disorder using transfer learning approaches for internet of things. IEEE Trans. Consum. Electron. 70, 4478–4489 (2024)

    Article  Google Scholar 

  13. Bangyal, W.H., Shakir, R., Rehman, N.U., Ashraf, A., Ahmad, J.: An improved seagull algorithm for numerical optimization problem. In: Tan, Y., Shi, Y., Luo, W. (eds.) Advances in Swarm Intelligence. ICSI 2023. LNCS, vol. 13968. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-36622-2_24

  14. Khalid, M., Ashraf, A., Bangyal, W.H., Iqbal, M.: An android application for unwanted vehicle detection and counting. In: 2023 International Conference on Human-Centered Cognitive Systems (HCCS); 16–17 Dec. 2023, pp. 1–6 (2023)

    Google Scholar 

  15. Choong, S.S., Wong, L.-P., Lim, C.P.: Automatic design of hyper-heuristic based on reinforcement learning. Inf. Sci. 436–437, 89–107 (2018)

    Article  MathSciNet  Google Scholar 

  16. Gölcük, İ, Ozsoydan, F.B.: Q-learning and hyper-heuristic based algorithm recommendation for changing environments. Eng. Appl. Artif. Intell. 102, 104284 (2021)

    Article  Google Scholar 

  17. Lin, J., Li, Y.-Y., Song, H.-B.: Semiconductor final testing scheduling using Q-learning based hyper-heuristic. Expert Syst. Appl. 187, 115978 (2022)

    Article  Google Scholar 

  18. Qian, B., She, M.Z., Hu, R., Guo, N., Xiang, F.H.: Hyper-heuristic cross-entropy algorithm for green distributed permutation flow-shop scheduling problem with fuzzy processing time. Control and Decision: 1–10 [2020–06–10.]. https://doi.org/10.13195/j.kzyjc. 2019. 1681

  19. Qian, B., Wang, L., Huang, D.X., Wang, W.L., Wang, X.: An effective hybrid DE-based algorithm for multi-objective flow shop scheduling with limited buffers. Comput. Oper. Res. 36(1), 209–233 (2009)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This research was supported by the National Natural Science Foundation of China (62173169 and 72201115) and the Basic Research Key Project of Yunnan Province (202201AS070030).

Author information

Authors and Affiliations

  1. School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, 650500, China

    Dong-Lin Yang, Bin Qian, Zi-Qi Zhang, Rong Hu & Kun Li

Authors
  1. Dong-Lin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zi-Qi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Qian .

Editor information

Editors and Affiliations

  1. Eastern Institute of Technology, Ningbo, China

    De-Shuang Huang

  2. Tianjin University of Science and Technology, Tianjin, China

    Xiankun Zhang

  3. China University of Mining and Technology, Xuzhou, China

    Wei Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yang, DL., Qian, B., Zhang, ZQ., Hu, R., Li, K. (2024). A Hyper-Heuristic Algorithm with Q-Learning for Distributed Flow Shop-Vehicle Transport-U-Assembly Integrated Scheduling Problem. In: Huang, DS., Zhang, X., Chen, W. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14862. Springer, Singapore. https://doi.org/10.1007/978-981-97-5578-3_24

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5578-3_24

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5577-6

  • Online ISBN: 978-981-97-5578-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics