Skip to main content
Springer Nature Link
Log in

Solving the Job Shop Scheduling Problem with Reinforcement Learning: A Statistical Analysis

  • Conference paper
  • First Online:
Intelligent Systems Design and Applications (ISDA 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1351))

  • 2454 Accesses

Abstract

This paper focuses on the application of modern reinforcement learning techniques to solve the job shop scheduling problem where the objective is to minimize the makespan. It starts by discussing the proposed system, which is based on an original architecture that proposes a novel approach based on the cooperation of different components, from the problem that needs to be solved to the job shop training environment or the reinforcement learning algorithm. Then, a methodical computational study is presented. The computational study is divided into two phases of performance analysis, namely of efficiency and effectiveness. In the efficiency analysis the execution times until a solution is obtained by the studied approaches are compared. In the efficiency analysis the quality of the solutions proposed by the chosen methods of optimization are evaluated. Both phases included a meticulous statistical inference analysis to validate the obtained results. A detailed discussion of the collected results is also included in this paper. Based on the statistical evidence that was found, it culminates in the conclusion that the proposed approach examined and evaluated in this paper is most likely one of the most efficient to date to solve job shop scheduling problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Cunha, B., Madureira, A., Fonseca, B.: Reinforcement Learning Environment for Job Shop Scheduling Problems. Int. J. Comput. Inf. Syst. Ind. Manag. Appl. 12, 231–238 (2020)

    Google Scholar 

  2. Cunha, B., Madureira, A. M., Fonseca, B., Coelho, D.: Deep reinforcement learning as a job shop scheduling solver: a literature review. In: Madureira, A. M., Abraham, A., Gandhi, N., Varela, M. L. (eds.) Hybrid Intelligent Systems: 18th International Conference on Hybrid Intelligent Systems (HIS 2018) Held in Porto, Portugal, December 13-15, 2018, pp. 350–359. Springer, Cham (2020)

    Chapter  Google Scholar 

  3. Cook, S.A.: The complexity of theorem-proving procedures. In: Proceedings of the third annual ACM symposium on Theory of computing, pp. 151–158 (1971)

    Google Scholar 

  4. Cunha, B., Madureira, A., Pereira, J.P., Pereira, I.: Evaluating the effectiveness of bayesian and neural networks for adaptive schedulling systems. In: 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016, pp. 1–6 (2017)

    Google Scholar 

  5. Ku, W.-Y., Beck, J.C.: Mixed integer programming models for job shop scheduling: a computational analysis. Comput. Oper. Res. 73, 165–173 (2016)

    Article  MathSciNet  Google Scholar 

  6. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, Second. The MIT Press, New York (2018)

    Google Scholar 

  7. Pinedo, M.L.: Scheduling: Theory, Algorithms, and Systems, 5th ed. Springer, Cham (2016)

    Google Scholar 

  8. Zhang, C.Y., Li, P., Rao, Y., Guan, Z.: A very fast TS/SA algorithm for the job shop scheduling problem. Comput. Oper. Res. 35(1), 282–294 (2008)

    Article  MathSciNet  Google Scholar 

  9. Peng, B., Lü, Z., Cheng, T.C.E.: A tabu search/path relinking algorithm to solve the job shop scheduling problem. Comput. Oper. Res. 53, 154–164 (2015)

    Article  MathSciNet  Google Scholar 

  10. Taillard, E.: Benchmarks for basic scheduling problems. Eur. J. Oper. Res. 64(2), 278–285 (1993)

    Article  MathSciNet  Google Scholar 

  11. Pardalos, P.M., Shylo, O.V.: An algorithm for the job shop scheduling problem based on global equilibrium search techniques. Comput. Manag. Sci. 3(4), 331–348 (2006)

    Article  MathSciNet  Google Scholar 

  12. Pardalos, P.M., Shylo, O.V., Vazacopoulos, A.: Solving job shop scheduling problems utilizing the properties of backbone and big valley. Comput. Optim. Appl. 47, 61–76 (2010)

    Google Scholar 

  13. van Hoorn, J.J.: The Current state of bounds on benchmark instances of the job-shop scheduling problem. J. Sched. 21(1), 127–128 (2018)

    Article  MathSciNet  Google Scholar 

  14. Vilím, P., Laborie, P., Shaw, P.: Failure-Directed Search for Constraint-Based Scheduling. In: CPAIOR (2015)

    Google Scholar 

  15. Siala, M., Artigues, C., Hebrard, E.: Two clause learning approaches for disjunctive scheduling. In: Pesant, G. (ed.) Principles and Practice of Constraint Programming: 21st International Conference, CP 2015, Cork, Ireland, August 31 -- September 4, 2015, Proceedings, pp. 393–402. Springer, Cham (2015)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Interdisciplinary Studies Research Center, Institute of Engineering - Polytechnic of Porto (ISEP/IPP), Porto, Portugal

    Bruno Cunha & Ana Madureira

  2. INESC TEC, University of Trás-Os-Montes and Alto Douro (UTAD), Vila Real, Portugal

    Benjamim Fonseca

Authors
  1. Bruno Cunha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Madureira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamim Fonseca
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruno Cunha .

Editor information

Editors and Affiliations

  1. Scientific Network for Innovation and Research Excellence, Machine Intelligence Research Labs (MIR Labs), Auburn, WA, USA

    Ajith Abraham

  2. Department of Computer Science, Università degli Studi di Milano, Milan, Milano, Italy

    Vincenzo Piuri

  3. Machine Intelligence Research Labs (MIR Labs), Auburn, WA, USA

    Niketa Gandhi

  4. Campus Centre de Créteil, Université Paris-Est Créteil, Créteil, France

    Patrick Siarry

  5. Department of Construction Management and Real Estate, Vilnius Gediminas Technical University, Vilnius, Lithuania

    Arturas Kaklauskas

  6. School of Engineering, Instituto Superior de Engenharia do Porto, Porto, Portugal

    Ana Madureira

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cunha, B., Madureira, A., Fonseca, B. (2021). Solving the Job Shop Scheduling Problem with Reinforcement Learning: A Statistical Analysis. In: Abraham, A., Piuri, V., Gandhi, N., Siarry, P., Kaklauskas, A., Madureira, A. (eds) Intelligent Systems Design and Applications. ISDA 2020. Advances in Intelligent Systems and Computing, vol 1351. Springer, Cham. https://doi.org/10.1007/978-3-030-71187-0_55

Download citation

Publish with us

Policies and ethics