Abstract
This paper addresses the integrated process planning and job-shop scheduling problem for mass customization in a reconfigurable manufacturing system. A bi-objective mixed-integer non-linear programming mathematical model for minimizing the total tardiness penalty of products and the total cost covering setup, machine reconfiguration as well as processing activities is built to formulate the problem. A Q-learning based reinforcement learning solution approach is presented to solve the formulated problem. Numerical experiments were carried out to validate the mathematical model and the solution approach. The computational results of the numerical examples show the great efficiency of the proposed solution approach in the aspect of computation time, compared with NSGA-II and the exhaustive search. The effectiveness of the problem-specific designed policies is also discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aheleroff, S., Philip, R., Zhong, R.Y., Xu, X.: The degree of mass personalisation under industry 4.0. Procedia CIRP 81, 1394–1399 (2019)
Leng, J., et al.: Digital twin-driven rapid reconfiguration of the automated manufacturing system via an open architecture model. Robot. Comput. Integr. Manuf. 63 (2020).https://doi.org/10.1016/j.rcim.2019.101895
Morgan, J., Halton, M., Qiao, Y., Breslin, J.G.: Industry 4.0 smart reconfigurable manufacturing machines. J. Manuf. Syst. 59, 481–506 (2021). https://doi.org/10.1016/j.jmsy.2021.03.001
Koren, Y., Gu, X., Guo, W.: Reconfigurable manufacturing systems: principles, design, and future trends. Front. Mech. Eng. 13(2), 121–136 (2017). https://doi.org/10.1007/s11465-018-0483-0
Bruccoleri, M., Nigro, G.L., Perrone, G., Renna, P., Diega, S.N.L.: Production planning in reconfigurable enterprises and reconfigurable production systems. CIRP Ann. 54(1), 433–436 (2005). https://doi.org/10.1016/S0007-8506(07)60138-3
Brahimi, N., Dolgui, A., Gurevsky, E., Yelles-Chaouche, A.R.: A literature review of optimization problems for reconfigurable manufacturing systems. IFAC-PapersOnLine 52, 433–438 (2019). https://doi.org/10.1016/j.ifacol.2019.11.097
Usuga Cadavid, J.P., Lamouri, S., Grabot, B., Pellerin, R., Fortin, A.: Machine learning applied in production planning and control: a state-of-the-art in the era of industry 4.0. J. Intell. Manuf. 31(6), 1531–1558 (2020). https://doi.org/10.1007/s10845-019-01531-7
Nassehi, A., Zhong, R.Y., Li, X., Epureanu, B.I.: Review of machine learning technologies and artificial intelligence in modern manufacturing systems. In: Design and Operation of Production Networks for Mass Personalization in the Era of Cloud Technology. pp. 317–348. Elsevier Inc. (2022)
Kardos, C., Laflamme, C., Gallina, V., Sihn, W.: Dynamic scheduling in a job-shop production system with reinforcement learning. Procedia CIRP 97, 104–109 (2021)
Tang, J., Salonitis, K.: A deep reinforcement learning based scheduling policy for reconfigurable manufacturing systems. Procedia CIRP 103, 1–7 (2021)
Tang, J., Haddad, Y., Salonitis, K.: Reconfigurable manufacturing system scheduling: a deep reinforcement learning approach. Procedia CIRP 107, 1198–1203 (2022). https://doi.org/10.1016/j.procir.2022.05.131
Yang, S., Zhigang, Xu.: Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing. Int. J. Prod. Res. 60(16), 4936–4953 (2021). https://doi.org/10.1080/00207543.2021.1943037
Khan, A.S., Homri, L., Dantan, J.Y., Siadat, A.: An analysis of the theoretical and implementation aspects of process planning in a reconfigurable manufacturing system. Int. J. Adv. Manuf. Technol. 119(9–10), 5615–5646 (2021). https://doi.org/10.1007/s00170-021-08522-0
He, Y., Xing, L., Chen, Y., Pedrycz, W., Wang, L., Wu, G.: A generic Markov decision process model and reinforcement learning method for scheduling agile earth observation satellites. IEEE Trans. Syst. Man Cybern. Syst. 52, 1463–1474 (2022). https://doi.org/10.1109/TSMC.2020.3020732
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992). https://doi.org/10.1007/BF00992698
Clifton, J., Laber, E.: Q-learning: theory and applications. Annu. Rev. Stat. Appl. 7, 279–301 (2020). https://doi.org/10.1146/annurev-statistics-031219-041220
Azab, A., ElMaraghy, H., Nyhuis, P., Pachow-Frauenhofer, J., Schmidt, M.: Mechanics of change: a framework to reconfigure manufacturing systems. CIRP J. Manuf. Sci. Technol. 6, 110–119 (2013). https://doi.org/10.1016/j.cirpj.2012.12.002
Morariu, C., Morariu, O., Răileanu, S., Borangiu, T.: Machine learning for predictive scheduling and resource allocation in large scale manufacturing systems. Comput. Ind. 120, 103244 (2020). https://doi.org/10.1016/j.compind.2020.103244
Gao, S., Daaboul, J., Le Duigou, J.: Process planning, scheduling, and layout optimization for multi-unit mass-customized products in sustainable reconfigurable manufacturing system. Sustainability 13, 13323 (2021). https://doi.org/10.3390/su132313323
Acknowledgments
The authors are grateful for the financial support (No. 201806280501) provided by China Scholarship Council (CSC).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, S., Daaboul, J., Le Duigou, J. (2023). A Reinforcement Learning Approach for Solving Integrated Mass Customization Process Planning and Job-Shop Scheduling Problem in a Reconfigurable Manufacturing System. In: Borangiu, T., Trentesaux, D., Leitão, P. (eds) Service Oriented, Holonic and Multi-Agent Manufacturing Systems for Industry of the Future. SOHOMA 2022. Studies in Computational Intelligence, vol 1083. Springer, Cham. https://doi.org/10.1007/978-3-031-24291-5_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-24291-5_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24290-8
Online ISBN: 978-3-031-24291-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)