Abstract
The complexity and dynamism of cyber-physical production systems (CPPS) pose significant coordination challenges. Traditional mechanisms are often insufficient to optimize these complex systems, due to rigid decision-making processes and limited human capacity. This paper presents a new approach to enhance the coordination of CPPS, based on reinforcement learning (RL). Faced with the challenges posed by the complexity and dynamics of modern production environments, this approach aims at increasing the performance of CPPS systems by overcoming the traditional limits of the coordinator. We demonstrate the practical implementation of this methodology through a simulated aluminium rim production line consisting of two workstations, using specialized simulation software. The goal is to demonstrate that the proposed approach can significantly improve the performance of CPPS systems by reducing the limitations faced by the coordinator. Finally, we analyse and compare the results of the approach that integrates the RL agent with those of a configuration without RL agent. This analysis allows to evaluate the effectiveness of the proposed solution, offering a new perspective on reducing the coordinator’s limitations in managing CPPS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Antons, O.: Data-driven and autonomous manufacturing control in cyber-physical production systems. Comput. Ind. 9 (2022)
Ansari, F., Hold, P., Sihn, W.: Human-centered cyber physical production system: how does industry 4.0 impact on decision-making tasks? In: 2018 IEEE Technology and Engineering Management Conference (TEMSCON), Evanston, IL, USA. IEEE (2018)
Dallel, M., Havard, V., Dupuis, Y., Baudry, D.: Digital twin of an industrial workstation: a novel method of an auto-labeled data generator using virtual reality for human action recognition in the context of human–robot collaboration. Eng. Appl. Artif. Intell. 118 (2023)
Ante, G., Facchini, F., Mossa, G., Digiesi, S.: Developing a key performance indicators tree for lean and smart production systems. IFAC-Papers 51(11), 13–18 (2018)
Cui, X.: Cyber-Physical System (CPS) architecture for real-time water sustainability management in manufacturing industry. Procedia CIRP 99, 543–548 (2021)
Cardin, O.: Contribution à la conception, l’évaluation et l’implémentation de systèmes de production cyber-physiques. Ph.D. thesis, Université de Nantes, fftel-01443318 (2017)
Amirshirzad, N., Kumru, A., Oztop, E.: Human adaptation to human-robot shared control. IEEE Trans. Hum.-Mach. Syst. 49(2), 126–136 (2019)
Wang, S., Wan, J., Zhang, D., Li, D., Zhang, C.: Towards smart factory for industry 4.0: a self-organized multi-agent system with big data based feedback and coordination. Comput. Netw. 101, 158–168 (2016)
Ouazzani-Chahidi, A., Jimenez, J.F., Berrah, L., Loukili, A.: Classification of coordinators’ limitations in cyber-physical production system management. In: Motahhir, S., Bossoufi, B. (eds.) ICDTA 2023. LNNS, vol. 669, pp. 200–209. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-29860-8_21
Ansari, F., Khobreh, M., Seidenberg, U., Sihn, W.: A problem-solving ontology for human-centered cyber physical production systems. CIRP J. Manuf. Sci. Technol. 22 (2018)
Ansari, F., Seidenberg, U.: A portfolio for optimal collaboration of human and cyber physical production systems in problem-solving. In: Proceedings of International Conference, CELDA 2016 (2016)
Jiao, J., Zhou, F., Gebraeel, N.Z., Duffy, V.: Towards augmenting cyber-physical-human collaborative cognition for human-automation interaction in complex manufacturing and operational environments. Int. J. Prod. Res. 58(16), 5089–5111 (2020)
Pacaux-Lemoine, M.-P., Habib, L., Berdal, Q., Trentesaux, D.: Cooperative patterns or how to support human-cyber-physical systems cooperation. In: International Conference, SMC 2021, p. 1501 (2021)
Bakakeu, J., et al.: An artificial intelligence approach for online optimization of flexible manufacturing systems. Appl. Mech. Mater. 882, 96–108 (2018)
Cardin, O., Trentesaux, D., Thomas, A., Castagna, P., Berger, T., Bril El-Haouzi, H.: Coupling predictive scheduling and reactive control in manufacturing hybrid control architectures: state of the art and future challenges. J. Intell. Manuf. 28(7) (2015)
Antons, O., Arlinghaus, J.C.: Designing decision-making authorities for smart factories. Procedia CIRP 93, 316–322 (2020)
Bhatia, M., Ahanger, T.A.: Intelligent Decision-making in Smart Food Industry: Quality Perspective. Pervasive Mob. Comput. 72, 101304 (2021)
Müller, R., Narciss, S., Urbas, L.: Interfacing Cyber-Physical Production Systems with Human Decision Makers, Intelligent Data-Centric Systems. Academic Press, Boston (2017)
Panzer, M., Bender, B.: Deep reinforcement learning in production systems: a systematic literature review. Int. J. Prod. Res. 60(13), 4316–4341 (2022)
Bakakeu, J., Bär, S., Klos, H.-H., Peschke, J., Brossog, M., Franke, J.: Multi-agent reinforcement learning for the energy optimization of cyber-physical production systems (2020)
Kuhnle, A., Röhrig, N., Lanza, G.: Autonomous order dispatching in the semiconductor industry using reinforcement learning. Procedia CIRP 79, 391–396 (2019)
Chen, S., Wang, J., Li, H., Wang, Z., Liu, F., Li, S.: Top-down human-cyber-physical data fusion based on reinforcement learning. IEEE Access 8, 134233–134245 (2020)
Zhang, Z., Liu, C., Zhang, J., Peng, T., Hu, X., Xu, Y.: Machine learning applications in cyber-physical production systems: a survey. In: 27th International Conference on Automation and Computing (ICAC), pp. 1–6 (2022)
Bignold, A., Vamplew, P., Dazeley, R., Foale, C.: Rule-based interactive assisted reinforcement learning. Ph.D. Thesis, Federation. University Australia (2019). https://doi.org/10.13140/RG.2.2.27422.48962
Dittrich, M.-A., Fohlmeister, S.: Cooperative multi-agent system for production control using reinforcement learning. CIRP Ann. 69(1), 389–392 (2020)
Liu, X., Xu, H., Liao, W., Yu, W.: Reinforcement learning for cyber-physical systems. In: 2019 IEEE International Conference on Industrial Internet (ICII), Orlando, FL, USA, p. 318–327. IEEE (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ouazzani-Chahidi, A., Jimenez, JF., Berrah, L., Loukili, A. (2024). Integration of Reinforcement Learning Agent to Reduce Coordinator Limitations in Cyber-Physical Production Systems. In: Borangiu, T., Trentesaux, D., Leitão, P., Berrah, L., Jimenez, JF. (eds) Service Oriented, Holonic and Multi-Agent Manufacturing Systems for Industry of the Future. SOHOMA 2023. Studies in Computational Intelligence, vol 1136. Springer, Cham. https://doi.org/10.1007/978-3-031-53445-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-53445-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-53444-7
Online ISBN: 978-3-031-53445-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)