Abstract
Assembly lines are at the core of many manufacturing systems, and planning for a well-balanced flow is key to ensure long-term efficiency. However, in flexible configurations such as Multi-Manned Assembly Lines (MMAL), the balancing problem also becomes more challenging. Due to the increased relevance of these assembly lines, this work aims to investigate the MMAL balancing problem, to contribute for a more effective decision-making process. Therefore, a new approach is proposed based on Deep Reinforcement Learning (DRL) embedded in a Digital Twin architecture. The proposed approach provides a close-to-reality training environment for the agent, using Discrete Event Simulation to simulate the production system dynamics. This methodology was tested on a real-world instance with preliminary results showing that similar solutions to the ones obtained using optimization-based strategies are achieved. This research provides evidence of success in terms of dynamic resource assignment to tasks and workers as a basis for future developments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Scholl, A., Becker, C.: State-of-the-art exact and heuristic solution procedures for simple assembly line balancing. Eur. J. Oper. Res. 168(3), 666–693 (2006)
Battaïa, O., Dolgui, A.: A taxonomy of line balancing problems and their solution approaches. Int. J. Prod. Econ. 142(2), 259–277 (2013)
Jiao, Y., Jin, H., Xing, X., Li, M., Liu, X.: Assembly line balance research methods, literature and development review. Concurr. Eng. 29(2), 183–194 (2021)
Dimitriadis, S.G.: Assembly line balancing and group working: a heuristic procedure for workers’ groups operating on the same product and workstation. Comput. Oper. Res. 33(9), 2757–2774 (2006)
Mnih, V., Silver, D., Riedmiller, M.: Playing Atari with Deep Reinforcement Learning. arXiv Prepr. arXiv1312.5602, pp. 1–9 (2013)
Silver, D., et al.: Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. no. arXiv:1712.01815. arXiv (2017)
Gomez, D., Quijano, N., Giraldo, L.F.: Learning Transferable Concepts in Deep Reinforcement Learning. no. arXiv:2005.07870. arXiv (2020)
Speck, D., Biedenkapp, A., Hutter, F., Mattmüller, R., Lindauer, M.: Learning heuristic selection with dynamic algorithm configuration. Proc. Int. Conf. Autom. Plann. Schedul. 31, 597–605 (2021)
Priore, P., Gómez, A., Pino, R., Rosillo, R.: Dynamic scheduling of manufacturing systems using machine learning: an updated review. Artif. Intell. Eng. Des. Anal. Manuf. 28(01), 83–97 (2014)
Cunha, B., Madureira, A.M., Fonseca, B., Coelho, D.: Deep reinforcement learning as a job shop scheduling solver: a literature review. In: Madureira, A.M., Abraham, A., Gandhi, N., Varela, M.L. (eds.) HIS 2018. AISC, vol. 923, pp. 350–359. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-14347-3_34
Lv, Y., Tan, Y., Zhong, R., Zhang, P., Wang, J., Zhang, J.: Deep reinforcement learning‐based balancing and sequencing approach for mixed model assembly lines. IET Collab. Intel. Manufact. 4(3), 181–193 (2022)
Papacharalampopoulos, A., Stavropoulos, P.: Manufacturing process optimization via digital twins: definitions and limitations. In: Kim, K.-Y., Monplaisir, L., Rickli, J. (eds.) Flexible Automation and Intelligent Manufacturing: The Human-Data-Technology Nexus: Proceedings of FAIM 2022, June 19–23, 2022, Detroit, Michigan, USA, pp. 342–350. Springer International Publishing, Cham (2023). https://doi.org/10.1007/978-3-031-18326-3_33
Arrais, R., et al.: Application of the open scalable production system to machine tending of additive manufacturing operations by a mobile manipulator. In: Moura Oliveira, P., Novais, P., Reis, L.P. (eds.) EPIA 2019. LNCS (LNAI), vol. 11805, pp. 345–356. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30244-3_29
Brockman, G., et al.: ‘Openai gym’, arXiv Prepr. arXiv1606.01540 (2016)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms
Şahin, M., Kellegöz, T.: Balancing multi-manned assembly lines with walking workers: problem definition, mathematical formulation, and an electromagnetic field optimisation algorithm. Int. J. Prod. Res. 57(20), 6487–6505 (2019)
Acknowledgments
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 952003.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Santos, R., Marques, C., Toscano, C., Ferreira, H.M., Ribeiro, J. (2024). Deep Reinforcement Learning-Based Approach to Dynamically Balance Multi-manned Assembly Lines. In: Silva, F.J.G., Pereira, A.B., Campilho, R.D.S.G. (eds) Flexible Automation and Intelligent Manufacturing: Establishing Bridges for More Sustainable Manufacturing Systems. FAIM 2023. Lecture Notes in Mechanical Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-38241-3_71
Download citation
DOI: https://doi.org/10.1007/978-3-031-38241-3_71
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-38240-6
Online ISBN: 978-3-031-38241-3
eBook Packages: EngineeringEngineering (R0)