Abstract
The manufacturing of large components is, in comparison to small components, cost intensive. This is due to the sheer size of the components and the limited scalability in number of produced items. To take advantage of the effects of small component production we segment the large components into smaller parts and schedule the production of these parts on regular-sized machine tools. We propose to apply and adapt recent developments in reinforcement learning in combination with heuristics to efficiently solve the resulting segmentation and assignment problem. In particular, we solve the assignment problem up to a factor of 8 faster and only a few percentages less accurate than a classic solver from operations research.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Afteni, C., Frumuşanu, G.: A review on optimization of manufacturing process performance. Int. J. Model. Optim. 7(3), 139–144 (2017). https://doi.org/10.7763/IJMO.2017.V7.573
Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34(6), 26–38 (2017). https://doi.org/10.1109/MSP.2017.2743240
Bambach, P., et al.: LEAP - legged exploration of the aristarchus plateau. Europlanet Sci. Congr. (2022). https://doi.org/10.5194/epsc2022-856
Bengio, Y., Lodi, A., Prouvost, A.: Machine learning for combinatorial optimization: a methodological tour d’horizon. Eur. J. Oper. Res. 290(2), 405–421 (2021). https://doi.org/10.1016/j.ejor.2020.07.063
Cattrysse, D.G., Van Wassenhove, L.N.: A survey of algorithms for the generalized assignment problem. Eur. J. Oper. Res. 60(3), 260–272 (1992). https://doi.org/10.1016/0377-2217(92)90077-M
Chen, X., Tian, Y.: Learning to perform local rewriting for combinatorial optimization. Adv. Neural. Inf. Process. Syst. 32, 6281–6292 (2019)
Deza, A., Khalil, E.B.: Machine learning for cutting planes in integer programming: a survey. arXiv preprint arXiv:2302.09166 (2023). https://doi.org/10.48550/arXiv.2302.09166
Dietterich, T.G., et al.: The MAXQ method for hierarchical reinforcement learning. In: ICML. vol. 98, pp. 118–126 (1998)
Hamzehi, S., Bogenberger, K., Franeck, P., Kaltenhäuser, B.: Combinatorial reinforcement learning of linear assignment problems. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3314–3321. IEEE (2019). https://doi.org/10.1109/ITSC.2019.8916920
Jagdev, H., Browne, J., Keogh, J.: Manufacturing process optimisation—a survey of techniques. In: Proceedings of the Twenty-eighth International: Matador Conference, pp. 205–215. Springer (1990). https://doi.org/10.1007/978-1-349-10890-9_29
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996). https://doi.org/10.1613/jair.301
Khalil, E.B.: Machine learning for integer programming. In: IJCAI, pp. 4004–4005 (2016)
Kool, W., van Hoof, H., Welling, M.: Attention, learn to solve routing problems! In: International Conference on Learning Representations (2019). https://doi.org/10.48550/arXiv.1803.08475
Kwon, Y.D., Choo, J., Kim, B., Yoon, I., Gwon, Y., Min, S.: POMO: policy optimization with multiple optima for reinforcement learning. Adv. Neural. Inf. Process. Syst. 33, 21188–21198 (2020)
Lee, M., Xiong, Y., Yu, G., Li, G.Y.: Deep neural networks for linear sum assignment problems. IEEE Wireless Commun. Lett. 7(6), 962–965 (2018). https://doi.org/10.1109/LWC.2018.2843359
Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J.E., Stoica, I.: Tune: A research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118 (2018). https://doi.org/10.48550/arXiv.1807.05118
Lin, X., Hou, Z.J., Ren, H., Pan, F.: Approximate mixed-integer programming solution with machine learning technique and linear programming relaxation. In: 2019 3rd International Conference on Smart Grid and Smart Cities (ICSGSC), pp. 101–107. IEEE (2019). https://doi.org/10.1109/ICSGSC.2019.00-11
Mazyavkina, N., Sviridov, S., Ivanov, S., Burnaev, E.: Reinforcement learning for combinatorial optimization: a survey. Comput. Oper. Res. 134, 105400 (2021). https://doi.org/10.1016/j.cor.2021.105400
Mirhoseini, A., et al.: A graph placement methodology for fast chip design. Nature 594(7862), 207–212 (2021). https://doi.org/10.1038/s41586-022-04657-6
Nazari, M., Oroojlooy, A., Snyder, L., Takác, M.: Reinforcement learning for solving the vehicle routing problem. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Ouyang, L., et al.: Training language models to follow instructions with human feedback. Adv. Neural. Inf. Process. Syst. 35, 27730–27744 (2022)
Pan, Z., Wang, L., Wang, J., Lu, J.: Deep reinforcement learning based optimization algorithm for permutation flow-shop scheduling. IEEE Trans. Emerging Top. Comput. Intell. (2021). https://doi.org/10.1109/TETCI.2021.3098354
Paul, N., Wirtz, T., Wrobel, S., Kister, A.: Multi-agent neural rewriter for vehicle routing with limited disclosure of costs. In: Presented at the Gamification and Multiagent Solutions Workshop within Tenth International Conference on Learning Representations, ICLR (2022). https://doi.org/10.48550/arXiv.2206.05990
Refaei Afshar, R., Zhang, Y., Firat, M., Kaymak, U.: A state aggregation approach for solving knapsack problem with deep reinforcement learning. In: Pan, S.J., Sugiyama, M. (eds.) Proceedings of The 12th Asian Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 129, pp. 81–96 (2020)
Schnellhardt, T., Hemschik, R., Weiß, A., Schoesau, R., Hellmich, A., Ihlenfeldt, S.: Scalable production of large components by industrial robots and machine tools through segmentation. Front. Robot. AI 9 (2022). https://doi.org/10.3389/frobt.2022.1021755
Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550(7676), 354–359 (2017). https://doi.org/10.1038/nature24270
Uriarte, L., et al.: Machine tools for large parts. CIRP Ann. 62(2), 731–750 (2013). https://doi.org/10.1016/j.cirp.2013.05.009
Volk, A.A., et al.: AlphaFlow: autonomous discovery and optimization of multi-step chemistry using a self-driven fluidic lab guided by reinforcement learning. Nat. Commun. 14(1), 1403 (2023). https://doi.org/10.1038/s41467-023-37139-y
Weichert, D., Link, P., Stoll, A., Rüping, S., Ihlenfeldt, S., Wrobel, S.: A review of machine learning for the optimization of production processes. Int. J. Adv. Manuf. Technol. 1889–1902 (2019). https://doi.org/10.1007/s00170-019-03988-5
Acknowledgements
The research of N. Paul, T. Schnellhardt and M. Fetz was funded by the Fraunhofer lighthouse project “SWAP - Hierarchical swarms as production architecture with optimized utilization”. D. Hecker contributed as part of the Fraunhofer Center for Machine Learning within the Fraunhofer Cluster for Cognitive Internet Technologies. The work of T. Wirtz was supported by the Federal Ministry of Education and Research of Germany and the state of North-Rhine Westphalia as part of the Lamarr-Institute for Machine Learning and Artificial Intelligence. We would also like to thank the reviewers for their valuable feedback which improved the presentation of our work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Paul, N., Kister, A., Schnellhardt, T., Fetz, M., Hecker, D., Wirtz, T. (2025). Reinforcement Learning for Segmented Manufacturing. In: Meo, R., Silvestri, F. (eds) Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2023. Communications in Computer and Information Science, vol 2136. Springer, Cham. https://doi.org/10.1007/978-3-031-74640-6_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-74640-6_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-74639-0
Online ISBN: 978-3-031-74640-6
eBook Packages: Artificial Intelligence (R0)