Abstract
Dynamic task assignment concerns the optimal assignment of resources to tasks in a business process. Recently, Deep Reinforcement Learning (DRL) has been proposed as the state of the art for solving assignment problems. DRL methods usually employ a neural network (NN) as an approximator for the policy function, which ingests the state of the process and outputs a valuation of the possible assignments. However, representing the state and the possible assignments so that they can serve as inputs and outputs for a policy NN remains an open challenge, especially when tasks or resources have features with an infinite number of possible values. To solve this problem, this paper proposes a method for representing and solving assignment problems with infinite state and action spaces. In doing so, it provides three contributions: (I) A graph-based feature representation of assignment problems, which we call assignment graph; (II) A mapping from marked Colored Petri Nets to assignment graphs; (III) An adaptation of the Proximal Policy Optimization algorithm that can learn to solve assignment problems represented through assignment graphs. To evaluate the proposed representation method, we model three archetypal assignment problems ranging from finite to infinite state and action space dimensionalities. The experiments show that the method is suitable for representing and learning close-to-optimal task assignment policies regardless of the state and action space dimensionalities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The code is publicly available at https://github.com/lobiaminor/BPM2024.
References
Lo Bianco, R., Dijkman, R., Nuijten, W., van Jaarsveld, W.: Action-evolution Petri nets: a framework for modeling and solving dynamic task assignment problems. In: Di Francescomarino, C., Burattin, A., Janiesch, C., Sadiq, S. (eds.) BPM 2023. LNCS, vol. 14159, pp. 216–231. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-41620-0_13
Żbikowski, K., Ostapowicz, M., Gawrysiak, P.: Deep reinforcement learning for resource allocation in business processes. In: Montali, M., Senderovich, A., Weidlich, M. (eds.) ICPM 2022. LNBIP, vol. 468, pp. 177–189. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-27815-0_13
Middelhuis, J., Lo Bianco, R., Scherzer, E., Bukhsh, Z., Adan, I., Dijkman, R.: Learning policies for resource allocation in business processes (2024)
Tavares, A.R., Anbalagan, S., Marcolino, L.S., Chaimowicz, L.: Algorithms or actions? A study in large-scale reinforcement learning. In IJCAI, pp. 2717–2723 (2018)
Psaraftis, H.N.: A dynamic programming solution to the single vehicle many-to-many immediate request dial-a-ride problem. Transp. Sci. 14(2), 130–154 (1980)
Rogers, D.F., Plante, R.D., Wong, R.T., Evans, J.R.: Aggregation and disaggregation techniques and methodology in optimization. Oper. Res. 39(4), 553–582 (1991)
Cheung, R., Powell, W.B.: Shape - a stochastic hybrid approximation procedure for two-stage stochastic programs. Oper. Res. 48(1), 73–79 (2000)
Spivey, M.Z., Powell, W.B.: The dynamic assignment problem. Transp. Sci. 38(4), 399–419 (2004)
Park, G., Song, M.: Prediction-based resource allocation using LSTM and minimum cost and maximum flow algorithm. In: 2019 International Conference on Process Mining (ICPM), pp. 121–128 (2019)
Park, G., Song, M.: Optimizing resource allocation based on predictive process monitoring. IEEE Access 11, 38309–38323 (2023)
Van Der Aalst, W.M.P., Kumar, A., Verbeek, E.M.W.: Dynamic work distribution in workflow management systems: how to balance quality and performance. J. Manag. Inf. Syst. 18(3), 157–193 (2002)
Dasht Bozorgi, Z., Teinemaa, I., Dumas, M., La Rosa, M., Polyvyanyy, A.: Prescriptive process monitoring based on causal effect estimation. Inf. Syst. 116, 102198 (2023)
Gammelli, D., Yang, K., Harrison, J., Rodrigues, F., Pereira, F.C., Pavone, M.: Graph neural network reinforcement learning for autonomous mobility-on-demand systems. In: 2021 60th IEEE Conference on Decision and Control (CDC), pp. 2996–3003 (2021)
Yan, Y., Deng, Y., Cui, S., Kuo, Y., Chow, A., Ying, C.: A policy gradient approach to solving dynamic assignment problem for on-site service delivery. Transp. Res. Part E: Logist. Transp. Rev. 178, 103260 (2023)
Begnardi, L., Baier, H., van Jaarsveld, W., Zhang, Y.: Deep reinforcement learning for two-sided online bipartite matching in collaborative order picking. In: Proceedings of the 15th Asian Conference on Machine Learning (ACML2023). Proceedings of Machine Learning Research (2023)
Chiorrini, A., Diamantini, C., Mircoli, A., Potena, D.: Exploiting instance graphs and graph neural networks for next activity prediction. In: Munoz-Gama, J., Lu, X. (eds.) ICPM 2021. LNBIP, vol. 433, pp. 115–126. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98581-3_9
Weinzierl, S.: Exploring gated graph sequence neural networks for predicting next process activities. In: Marrella, A., Weber, B. (eds.) BPM 2021. LNBIP, vol. 436, pp. 30–42. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-94343-1_3
Sommers, D., Menkovski, V., Fahland, D.: Process discovery using graph neural networks (2021)
Hu, L., Liu, Z., Hu, W., Wang, Y., Tan, J., Wu, F.: Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network. J. Manuf. Syst. 55, 1–14 (2020)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017)
Schulman, J., Moritz, P., Levine, S., Jordan, M., Abbeel, P.: High-dimensional continuous control using generalized advantage estimation (2018)
Wang, X., et al.: Heterogeneous graph attention network (2021)
Fey, M., Lenssen, J.E.: Fast graph representation learning with PyTorch geometric. CoRR, abs/1903.02428 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lo Bianco, R., Dijkman, R., Nuijten, W., van Jaarsveld, W. (2024). A Universal Approach to Feature Representation in Dynamic Task Assignment Problems. In: Marrella, A., Resinas, M., Jans, M., Rosemann, M. (eds) Business Process Management Forum. BPM 2024. Lecture Notes in Business Information Processing, vol 526. Springer, Cham. https://doi.org/10.1007/978-3-031-70418-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-70418-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70417-8
Online ISBN: 978-3-031-70418-5
eBook Packages: Computer ScienceComputer Science (R0)