Abstract
Applications in various fields such as embedded systems or High-Performance-Computing are often represented as Directed Acyclic Graphs (DAG), also known as taskgraphs. DAGs represent the data flow between tasks in an application and can be used for scheduling. When scheduling taskgraphs, a scheduler needs to decide when and on which core each task is executed, while minimising the runtime of the schedule.
This paper explores offline scheduling of dependent tasks using a Reinforcement Learning (RL) approach. We propose two RL schedulers, one using a Fully Connected Network (FCN) and another one using a Graph Convolutional Network (GCN). First, we detail the different components of our two RL schedulers and illustrate how they schedule a task. Then, we compare our RL schedulers to a Forward List Scheduling (FLS) approach based on two different datasets. We demonstrate that our GCN-based scheduler produces schedules that are as good or better than the schedules produced by the FLS approach in over 85% of the cases for a dataset with small taskgraphs. The same scheduler performs very similar to the FLS scheduler (at most 5% degradation) in almost 76% of the cases for a more challenging dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jetson, N.: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/. Accessed 21 Jan 2023
Andreozzi, M., Gabrielli, G., Venu, B., Travaglini, G.: Industrial challenge 2022: a high-performance real-time case study on arm. In: ECRTS 2022. Schloss Dagstuhl-Leibniz-Zentrum für Informatik (2022)
Bambagini, M., Marinoni, M., Aydin, H., Buttazzo, G.: Energy-aware scheduling for real-time systems. TECS 15(1), 1–34 (2016)
Biewald, L.: Experiment tracking with weights and biases (2020). https://www.wandb.com/
Clevert, D., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUS). arXiv preprint arXiv:1511.07289 (2015)
Cooper, K.D., Schielke, P.J., Subramanian, D.: An Experimental Evaluation of List Scheduling. TR98 326 (1998)
Dick, R., Rhodes, D., Wolf, W.: TGFF: task graphs for free. In: 6th CODES/CASHE. IEEE (1998)
Gerards, M.E.T., Hurink, J.L., Hölzenspies, P.K.F.: A survey of offline algorithms for energy minimization under deadline constraints. J. Sched. 19(1), 3–19 (2016). https://doi.org/10.1007/s10951-015-0463-8
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Hardkernel Co., Ltd: Odroid-XU4. https://wiki.odroid.com/odroid-xu4/odroid-xu4. Accessed 06 Sep 2019
Hasselt, H.: Double Q-learning. In: 24th NIPS, vol. 23, pp. 2613–2621 (2010)
Hu, Y., de Laat, C., Zhao, Z.: Learning workflow scheduling on multi-resource clusters. In: 2019 NAS, pp. 1–8. IEEE (2019)
Hu, Z., Tu, J., Li, B.: Spear: optimized dependency-aware task scheduling with deep reinforcement learning. In: 39th ICDCS, pp. 2037–2046. IEEE (2019)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Kool, W., van Hoof, H., Welling, M.: Attention, learn to solve routing problems! In: 7th ICLR (2019)
Mao, H., Alizadeh, M., Menache, I., Kandula, S.: Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM hot topics in Workshop on Networks, pp. 50–56 (2016)
Patrício, D.I., Rieder, R.: Computer vision and artificial intelligence in precision agriculture for grain crops: a systematic review. Comput. Electron. Agric. 153, 69–81 (2018)
Roeder, J., Rouxel, B., Altmeyer, S., Grelck, C.: Energy-aware scheduling of multi-version tasks on heterogeneous real-time systems. In: 2021 36th SAC, pp. 501–510 (2021)
Rouxel, B., Skalistis, S., Derrien, S., Puaut, I.: Hiding communication delays in contention-free execution for SPM-based multi-core architectures. In: 31st ECRTS19 (2019)
Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized Experience Replay. arXiv:1511.05952 (2015)
Silver, D., et al.: Mastering the game of Go without human knowledge. Nature 550, 354–379 (2017)
Singh, A.K., Dziurzanski, P., Mendis, H.R., Indrusiak, L.S.: A survey and comparative study of hard and soft real-time dynamic resource allocation strategies for multi-/many-core systems. ACM Comput. Surv. (CSUR) 50(2), 1–40 (2017)
Singh, A.K., Shafique, M., Kumar, A., Henkel, J.: Mapping on multi/many-core systems: survey of current and emerging trends. In: 2013 50th DAC, pp. 1–10. IEEE (2013)
Ullah Tariq, U., Ali, H., Liu, L., Panneerselvam, J., Zhai, X.: Energy-efficient static task scheduling on VFI based NoC-HMPSoCs for intelligent edge devices in cyber-physical systems. TIST 1(1) (2019)
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)
Wu, Q., Wu, Z., Zhuang, Y., Cheng, Y.: Adaptive DAG tasks scheduling with deep reinforcement learning. In: Vaidya, J., Li, J. (eds.) ICA3PP 2018. LNCS, vol. 11335, pp. 477–490. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-05054-2_37
Acknowledgement
We would like to thank the reviewers for their time and feedback.
This work has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 871259 (ADMORPH project).
This article is based upon work from COST Action CERCIRAS, supported by COST (European Cooperation in Science and Technology)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Roeder, J., Pimentel, A.D., Grelck, C. (2023). GCN-based Reinforcement Learning Approach for Scheduling DAG Applications. In: Maglogiannis, I., Iliadis, L., MacIntyre, J., Dominguez, M. (eds) Artificial Intelligence Applications and Innovations. AIAI 2023. IFIP Advances in Information and Communication Technology, vol 676. Springer, Cham. https://doi.org/10.1007/978-3-031-34107-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-34107-6_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34106-9
Online ISBN: 978-3-031-34107-6
eBook Packages: Computer ScienceComputer Science (R0)
