Abstract
The edge computing paradigm is featured by the ability to off-load computing tasks from mobile devices to edge clouds and provide high cost-efficient computing resources, storage and network services closer to the edge. A key question for workflow scheduling in the edge computing environment is how to guarantee user-perceived quality of services when the supporting edge services and resources are with unstable, time-variant, and fluctuant performance. In this work, we study the workflow scheduling problem in the multi-user edge computing environment and propose a Deep-Q-Network (DQN) -based multi-workflow scheduling approach which is capable of handling time-varying performance of edge services. To validate our proposed approach, we conduct a simulative case study and compare ours with other existing methods. Results clearly demonstrate that our proposed method beats its peers in terms of convergence speed and workflow completion time.
H. Liu and Y. Ma—Contribute equally to this article and should be considered co-first authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, X., Liu, Z., Chen, Y., Li, Z.: Mobile edge computing based task offloading and resource allocation in 5g ultra-dense networks. IEEE Access 7, 184172–184182 (2019)
Ciobanu, R., Dobre, C., Balanescu, M., Suciu, G.: Data and task offloading in collaborative mobile fog-based networks. IEEE Access 7, 104405–104422 (2019)
Li, G., Lin, Q., Wu, J., Zhang, Y., Yan, J.: Dynamic computation offloading based on graph partitioning in mobile edge computing. IEEE Access 7, 185131–185139 (2019)
Luo, S., Wen, Y., Xu, W., Puthal, D.: Adaptive task offloading auction for industrial CPS in mobile edge computing. IEEE Access 7, 169055–169065 (2019)
Zhou, J., Fan, J., Wang, J., Zhu, J.: Task offloading for social sensing applications in mobile edge computing. In: Seventh International Conference on Advanced Cloud and Big Data, CBD 2019, Suzhou, China, 21–22 September 2019, pp. 333–338. IEEE (2019)
Chen, H., Zhu, X., Liu, G., Pedrycz, W.: Uncertainty-aware online scheduling for real-time workflows in cloud service environment. IEEE Trans. Serv. Comput. 1, 1 (2018)
Zhang, Y., Du, P.: Delay-driven computation task scheduling in multi-cell cellular edge computing systems. IEEE Access 7, 149156–149167 (2019)
Cao, H., Xu, X., Liu, Q., Xue, Y., Qi, L.: Uncertainty-aware resource provisioning for workflow scheduling in edge computing environment. In: 18th IEEE International Conference On Trust, Security and Privacy in Computing and Communications / 13th IEEE International Conference On Big Data Science And Engineering, TrustCom/BigDataSE 2019, Rotorua, New Zealand, 5–8 August 2019, pp. 734–739. IEEE (2019)
Deng, Y., Chen, Z., Yao, X., Hassan, S., Wu, J.: Task scheduling for smart city applications based on multi-server mobile edge computing. IEEE Access 7, 14410–14421 (2019)
Jian, C., Chen, J., Ping, J., Zhang, M.: An improved chaotic bat swarm scheduling learning model on edge computing. IEEE Access 7, 58602–58610 (2019)
Ma, Y., et al.: A novel approach to cost-efficient scheduling of multi-workflows in the edge computing environment with the proximity constraint. In: Wen, S., Zomaya, A.Y., Yang, L.T. (eds.) Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Melbourne, VIC, Australia, 9–11 December 2019, Proceedings, Part I. Volume 11944 of Lecture Notes in Computer Science, pp. 655–668. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-38991-8_43
Peng, Q., Jiang, H., Chen, M., Liang, J., Xia, Y.: Reliability-aware and deadline-constrained workflow scheduling in mobile edge computing. In: 16th IEEE International Conference on Networking, Sensing and Control, ICNSC 2019, Banff, AB, Canada, May 9–11, 2019, pp. 236–241. IEEE (2019)
Bernal, J., et al.: Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review. Artif. Intell. Med. 95, 64–81 (2019)
Bouwmans, T., Javed, S., Sultana, M., Jung, S.K.: Deep neural network concepts for background subtraction: a systematic review and comparative evaluation. Neural Netw. 117, 8–66 (2019)
Grekousis, G.: Artificial neural networks and deep learning in urban geography: a systematic review and meta-analysis. Comput. Environ. Urban Syst. 74, 244–256 (2019)
Kaur, M., Kadam, S.: A novel multi-objective bacteria foraging optimization algorithm (MOBFOA) for multi-objective scheduling. Appl. Soft Comput. J. 66, 183–195 (2018)
Zhang, L., Li, K., Li, C., Li, K.: Bi-objective workflow scheduling of the energy consumption and reliability in heterogeneous computing systems. Inf. Sci. 379, 241–256 (2017)
Casas, I., Taheri, J., Ranjan, R., Wang, L., Zomaya, A.Y.: GA-ETI: an enhanced genetic algorithm for the scheduling of scientific workflows in cloud environments. J. Comput. Sci. 26, 318–331 (2018)
Verma, A., Kaushal, S.: A hybrid multi-objective Particle Swarm Optimization for scientific workflow scheduling. Parallel Comput. 62, 1–19 (2017)
Zhou, X., Zhang, G., Sun, J., Zhou, J., Wei, T., Hu, S.: Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT. Fut. Generation Comput. Syst. 93, 278–289 (2019)
Bertsekas, D.P.: Feature-based aggregation and deep reinforcement learning: a survey and some new implementations. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing, pp. 1–31 (2018)
Mao, H., Alizadeh, M., Menache, I., Kandula, S.: Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks - HotNets 2016, pp. 50–56 (2016)
Xue, L., Sun, C., Wunsch, D., Zhou, Y., Yu, F.: An adaptive strategy via reinforcement learning for the prisoner’s dilemma game. IEEE/CAA J. Automatica Sinica 5(1), 301–310 (2018)
Zhan, Y., Ammar, H.B., Taylor, M.E.: Theoretically-grounded policy advice from multiple teachers in reinforcement learning settings with applications to negative transfer. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. IJCAI’16, AAAI Press, pp. 2315–2321 (2016)
Wang, H., Huang, T., Liao, X., Abu-Rub, H., Chen, G.: Reinforcement learning for constrained energy trading games with incomplete information. IEEE Trans. Cybern. 47(10), 3404–3416 (2017)
Zheng, L., Yang, J., Cai, H., Zhang, W., Wang, J., Yu, Y.: MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence, pp. 1–2 (2017)
Lowe, R., Wu, Y., Tamar, A., Harb, J., Pieter Abbeel, O., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30. Curran Associates, Inc. pp. 6379–6390 (2017)
Duan, R., Prodan, R., Li, X.: Multi-objective game theoretic scheduling of bag-of-tasks workflows on hybrid clouds. IEEE Trans. Cloud Comput. 2(1), 29–42 (2014)
Cui, D., Ke, W., Peng, Z., Zuo, J.: Multiple DAGs Workflow Scheduling Algorithm Based on Reinforcement Learning in Cloud Computing, pp. 305–311. Springer, Singapore (2016). https://doi.org/10.1007/978-981-10-0356-1_31
Iranpour, E., Sharifian, S.: A distributed load balancing and admission control algorithm based on Fuzzy type-2 and Game theory for large-scale SaaS cloud architectures. Future Generation Comput. Syst. 86 81–98 (2018)
Jiahao, W., Zhiping, P., Delong, C., Qirui, L., Jieguang, H.: A Multi-object Optimization Cloud Workflow Scheduling Algorithm Based on Reinforcement Learning, pp. 550–559. Springer, Cham (aug (2018). https://doi.org/10.1007/978-3-319-95933-7_64
Guo, S., Liu, J., Yang, Y., Xiao, B., Li, Z.: Energy-efficient dynamic computation offloading and cooperative task scheduling in mobile cloud computing. IEEE Trans. Mob. Comput. 18(2), 319–333 (2019)
Mnih, V., et al.: Playing atari with deep reinforcement. Learning 2055, 1–9 (2013)
Lai, P., et al.: Optimal edge user allocation in edge computing with variable sized vector bin packing. CoRR abs/1904.05553 (2019)
Li, W., Xia, Y., Zhou, M., Sun, X., Zhu, Q.: Fluctuation-aware and predictive workflow scheduling in cost-effective infrastructure-as-a-service clouds. IEEE Access 6, 61488–61502 (2018)
Beegom, A.S.A., Rajasree, M.S.: Non-dominated sorting based PSO algorithm for workflow task scheduling in cloud computing systems. J. Intell. Fuzzy Syst. 37(5), 6801–6813 (2019)
Mollajafari, M., Shahhoseini, H.S.: Cost-optimized ga-based heuristic for scheduling time-constrained workflow applications in infrastructure clouds using an innovative feasibility-assured decoding mechanism. J. Inf. Sci. Eng. 32(6), 1541–1560 (2016)
Acknowledgement
This work is supported in part by Science and Technology Program of Sichuan Province under Grant 2020 JDRC0067.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, H. et al. (2020). Scheduling Multi-workflows over Edge Computing Resources with Time-Varying Performance, A Novel Probability-Mass Function and DQN-Based Approach. In: Ku, WS., Kanemasa, Y., Serhani, M.A., Zhang, LJ. (eds) Web Services – ICWS 2020. ICWS 2020. Lecture Notes in Computer Science(), vol 12406. Springer, Cham. https://doi.org/10.1007/978-3-030-59618-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-59618-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59617-0
Online ISBN: 978-3-030-59618-7
eBook Packages: Computer ScienceComputer Science (R0)