Abstract
With the development of 5 G communication and Internet of Things (IoT) technology, increasing data is generated by a large number of IoT devices at edge networks. Therefore, increasing need for distributed Data Centers (DCs) are seen from enterprises and building elastic applications upon DCs deployed over decentralized edge infrastructures is becoming popular. Nevertheless, it remains a great difficulty to effectively schedule computational tasks to appropriate DCs at the edge end with low energy consumption and satisfactory user-perceived Quality of Service. It is especially true when DCs deployed over an edge environment, which can be highly inhomogeneous in terms of resource configurations and computing capabilities. To this end, we develop an edge task scheduling method by synthesizing a M/G/1/PR queuing model for characterizing the workload distribution and a Deep Deterministic Policy Gradient algorithm for yielding high-quality schedules with low energy cost. We conduct extensive numerical analysis as well and show that our proposed method outperforms state-of-the-art methods in terms of average task response time and energy consumption.






Similar content being viewed by others
Data availability
Not applicable.
Code availability
Not applicable.
Notes
www.juniperresearch.com/research/telco-connectivity/operator-strategies/cellular-iot-strategies-research-report.
https://github.com/google/cluster-data.
https://www.nyiso.com/energy-market-operational-data.
References
Fang, W., Yao, X., Zhao, X., Yin, J., & Xiong, N. (2016). A stochastic control approach to maximize profit on service provisioning for mobile cloudlet platforms. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 48(4), 522–534. https://doi.org/10.1109/TSMC.2016.2606400
Fazio, M., Celesti, A., Ranjan, R., Liu, C., Chen, L., & Villari, M. (2016). Open issues in scheduling microservices in the cloud. IEEE Cloud Computing, 3(5), 81–88. https://doi.org/10.1109/MCC.2016.112
Masanet, E., Shehabi, A., Lei, N., Smith, S., & Koomey, J. (2020). Recalibrating global data center energy-use estimates. Science, 367(6481), 984–986. https://doi.org/10.1126/science.aba3758
Kiani, A., & Ansari, N. (2016). A fundamental tradeoff between total and brown power consumption in geographically dispersed data centers. IEEE Communications Letters, 20(10), 1955–1958. https://doi.org/10.1109/LCOMM.2016.2598535
Zhao, Z., Peng, M., Ding, Z., Wang, W., & Poor, H. V. (2016). Cluster content caching: An energy-efficient approach to improve quality of service in cloud radio access networks. IEEE Journal on Selected Areas in Communications, 34(5), 1207–1221. https://doi.org/10.1109/JSAC.2016.2545384
Li, P., Xiao, Z., Wang, X., Huang, K., Huang, Y., & Gao, H. (2023). Eptask: Deep reinforcement learning based energy-efficient and priority-aware task scheduling for dynamic vehicular edge computing. IEEE Transactions on Intelligent Vehicles. https://doi.org/10.1109/TIV.2023.3321679
Luo, J., Rao, L., & Liu, X. (2015). Spatio-temporal load balancing for energy cost optimization in distributed internet data centers. IEEE Transactions on Cloud Computing, 3(3), 387–397. https://doi.org/10.1109/TCC.2015.2415798
Nosrati, M., & Karimi, R. (2016). Energy efficient and latency optimized media resource allocation. International Journal of Web Information Systems, 12(1), 2–17. https://doi.org/10.1108/IJWIS-10-2015-0031
Wu, C., Peng, Q., Xia, Y., Jin, Y., & Hu, Z. (2023). Towards cost-effective and robust ai microservice deployment in edge computing environments. Future Generation Computer Systems, 141, 129–142. https://doi.org/10.1016/j.future.2022.10.015
Zeng, H., Zhu, Z., Wang, Y., Xiang, Z., & Gao, H. (2024). Periodic collaboration and real-time dispatch using an actor-critic framework for uav movement in mobile edge computing. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2024.3366506
Gao, H., Shao, J., Iqbal, M., Wang, Y., & Xiang, Z. (2024). Cfpc: The curbed fake point collector to pseudo-lidar-based 3d object detection for autonomous vehicles. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2024.3372940
Huang, Z., Zhong, W., Li, D., & Lu, H. (2023). Delay constrained sfc orchestration for edge intelligence-enabled iiot: A drl approach. Journal of Network and Systems Management, 31(3), 53. https://doi.org/10.1007/s10922-023-09743-2
Gao, H., Wang, X., Wei, W., Al-Dulaimi, A., & Xu, Y. (2023). Com-ddpg: task offloading based on multiagent reinforcement learning for information-communication-enhanced mobile edge computing in the internet of vehicles. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2023.3309321
Liu, M., Mao, Y., & Leng, S. (2018). Cooperative fog-cloud computing enhanced by full-duplex communications. IEEE Communications Letters, 22(10), 2044–2047. https://doi.org/10.1109/LCOMM.2018.2866145
Ghamkhari, M., & Mohsenian-Rad, H. (2013). Energy and performance management of green data centers: A profit maximization approach. IEEE Transactions on Smart Grid, 4(2), 1017–1025. https://doi.org/10.1109/TSG.2013.2237929
Ma, Y., Dai, M., Shao, S., Xia, Y., Li, F., Shen, Y., Li, J., Li, Y., & Peng, H. (2023). A performance and reliability-guaranteed predictive approach to service migration path selection in mobile computing. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2023.3278884
Peng, Q., Xia, Y., Zhou, M., Luo, X., Wang, S., Wang, Y., Wu, C., Pang, S., & Lin, M. (2020). Reliability-aware and deadline-constrained mobile service composition over opportunistic networks. IEEE Transactions on Automation Science and Engineering, 18(3), 1012–1025. https://doi.org/10.1109/TASE.2020.2993218
Liu, W., Geng, J., Zhu, Z., Zhao, Y., Ji, C., Li, C., Lian, Z., & Zhou, X. (2023). Ace-sniper: Cloud-edge collaborative scheduling framework with dnn inference latency modeling on heterogeneous devices. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. https://doi.org/10.1109/TCAD.2023.3314388
Kiani, A., & Ansari, N. (2016). Profit maximization for geographically dispersed green data centers. IEEE Transactions on Smart Grid, 9(2), 703–711. https://doi.org/10.1109/TSG.2016.2562565
Yuan, H., Bi, J., & Zhou, M. (2020). Energy-efficient and qos-optimized adaptive task scheduling and management in clouds. IEEE Transactions on Automation Science and Engineering, 19(2), 1233–1244. https://doi.org/10.1109/TASE.2020.3042409
Yuan, H., Bi, J., Zhang, J., & Zhou, M. (2021). Energy consumption and performance optimized task scheduling in distributed data centers. IEEE transactions on systems, man, and cybernetics: systems, 52(9), 5506–5517. https://doi.org/10.1109/TSMC.2021.3128430
Cai, J., Zhou, Z., Huang, Z., Dai, W., & Yu, F. R. (2023). Privacy-preserving deployment mechanism for service function chains across multiple domains. IEEE Transactions on Network and Service Management. https://doi.org/10.1109/TNSM.2023.3311587
Masadeh, R., Alsharman, N., Sharieh, A., Mahafzah, B. A., & Abdulrahman, A. (2021). Task scheduling on cloud computing based on sea lion optimization algorithm. International Journal of Web Information Systems, 17(2), 99–116. https://doi.org/10.1108/IJWIS-11-2020-0071
Gao, H., Qiu, B., Wang, Y., Yu, S., Xu, Y., & Wang, X. (2023). Tbdb: Token bucket-based dynamic batching for resource scheduling supporting neural network inference in intelligent consumer electronics. IEEE Transactions on Consumer Electronics. https://doi.org/10.1109/TCE.2023.3339633
Hu, X., Li, P., Wang, K., Sun, Y., Zeng, D., Wang, X., & Guo, S. (2019). Joint workload scheduling and energy management for green data centers powered by fuel cells. IEEE Transactions on Green Communications and Networking, 3(2), 397–406. https://doi.org/10.1109/TGCN.2019.2893712
Gao, H., Wu, Y., Xu, Y., Li, R., & Jiang, Z. (2023). Neural collaborative learning for user preference discovery from biased behavior sequences. IEEE Transactions on Computational Social Systems. https://doi.org/10.1109/TCSS.2023.3268682
Xu, Y., Qiu, Z., Gao, H., Zhao, X., Wang, L., & Li, R. (2023). Heterogeneous data-driven failure diagnosis for microservice-based industrial clouds towards consumer digital ecosystems. IEEE Transactions on Consumer Electronics. https://doi.org/10.1109/TCE.2023.3337351
Kaur, K., Garg, S., Kaddoum, G., Bou-Harb, E., & Choo, K.-K.R. (2019). A big data-enabled consolidated framework for energy efficient software defined data centers in iot setups. IEEE Transactions on Industrial Informatics, 16(4), 2687–2697. https://doi.org/10.1109/TII.2019.2939573
Hogade, N., Pasricha, S., Siegel, H. J., Maciejewski, A. A., Oxley, M. A., & Jonardi, E. (2018). Minimizing energy costs for geographically distributed heterogeneous data centers. IEEE Transactions on Sustainable Computing, 3(4), 318–331. https://doi.org/10.1109/TSUSC.2018.2822674
Canali, C., Chiaraviglio, L., Lancellotti, R., & Shojafar, M. (2018). Joint minimization of the energy costs from computing, data transmission, and migrations in cloud data centers. IEEE Transactions on Green Communications and Networking, 2(2), 580–595. https://doi.org/10.1109/TGCN.2018.2796613
Yadav, R., Zhang, W., Kaiwartya, O., Singh, P. R., Elgendy, I. A., & Tian, Y.-C. (2018). Adaptive energy-aware algorithms for minimizing energy consumption and sla violation in cloud computing. IEEE Access, 6, 55923–55936. https://doi.org/10.1109/ACCESS.2018.287275
Chen, T., Zhang, Y., Wang, X., & Giannakis, G. B. (2016). Robust workload and energy management for sustainable data centers. IEEE Journal on Selected Areas in Communications, 34(3), 651–664. https://doi.org/10.1109/JSAC.2016.2525618
Paul Beran, P., Vinek, E., & Schikuta, E. (2013). An adaptive framework for qos-aware service selection optimization. International Journal of Web Information Systems, 9(1), 32–52. https://doi.org/10.1108/17440081311316370
Garg, S., Modi, K., & Chaudhary, S. (2016). A qos-aware approach for runtime discovery, selection and composition of semantic web services. International Journal of Web Information Systems, 12(2), 177–200. https://doi.org/10.1108/IJWIS-12-2015-0040
Wei, L., Cai, J., Foh, C. H., & He, B. (2016). Qos-aware resource allocation for video transcoding in clouds. IEEE Transactions on Circuits and Systems for Video Technology, 27(1), 49–61. https://doi.org/10.1109/TCSVT.2016.2589621
Li, K. (2018). Optimal power and performance management for heterogeneous and arbitrary cloud servers. IEEE Access, 7, 5071–5084. https://doi.org/10.1109/ACCESS.2018.2889220
Farhat, F., Tootaghaj, D. Z., He, Y., Sivasubramaniam, A., Kandemir, M., & Das, C. R. (2016). Stochastic modeling and optimization of stragglers. IEEE Transactions on Cloud Computing, 6(4), 1164–1177. https://doi.org/10.1109/TCC.2016.2552516
Yuan, H., Bi, J., Zhou, M., Liu, Q., & Ammari, A. C. (2020). Biobjective task scheduling for distributed green data centers. IEEE Transactions on Automation Science and Engineering, 18(2), 731–742. https://doi.org/10.1109/TASE.2019.2958979
Li, P., Guo, S., Miyazaki, T., Liao, X., Jin, H., Zomaya, A. Y., & Wang, K. (2016). Traffic-aware geo-distributed big data analytics with predictable job completion time. IEEE Transactions on Parallel and Distributed Systems, 28(6), 1785–1796. https://doi.org/10.1109/TPDS.2016.2626285
Yu, L., Jiang, T., & Zou, Y. (2016). Distributed real-time energy management in data center microgrids. IEEE Transactions on Smart Grid, 9(4), 3748–3762. https://doi.org/10.1109/TSG.2016.2640453
Dong, C., Wen, W., Xu, T., & Yang, X. (2019). Joint optimization of data-center selection and video-streaming distribution for crowdsourced live streaming in a geo-distributed cloud platform. IEEE Transactions on Network and Service Management, 16(2), 729–742. https://doi.org/10.1109/TNSM.2019.2907785
Hu, Z., Li, B., & Luo, J. (2017). Time-and cost-efficient task scheduling across geo-distributed data centers. IEEE Transactions on Parallel and Distributed Systems, 29(3), 705–718. https://doi.org/10.1109/TPDS.2017.2773504
Liu, J., Li, G., Huang, Q., Bilal, M., Xu, X., & Song, H. (2022). Cooperative resource allocation for computation-intensive iiot applications in aerial computing. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2022.3222340
Liu, C., Zhao, M., Wang, H., Cheng, B., Liu, J., & Yuan, P. (2024). Stackelberg-game computation offloading scheme for parked vehicle-assisted vec and experiment analysis. IEEE Transactions on Intelligent Vehicles. https://doi.org/10.1109/TIV.2024.3357076
Schneider, S., Khalili, R., Manzoor, A., Qarawlus, H., Schellenberg, R., Karl, H., & Hecker, A. (2021). Self-learning multi-objective service coordination using deep reinforcement learning. IEEE Transactions on Network and Service Management, 18(3), 3829–3842. https://doi.org/10.1109/TNSM.2021.3076503
Qu, C., Calheiros, R. N., & Buyya, R. (2018). Auto-scaling web applications in clouds: A taxonomy and survey. ACM Computing Surveys (CSUR), 51(4), 1–33. https://doi.org/10.1145/3148149
Gu, L., Zeng, D., Barnawi, A., Guo, S., & Stojmenovic, I. (2014). Optimal task placement with qos constraints in geo-distributed data centers using dvfs. IEEE Transactions on Computers, 64(7), 2049–2059. https://doi.org/10.1109/TC.2014.2349510
Clouds, M.S.P. Optimal power allocation and load distribution for multiple heterogeneous multicore server processors across clouds and data centers. https://doi.org/10.1109/TC.2013.122
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014). Deterministic policy gradient algorithms. In: International Conference on Machine Learning, pp. 387–395 Pmlr
Acknowledgements
This work was supported by the Postgraduate Scientific Research and Innovation Foundation of Chongqing under Grant No. CYB22064; This work was supported in part by the Foundation of Yunnan Key Laboratory of Science Computing No. YNSC23104; and in part by the Science Foundation of Chongqing under Grant No. CSTB2023NSCQ-MSX0782 and Grant No. KJQN202300533; This work was supported in part by the Key Research and Development Project of Henan Province under Grant No. 231111211900; and in part by the Key Research projects of Henan Colleges and Universities under Grant No. 24A520005; This work was supported in part by the Science and Technology Program of Sichuan Province under Grant No.24NSFTD0025
Author information
Authors and Affiliations
Contributions
All authors have equal contributions in this work.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no Conflict of interest.
Ethics approval and consent to participate
This article does not contain any studies with human participants or animals performed by any of the authors. All the authors involved have agreed to participate in this submitted article.
Consent for publication
All the authors involved in this manuscript give full consent for publication of this submitted article.
Materials availability
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhao, J., Xia, Y., Sun, X. et al. Ets-ddpg: an energy-efficient and QoS-guaranteed edge task scheduling approach based on deep reinforcement learning. Wireless Netw 31, 1241–1254 (2025). https://doi.org/10.1007/s11276-024-03820-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-024-03820-3