Skip to main content

Advertisement

Log in

Ets-ddpg: an energy-efficient and QoS-guaranteed edge task scheduling approach based on deep reinforcement learning

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

With the development of 5 G communication and Internet of Things (IoT) technology, increasing data is generated by a large number of IoT devices at edge networks. Therefore, increasing need for distributed Data Centers (DCs) are seen from enterprises and building elastic applications upon DCs deployed over decentralized edge infrastructures is becoming popular. Nevertheless, it remains a great difficulty to effectively schedule computational tasks to appropriate DCs at the edge end with low energy consumption and satisfactory user-perceived Quality of Service. It is especially true when DCs deployed over an edge environment, which can be highly inhomogeneous in terms of resource configurations and computing capabilities. To this end, we develop an edge task scheduling method by synthesizing a M/G/1/PR queuing model for characterizing the workload distribution and a Deep Deterministic Policy Gradient algorithm for yielding high-quality schedules with low energy cost. We conduct extensive numerical analysis as well and show that our proposed method outperforms state-of-the-art methods in terms of average task response time and energy consumption.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Not applicable.

Code availability

Not applicable.

Notes

  1. www.juniperresearch.com/research/telco-connectivity/operator-strategies/cellular-iot-strategies-research-report.

  2. https://github.com/google/cluster-data.

  3. https://www.nyiso.com/energy-market-operational-data.

References

  1. Fang, W., Yao, X., Zhao, X., Yin, J., & Xiong, N. (2016). A stochastic control approach to maximize profit on service provisioning for mobile cloudlet platforms. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 48(4), 522–534. https://doi.org/10.1109/TSMC.2016.2606400

    Article  MATH  Google Scholar 

  2. Fazio, M., Celesti, A., Ranjan, R., Liu, C., Chen, L., & Villari, M. (2016). Open issues in scheduling microservices in the cloud. IEEE Cloud Computing, 3(5), 81–88. https://doi.org/10.1109/MCC.2016.112

    Article  Google Scholar 

  3. Masanet, E., Shehabi, A., Lei, N., Smith, S., & Koomey, J. (2020). Recalibrating global data center energy-use estimates. Science, 367(6481), 984–986. https://doi.org/10.1126/science.aba3758

    Article  Google Scholar 

  4. Kiani, A., & Ansari, N. (2016). A fundamental tradeoff between total and brown power consumption in geographically dispersed data centers. IEEE Communications Letters, 20(10), 1955–1958. https://doi.org/10.1109/LCOMM.2016.2598535

    Article  MATH  Google Scholar 

  5. Zhao, Z., Peng, M., Ding, Z., Wang, W., & Poor, H. V. (2016). Cluster content caching: An energy-efficient approach to improve quality of service in cloud radio access networks. IEEE Journal on Selected Areas in Communications, 34(5), 1207–1221. https://doi.org/10.1109/JSAC.2016.2545384

    Article  MATH  Google Scholar 

  6. Li, P., Xiao, Z., Wang, X., Huang, K., Huang, Y., & Gao, H. (2023). Eptask: Deep reinforcement learning based energy-efficient and priority-aware task scheduling for dynamic vehicular edge computing. IEEE Transactions on Intelligent Vehicles. https://doi.org/10.1109/TIV.2023.3321679

    Article  MATH  Google Scholar 

  7. Luo, J., Rao, L., & Liu, X. (2015). Spatio-temporal load balancing for energy cost optimization in distributed internet data centers. IEEE Transactions on Cloud Computing, 3(3), 387–397. https://doi.org/10.1109/TCC.2015.2415798

    Article  MATH  Google Scholar 

  8. Nosrati, M., & Karimi, R. (2016). Energy efficient and latency optimized media resource allocation. International Journal of Web Information Systems, 12(1), 2–17. https://doi.org/10.1108/IJWIS-10-2015-0031

    Article  MATH  Google Scholar 

  9. Wu, C., Peng, Q., Xia, Y., Jin, Y., & Hu, Z. (2023). Towards cost-effective and robust ai microservice deployment in edge computing environments. Future Generation Computer Systems, 141, 129–142. https://doi.org/10.1016/j.future.2022.10.015

    Article  MATH  Google Scholar 

  10. Zeng, H., Zhu, Z., Wang, Y., Xiang, Z., & Gao, H. (2024). Periodic collaboration and real-time dispatch using an actor-critic framework for uav movement in mobile edge computing. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2024.3366506

    Article  Google Scholar 

  11. Gao, H., Shao, J., Iqbal, M., Wang, Y., & Xiang, Z. (2024). Cfpc: The curbed fake point collector to pseudo-lidar-based 3d object detection for autonomous vehicles. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2024.3372940

    Article  Google Scholar 

  12. Huang, Z., Zhong, W., Li, D., & Lu, H. (2023). Delay constrained sfc orchestration for edge intelligence-enabled iiot: A drl approach. Journal of Network and Systems Management, 31(3), 53. https://doi.org/10.1007/s10922-023-09743-2

    Article  Google Scholar 

  13. Gao, H., Wang, X., Wei, W., Al-Dulaimi, A., & Xu, Y. (2023). Com-ddpg: task offloading based on multiagent reinforcement learning for information-communication-enhanced mobile edge computing in the internet of vehicles. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2023.3309321

    Article  Google Scholar 

  14. Liu, M., Mao, Y., & Leng, S. (2018). Cooperative fog-cloud computing enhanced by full-duplex communications. IEEE Communications Letters, 22(10), 2044–2047. https://doi.org/10.1109/LCOMM.2018.2866145

    Article  MATH  Google Scholar 

  15. Ghamkhari, M., & Mohsenian-Rad, H. (2013). Energy and performance management of green data centers: A profit maximization approach. IEEE Transactions on Smart Grid, 4(2), 1017–1025. https://doi.org/10.1109/TSG.2013.2237929

    Article  Google Scholar 

  16. Ma, Y., Dai, M., Shao, S., Xia, Y., Li, F., Shen, Y., Li, J., Li, Y., & Peng, H. (2023). A performance and reliability-guaranteed predictive approach to service migration path selection in mobile computing. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2023.3278884

    Article  MATH  Google Scholar 

  17. Peng, Q., Xia, Y., Zhou, M., Luo, X., Wang, S., Wang, Y., Wu, C., Pang, S., & Lin, M. (2020). Reliability-aware and deadline-constrained mobile service composition over opportunistic networks. IEEE Transactions on Automation Science and Engineering, 18(3), 1012–1025. https://doi.org/10.1109/TASE.2020.2993218

    Article  Google Scholar 

  18. Liu, W., Geng, J., Zhu, Z., Zhao, Y., Ji, C., Li, C., Lian, Z., & Zhou, X. (2023). Ace-sniper: Cloud-edge collaborative scheduling framework with dnn inference latency modeling on heterogeneous devices. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. https://doi.org/10.1109/TCAD.2023.3314388

    Article  Google Scholar 

  19. Kiani, A., & Ansari, N. (2016). Profit maximization for geographically dispersed green data centers. IEEE Transactions on Smart Grid, 9(2), 703–711. https://doi.org/10.1109/TSG.2016.2562565

    Article  MATH  Google Scholar 

  20. Yuan, H., Bi, J., & Zhou, M. (2020). Energy-efficient and qos-optimized adaptive task scheduling and management in clouds. IEEE Transactions on Automation Science and Engineering, 19(2), 1233–1244. https://doi.org/10.1109/TASE.2020.3042409

    Article  MATH  Google Scholar 

  21. Yuan, H., Bi, J., Zhang, J., & Zhou, M. (2021). Energy consumption and performance optimized task scheduling in distributed data centers. IEEE transactions on systems, man, and cybernetics: systems, 52(9), 5506–5517. https://doi.org/10.1109/TSMC.2021.3128430

    Article  MATH  Google Scholar 

  22. Cai, J., Zhou, Z., Huang, Z., Dai, W., & Yu, F. R. (2023). Privacy-preserving deployment mechanism for service function chains across multiple domains. IEEE Transactions on Network and Service Management. https://doi.org/10.1109/TNSM.2023.3311587

    Article  MATH  Google Scholar 

  23. Masadeh, R., Alsharman, N., Sharieh, A., Mahafzah, B. A., & Abdulrahman, A. (2021). Task scheduling on cloud computing based on sea lion optimization algorithm. International Journal of Web Information Systems, 17(2), 99–116. https://doi.org/10.1108/IJWIS-11-2020-0071

    Article  Google Scholar 

  24. Gao, H., Qiu, B., Wang, Y., Yu, S., Xu, Y., & Wang, X. (2023). Tbdb: Token bucket-based dynamic batching for resource scheduling supporting neural network inference in intelligent consumer electronics. IEEE Transactions on Consumer Electronics. https://doi.org/10.1109/TCE.2023.3339633

    Article  MATH  Google Scholar 

  25. Hu, X., Li, P., Wang, K., Sun, Y., Zeng, D., Wang, X., & Guo, S. (2019). Joint workload scheduling and energy management for green data centers powered by fuel cells. IEEE Transactions on Green Communications and Networking, 3(2), 397–406. https://doi.org/10.1109/TGCN.2019.2893712

    Article  MATH  Google Scholar 

  26. Gao, H., Wu, Y., Xu, Y., Li, R., & Jiang, Z. (2023). Neural collaborative learning for user preference discovery from biased behavior sequences. IEEE Transactions on Computational Social Systems. https://doi.org/10.1109/TCSS.2023.3268682

    Article  MATH  Google Scholar 

  27. Xu, Y., Qiu, Z., Gao, H., Zhao, X., Wang, L., & Li, R. (2023). Heterogeneous data-driven failure diagnosis for microservice-based industrial clouds towards consumer digital ecosystems. IEEE Transactions on Consumer Electronics. https://doi.org/10.1109/TCE.2023.3337351

    Article  Google Scholar 

  28. Kaur, K., Garg, S., Kaddoum, G., Bou-Harb, E., & Choo, K.-K.R. (2019). A big data-enabled consolidated framework for energy efficient software defined data centers in iot setups. IEEE Transactions on Industrial Informatics, 16(4), 2687–2697. https://doi.org/10.1109/TII.2019.2939573

    Article  MATH  Google Scholar 

  29. Hogade, N., Pasricha, S., Siegel, H. J., Maciejewski, A. A., Oxley, M. A., & Jonardi, E. (2018). Minimizing energy costs for geographically distributed heterogeneous data centers. IEEE Transactions on Sustainable Computing, 3(4), 318–331. https://doi.org/10.1109/TSUSC.2018.2822674

    Article  MATH  Google Scholar 

  30. Canali, C., Chiaraviglio, L., Lancellotti, R., & Shojafar, M. (2018). Joint minimization of the energy costs from computing, data transmission, and migrations in cloud data centers. IEEE Transactions on Green Communications and Networking, 2(2), 580–595. https://doi.org/10.1109/TGCN.2018.2796613

    Article  Google Scholar 

  31. Yadav, R., Zhang, W., Kaiwartya, O., Singh, P. R., Elgendy, I. A., & Tian, Y.-C. (2018). Adaptive energy-aware algorithms for minimizing energy consumption and sla violation in cloud computing. IEEE Access, 6, 55923–55936. https://doi.org/10.1109/ACCESS.2018.287275

    Article  Google Scholar 

  32. Chen, T., Zhang, Y., Wang, X., & Giannakis, G. B. (2016). Robust workload and energy management for sustainable data centers. IEEE Journal on Selected Areas in Communications, 34(3), 651–664. https://doi.org/10.1109/JSAC.2016.2525618

    Article  MATH  Google Scholar 

  33. Paul Beran, P., Vinek, E., & Schikuta, E. (2013). An adaptive framework for qos-aware service selection optimization. International Journal of Web Information Systems, 9(1), 32–52. https://doi.org/10.1108/17440081311316370

    Article  Google Scholar 

  34. Garg, S., Modi, K., & Chaudhary, S. (2016). A qos-aware approach for runtime discovery, selection and composition of semantic web services. International Journal of Web Information Systems, 12(2), 177–200. https://doi.org/10.1108/IJWIS-12-2015-0040

    Article  MATH  Google Scholar 

  35. Wei, L., Cai, J., Foh, C. H., & He, B. (2016). Qos-aware resource allocation for video transcoding in clouds. IEEE Transactions on Circuits and Systems for Video Technology, 27(1), 49–61. https://doi.org/10.1109/TCSVT.2016.2589621

    Article  MATH  Google Scholar 

  36. Li, K. (2018). Optimal power and performance management for heterogeneous and arbitrary cloud servers. IEEE Access, 7, 5071–5084. https://doi.org/10.1109/ACCESS.2018.2889220

    Article  MATH  Google Scholar 

  37. Farhat, F., Tootaghaj, D. Z., He, Y., Sivasubramaniam, A., Kandemir, M., & Das, C. R. (2016). Stochastic modeling and optimization of stragglers. IEEE Transactions on Cloud Computing, 6(4), 1164–1177. https://doi.org/10.1109/TCC.2016.2552516

    Article  Google Scholar 

  38. Yuan, H., Bi, J., Zhou, M., Liu, Q., & Ammari, A. C. (2020). Biobjective task scheduling for distributed green data centers. IEEE Transactions on Automation Science and Engineering, 18(2), 731–742. https://doi.org/10.1109/TASE.2019.2958979

    Article  MATH  Google Scholar 

  39. Li, P., Guo, S., Miyazaki, T., Liao, X., Jin, H., Zomaya, A. Y., & Wang, K. (2016). Traffic-aware geo-distributed big data analytics with predictable job completion time. IEEE Transactions on Parallel and Distributed Systems, 28(6), 1785–1796. https://doi.org/10.1109/TPDS.2016.2626285

    Article  Google Scholar 

  40. Yu, L., Jiang, T., & Zou, Y. (2016). Distributed real-time energy management in data center microgrids. IEEE Transactions on Smart Grid, 9(4), 3748–3762. https://doi.org/10.1109/TSG.2016.2640453

    Article  MATH  Google Scholar 

  41. Dong, C., Wen, W., Xu, T., & Yang, X. (2019). Joint optimization of data-center selection and video-streaming distribution for crowdsourced live streaming in a geo-distributed cloud platform. IEEE Transactions on Network and Service Management, 16(2), 729–742. https://doi.org/10.1109/TNSM.2019.2907785

    Article  MATH  Google Scholar 

  42. Hu, Z., Li, B., & Luo, J. (2017). Time-and cost-efficient task scheduling across geo-distributed data centers. IEEE Transactions on Parallel and Distributed Systems, 29(3), 705–718. https://doi.org/10.1109/TPDS.2017.2773504

    Article  MATH  Google Scholar 

  43. Liu, J., Li, G., Huang, Q., Bilal, M., Xu, X., & Song, H. (2022). Cooperative resource allocation for computation-intensive iiot applications in aerial computing. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2022.3222340

    Article  MATH  Google Scholar 

  44. Liu, C., Zhao, M., Wang, H., Cheng, B., Liu, J., & Yuan, P. (2024). Stackelberg-game computation offloading scheme for parked vehicle-assisted vec and experiment analysis. IEEE Transactions on Intelligent Vehicles. https://doi.org/10.1109/TIV.2024.3357076

    Article  MATH  Google Scholar 

  45. Schneider, S., Khalili, R., Manzoor, A., Qarawlus, H., Schellenberg, R., Karl, H., & Hecker, A. (2021). Self-learning multi-objective service coordination using deep reinforcement learning. IEEE Transactions on Network and Service Management, 18(3), 3829–3842. https://doi.org/10.1109/TNSM.2021.3076503

    Article  Google Scholar 

  46. Qu, C., Calheiros, R. N., & Buyya, R. (2018). Auto-scaling web applications in clouds: A taxonomy and survey. ACM Computing Surveys (CSUR), 51(4), 1–33. https://doi.org/10.1145/3148149

    Article  MATH  Google Scholar 

  47. Gu, L., Zeng, D., Barnawi, A., Guo, S., & Stojmenovic, I. (2014). Optimal task placement with qos constraints in geo-distributed data centers using dvfs. IEEE Transactions on Computers, 64(7), 2049–2059. https://doi.org/10.1109/TC.2014.2349510

    Article  MathSciNet  MATH  Google Scholar 

  48. Clouds, M.S.P. Optimal power allocation and load distribution for multiple heterogeneous multicore server processors across clouds and data centers. https://doi.org/10.1109/TC.2013.122

  49. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014). Deterministic policy gradient algorithms. In: International Conference on Machine Learning, pp. 387–395 Pmlr

Download references

Acknowledgements

This work was supported by the Postgraduate Scientific Research and Innovation Foundation of Chongqing under Grant No. CYB22064; This work was supported in part by the Foundation of Yunnan Key Laboratory of Science Computing No. YNSC23104; and in part by the Science Foundation of Chongqing under Grant No. CSTB2023NSCQ-MSX0782 and Grant No. KJQN202300533; This work was supported in part by the Key Research and Development Project of Henan Province under Grant No. 231111211900; and in part by the Key Research projects of Henan Colleges and Universities under Grant No. 24A520005; This work was supported in part by the Science and Technology Program of Sichuan Province under Grant No.24NSFTD0025

Author information

Authors and Affiliations

Authors

Contributions

All authors have equal contributions in this work.

Corresponding author

Correspondence to Yunni Xia.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Ethics approval and consent to participate

This article does not contain any studies with human participants or animals performed by any of the authors. All the authors involved have agreed to participate in this submitted article.

Consent for publication

All the authors involved in this manuscript give full consent for publication of this submitted article.

Materials availability

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Xia, Y., Sun, X. et al. Ets-ddpg: an energy-efficient and QoS-guaranteed edge task scheduling approach based on deep reinforcement learning. Wireless Netw 31, 1241–1254 (2025). https://doi.org/10.1007/s11276-024-03820-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-024-03820-3

Keywords