Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Reinforcement Learning-Based Resource Allocation in Edge Computing

  • Conference paper
  • First Online:
Artificial Intelligence and Security (ICAIS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12240))

Included in the following conference series:

  • 1215 Accesses

Abstract

The problem of online resource allocation in edge computing has become a research hotspot. Meanwhile, reinforcement learning (RL) is suitable for solving online problems. In this paper, we combine edge computing online resource allocation with RL. This combination enables edge computing resource providers to obtain more social welfare and improve resource utilization. Specifically, we define a dynamic resource allocation problem for edge computing: edge equipment users request resources from a nearby edge computing server (ECS), the amount of resources required varies among the users, and there are time limits for the completion of the requested tasks. Since this resource allocation problem is NP-hard, it cannot be solved in polynomial time. Therefore, we propose an algorithm based on the policy-gradient algorithm in RL to solve the problem. Our approach is experimentally compared with existing research in terms of social welfare and resource utilization, for which it achieves good results.

This work is supported in part by the National Natural Science Foundation of China (Nos. 61762091, 61662088 and 11663007), the Project of the Natural Science Foundation of Yunnan Province of China (2019FB142), and the Program for Excellent Young Talents, Yunnan University, China.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Satyanarayanan, M., Bahl, P., Cceres, R., Davies, N.: The case for vm-based cloudlets in mobile computing. IEEE Pervasive Comput. 8(4), 14–23 (2009)

    Article  Google Scholar 

  2. Huang, X., Rong, Y., Kang, J., He, Y., Yan, Z.: Exploring mobile edge computing for 5G-enabled software defined vehicular networks. IEEE Wirel. Commun. 24(6), 55–63 (2018)

    Article  Google Scholar 

  3. Zeng, D., Lin, G., Song, G., Cheng, Z., Shui, Y.: Joint optimization of task scheduling and image placement in fog computing supported software-defined embedded system. IEEE Trans. Comput. 65(12), 3702–3712 (2016)

    Article  MathSciNet  Google Scholar 

  4. Gu, L., Zeng, D., Guo, S., Barnawi, A., Xiang, Y.: Cost-efficient resource management in fog computing supported medical CPS. IEEE Trans. Emerg. Top. Comput. 99, 113–123 (2017)

    Google Scholar 

  5. Chi, K., Zhu, Y.H., Li, Y., Zhang, D., Leung, V.: Coding schemes to minimize energy consumption of communication links in wireless nanosensor networks. IEEE IoT J. 3(4), 480–493 (2016)

    Google Scholar 

  6. Marinescu, D.C.: Cloud Computing: Theory and Practice. Elsevier, Amsterdam (2013)

    Google Scholar 

  7. Kiani, A., Ansari, N.: Toward hierarchical mobile edge computing: an auction-based profit maximization approach. IEEE IoT J. 99, 2082–2091 (2016)

    Google Scholar 

  8. Nejad, M.M., Mashayekhy, L., Grosu, D.: A family of truthful greedy mechanisms for dynamic virtual machine provisioning and allocation in clouds. In: IEEE Sixth International Conference on Cloud Computing, pp. 594–603 (2013)

    Google Scholar 

  9. Mashayekhy, L., Nejad, M.M., Grosu, D.: A ptas mechanism for provisioning and allocation of heterogeneous cloud resources. IEEE Trans. Parallel Distrib. Syst. 26(9), 2386–2399 (2014)

    Article  Google Scholar 

  10. Shi, W., Zhang, L., Wu, C., Li, Z., Lau, F., An online auction framework for dynamic resource provisioning in cloud computing. In: ACM SIGMETRICS Performance Evaluation Review, vol. 42, no. 1, pp. 71–83. ACM (2014)

    Google Scholar 

  11. Liu, X., Li, W., Zhang, X.: Strategy-proof mechanism for provisioning and allocation virtual machines in heterogeneous clouds. IEEE Trans. Parallel Distrib. Syst. 29(7), 1650–1663 (2017)

    Article  Google Scholar 

  12. Zhang, J., Xie, N., Zhang, X., Li, W.: Supervised learning based truthful auction mechanism design in cloud computing. J. Electron. Inf. Technol. 41(5), 1243 (2019)

    Google Scholar 

  13. Zhang, J., Xie, N., Zhang, X., Li, W.: An online auction mechanism for cloud computing resource allocation and pricing based on user evaluation and cost. Future Gener. Comput. Syst. 89, 286–299 (2018)

    Article  Google Scholar 

  14. Zhang, J., Xie, N., Li, W., Yue, K., Zhang, X.: Truthful multi requirements auction mechanism for virtual resource allocation of cloud computing. J. Electron. Inf. Technol. 40(1), 25–34 (2018)

    Google Scholar 

  15. Guo, Y., Liu, F., Xiao, N., Chen, Z.: Task-based resource allocation bid in edge computing micro datacenter. Comput. Mater. Continua 58, 777–792 (2019)

    Article  Google Scholar 

  16. Zhang, J., Xie, N., Zhang, X.: Machine learning based resource allocation of cloud computing in auction. Comput. Mater. Continua 56, 123–135 (2018)

    Google Scholar 

  17. Nisan, E.B.N., Roughgarden, T., Tardos, E., Vazirani, V.V.: Algorithmic game theory. Commun. ACM 53(7), 78–86 (2009)

    Google Scholar 

  18. Mashayekhy, L., Fisher, N., Grosu, D.: Truthful mechanisms for competitive reward-based scheduling. IEEE Trans. Comput. 65(7), 2299–2312 (2016)

    Article  MathSciNet  Google Scholar 

  19. Azar, Y., Kalp-shaltiel, I., Lucier, B., Menache, I., Naor, J., Yaniv, J.: Truthful online scheduling with commitments. In: Sixteenth ACM Conference on Economics and Computation, pp. 715–732 (2015)

    Google Scholar 

  20. Porter, R.: Mechanism design for online real-time scheduling. In: Proceedings of the 5th ACM Conference on Electronic Commerce, pp. 61–70. ACM (2004)

    Google Scholar 

  21. Parkes, D.C., Singh, S.: An MDP-based approach to online mechanism design. In: International Conference on Neural Information Processing Systems, pp. 791–798 (2003)

    Google Scholar 

  22. Stein, S., Gerding, E., Robu, V., Jennings, N.R.: A model-based online mechanism with pre-commitment and its application to electric vehicle charging. In: International Conference on Autonomous Agents and Multiagent Systems (2012)

    Google Scholar 

  23. Sandholm, T.: Automated mechanism design: a new application area for search algorithms. In: International Conference on Principles and Practice of Constraint Programming, pp. 19–36 (2003)

    Google Scholar 

  24. Yamada, K.: Layered learning in multiagent systems: a winning approach to soccer. Syst. Control Inf. 45, 285 (2001)

    Google Scholar 

  25. Moore, A.W.: Variable resolution dynamic programming: efficiently learning action maps in multivariate real-valued state-spaces. In: Machine Learning Proceedings, vol. 1991, pp. 333–337. Elsevier (1991)

    Google Scholar 

  26. Jiang, G., Wu, C.: Learning to control an inverted pendulum using q-learning and neural networks. Acta Automatica Sinica 24, 666–669 (1998)

    Google Scholar 

  27. Crites, R.H., Barto, A.G.: Elevator group control using multiple reinforcement learning agents. Mach. Learn. 33(2–3), 235–262 (1998)

    Article  Google Scholar 

  28. Hernandez-Leal, P., Kaisers, M., Baarslag, T., de Cote, E.M.: A survey of learning in multiagent environments: dealing with non-stationarity. arXiv preprint arXiv:1707.09183 (2017)

  29. Mao, H., Alizadeh, M., Menache, I., Kandula, S.: Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks, pp. 50–56. ACM (2016)

    Google Scholar 

  30. Mashayekhy, L., Nejad, M.M., Grosu, D., Vasilakos, A.V.: An online mechanism for resource allocation and pricing in clouds. IEEE Trans. Comput. 65(4), 1172–1184 (2015)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yunnan University, Kunming, 650504, Yunnan, People’s Republic of China

    Qianyu Xie, Xutao Yang, Laixin Chi, Xuejie Zhang & Jixian Zhang

Authors
  1. Qianyu Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xutao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laixin Chi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuejie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jixian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jixian Zhang .

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science, Nanjing, China

    Xingming Sun

  2. Nanjing University of Information Science, Nanjing, China

    Jinwei Wang

  3. Purdue University, West Lafayette, IN, USA

    Elisa Bertino

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xie, Q., Yang, X., Chi, L., Zhang, X., Zhang, J. (2020). Reinforcement Learning-Based Resource Allocation in Edge Computing. In: Sun, X., Wang, J., Bertino, E. (eds) Artificial Intelligence and Security. ICAIS 2020. Lecture Notes in Computer Science(), vol 12240. Springer, Cham. https://doi.org/10.1007/978-3-030-57881-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-57881-7_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-57880-0

  • Online ISBN: 978-3-030-57881-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation