Abstract
The problem of online resource allocation in edge computing has become a research hotspot. Meanwhile, reinforcement learning (RL) is suitable for solving online problems. In this paper, we combine edge computing online resource allocation with RL. This combination enables edge computing resource providers to obtain more social welfare and improve resource utilization. Specifically, we define a dynamic resource allocation problem for edge computing: edge equipment users request resources from a nearby edge computing server (ECS), the amount of resources required varies among the users, and there are time limits for the completion of the requested tasks. Since this resource allocation problem is NP-hard, it cannot be solved in polynomial time. Therefore, we propose an algorithm based on the policy-gradient algorithm in RL to solve the problem. Our approach is experimentally compared with existing research in terms of social welfare and resource utilization, for which it achieves good results.
This work is supported in part by the National Natural Science Foundation of China (Nos. 61762091, 61662088 and 11663007), the Project of the Natural Science Foundation of Yunnan Province of China (2019FB142), and the Program for Excellent Young Talents, Yunnan University, China.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Satyanarayanan, M., Bahl, P., Cceres, R., Davies, N.: The case for vm-based cloudlets in mobile computing. IEEE Pervasive Comput. 8(4), 14–23 (2009)
Huang, X., Rong, Y., Kang, J., He, Y., Yan, Z.: Exploring mobile edge computing for 5G-enabled software defined vehicular networks. IEEE Wirel. Commun. 24(6), 55–63 (2018)
Zeng, D., Lin, G., Song, G., Cheng, Z., Shui, Y.: Joint optimization of task scheduling and image placement in fog computing supported software-defined embedded system. IEEE Trans. Comput. 65(12), 3702–3712 (2016)
Gu, L., Zeng, D., Guo, S., Barnawi, A., Xiang, Y.: Cost-efficient resource management in fog computing supported medical CPS. IEEE Trans. Emerg. Top. Comput. 99, 113–123 (2017)
Chi, K., Zhu, Y.H., Li, Y., Zhang, D., Leung, V.: Coding schemes to minimize energy consumption of communication links in wireless nanosensor networks. IEEE IoT J. 3(4), 480–493 (2016)
Marinescu, D.C.: Cloud Computing: Theory and Practice. Elsevier, Amsterdam (2013)
Kiani, A., Ansari, N.: Toward hierarchical mobile edge computing: an auction-based profit maximization approach. IEEE IoT J. 99, 2082–2091 (2016)
Nejad, M.M., Mashayekhy, L., Grosu, D.: A family of truthful greedy mechanisms for dynamic virtual machine provisioning and allocation in clouds. In: IEEE Sixth International Conference on Cloud Computing, pp. 594–603 (2013)
Mashayekhy, L., Nejad, M.M., Grosu, D.: A ptas mechanism for provisioning and allocation of heterogeneous cloud resources. IEEE Trans. Parallel Distrib. Syst. 26(9), 2386–2399 (2014)
Shi, W., Zhang, L., Wu, C., Li, Z., Lau, F., An online auction framework for dynamic resource provisioning in cloud computing. In: ACM SIGMETRICS Performance Evaluation Review, vol. 42, no. 1, pp. 71–83. ACM (2014)
Liu, X., Li, W., Zhang, X.: Strategy-proof mechanism for provisioning and allocation virtual machines in heterogeneous clouds. IEEE Trans. Parallel Distrib. Syst. 29(7), 1650–1663 (2017)
Zhang, J., Xie, N., Zhang, X., Li, W.: Supervised learning based truthful auction mechanism design in cloud computing. J. Electron. Inf. Technol. 41(5), 1243 (2019)
Zhang, J., Xie, N., Zhang, X., Li, W.: An online auction mechanism for cloud computing resource allocation and pricing based on user evaluation and cost. Future Gener. Comput. Syst. 89, 286–299 (2018)
Zhang, J., Xie, N., Li, W., Yue, K., Zhang, X.: Truthful multi requirements auction mechanism for virtual resource allocation of cloud computing. J. Electron. Inf. Technol. 40(1), 25–34 (2018)
Guo, Y., Liu, F., Xiao, N., Chen, Z.: Task-based resource allocation bid in edge computing micro datacenter. Comput. Mater. Continua 58, 777–792 (2019)
Zhang, J., Xie, N., Zhang, X.: Machine learning based resource allocation of cloud computing in auction. Comput. Mater. Continua 56, 123–135 (2018)
Nisan, E.B.N., Roughgarden, T., Tardos, E., Vazirani, V.V.: Algorithmic game theory. Commun. ACM 53(7), 78–86 (2009)
Mashayekhy, L., Fisher, N., Grosu, D.: Truthful mechanisms for competitive reward-based scheduling. IEEE Trans. Comput. 65(7), 2299–2312 (2016)
Azar, Y., Kalp-shaltiel, I., Lucier, B., Menache, I., Naor, J., Yaniv, J.: Truthful online scheduling with commitments. In: Sixteenth ACM Conference on Economics and Computation, pp. 715–732 (2015)
Porter, R.: Mechanism design for online real-time scheduling. In: Proceedings of the 5th ACM Conference on Electronic Commerce, pp. 61–70. ACM (2004)
Parkes, D.C., Singh, S.: An MDP-based approach to online mechanism design. In: International Conference on Neural Information Processing Systems, pp. 791–798 (2003)
Stein, S., Gerding, E., Robu, V., Jennings, N.R.: A model-based online mechanism with pre-commitment and its application to electric vehicle charging. In: International Conference on Autonomous Agents and Multiagent Systems (2012)
Sandholm, T.: Automated mechanism design: a new application area for search algorithms. In: International Conference on Principles and Practice of Constraint Programming, pp. 19–36 (2003)
Yamada, K.: Layered learning in multiagent systems: a winning approach to soccer. Syst. Control Inf. 45, 285 (2001)
Moore, A.W.: Variable resolution dynamic programming: efficiently learning action maps in multivariate real-valued state-spaces. In: Machine Learning Proceedings, vol. 1991, pp. 333–337. Elsevier (1991)
Jiang, G., Wu, C.: Learning to control an inverted pendulum using q-learning and neural networks. Acta Automatica Sinica 24, 666–669 (1998)
Crites, R.H., Barto, A.G.: Elevator group control using multiple reinforcement learning agents. Mach. Learn. 33(2–3), 235–262 (1998)
Hernandez-Leal, P., Kaisers, M., Baarslag, T., de Cote, E.M.: A survey of learning in multiagent environments: dealing with non-stationarity. arXiv preprint arXiv:1707.09183 (2017)
Mao, H., Alizadeh, M., Menache, I., Kandula, S.: Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks, pp. 50–56. ACM (2016)
Mashayekhy, L., Nejad, M.M., Grosu, D., Vasilakos, A.V.: An online mechanism for resource allocation and pricing in clouds. IEEE Trans. Comput. 65(4), 1172–1184 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, Q., Yang, X., Chi, L., Zhang, X., Zhang, J. (2020). Reinforcement Learning-Based Resource Allocation in Edge Computing. In: Sun, X., Wang, J., Bertino, E. (eds) Artificial Intelligence and Security. ICAIS 2020. Lecture Notes in Computer Science(), vol 12240. Springer, Cham. https://doi.org/10.1007/978-3-030-57881-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-57881-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-57880-0
Online ISBN: 978-3-030-57881-7
eBook Packages: Computer ScienceComputer Science (R0)