Abstract
Multi-Access Edge Computing (MEC) has emerged as a promising new paradigm allowing low latency access to services deployed on edge servers to avert network latencies often encountered in accessing cloud services. A key component of the MEC environment is an auto-scaling policy which is used to decide the overall management and scaling of container instances corresponding to individual services deployed on MEC servers to cater to traffic fluctuations. In this work, we propose a Safe Reinforcement Learning (RL)-based auto-scaling policy agent that can efficiently adapt to traffic variations to ensure adherence to service specific latency requirements. We model the MEC environment using a Markov Decision Process (MDP). We demonstrate how latency requirements can be formally expressed in Linear Temporal Logic (LTL). The LTL specification acts as a guide to the policy agent to automatically learn auto-scaling decisions that maximize the probability of satisfying the LTL formula. We introduce a quantitative reward mechanism based on the LTL formula to tailor service specific latency requirements. We prove that our reward mechanism ensures convergence of standard Safe-RL approaches. We present experimental results in practical scenarios on a test-bed setup with real-world benchmark applications to show the effectiveness of our approach in comparison to other state-of-the-art methods in literature. Furthermore, we perform extensive simulated experiments to demonstrate the effectiveness of our approach in large scale scenarios.
- [1] 2020. Existing Commercial Wireless Telecommunication Services Facilities in San Francisco: DataSF: City and County of San Francisco. (
May 2020). https://data.sfgov.org/Geographic-Locations-and-Boundaries/Existing-Commercial-Wireless-Telecommunication-Ser/aa26-h926.Google Scholar - [2] . 2018. Safe reinforcement learning via shielding. In Proceedings of the AAAI Conference on Artificial Intelligence. Google ScholarDigital Library
- [3] . 2020. Risk-aware data offloading in multi-server multi-access edge computing environment. IEEE/ACM Transactions on Networking 28, 3 (2020), 1405–1418.Google ScholarDigital Library
- [4] . 2019. Hierarchical game-theoretic and reinforcement learning framework for computational offloading in UAV-enabled mobile edge computing networks with multiple service providers. IEEE Internet of Things Journal 6, 5 (2019), 8753–8769.Google ScholarCross Ref
- [5] . 2008. Principles of Model Checking. The MIT Press. Google ScholarDigital Library
- [6] . 2008. Introduction to probability, ser. Athena Scientific Optimization and Computation Series. Athena Scientific.Google Scholar
- [7] . 2020. YOLOv4: Optimal Speed and Accuracy of Object Detection. (2020). arXiv:cs.CV/2004.10934. Retrieved from https://arxiv.org/abs/2004.10934.Google Scholar
- [8] . 2020. Control synthesis from linear temporal logic specifications using model-free reinforcement learning. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation. IEEE, 10349–10355.Google ScholarCross Ref
- [9] . 2019. Multi-user multi-task computation offloading in green mobile edge cloud computing. IEEE Transactions on Services Computing 12, 5 (2019), 726–738.Google ScholarCross Ref
- [10] . 2015. Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Transactions on Networking 24, 5 (2015), 2795–2808. Google ScholarDigital Library
- [11] . 2020. Trading off between user coverage and network robustness for edge server placement. IEEE Transactions on Cloud Computing (2020).Google ScholarCross Ref
- [12] . 2020. An efficient service dispersal mechanism for fog and cloud computing using deep reinforcement learning. In Proceedings of the 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing. IEEE, 589–598.Google Scholar
- [13] . 2018. Learning for computation offloading in mobile edge computing. IEEE Transactions on Communications 66, 12 (2018), 6353–6367.Google ScholarCross Ref
- [14] . 2020. Docker Hub Retrieved from https://hub.docker.com/.Google Scholar
- [15] . 2018. Performance modelling and verification of cloud-based auto-scaling policies. Future Generation Computer Systems 87 (2018), 629–638.Google ScholarCross Ref
- [16] . 2019. Service placement and request scheduling for data-intensive applications in edge clouds. In Proceedings of the IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 1279–1287.Google ScholarCross Ref
- [17] . 2013. Mobile cloud computing: A survey. Future Generation Computer Systems 29, 1 (2013), 84–106. Google ScholarDigital Library
- [18] . 2019. An open-source benchmark suite for microservices and their hardware-software implications for cloud and edge systems. In Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 3–18. Google ScholarDigital Library
- [19] . 2019. Leveraging deep learning to improve performance predictability in cloud microservices with seer. ACM SIGOPS Operating Systems Review 53, 1 (2019), 34–39. Google ScholarDigital Library
- [20] . 2019. Seer: Leveraging big data to navigate the complexity of performance debugging in cloud microservices. In Proceedings of the T24th International Conference on Architectural Support for Programming Languages and Operating Systems. 19–33. Google ScholarDigital Library
- [21] . 2020. Cautious reinforcement learning with logical constraints. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, AAMAS. 483–491. Google ScholarDigital Library
- [22] . 2017. Enhancing datacenter resource management through temporal logic constraints. In Proceedings of the 2017 IEEE International Parallel and Distributed Processing Symposium. IEEE, 133–142.Google ScholarCross Ref
- [23] . 2019. A game-theoretical approach for user allocation in edge computing environment. IEEE Transactions on Parallel and Distributed Systems 31, 3 (2019), 515–529.Google ScholarCross Ref
- [24] . 2020. Game-based task offloading of multiple mobile devices with QoS in mobile edge computing systems of limited computation capacity. ACM Transactions on Embedded Computing Systems 19, 4, Article
29 (July 2020), 21 pages. Google ScholarDigital Library - [25] . 2019. Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Transactions on Mobile Computing 19, 11 (2019), 2581–2593.Google ScholarCross Ref
- [26] . 2018. Machine learning and intelligent communications. Mobile Networks and Applications 23, 1 (2018), 68–70. Google ScholarDigital Library
- [27] . 2018. Optimal edge user allocation in edge computing with variable sized vector bin packing. In Proceedings of the International Conference on Service-Oriented Computing. Springer, 230–245.Google ScholarDigital Library
- [28] . 2020. Cost-effective app user allocation in an edge computing environment. IEEE Transactions on Cloud Computing (2020).Google ScholarCross Ref
- [29] . 2021. Heuristic computation offloading algorithms for mobile users in fog computing. ACM Transactions on Embedded Computing Systems 20, 2 (2021), 1–28. Google ScholarDigital Library
- [30] . 2020. Amoeba: QoS-awareness and reduced resource usage of microservices with serverless computing. In Proceedings of the 2020 IEEE International Parallel and Distributed Processing Symposium. IEEE, 399–408.Google ScholarCross Ref
- [31] . 2019. Optimizing serverless computing: Introducing an adaptive function placement algorithm. In Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering. 203–213. Google ScholarDigital Library
- [32] . 2015. Dependable horizontal scaling based on probabilistic model checking. In Proceedings of the 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE, 31–40. Google ScholarDigital Library
- [33] . 2018. ULOOF: A user level online offloading framework for mobile edge computing. IEEE Transactions on Mobile Computing 17, 11 (2018), 2660–2674.Google ScholarCross Ref
- [34] . 2019. Adaptive user-managed service placement for mobile edge computing: An online learning approach. In Proceedings of the IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 1468–1476.Google ScholarCross Ref
- [35] . 2019. Service placement with provable guarantees in heterogeneous edge computing systems. In Proceedings of the IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 514–522.Google ScholarCross Ref
- [36] . 2019. Mobility-aware and migration-enabled online edge user allocation in mobile edge computing. In Proceedings of the 2019 IEEE International Conference on Web Services. IEEE, 91–98.Google ScholarCross Ref
- [37] . 2020. Service placement and request routing in MEC networks with storage, computation, and communication constraints. IEEE/ACM Transactions on Networking 28, 3 (2020), 1047–1060.Google ScholarDigital Library
- [38] . 2019. Horizontal and vertical scaling of container-based applications using reinforcement learning. In Proceedings of the 2019 IEEE 12th International Conference on Cloud Computing. IEEE, 329–338.Google ScholarCross Ref
- [39] . 2014. A learning based approach to control synthesis of markov decision processes for linear temporal logic specifications. In Proceedings of the 53rd IEEE Conference on Decision and Control. IEEE, 1091–1096.Google ScholarCross Ref
- [40] . 2016. Edge computing: Vision and challenges. IEEE Internet of Things 3, 5 (2016), 637–646.Google ScholarCross Ref
- [41] . 2018. Reinforcement Learning: An Introduction. The MIT Press. Google ScholarDigital Library
- [42] . 2020. Metis: Learning to schedule long-running applications in shared container clusters at scale. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Google ScholarDigital Library
- [43] . 2019. Dynamic service migration in mobile edge computing based on markov decision process. IEEE/ACM Transactions on Networking 27, 3 (2019), 1272–1288. Google ScholarDigital Library
- [44] . 2020. Cloud-edge orchestration for the internet-of-things: Architecture and ai-powered data processing. IEEE Internet of Things Journal 8, 16 (2020), 12792–12805.Google ScholarCross Ref
- [45] . 2021. Constrained app data caching over edge server graphs in edge computing environment. IEEE Transactions on Services Computing (2021).Google ScholarCross Ref
- [46] . 2020. Online collaborative data caching in edge computing. IEEE Transactions on Parallel and Distributed Systems 32, 2 (2020), 281–294.Google ScholarCross Ref
- [47] . 2017. Online learning for offloading and autoscaling in energy harvesting mobile edge computing. IEEE Transactions on Cognitive Communications and Networking 3, 3 (2017), 361–373.Google ScholarCross Ref
- [48] . 2018. Joint service caching and task offloading for mobile edge computing in dense networks. In Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 207–215.Google ScholarCross Ref
- [49] . 2020. RJCC: Reinforcement-learning-based joint communicational-and-computational resource allocation mechanism for smart city IoT. IEEE Internet of Things Journal 7, 9 (2020), 8059–8076.Google ScholarCross Ref
- [50] . 2020. A joint optimization scheme for task offloading and resource allocation based on edge computing in 5G communication networks. Computer Communications 160 (2020), 759–768.Google ScholarCross Ref
- [51] . 2016. Energy-efficient resource allocation for mobile-edge computation offloading. IEEE Transactions on Wireless Communications 16, 3 (2016), 1397–1411.Google ScholarDigital Library
Index Terms
- Horizontal Auto-Scaling for Multi-Access Edge Computing Using Safe Reinforcement Learning
Recommendations
Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey
Web application providers have been migrating their applications to cloud data centers, attracted by the emerging cloud computing paradigm. One of the appealing features of the cloud is elasticity. It allows cloud users to acquire or release computing ...
Auto-scaling techniques in container-based cloud and edge/fog computing: Taxonomy and survey
AbstractThe long-held dream of computing as a service was realized with the emergence of cloud computing. Recently, fog and edge computing have been introduced as extensions of cloud networks, providing networking, processing, data management, ...
All one needs to know about fog computing and related edge computing paradigms: A complete survey
AbstractWith the Internet of Things (IoT) becoming part of our daily life and our environment, we expect rapid growth in the number of connected devices. IoT is expected to connect billions of devices and humans to bring promising advantages ...
Comments