A game theory-based dynamic resource allocation strategy in Geo-distributed Datacenter Clouds

https://doi.org/10.1016/j.future.2017.04.046Get rights and content

Highlights

  • We propose an traffic interaction model to discloses the traffic interaction between Geo-distributed datacentre clouds and physical networks.

  • We propose a resource allocation model among datecenters for the Geo-distributed Datacenter Cloud systems.

  • We propose an optimum resource allocation strategy that can allocate the system resources among different links.

Abstract

Geo-distributed Datacenter Cloud is an effective solution to store, process and transfer the big data produced by Internet-of-Things (IoT). A key challenge in this distributed system is how to allocate the bandwidth resources among these geo-distributed datacenters of this cloud efficiently. This paper aims to address this challenge by optimizing the transfer bandwidth resources among different geo-distributed datacenters. To this end, we firstly analyze the interaction between the traffic of physical networks and the data flow of Geo-distributed Datacenter Clouds, and then establish a game theory-based model for cloud resource allocation. Based on this model, a dynamic resource allocation strategy and its corresponding algorithm that are adaptable to the Internet conditions are proposed. Since the background traffic, capacity limit of physical networks as well as the flows and resource demands of geo-distributed datacenters are taken into account, this new strategy can achieve the load balance of the physical networks and content transferring among different geo-distributed datacenters effectively. The real-world trace data is adopted to validate the effectiveness and efficiency of the proposed resource allocation strategy. Compared with existing strategies, the evaluation results demonstrate that our proposed strategy can balance the workloads of physical networks, reduce the response delay of cloud applications, and possess an excellent adaptability.

Introduction

As an important and challenging evolution of Future Internet, Internet-of-Things (IoT) [1], [2] has attracted significant attention from both industry and academia. The main idea of it is to connect things to Internet with Internet technologies to store, manage, transfer and process the data produced by Things, making Things more smart and convenient for people to handle [3]. Due to these features, IoT was developed rapidly and became more and more popular. According to Cisco IoT White Paper, nowadays, there are 9 billion electronic devices connecting with the Internet, but this figure is expected to reach 50 billion by 2020 [4]. The rapid development of IoT also poses great challenges. Firstly, huge data generated by IoT scales much faster than CPU, so new technologies is required to handle the heterogeneous structured, semi-structured and unstructured data. Secondly, Requirements such as real-time services in some IoT applications should be met, which means that high QoS services should be provided for these applications.

To address these challenges, Geo-distributed Datacenter Cloud Computing has been introduced to process huge data generated in IoT [5]. Geo-distribution of datacenters of this kind clouds makes it possible to collect data quickly from different data sources. Besides, with the virtues of VM technologies, all resources such as CPU, bandwidth and storage hosted on interconnected servers in multiple datacenters are integrated into one virtual cloud platform to deploy the big data streaming processing and management applications based on the pay-as-you-go model. For instance, sensing service providers can access Internet and store their data using Cloud Computing, analytic tool developers can deploy their data mining and analyzing softwares in converting information to knowledge. In addition, new parallel and distributed algorithms can be developed and adopted to process the huge amount of data effectively [6].

However, most datacenters of one cloud are located in the networks offered by different Internet Service Providers (ISPs). In such a heterogeneous network environment, it is a great challenge to allocate the bandwidth resources among these datacenters efficiently. As one hot issue of Cloud Computing, the bandwidth resource allocation have been well investigated. For example, Beloglazov et al. [7] proposed an energy-aware resource allocation algorithm, which could efficiently assign datacenter resources to client applications without violating the negotiated Service-Level Agreements (SLAs). By applying virtualization technologies, Xiao et al. [8] developed a resource allocation system, which could avoid overload in cloud systems effectively while minimize the number of servers. Warneke and Kao  [9] discussed the opportunities and challenges in efficient parallel data processing in clouds and proposed a data processing framework to exploit the dynamic resource allocation for both task scheduling and execution. Teng and Magoul [10] analyzed the purchasing and consuming manners between providers and users and presented a Bayesian Nash equilibrium allocation algorithm for resource management in Cloud Computing. Obviously, these works focused on the bandwidth resource allocation in centralized cloud systems and did not yield their excepted performances under distributed environment. So recently, some works focused on the resource allocation under distributed environment. J. Guo et al. [11] focused on the bandwidth resource allocation among VMs in one datacenter and designed a distributed rate allocation algorithm based on the Logistic model for these VMs. These studies included two fields of cloud service applications: (1) profit-maximized resource allocation for cloud providers, (2) energy efficiency of datacenters and (3) quality of service guarantees for VMs. Then the resource allocation was formulated as optimization problems and the corresponding allocation strategies were designed for some specific condition. Although these strategies achieved their excepted performance under given conditions, they did not take Internet conditions into account. In fact, Internet condition, such as jitter, delay etc., which reflects the congestion of physical links and the load of network devices such as routers and switches, is one of the core factors who influence the performance of resource allocation. Like other overlay networks, one cloud, especially Geo-distributed Datacenter Clouds, should share the same physical communication devices with other Internet service applications such as P2P, multi-media services etc. Since these datacenters are interconnected through Internet, their response delay is affected by the efficiency of the content transfer, which replies on the bandwidth resource allocation among these datacenters. Specifically, resource allocation of cloud computing includes two steps: mapping and scheduling. Mapping aims to locate the resources of application layer to the network elements such as servers, routers and links of physical network, while scheduling assigns virtual resource of clouds to these elements. Moreover, both mapping and scheduling depend on the physical conditions (such as congestion of links and load of nodes, etc.). Therefore, these resource allocation strategies without considering the Internet conditions cannot achieve their desirable performance.

This paper targets to develop an effective resource management strategy in Geo-distributed Datacenter Clouds. As we well know, all overlay network service applications share the same physical network bandwidth resources, but few ISPs publish their traffic management and scheduling strategies. As a result, each of these overlay networks will optimize their content transferring independently. Thus, the bandwidth resource allocation among these datacenters can be considered as a noncooperative game among ISPs and all Overlay networks. If we take the traffic of other overlay networks flowing on one physical link as background traffic, the resource allocation among Geo-distributed Datacenters of this Cloud can be formulated as a noncooperative no zero-sum game with limited link capacities. Thus, M/M/1 queue theory is introduced to investigate the bandwidth resource sharing for all Internet service applications, and an interaction model, which takes the data flows of this cloud, the background traffic and link capacity limit of the physical networks into account, is proposed to disclose the relationship between content transfer efficiency and bandwidth resource consumption of physical links. Then the game theory is applied to our proposed interaction model, and our resource allocation model is proposed to optimize the service performance of both the Internet and Geo-distributed Datacenters Clouds. Based on this model, a dynamic resource allocation algorithm adaptable to the Internet conditions is proposed to allocate the cloud resources effectively. Our resource allocation algorithm treats the traffic of other Overlay networks as background traffic, this means our algorithm can be carried out with measurement data such as service delay, and the traffic management and scheduling strategies of ISPs are not necessary. Furthermore, our algorithm synthesizes Internet conditions and cloud operating states, leading to the global optimization of resource allocation strategy for Geo-distributed Datacenter Clouds. Thus, the proposed resource allocation strategy in the paper not only allocates the bandwidth resources among geo-distributed datacenters, but also achieves the load balance of Internet, voiding the problem of local overload in Internet. The major contributions of this paper are summarized as follows:

(1) We investigate the relationship between Geo-distributed Datacenters Clouds and physical networks, and propose an interaction model in order to disclose the interaction among the data flows of Geo-distributed Datacenter Clouds, the background traffic of the physical networks and the link capacity limit of the physical networks.

(2) Based on this interaction model and Game Theory, we establish a resource allocation model for Geo-distributed Datacenters Clouds and validate the effectiveness of this model.

(3) We propose an optimal resource allocation strategy that can allocate the system resources among different links efficiently according to the background traffic of each physical network link, the capacity limit of the link and the resource demand of Geo-distributed Datacenters Clouds.

(4) Based on this optimum strategy, we design a new dynamic resource allocation algorithm that periodically allocates the system resources according to the Internet conditions and cloud operation states. This algorithm can allocate the bandwidth resources among the paths of geo-distributed datacenters to transfer the content more quickly and effectively. Besides, our bandwidth resource allocation algorithm also can balance the load on the links of physical networks.

The rest of this paper is organized as follows. Section 2 reviews the related work on Datacenter Cloud Computing and resource allocation, respectively. Section 3 presents the system model and the framework of associated solutions, and formulates the resource allocation problem of Geo-distributed Datacenter Clouds. Section 4 investigates the interaction between the background traffic and the flow of Geo-distributed Datacenter Clouds. Then a dynamic resource allocation algorithm is developed accordingly. Section 5 evaluates the performance of the proposed algorithm via simulation experiments. Finally, Section 6 concludes this paper.

Section snippets

Preliminaries and related work

This section reviews the preliminaries and related work on Datacenter Cloud Computing and resource allocation.

The system model and resource allocation

This section presents the basic Geo-distributed Datacenter Cloud system model. We firstly describe our framework for the resource allocation of Geo-distributed Datacenter Clouds, investigate the relationship between these clouds and physical networks. Then Queuing Theory is used to develop an interaction model and the resource allocation problem is formulated (see Table 1).

Dynamic resource optimization among datacenters

In this session, we focus on how to allocate resources adaptively among different links to optimize the system performance.

Experiment validation and performance analysis

To evaluate the performance of the proposed algorithm under real network environments, we designed our inputs based on China Internet Development Report (the fourth quarter of 2012 and the fourth quarter of 2014) [37]. China Internet is the typical Internet consisting of more than a dozen heterogeneous networks, such as Mobile network, wireless networks etc., where the world largest population of Internet users or Things reside and the world’s largest e-commercial service platforms (Taobao),

Conclusion

Geo-distributed Datacenter clouds is widely known as the effective technology for IoT, but it is a great challenge to allocate the bandwidth resources among geo-distributed datacenters to achieve good performance. In this paper, we proposed a effective bandwidth resource allocation solution among different datacenters for Geo-distributed Datacenter Clouds. To this end, we firstly investigated the interaction between the background traffic of physical network and the flow of Geo-distributed

Acknowledgments

This paper is supported by National Social Science Foundation of China under the agreement of 71673208 and National Key Technology Support Program of China under the agreement of 2012BAH89F03.

Xiaoqun Yuan was an Associate Professor of Information Management in the school of Information Management at Wuhan University. His current research interests include Overlay Networks, Peer to Peer Systems, Live Streaming, Source Optimization and Digital Publishing.

References (38)

  • AtzoriaL. et al.

    The Internet of Things: a survey

    Comput. Netw.

    (2010)
  • GubbiaJ. et al.

    Internet of Things (IoT): A vision, architectural elements, and future directions

    Future Gener. Comput. Syst.

    (2013)
  • KortuemG. et al.

    Smart objects as building blocks for the Internet of Things

    IEEE Internet Comput.

    (2010)
  • D. Evans, The Internet of Things How the Next Evolution of the Internet is Changing Everything, available:...
  • LeeaI. et al.

    The Internet of Things (IoT): Applications, investments, and challenges for enterprises

    Bus. Horiz.

    (2015)
  • ZhangF. et al.

    Evolutionary scheduling of dynamic multitasking workloads for bigdata analytics in elastic cloud

    IEEE Trans. Emerg. Top. Comput.

    (2014)
  • BeloglazovA. et al.

    Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing

    Future Gener. Comput. Syst.

    (2012)
  • XiaoZ. et al.

    Dynamic resource allocation using virtual machines for cloud computing environment

    IEEE Trans. Parallel Distrib. Syst.

    (2013)
  • WarnekeD. et al.

    Exploiting dynamic resource allocation for efficient parallel data processing in the cloud

    IEEE Trans. Parallel Distrib. Syst.

    (2011)
  • Cited by (32)

    • Ayo game approach to mitigate free riding in peer-to-peer networks

      2022, Journal of King Saud University - Computer and Information Sciences
    • A reputation score policy and Bayesian game theory based incentivized mechanism for DDoS attacks mitigation and cyber defense

      2021, Future Generation Computer Systems
      Citation Excerpt :

      Apart from this, marginal utility based malicious user detection mechanism also has been proposed. In [31], authors have addressed the issue of allocation of bandwidth to geo-distributed cloud data centres. Interaction between traffic flow of physical network and cloud data centres has been modelled through game theory.

    • Evolutionary game for task mapping in resource constrained heterogeneous environments

      2020, Future Generation Computer Systems
      Citation Excerpt :

      Application of game theory in distributed computing domain is not new because it offers an efficient framework for tackling the task mapping problems. Specifically, game theory is already been used for: (i) efficient resource management (or load balancing) [10–17], (ii) energy optimization [18–20]; (iii) price optimization [21–23]. Interestingly, evolutionary game theory (EGT) provides a dynamical point-of-view on the standard game theory.

    • A comprehensive review and open issues on energy aware resource allocation in cloud

      2023, International Journal of Web Engineering and Technology
    View all citing articles on Scopus

    Xiaoqun Yuan was an Associate Professor of Information Management in the school of Information Management at Wuhan University. His current research interests include Overlay Networks, Peer to Peer Systems, Live Streaming, Source Optimization and Digital Publishing.

    Geyong Min is the Chair and Director of High Performance Computing and Networking (HPCN) Research Group at the University of Exeter, UK, United Kingdom. He received the Ph.D. degree in Computing Science from the University of Glasgow, United Kingdom, in 2003, and the B.Sc. degree in Computer Science from Huazhong University of Science and Technology, China, in 1995. His research interests include Next Generation Internet, Wireless Communications, Multimedia Systems, Information Security, Ubiquitous Computing, Modeling and Performance Engineering.

    Laurence T. Yang graduated from Tsinghua University, China and got his Ph.D. in Computer Science from University of Victoria, Canada. He joined St. Francis Xavier University in 1999. His current research includes parallel and distributed computing, embedded and ubiquitous/pervasive computing.

    Yi Ding now is an Associate Professor of Huazhong University of Science and Technology. He won the scholarship from China Scholarship Council and visited the University of Wollongong from 2009 to 2011. His main research field includes optoelectronics signal processing and streaming media resource allocation.

    Qing Fang is a Professor of Information Management in the school of Information Management at Wuhan University. He receives the Ph.D. degree and B.Sc. degree in information management from the Wuhan University, China. His research interests include Digital Publishing, Information resource management, Cloud Computing, Network Communication and Multimedia Analysis.

    View full text