Elsevier

Computer Communications

Volume 50, 1 September 2014, Pages 162-174
Computer Communications

JET: Electricity cost-aware dynamic workload management in geographically distributed datacenters

https://doi.org/10.1016/j.comcom.2014.02.011Get rights and content

Abstract

The ever-increasing operational cost of geographically distributed datacenters has become a critical issue for cloud service providers. To cut the electricity cost of geographically distributed datacenters, several workload management schemes have been proposed. These include Electricity price-aware InteR-datacenter load balancing (EIR), which reduces the electricity cost of active servers by dispatching the workload to datacenters with lower electricity prices, and Cooling-aware IntrA-datacenter load balancing (CIA), which decreases the power consumption of a datacenter by consolidating the workload on servers with high cooling efficiency. However, these existing schemes could incur some undesired results. For example, EIR may result in high electricity cost of cooling systems due to random workload distribution in datacenters. CIA could lead to high electricity cost of active servers since it does not consider the variation of electricity prices. In this paper, we propose a joint inter- and intra-datacenter workload management scheme, Joint ElectriciTy price-aware and cooling efficiency-aware load balancing (JET), to cut the electricity cost of geographically distributed datacenters. JET uses a short processing time to calculate the optimal workload distribution, which trades off the electricity cost of active servers and cooling systems by alternately selecting the electricity prices or the efficiency of a cooling system as the dominating factor to the electricity cost of geographically distributed datacenters. Extensive evaluations show that JET outperforms the existing schemes and achieves substantial reduction in the electricity cost of geographically distributed datacenters.

Introduction

As the demand for resilient and low-latency cloud services increases, cloud service providers, such as Amazon, Google, and Microsoft, have been rapidly deploying and expanding their geographically distributed datacenters. A recent report indicates that the electricity demand of worldwide datacenters increased by about 56% from 2005 to 2010, and the electricity usage of datacenters accounted for 1.1%–1.5% of the worldwide electricity usage in 2010 [1]. Generally, a datacenter spends 30%–50% of its operational expense toward electricity [2]. Therefore, cutting down on the electricity cost of geographically distributed datacenters has become a major effort of cloud service providers.

Many studies have sought the optimal datacenter workload management scheme for reducing the electricity cost of geographically distributed datacenters. Some studies focused on intra-datacenter workload management, which decreases the power consumption of active servers in a datacenter through dynamic server demand management mechanisms [3], [4]. Since the power consumption of a cooling system can take up to 50% of the total power consumption of a datacenter [5], other studies considered the impact of datacenter workload distribution on the power consumption of a cooling system and proposed intra-datacenter workload management mechanisms to reduce the power consumption of a cooling system [6], [7]. Due to the location and time diversities of electricity prices in the United States, the inter-datacenter workload management mechanism was proposed to minimize the electricity cost1 of active servers of geographically distributed datacenters by periodically distributing workload to datacenters with lower electricity prices [8], [9].

In recent publications, server facilities and cooling systems are jointly considered to lower the electricity cost of geographically distributed datacenters [10], [11]. However, these works only consider a cooling system as a device consuming constant power. They neglect variant efficiency of a cooling system resulting from diverse workload distribution in a datacenter. [12] exploited the impact of workload distribution in a datacenter on the efficiency of a cooling system and proposed that a datacenter can be divided into three nearly temperature-isolated zones (i.e., cool, warm, and hot zones) based on air-flow patterns in the datacenter. The power consumption for cooling the same number of servers in the hot zone is larger than that for the other two zones. Therefore, the workload distribution in different zones of a datacenter significantly affects the power consumption of a cooling system, and it should be taken into consideration as an important factor by cloud service providers to reduce the electricity cost of cooling systems of geographically distributed datacenters.

In this paper, we propose a joint inter- and intra-datacenter workload management scheme, Joint ElectriciTy price-aware and cooling efficiency-aware load balancing (JET), to minimize the total electricity cost of geographically distributed datacenters. We first model the electricity cost of active servers as a function of electricity prices and the number of active servers. We also model the electricity cost of a cooling system as a function of Computer Room Air Conditioner (CRAC) output temperature. Based on these models, we propose an electricity cost minimization model of geographically distributed datacenters that effectively integrates both electricity price and cooling system management, and formulates the Electricity Cost Minimization (ECM) problem of geographically distributed datacenters as a constrained nonlinear optimization problem, subject to Quality of Service (QoS) constraints. We then simplify the complicated ECM problem to the Transformed ECM (TECM) problem (a convex optimization problem with linear constraints) by reasonable transformations and assumptions and solve the TECM problem with the proposed scheme JET. Extensive evaluations based on real-life workload trace and electricity prices of multiple datacenter locations show that JET calculates the optimal workload distribution with a short processing time and substantially reduces the electricity cost of geographically distributed datacenters, as compared with existing schemes. To the best of our knowledge, our work presents the first study on cutting the electricity cost of geographically distributed datacenters with a joint inter- and intra-datacenter workload management scheme.

The major contributions of this paper are summarized as follows:

  • 1.

    We build a model for the efficiency of a cooling system in a datacenter with respect to CRAC output temperature and apply this model to geographically distributed datacenters. Using this model, we further propose the ECM problem through explicitly enforcing the impact of workload distribution in a datacenter on the efficiency of a cooling system.

  • 2.

    We make reasonable transformations and assumptions on variables and constraints to transform the ECM problem (a complicated constrained nonlinear optimization problem) to the TECM problem (a convex optimization problem with linear constraints). We solve the TECM problem with the proposed JET, which dynamically dispatches incoming service requests2 to active servers in three temperature-isolated zones of geographically distributed datacenters by jointly considering time-varying locational electricity prices and the impact of workload distribution in a datacenter on the efficiency of a cooling system.

  • 3.

    We evaluate the performance of JET against existing schemes based on real-life traces. Extensive evaluations show that JET performs much better than existing schemes and substantially reduces the electricity cost of geographically distributed datacenters. Compared with service deadlines in real-life applications, the processing time of JET is short, and it changes very slightly as the amount of the workload changes.

The rest of the paper is organized as follows. Section 2 discusses the motivation for this paper. Section 3 presents the system framework. Section 4 describes electricity cost models and QoS constraints used for problem formulation. Section 5 formally proposes the ECM problem and proposes JET to solve the problem. Section 6 discusses the evaluation strategy and compares JET with existing schemes using real-life traces. Section 7 reviews the related work and Section 8 concludes the paper.

Section snippets

Variation of electricity prices

Electricity is produced by government utilities and independent power producers from different sources, such as coal, natural gas, and renewable energy. Providers and consumers of electric power are usually connected to an electricity grid, which is a complex electricity transmission and distribution network. Consider the United States as an example. Its electricity grid is divided into eight regional grids, each of which is operated and managed by a Regional Transmission Organization (RTO).

System architecture

This section provides a high-level description of our system architecture. In this paper, we assume our system is a centralized system that manages a datacenter network for minimizing the electricity cost. While such a centralized architecture is commonly used in the management of geographically distributed datacenters [8], [9], our system can be extended to work in a hierarchical/distributed way, which is our future work. Fig. 5 shows our system architecture, which includes three components:

Electricity cost models and QoS constraints of geographically distributed datacenters

This section presents the modeling for electricity cost of active servers and cooling systems of geographically distributed datacenters in detail. We also models different components for QoS constraints of geographically distributed datacenters.

Problem formulation and solution

In this section, we first formulate the electricity cost minimization problem of geographically distributed datacenters as a constrained nonlinear optimization problem. We then transform the complicated original problem to a convex problem with linear constraints and finally solve this problem with the proposed scheme, JET.

Evaluation

In this section, we use real-life workload and electricity price traces to evaluate the performance of JET against the existing schemes on electricity cost reduction of geographically distributed datacenters at two specific situations. To understand the cost reduction, we analyze the composition of electricity cost, the variation of COPs, and the ratio of the number of active servers and the maximum number of servers on different schemes in detail. These evaluations are primarily targeted

Related work

Electricity cost has become the major factor in the operational cost of geographically distributed datacenters. Many efforts have tried to reduce the electricity cost of geographically distributed datacenters by optimal datacenter workload management mechanisms. Some studies focused on the reduction of the power consumption of active servers in a datacenter by intra-datacenter workload management. Elnozahy et al. [4] reduced the aggregate power consumption of server farms by dynamically

Conclusion

One of the key questions faced by many cloud service providers is how to reduce the electricity cost of geographically distributed datacenters. Current studies mainly focus on either the impact of variant electricity prices on the electricity cost of active servers or the impact of datacenter workload distribution in a datacenter on the electricity cost of a cooling system, but neglect the joint optimization of these two aspects. In this paper, we propose a joint inter- and intra-datacenter

Acknowledgments

We thank Junjie Zhang at the New York University Polytechnic School of Engineering, for his help on the problem formulation.

References (30)

  • J. Koomey

    Growth in Data Center Electricity Use 2005 to 2010

    (2011)
  • L. Barroso et al.

    The datacenter as a computer: an introduction to the design of warehouse-scale machines

    Synth. Lect. Comput. Archit.

    (2009)
  • J. Heo, D. Henriksson, X. Liu, T. Abdelzaher, Integrating adaptive components: an emerging challenge in...
  • E. Elnozahy et al.

    Energy-efficient server clusters

    Power-Aware Comput. Syst.

    (2003)
  • U.E.P. Agency, Report to congress on server and data center energy efficiency, Public law 109-431,...
  • R. Sharma et al.

    Balance of power: dynamic thermal management for internet data centers

    IEEE Internet Comput.

    (2005)
  • C. Bash, G. Forman, Cool job allocation: measuring the power savings of placing jobs at cooling-efficient locations in...
  • A. Qureshi et al.

    Cutting the electric bill for internet-scale systems

    ACM SIGCOMM Comput. Commun. Rev.

    (2009)
  • L. Rao, X. Liu, L. Xie, W. Liu, Minimizing electricity cost: optimization of distributed internet data centers in a...
  • J. Li et al.

    Towards optimal electric demand management for internet data centers

    IEEE Trans. Smart Grid

    (2012)
  • Y. Zhang, Y. Wang, X. Wang, Electricity bill capping for cloud-scale data centers that impact the power markets, in:...
  • F. Ahmad et al.

    Joint optimization of idle and cooling power in data centers while maintaining response time

  • D. Anderson, J. Dykes, E. Riedel, More than an interface – scsi vs. ata, in: FAST03, USENIX, 2003, pp....
  • J. Moore, J. Chase, P. Ranganathan, R. Sharma, Making scheduling cool: temperature-aware workload placement in data...
  • M. Pióro et al.

    Routing, Flow, and Capacity Design in Communication and Computer Networks

    (2004)
  • Cited by (31)

    • SmartFCT: Improving power-efficiency for data center networks with deep reinforcement learning

      2020, Computer Networks
      Citation Excerpt :

      High power consumption of data centers has become a big concern for data center operators [1].

    • Secure virtual machine placement in cloud data centers

      2019, Future Generation Computer Systems
      Citation Excerpt :

      Finally, in Section 6, we conclude the paper and provide some directions for future research work. Recent works [11,12] have focused on increasing the power-efficiency and thereby reducing the operational cost of data centers. However, the problem of increasing resource efficiency while maintaining co-location security is still a major issue.

    View all citing articles on Scopus
    View full text