JET: Electricity cost-aware dynamic workload management in geographically distributed datacenters
Introduction
As the demand for resilient and low-latency cloud services increases, cloud service providers, such as Amazon, Google, and Microsoft, have been rapidly deploying and expanding their geographically distributed datacenters. A recent report indicates that the electricity demand of worldwide datacenters increased by about 56% from 2005 to 2010, and the electricity usage of datacenters accounted for 1.1%–1.5% of the worldwide electricity usage in 2010 [1]. Generally, a datacenter spends 30%–50% of its operational expense toward electricity [2]. Therefore, cutting down on the electricity cost of geographically distributed datacenters has become a major effort of cloud service providers.
Many studies have sought the optimal datacenter workload management scheme for reducing the electricity cost of geographically distributed datacenters. Some studies focused on intra-datacenter workload management, which decreases the power consumption of active servers in a datacenter through dynamic server demand management mechanisms [3], [4]. Since the power consumption of a cooling system can take up to 50% of the total power consumption of a datacenter [5], other studies considered the impact of datacenter workload distribution on the power consumption of a cooling system and proposed intra-datacenter workload management mechanisms to reduce the power consumption of a cooling system [6], [7]. Due to the location and time diversities of electricity prices in the United States, the inter-datacenter workload management mechanism was proposed to minimize the electricity cost1 of active servers of geographically distributed datacenters by periodically distributing workload to datacenters with lower electricity prices [8], [9].
In recent publications, server facilities and cooling systems are jointly considered to lower the electricity cost of geographically distributed datacenters [10], [11]. However, these works only consider a cooling system as a device consuming constant power. They neglect variant efficiency of a cooling system resulting from diverse workload distribution in a datacenter. [12] exploited the impact of workload distribution in a datacenter on the efficiency of a cooling system and proposed that a datacenter can be divided into three nearly temperature-isolated zones (i.e., cool, warm, and hot zones) based on air-flow patterns in the datacenter. The power consumption for cooling the same number of servers in the hot zone is larger than that for the other two zones. Therefore, the workload distribution in different zones of a datacenter significantly affects the power consumption of a cooling system, and it should be taken into consideration as an important factor by cloud service providers to reduce the electricity cost of cooling systems of geographically distributed datacenters.
In this paper, we propose a joint inter- and intra-datacenter workload management scheme, Joint ElectriciTy price-aware and cooling efficiency-aware load balancing (JET), to minimize the total electricity cost of geographically distributed datacenters. We first model the electricity cost of active servers as a function of electricity prices and the number of active servers. We also model the electricity cost of a cooling system as a function of Computer Room Air Conditioner (CRAC) output temperature. Based on these models, we propose an electricity cost minimization model of geographically distributed datacenters that effectively integrates both electricity price and cooling system management, and formulates the Electricity Cost Minimization (ECM) problem of geographically distributed datacenters as a constrained nonlinear optimization problem, subject to Quality of Service (QoS) constraints. We then simplify the complicated ECM problem to the Transformed ECM (TECM) problem (a convex optimization problem with linear constraints) by reasonable transformations and assumptions and solve the TECM problem with the proposed scheme JET. Extensive evaluations based on real-life workload trace and electricity prices of multiple datacenter locations show that JET calculates the optimal workload distribution with a short processing time and substantially reduces the electricity cost of geographically distributed datacenters, as compared with existing schemes. To the best of our knowledge, our work presents the first study on cutting the electricity cost of geographically distributed datacenters with a joint inter- and intra-datacenter workload management scheme.
The major contributions of this paper are summarized as follows:
- 1.
We build a model for the efficiency of a cooling system in a datacenter with respect to CRAC output temperature and apply this model to geographically distributed datacenters. Using this model, we further propose the ECM problem through explicitly enforcing the impact of workload distribution in a datacenter on the efficiency of a cooling system.
- 2.
We make reasonable transformations and assumptions on variables and constraints to transform the ECM problem (a complicated constrained nonlinear optimization problem) to the TECM problem (a convex optimization problem with linear constraints). We solve the TECM problem with the proposed JET, which dynamically dispatches incoming service requests2 to active servers in three temperature-isolated zones of geographically distributed datacenters by jointly considering time-varying locational electricity prices and the impact of workload distribution in a datacenter on the efficiency of a cooling system.
- 3.
We evaluate the performance of JET against existing schemes based on real-life traces. Extensive evaluations show that JET performs much better than existing schemes and substantially reduces the electricity cost of geographically distributed datacenters. Compared with service deadlines in real-life applications, the processing time of JET is short, and it changes very slightly as the amount of the workload changes.
The rest of the paper is organized as follows. Section 2 discusses the motivation for this paper. Section 3 presents the system framework. Section 4 describes electricity cost models and QoS constraints used for problem formulation. Section 5 formally proposes the ECM problem and proposes JET to solve the problem. Section 6 discusses the evaluation strategy and compares JET with existing schemes using real-life traces. Section 7 reviews the related work and Section 8 concludes the paper.
Section snippets
Variation of electricity prices
Electricity is produced by government utilities and independent power producers from different sources, such as coal, natural gas, and renewable energy. Providers and consumers of electric power are usually connected to an electricity grid, which is a complex electricity transmission and distribution network. Consider the United States as an example. Its electricity grid is divided into eight regional grids, each of which is operated and managed by a Regional Transmission Organization (RTO).
System architecture
This section provides a high-level description of our system architecture. In this paper, we assume our system is a centralized system that manages a datacenter network for minimizing the electricity cost. While such a centralized architecture is commonly used in the management of geographically distributed datacenters [8], [9], our system can be extended to work in a hierarchical/distributed way, which is our future work. Fig. 5 shows our system architecture, which includes three components:
Electricity cost models and QoS constraints of geographically distributed datacenters
This section presents the modeling for electricity cost of active servers and cooling systems of geographically distributed datacenters in detail. We also models different components for QoS constraints of geographically distributed datacenters.
Problem formulation and solution
In this section, we first formulate the electricity cost minimization problem of geographically distributed datacenters as a constrained nonlinear optimization problem. We then transform the complicated original problem to a convex problem with linear constraints and finally solve this problem with the proposed scheme, JET.
Evaluation
In this section, we use real-life workload and electricity price traces to evaluate the performance of JET against the existing schemes on electricity cost reduction of geographically distributed datacenters at two specific situations. To understand the cost reduction, we analyze the composition of electricity cost, the variation of COPs, and the ratio of the number of active servers and the maximum number of servers on different schemes in detail. These evaluations are primarily targeted
Related work
Electricity cost has become the major factor in the operational cost of geographically distributed datacenters. Many efforts have tried to reduce the electricity cost of geographically distributed datacenters by optimal datacenter workload management mechanisms. Some studies focused on the reduction of the power consumption of active servers in a datacenter by intra-datacenter workload management. Elnozahy et al. [4] reduced the aggregate power consumption of server farms by dynamically
Conclusion
One of the key questions faced by many cloud service providers is how to reduce the electricity cost of geographically distributed datacenters. Current studies mainly focus on either the impact of variant electricity prices on the electricity cost of active servers or the impact of datacenter workload distribution in a datacenter on the electricity cost of a cooling system, but neglect the joint optimization of these two aspects. In this paper, we propose a joint inter- and intra-datacenter
Acknowledgments
We thank Junjie Zhang at the New York University Polytechnic School of Engineering, for his help on the problem formulation.
References (30)
Growth in Data Center Electricity Use 2005 to 2010
(2011)- et al.
The datacenter as a computer: an introduction to the design of warehouse-scale machines
Synth. Lect. Comput. Archit.
(2009) - J. Heo, D. Henriksson, X. Liu, T. Abdelzaher, Integrating adaptive components: an emerging challenge in...
- et al.
Energy-efficient server clusters
Power-Aware Comput. Syst.
(2003) - U.E.P. Agency, Report to congress on server and data center energy efficiency, Public law 109-431,...
- et al.
Balance of power: dynamic thermal management for internet data centers
IEEE Internet Comput.
(2005) - C. Bash, G. Forman, Cool job allocation: measuring the power savings of placing jobs at cooling-efficient locations in...
- et al.
Cutting the electric bill for internet-scale systems
ACM SIGCOMM Comput. Commun. Rev.
(2009) - L. Rao, X. Liu, L. Xie, W. Liu, Minimizing electricity cost: optimization of distributed internet data centers in a...
- et al.
Towards optimal electric demand management for internet data centers
IEEE Trans. Smart Grid
(2012)
Joint optimization of idle and cooling power in data centers while maintaining response time
Routing, Flow, and Capacity Design in Communication and Computer Networks
Cited by (31)
Spatial and thermal aware methods for efficient workload management in distributed data centers
2024, Future Generation Computer SystemsSmartFCT: Improving power-efficiency for data center networks with deep reinforcement learning
2020, Computer NetworksCitation Excerpt :High power consumption of data centers has become a big concern for data center operators [1].
Crystal: A scalable and fault-tolerant Archimedean-based server-centric cloud data center network architecture
2019, Computer CommunicationsVirtual machine allocation and migration based on performance-to-power ratio in energy-efficient clouds
2019, Future Generation Computer SystemsSecure virtual machine placement in cloud data centers
2019, Future Generation Computer SystemsCitation Excerpt :Finally, in Section 6, we conclude the paper and provide some directions for future research work. Recent works [11,12] have focused on increasing the power-efficiency and thereby reducing the operational cost of data centers. However, the problem of increasing resource efficiency while maintaining co-location security is still a major issue.
Fog-Based Data Distribution Service (F-DAD) for Internet of Things (IoT) applications
2019, Future Generation Computer Systems