Skip to main content

Advertisement

Springer Nature Link
Log in

A two-time-scale load balancing framework for minimizing electricity bills of Internet Data Centers

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Internet Data Center (IDC) is one of important emerging cyber-physical systems. To guarantee the quality of service for their worldwide users, large Internet service providers usually operate multiple geographically distributed IDCs. The enormous power consumption of these data centers may lead to both huge electricity bills and considerable carbon emissions. To mitigate these problems, on-site renewable energy plants are emerging in recent years. Since the renewable energy is intermittent, greening geographical load balancing (GGLB for short) has been proposed to reduce both the electricity bills and carbon emissions by following the renewables. However, GGLB is not able to well deal with the wildly fluctuating wind power when applied into IDCs with on-site wind energy plants. It may either fail to minimize the total electricity bills or incur the costly frequent on–off switching of servers. In order to minimize the total electricity bills of geographically distributed IDCs with on-site wind energy plants, we formulate the total electricity bills minimization problem and propose a novel two-time-scale load balancing framework TLB to solve it. First, TLB models the runtime cooling efficiency for each IDC. Then it predicts the future fine-grained (e.g., 10-min) on-site wind power output at the beginning of each scheduling period (e.g., an hour). After that, TLB transforms the primal optimization problem into a typical mixed-integer linear programming problem and solves it to finally obtain the optimal scheduling policy including the open server number as well as the request routing policy. It is worth noting that the open server number of each IDC will remain the same during each scheduling period. As an application instance of TLB, we also design a two-time-scale load balancing algorithm TLB-ARMA for our experimental scenario. Evaluation results based on real-life traces show that TLB-ARMA is able to reduce the total electricity bills by as much as 12.58 % compared with the hourly executed GGLB without incurring the costly repeated on–off switching of servers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Lee E et al (2008) Cyber physical systems: design challenges. In: 11th IEEE international symposium on object oriented real-time distributed computing (ISORC), 2008. IEEE, pp 363–369

  2. Rao L, Liu X, Ilic M, Liu J (2010) Mec-idc: joint load balancing and power control for distributed internet data centers. In: Proceedings of the 1st ACM/IEEE international conference on cyber-physical systems. ACM, pp 188–197

  3. Qureshi A, Weber R, Balakrishnan H, Guttag J, Maggs B (2009) Cutting the electric bill for internet-scale systems. ACM SIGCOMM Comput Commun Rev 39(4):123–134

    Article  Google Scholar 

  4. Google Green. http://www.google.com/green/

  5. America’s Data Centers Consuming and Wasting Growing Amounts of Energy. http://www.nrdc.org/energy/data-center-efficiency-assessment.asp

  6. Goiri Í, Katsak W, Le K, Nguyen TD, Bianchini R (2013) Parasol and greenswitch: managing datacenters powered by renewable energy. In: Proceedings of the eighteenth international conference on architectural support for programming languages and operating systems. ACM, pp 51–64

  7. Li C, Qouneh A, Li T (2012) iSwitch: coordinating and optimizing renewable energy powered server clusters. In: 39th annual international symposium on computer architecture (ISCA), 2012. IEEE, pp 512–523

  8. Ren C, Wang D, Urgaonkar B, Sivasubramaniam A (2012) Carbon-aware energy capacity planning for datacenters. In: IEEE 20th international symposium on modeling, analysis and simulation of computer and telecommunication systems (MASCOTS), 2012. IEEE, pp 391–400

  9. Apple Environmental Responsibility. http://www.apple.com/environment/

  10. Stewart C, Shen K (2009) Some joules are more precious than others: managing renewable energy in the datacenter. In: Proceedings of the workshop on power aware computing and systems

  11. Gao PX, Curtis AR, Wong B, Keshav S (2012) It’s not easy being green. ACM SIGCOMM Comput Commun Rev 42(4):211–222

    Article  Google Scholar 

  12. Liu Z, Lin M, Wierman A, Low SH, Andrew LL (2011) Greening geographical load balancing. In: Proceedings of the ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems. ACM, pp 233–244

  13. Wind Integration Datasets. http://www.nrel.gov/electricity/transmission/wind_integration_dataset.html

  14. Gandhi A, Doroudi S, Harchol-Balter M, Scheller-Wolf A (2013) Exact analysis of the m/m/k/setup class of Markov chains via recursive renewal reward. In: Proceedings of the ACM SIGMETRICS/international conference on measurement and modeling of computer systems. ACM, pp 153–166

  15. Gandhi A, Harchol-Balter M, Adan I (2010) Server farms with setup costs. Perform Eval 67(11):1123–1138

    Article  Google Scholar 

  16. Deng W, Liu F, Jin H, Liao X, Liu H, Chen L (2012) Lifetime or energy: Consolidating servers with reliability control in virtualized cloud datacenters. In: IEEE 4th international conference on cloud computing technology and science (CloudCom), 2012. IEEE, pp 18–25

  17. Guenter B, Jain N, Williams C (2011) Managing cost, performance, and reliability tradeoffs for energy-aware server provisioning. In: INFOCOM, 2011 Proceedings IEEE. IEEE, pp 1332–1340

  18. Xie T, Sun Y (2008) Sacrificing reliability for energy saving: is it worthwhile for disk arrays? In: IEEE international symposium on parallel and distributed processing, 2008. IPDPS 2008. IEEE, pp 1–12

  19. Horvath T, Abdelzaher T, Skadron K, Liu X (2007) Dynamic voltage scaling in multitier web servers with end-to-end delay control. IEEE Trans Comput 56(4):444–458

    Article  MathSciNet  Google Scholar 

  20. NYISO Pricing Data. http://www.nyiso.com/public/markets_operations/mar-ket_data/pricing_data/index.jsp

  21. Traces available in the Internet Traffic Archive. http://ita.ee.lbl.gov/html/traces.html

  22. Measurement and Instrumentation Data Center. http://www.nrel.gov/midc/

  23. Liu Z, Chen Y, Bash C, Wierman A, Gmach D, Wang Z, Marwah M, Hyser C (2012) Renewable and cooling aware workload management for sustainable data centers. In: ACM SIGMETRICS Perform Eval Rev 40(1):175–186

    Article  Google Scholar 

  24. Xu H, Feng C, Li B (2013) Temperature aware workload management in geo-distributed datacenters. In: Proceedings of the 10th USENIX international conference on autonomic computing (ICAC). USENIX, pp 303–314

  25. Rao L, Liu X, Xie L, Liu W (2010) Minimizing electricity cost: optimization of distributed internet data centers in a multi-electricity-market environment. In: INFOCOM, 2010 Proceedings IEEE. IEEE, pp 1–9

  26. Zhang Y, Wang Y, Wang X (2011) Greenware: greening cloud-scale data centers to maximize the use of renewable energy. In: ACM/IFIP/USENIX international conference on distributed systems platforms and open distributed processing. Springer, pp 143–164 

  27. Wei W, Xu Q, Wang L, Hei X, Shen P, Shi W, Shan L (2014) Gi/geom/1 queue based on communication model for mesh networks. Int J Commun Syst 27(11):3013–3029

    Google Scholar 

  28. Barroso LA, Clidaras J, Hölzle U (2013) The datacenter as a computer: an introduction to the design of warehouse-scale machines. Synth Lect Comput Archit 8(3):1–154

    Article  Google Scholar 

  29. Goiri Í, Nguyen TD, Bianchini R, Presa Í G (2015) Coolair: temperature-and variation-aware management for free-cooled datacenters. In: Proceedings of the twentieth international conference on architectural support for programming languages and operating systems. ACM, pp 253–265

  30. Aksanli B, Venkatesh J, Zhang L, Rosing T (2012) Utilizing green energy prediction to schedule mixed batch and service jobs in data centers. ACM SIGOPS Operating Systems Review 45(3):53–57

    Article  Google Scholar 

  31. Sharma N, Gummeson J, Irwin D, Zhu T, Shenoy P (2014) Leveraging weather forecasts in renewable energy systems. Sustain Comput Inform Syst 4(3):160–171

    Google Scholar 

  32. Soman SS, Zareipour H, Malik O, Mandal P (2010) A review of wind power and wind speed forecasting methods with different time horizons. In: North American Power Symposium (NAPS), 2010. IEEE, pp 1–8

  33. Linderoth JT, Savelsbergh MW (1999) A computational study of search strategies for mixed integer programming. INFORMS J Comput 11(2):173–187

    Article  MathSciNet  MATH  Google Scholar 

  34. Kirkpatrick S, Gelatt CD, Vecchi MP et al (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  MathSciNet  MATH  Google Scholar 

  35. Dorigo M, Birattari M, Stutzle T (2006) Ant colony optimization. IEEE Comput Intell Mag 1(4):28–39

    Article  Google Scholar 

  36. IBM ILOG CPLEX (2009) V12.1: users manual for cplex. Int Bus Mach Corpor 46(53):157

    Google Scholar 

  37. Löfberg J (2004) YALMIP: a toolbox for modeling and optimization in MATLAB. In: IEEE international symposium on computer aided control systems design, 2004. IEEE, pp 284–289

  38. EMERSON Network Power. http://www.emersonnetworkpower.com/documentation/en-US/Products/PrecisionCooling/LargeRoomCooling/Documents/SL-18927.pdf

  39. Milligan M, Schwartz M, Wan Y (2003) Statistical wind power forecasting models: results for us wind farms. National Renewable Energy Laboratory, Golden

    Google Scholar 

  40. Li Y, Yao M, Lin C (2013) Joint study on optimizations of data center deployment, VM assignment and migration. In: IEEE/ACM 21st international symposium on quality of service (IWQoS), 2013. IEEE, pp 1–10

  41. The Siemens G4 Platform. http://www.energy.siemens.com/hq/en/renewable-energy/wind-power/platforms/g4-platform/

  42. Sun Y, Jara AJ (2014) An extensible and active semantic model of information organizing for the internet of things. Pers Ubiquitous Comput 18(8):1821–1833

    Article  Google Scholar 

  43. Liu J, Jiang J, Cui X, Yang W, Liu X (2015) Power consumption prediction of web services for energy-efficient service selection. Pers Ubiquitous Comput 19(7):1063–1073

    Article  Google Scholar 

  44. Wang J, Kuang Q, Duan S (2015) A new online anomaly learning and detection for large-scale service of internet of thing. Pers Ubiquitous Comput 19(7):1021–1031

    Article  Google Scholar 

  45. Urgaonkar R, Urgaonkar B, Neely MJ, Sivasubramaniam A (2011) Optimal power cost management using stored energy in data centers. In: Proceedings of the ACM SIGMETRICS joint international conference on measurement and modeling of computer systems. ACM, pp 221–232

  46. Yu L, Chen L, Cai Z, Shen H, Liang Y, Pan Y (2016) Stochastic load balancing for virtual resource management in datacenters. IEEE Trans Cloud Comput PP(99):1

    Google Scholar 

  47. Huang R, Huang T, Gadh R, Li N (2012) Solar generation prediction using the arma model in a laboratory-level micro-grid. In: IEEE third international conference on smart grid communications (SmartGridComm), 2012. IEEE, pp 528–533

  48. Inman RH, Pedro HT, Coimbra CF (2013) Solar forecasting methods for renewable energy integration. Prog Energy Combust Sci 39(6):535–576

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant No. 61272460 and the Specialized Research Fund for the Doctoral Program of Higher Education under Grant No. 20120201110010. This work was also supported by Xi’an science and technology project (CXY1440(6)) and Beilin District 2012 High-tech Plan, Xi’an, China (No. GX1504) and supported by the Specialized Research Fund for the Doctoral Program of Higher Education of China (Grant No. 20136118120010).

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, China

    Hui Dou & Yong Qi

  2. School of Computer Science and Engineering, Xi’an University of Technology, Xi’an, China

    Wei Wei

  3. Department of Electrical and Computer Engineering, West Virginia University, Montgomery, WV, USA

    Houbing Song

Authors
  1. Hui Dou
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yong Qi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wei Wei
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Houbing Song
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Wei Wei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dou, H., Qi, Y., Wei, W. et al. A two-time-scale load balancing framework for minimizing electricity bills of Internet Data Centers. Pers Ubiquit Comput 20, 681–693 (2016). https://doi.org/10.1007/s00779-016-0941-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-016-0941-9

Keywords