Skip to main content

Advertisement

SpringerLink
Log in

A survey: ICT enabled energy efficiency techniques for big data applications

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The growth and development of the information and communication technology industry has led to a rapid rise in big data applications. With the development of cloud data centers, cloud computing serves as an appropriate approach for delivering services to these applications. Such centers are equipped with a large number of servers which consume high energy and thus energy efficiency has become a major concern. To achieve sustainability, it is imperative to construct green data centers. This paper surveys big data applications and related challenges in the cloud environment. Energy efficiency has been recognised as the prime concern, and the techniques to achieve it have been categorised as infrastructure, storage, analytical, networking, scheduling and hybrid. The limitations in each energy saving techniques have been discussed. The importance of performance parameters, along with the energy efficiency, has been highlighted. The article has been concluded with valuable insights for future enhancements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://www.datacenterknowledge.com/industry-perspectives/achieving-data-center-energy-efficiency

  2. https://www.forbes.com/sites/forbestechcouncil/2017/12/15/why-energy-is-a-big-and-rapidly-growing-problem-for-data-centers/#53e00df25a30

  3. http://www.us.jll.com/united-states/en-us/Research/US-North-America-Data-Center-Outlook-2016-JLL

  4. https://en.wikipedia.org/wiki/Exabyte

  5. https://www.cirrusinsight.com/blog/much-data-google-store

  6. https://energy.stanford.edu/news/data-centers-can-slash-CO2-emissions-88-or-more

  7. https://www.google.com/about/datacenters/efficiency/internal/

References

  1. Addo-Tenkorang, R., Helo, P.T.: Big data applications in operations/supply-chain management: a literature review. Comput. Ind. Eng. 101, 528–543 (2016)

    Google Scholar 

  2. Ahmad, R.W., Gani, A., Hamid, S.H.A., Shiraz, M., Yousafzai, A., Xia, F.: A survey on virtual machine migration and server consolidation frameworks for cloud data centers. J. Netw. Comput. Appl. 52, 11–25 (2015)

    Google Scholar 

  3. Aksanli, B., Venkatesh, J., Zhang, L., Rosing, T.: Utilizing green energy prediction to schedule mixed batch and service jobs in data centers. ACM SIGOPS Oper. Syst. Rev. 45(3), 53–57 (2012)

    Google Scholar 

  4. Alkhater, N., Walters, R., Wills, G.: An empirical study of factors influencing cloud adoption among private sector organisations. Telemat. Inform. 35(1), 38–54 (2018)

    Google Scholar 

  5. Atat, R., Liu, L., Wu, J., Li, G., Ye, C., Yang, Y.: Big data meet cyber-physical systems: a panoramic survey. IEEE Access 6, 73603–73636 (2018)

    Google Scholar 

  6. Baker, T., Al-Dawsari, B., Tawfik, H., Reid, D., Ngoko, Y.: Greedi: an energy efficient routing algorithm for big data on cloud. Ad Hoc Netw. 35, 83–96 (2015)

    Google Scholar 

  7. Baker, T., Asim, M., Tawfik, H., Aldawsari, B., Buyya, R.: An energy-aware service composition algorithm for multiple cloud-based iot applications. J. Netw. Comput. Appl. 89, 96–108 (2017)

    Google Scholar 

  8. Baker, T., García-Campos, J.M., Reina, D.G., Toral, S., Tawfik, H., Al-Jumeily, D., Hussain, A.: Greeaodv: an energy efficient routing protocol for vehicular ad hoc networks. In: International Conference on Intelligent Computing, pp. 670–681. Springer (2018)

  9. Baker, T., Ngoko, Y., Tolosana-Calasanz, R., Rana, O.F., Randles, M.: Energy efficient cloud computing environment via autonomic meta-director framework. In: 2013 Sixth International Conference on Developments in eSystems Engineering, pp. 198–203. IEEE (2013)

  10. Barbagallo, D., Di Nitto, E., Dubois, D.J., Mirandola, R.: A bio-inspired algorithm for energy optimization in a self-organizing data center. In: Self-Organizing Architectures, pp. 127–151. Springer (2010)

  11. Bello-Orgaz, G., Jung, J.J., Camacho, D.: Social big data: Recent achievements and new challenges. Inf. Fusion 28, 45–59 (2016)

    Google Scholar 

  12. Beloglazov, A., Buyya, R., Lee, Y.C., Zomaya, A., et al.: A taxonomy and survey of energy-efficient data centers and cloud computing systems. Adv. Comput. 82(2), 47–111 (2011)

    Google Scholar 

  13. Berl, A., Gelenbe, E., Di Girolamo, M., Giuliani, G., De Meer, H., Dang, M.Q., Pentikousis, K.: Energy-efficient cloud computing. Comput. J. 53(7), 1045–1051 (2010)

    Google Scholar 

  14. Bostoen, T., Mullender, S., Berbers, Y.: Power-reduction techniques for data-center storage systems. ACM Comput. Surv. CSUR 45(3), 33 (2013)

    Google Scholar 

  15. Bouley, D.: Estimating a data center’s electrical carbon footprint. Schneider Electric White Paper Library (2011)

  16. Buttazzo, G.C.: Scalable applications for energy-aware processors. In: EMSOFT, pp. 153–165. Springer (2002)

  17. Castro, P.H., Barreto, V.L., Corrêa, S.L., Granville, L.Z., Cardoso, K.V.: A joint cpu-ram energy efficient and sla-compliant approach for cloud data centers. Comput. Netw. 94, 1–13 (2016)

    Google Scholar 

  18. Dai, L., Gao, X., Guo, Y., Xiao, J., Zhang, Z.: Bioinformatics clouds for big data manipulation. Biol. Direct 7(1), 43 (2012)

    Google Scholar 

  19. Dayal, M., Singh, N.: Indian health care analysis using big data programming tool. Proc. Comput. Sci. 89, 521–527 (2016)

    Google Scholar 

  20. Dayarathna, M., Wen, Y., Fan, R.: Data center energy consumption modeling: a survey. IEEE Commun. Surv. Tutor. 18(1), 732–794 (2016)

    Google Scholar 

  21. Devadas, S., Malik, S.: A survey of optimization techniques targeting low power vlsi circuits. In: Proceedings of the 32nd annual ACM/IEEE Design Automation Conference, pp. 242–247. ACM (1995)

  22. Ebrahimi, K., Jones, G.F., Fleischer, A.S.: A review of data center cooling technology, operating conditions and the corresponding low-grade waste heat recovery opportunities. Renew. Sustain. Energy Rev. 31, 622–638 (2014)

    Google Scholar 

  23. Esfandiarpoor, S., Pahlavan, A., Goudarzi, M.: Structure-aware online virtual machine consolidation for datacenter energy improvement in cloud computing. Computers & Electrical Engineering 42, 74–89 (2015)

    Google Scholar 

  24. Fahim, M., Baker, T.: Knowledge-based decision support systems for personalized u-lifecare big data services. In: Current Trends on Knowledge-Based Systems, pp. 187–203. Springer (2017)

  25. Faroqi, M.G., Siddiquee, N.A., Ullah, S.: Sustainability of telecentres in developing countries: lessons from union digital centre in Bangladesh. Telemat. Inform. 37, 113–127 (2019)

    Google Scholar 

  26. Feller, E., Ramakrishnan, L., Morin, C.: Performance and energy efficiency of big data applications in cloud environments: a hadoop case study. J. Parallel Distrib. Comput. 79, 80–89 (2015)

    Google Scholar 

  27. Gautham, A., Korgaonkar, K., Slpsk, P., Balachandran, S., Veezhinathan, K.: The implications of shared data synchronization techniques on multi-core energy efficiency. In: HotPower, pp. 1–5

  28. Geist, A., Reed, D.A.: A survey of high-performance computing scaling challenges. Int. J. High Perform. Comput. Appl. 31(1), 104–113 (2017)

    Google Scholar 

  29. Gill, S.S., Buyya, R.: A taxonomy and future directions for sustainable cloud computing: 360 degree view. ACM Comput. Surv. CSUR 51(5), 104 (2018)

    Google Scholar 

  30. Guerra, J., Belluomini, W., Glider, J., Gupta, K., Pucha, H.: Energy proportionality for storage: Impact and feasibility. ACM SIGOPS Oper. Syst. Rev. 44(1), 35–39 (2010)

    Google Scholar 

  31. Hammadi, A., Mhamdi, L.: A survey on architectures and energy efficiency in data center networks. Comput. Commun. 40, 1–21 (2014)

    Google Scholar 

  32. Haridas, M.: Redefining Military Intelligence Using Big Data Analytics, pp. 72–78. Scholar Warrior, Autum (2015)

    Google Scholar 

  33. Higón, D.A., Gholami, R., Shirazi, F.: Ict and environmental sustainability: a global perspective. Telemat. Inform. 34(4), 85–95 (2017)

    Google Scholar 

  34. Hossain, M.S., Hoda, M., Muhammad, G., Almogren, A., Alamri, A.: Cloud-supported framework for patients in post-stroke disability rehabilitation. Telemat. Inform. 35(4), 826–836 (2018)

    Google Scholar 

  35. Ibrahim, S., Phan, T.D., Carpen-Amarie, A., Chihoub, H.E., Moise, D., Antoniu, G.: Governing energy consumption in hadoop through cpu frequency scaling: an analysis. Future Gener. Comput. Syst. 54, 219–232 (2016)

    Google Scholar 

  36. Kachris, C., Sirakoulis, G.C., Soudris, D.: A mapreduce scratchpad memory for multi-core cloud computing applications. Microprocess. Microsyst. 39(8), 599–608 (2015)

    Google Scholar 

  37. Kansal, N.J., Chana, I.: An empirical evaluation of energy-aware load balancing technique for cloud data center. Clust. Comput. 21(2), 1311–1329 (2018)

    Google Scholar 

  38. Karakoyunlu, C., Chandy, J.A.: Exploiting user metadata for energy-aware node allocation in a cloud storage system. J. Comput. Syst. Sci. 82(2), 282–309 (2016)

    MathSciNet  MATH  Google Scholar 

  39. Kaur, P.D., Chana, I.: A resource elasticity framework for qos-aware execution of cloud applications. Future Gener. Comput. Syst. 37, 14–25 (2014)

    Google Scholar 

  40. Kaur, T., Chana, I.: Energy efficiency techniques in cloud computing: A survey and taxonomy. ACM Comput. Surv. CSUR 48(2), 22 (2015)

    Google Scholar 

  41. Kaushik, R.T., Bhandarkar, M.: Greenhdfs: towards an energy-conserving, storage-efficient, hybrid hadoop compute cluster. In: Proceedings of the USENIX annual technical conference, vol. 109, p. 34 (2010)

  42. Koller, R., Verma, A., Neogi, A.: Wattapp: an application aware power meter for shared data centers. In: Proceedings of the 7th international conference on Autonomic computing, pp. 31–40. ACM (2010)

  43. Kumar, A., Bawa, S.: Generalized ant colony optimizer: swarm-based meta-heuristic algorithm for cloud services execution. Computing. (2018). https://doi.org/10.1007/s00607-018-0674-x

    Article  Google Scholar 

  44. Kusic, D., Kephart, J.O., Hanson, J.E., Kandasamy, N., Jiang, G.: Power and performance management of virtualized computing environments via lookahead control. Clust. Comput. 12(1), 1–15 (2009)

    Google Scholar 

  45. Li, C., Zhang, W., Cho, C.B., Li, T.: Solarcore: solar energy driven multi-core architecture power management. In: High Performance Computer Architecture (HPCA), 2011 IEEE 17th International Symposium on, pp. 205–216. IEEE (2011)

  46. Liao, J.S., Chang, C.C., Hsu, Y.L., Zhang, X.W., Lai, K.C., Hsu, C.H.: Energy-efficient resource provisioning with sla consideration on cloud computing. In: 2012 41st International Conference on Parallel Processing Workshops (ICPPW), pp. 206–211. IEEE (2012)

  47. Lima, J.V., Raïs, I., Lefèvre, L., Gautier, T.: Performance and energy analysis of openmp runtime systems with dense linear algebra algorithms. In: 2017 International Symposium on Computer Architecture and High Performance Computing Workshops (SBAC-PADW), pp. 7–12. IEEE (2017)

  48. Lin, W., Wang, H., Zhang, Y., Qi, D., Wang, J.Z., Chang, V.: A cloud server energy consumption measurement system for heterogeneous cloud environments. Inf. Sci. 468, 47–62 (2018)

    Google Scholar 

  49. Long, S., Zhao, Y., Chen, W.: A three-phase energy-saving strategy for cloud storage systems. J. Syst. Softw. 87, 38–47 (2014)

    Google Scholar 

  50. Lorenzon, A.F., Cera, M.C., Beck, A.C.S.: Investigating different general-purpose and embedded multicores to achieve optimal trade-offs between performance and energy. J. Parallel Distrib. Comput. 95, 107–123 (2016)

    Google Scholar 

  51. Mastelic, T., Oleksiak, A., Claussen, H., Brandic, I., Pierson, J.M., Vasilakos, A.V.: Cloud computing: survey on energy efficiency. ACM Comput. Surv. CSUR 47(2), 33 (2015)

    Google Scholar 

  52. Mehdipour, F., Noori, H., Javadi, B.: Chapter two-energy-efficient big data analytics in datacenters. Adv. Comput. 100, 59–101 (2016)

    Google Scholar 

  53. Meisner, D., Gold, B.T., Wenisch, T.F.: Powernap: eliminating server idle power. In: ACM Sigplan Notices, vol. 44, pp. 205–216. ACM (2009)

  54. Nakano, T.: Biologically inspired network systems: a review and future prospects. IEEE Trans. Syst. Man Cybern. Part C 41(5), 630–643 (2011)

    Google Scholar 

  55. Njenga, K., Garg, L., Bhardwaj, A.K., Prakash, V., Bawa, S.: The cloud computing adoption in higher learning institutions in kenya: hindering factors and recommendations for the way forward. Telemat. Inform. 38, 225–246 (2018)

    Google Scholar 

  56. Orgerie, A.C., Assuncao, M.D., Lefevre, L.: A survey on techniques for improving the energy efficiency of large-scale distributed systems. ACM Comput. Surv. CSUR 46(4), 47 (2014)

    Google Scholar 

  57. Park, C., Kim, Y., Jeong, M.: Influencing factors on risk perception of iot-based home energy management services. Telemat. Inform. 35(8), 2355–2365 (2018)

    Google Scholar 

  58. Pérez, J.L., et al.: A resilient and distributed near real-time traffic forecasting application for Fog computing environments. Future Gener Comput Syst 87, 198–212 (2018)

    Google Scholar 

  59. Rivoire, S., Shah, M.A., Ranganathan, P., Kozyrakis, C.: Joulesort: a balanced energy-efficiency benchmark. In: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, pp. 365–376. ACM (2007)

  60. Rong, H., Zhang, H., Xiao, S., Li, C., Hu, C.: Optimizing energy consumption for data centers. Renew. Sustain. Energy Rev. 58, 674–691 (2016)

    Google Scholar 

  61. Rossi, F.D., Xavier, M.G., De Rose, C.A., Calheiros, R.N., Buyya, R.: E-eco: performance-aware energy-efficient cloud data center orchestration. J. Netw. Comput. Appl. 78, 83–96 (2017)

    Google Scholar 

  62. Sun, H., Lee, S.: Case study of data centers’ energy performance. Energy Build. 38(5), 522–533 (2006)

    Google Scholar 

  63. Sundriyal, V., Keipert, K., Sosonkina, M., Gordon, M.S.: Effect of frequency scaling granularity on energy-saving strategies. Int. J. High Perform. Comput. Appl. (2016). https://doi.org/10.1177/1094342018774405

    Article  Google Scholar 

  64. Tien, J.M.: Big data: unleashing information. J. Syst. Sci. Syst. Eng. 22(2), 127–151 (2013)

    Google Scholar 

  65. Vasudevan, M., Tian, Y.C., Tang, M., Kozan, E.: Profile-based application assignment for greener and more energy-efficient data centers. Future Gener. Comput. Syst. 67, 94–108 (2017)

    Google Scholar 

  66. Verma, A., Koller, R., Useche, L., Rangaswami, R.: Energy proportional storage using dynamic consolidation. In: In Proceedings of the File and Storage Systems, pp. 23–26. Citeseer (2010)

  67. Vohl, D., Fluke, C.J., Vernardos, G.: Data compression in the petascale astronomy era: a gerlumph case study. Astron. Comput. 12, 200–211 (2015)

    Google Scholar 

  68. Wang, K., Wang, Y., Sun, Y., Guo, S., Wu, J.: Green industrial internet of things architecture: an energy-efficient perspective. IEEE Commun. Mag. 54(12), 48–54 (2016)

    Google Scholar 

  69. Wu, J., Guo, S., Huang, H., Liu, W., Xiang, Y.: Information and communications technologies for sustainable development goals: state-of-the-art, needs and perspectives. IEEE Commun. Surv. Tut. 20(3), 2389–2406 (2018)

    Google Scholar 

  70. Wu, J., Guo, S., Li, J., Zeng, D.: Big data meet green challenges: big data toward green applications. IEEE Syst. J. 10(3), 888–900 (2016)

    Google Scholar 

  71. Wu, J., Guo, S., Li, J., Zeng, D.: Big data meet green challenges: greening big data. IEEE Syst. J. 10(3), 873–887 (2016)

    Google Scholar 

  72. Wu, W., Li, W., Law, D., Na, W.: Improving data center energy efficiency using a cyber-physical systems approach: integration of building information modeling and wireless sensor networks. Procedia Eng. 118, 1266–1273 (2015)

    Google Scholar 

  73. Yao, X., Wang, J.: Rimac: a novel redundancy-based hierarchical cache architecture for energy efficient, high performance storage systems. In: ACM SIGOPS Operating Systems Review, vol. 40, pp. 249–262. ACM (2006)

  74. Yoon, M.S., Kamal, A.E., Zhu, Z.: Adaptive data center activation with user request prediction. Comput. Netw. 122, 191–204 (2017)

    Google Scholar 

  75. Zhang, A.X., Safai, F., Beyer, D.M., Rolia, J., Fremont, M.J.L.: Performance-data based server consolidation (2012). US Patent 8,255,516

  76. Zhou, Y., Taneja, S., Qin, X., Ku, W.S., Zhang, J.: Edom: Improving energy efficiency of database operations on multicore servers. Future Gener. Comput, Syst (2017)

    Google Scholar 

  77. Zhu, A.W., Pi, H.: A method for improving the accuracy of weather forecasts based on a comprehensive statistical analysis of historical data for the contiguous united states. J. Climatol. Weather Forecast. 2(1), 1–4 (2014)

    Google Scholar 

  78. Zhu, Q., David, F.M., Devaraj, C.F., Li, Z., Zhou, Y., Cao, P.: Reducing energy consumption of disk storage using power-aware cache management. In: Software, IEEE Proceedings, pp. 118–129. IEEE (2004)

Download references

Acknowledgements

One of the authors, Sumedha Arora offers the sincerest gratitude to Council of Scientific and Industrial Research (CSIR), Government of India, for funding the research and providing required resources to carry out this research work with the ACK.NO.: 143253/2K17/1 and File No.: 09/677(0030)/2018-EMR-I.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, India

    Sumedha Arora

  2. Department of Computer Science and Engineering, Thapar University, Patiala, India

    Anju Bala

Authors
  1. Sumedha Arora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anju Bala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sumedha Arora.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arora, S., Bala, A. A survey: ICT enabled energy efficiency techniques for big data applications. Cluster Comput 23, 775–796 (2020). https://doi.org/10.1007/s10586-019-02958-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-019-02958-6

Keywords

Search

Navigation