Skip to main content
SpringerLink
Log in

Towards Enabling Live Thresholding as Utility to Manage Elastic Master-Slave Applications in the Cloud

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The elasticity feature of cloud computing has been proved as pertinent for parallel applications, since users do not need to take care about the best choice for the number of processes/resources beforehand. To accomplish this, the most common approaches use threshold-based reactive elasticity or time-consuming proactive elasticity. However, both present at least one problem related to: the need of a previous user experience, lack on handling load peaks, completion of parameters or design for a specific infrastructure and workload setting. In this regard, we developed a hybrid elasticity service for Master-Slave parallel applications named Helpar (Hybrid Elasticity Model for Parallel Applications). As parameterless model, Helpar presents a closed control loop elasticity architecture that adapts at runtime the values of lower and upper thresholds. Thus, we intend to provide a practical and effortless realization of the cloud elasticity and parallel computing duet, so delivering this capability as a plug-and-play utility to end users. Besides presenting Helpar, our purpose is to provide a comparison between Helpar and our previous work on reactive elasticity called AutoElastic. We will explore different metrics, including applications’ time, energy consumption and cost, as well as distinct types of workloads when executing a scientific HPC application. The results present the Helpar’s lightweight feature, besides highlighting its performance competitiveness in terms of application time and cost (performance × energy) metrics. In other words, the hand-tuning of thresholds in AutoElastic often is responsible for the best results, but this procedure may be time-consuming besides optimized for a particular set of application and infrastructure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ali-Eldin, A., Tordsson, J., Elmroth, E.: An adaptive hybrid elasticity controller for cloud infrastructures Network Operations and Management Symposium (NOMS), 2012 IEEE, pp 204–212 (2012)

    Chapter  Google Scholar 

  2. Banas, K., Kruzel, F.: Comparison of xeon phi and kepler gpu performance for finite element numerical integration. In: Proceedings of the 2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS), HPCC ’14, pp 145–148. IEEE Computer Society, Washington, DC, USA (2014)

  3. Beloglazov, A., Buyya, R.: Adaptive Threshold-Based Approach for Energy-Efficient Consolidation of Virtual Machines in Cloud Data Centers Proceedings of the 8Th International Workshop on Middleware for Grids, Clouds and e-Science, MGC ’10, pp 4:1–4:6. ACM, New York, NY, USA (2010)

  4. Beloglazov, A., Buyya, R.: Energy Efficient Resource Management in Virtualized Cloud Data Centers proceedings of the 2010 10Th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, CCGRID ’10, pp 826–831. IEEE Computer Society, Washington, DC, USA (2010)

    Chapter  Google Scholar 

  5. Bing, H., Ying-lan, F., Bai, L.Y.: Research and improvement of congestion control algorithms based on tcp protocol WRI World Congress on Software Engineering, 2009 WCSE ’09, vol. 1, pp 440–443 (2009)

  6. Breitgand, D., Henis, E., Shehory, O.: Automated and adaptive threshold setting: Enabling technology for autonomy and self-management ICAC 2005 Proceedings. Second International Conference on Autonomic Computing, pp 204–215 (2005)

    Google Scholar 

  7. Cai, B., Xu, F., Ye, F., Zhou, W.: Research and application of migrating legacy systems to the private cloud platform with cloudstack 2012 IEEE International Conference on Automation and Logistics (ICAL), pp 400–404 (2012)

    Chapter  Google Scholar 

  8. Caron, E., Desprez, F., Muresan, A.: Forecasting for grid and cloud computing on-demand resources based on pattern matching 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pp 456–463 (2010)

    Chapter  Google Scholar 

  9. Changqing, G., qinghui, W., Guangxing, W.: The impact of tcp segment size and routing change on congestion control protocol performance in mobile ad hoc networks Proceedings. 2005 International Conference on Wireless Communications, Networking and Mobile Computing, 2005, vol. 2, pp 820–823 (2005)

  10. Chen, F., Grundy, J., Schneider, J.-G., Yang, Y., He, Q.: Automated Analysis of Performance and Energy Consumption for Cloud Applications Proceedings of the 5Th ACM/ SPEC International Conference on Performance Engineering, ICPE ’14, pp 39–50. ACM, New York, NY, USA (2014)

    Chapter  Google Scholar 

  11. Comanescu, M.: Implementation of time-varying observers used in direct field orientation of motor drives by trapezoidal integration 6th IET International Conference on Power electronics, Machines and Drives (PEMD 2012), pp 1–6 (2012)

    Google Scholar 

  12. Dustdar, S., Gambi, A., Krenn, W., Nickovic, D.: A Pattern-Based Formalization of Cloud-Based Elastic Systems Proceedings of the Seventh International Workshop on Principles of Engineering Service-Oriented and Cloud Systems, PESOS ’15, pp 31–37. IEEE Press, Piscataway, NJ, USA (2015)

    Google Scholar 

  13. Farokhi, S., Jamshidi, P., Brandic, I., Elmroth, E.: Self-adaptation challenges for cloud-based applications : A control theoretic perspective 10th International Workshop on Feedback Computing (Feedback Computing 2015). ACM (2015)

  14. Galante, G., De Bona, L.C.E.: A programming-level approach for elasticizing parallel scientific applications. J. Syst. Softw. 110, 239–252 (2015)

    Article  Google Scholar 

  15. Galante, G., De Bona, L.C.E., Mury, A.R., Schulze, B., da Rosa Righi, R.: An analysis of public clouds elasticity in the execution of scientific applications: a survey. Journal of Grid Computing 14(2), 193–216 (2016)

    Article  Google Scholar 

  16. Ghanbari, H., Simmons, B., Litoiu, M., Iszlai, G.: Exploring alternative approaches to implement an elasticity policy 2011 IEEE International Conference on Cloud Computing (CLOUD), pp 716–723 (2011)

    Chapter  Google Scholar 

  17. Gorlach, K., Leymann, F.: Dynamic service provisioning for the cloud 2012 IEEE Ninth International Conference on Services Computing (SCC), pp 555–561 (2012)

    Chapter  Google Scholar 

  18. Herbst, N.R., Huber, N., Kounev, S., Amrehn, E.: Self-adaptive Workload Classification and Forecasting for Proactive Resource Provisioning Proceedings of the 4Th ACM/SPEC International Conference on Performance Engineering, ICPE ’13, pp 187–198. ACM, New York, NY, USA (2013)

    Chapter  Google Scholar 

  19. Herbst, N.R., Kounev, S., Weber, A., Bungee, H.G.: An Elasticity Benchmark for Self-Adaptive Iaas Cloud Environments Proceedings of the 10Th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS ’15, pp 46–56. IEEE Press, Piscataway, NJ, USA (2015)

    Google Scholar 

  20. Hirsch, M., Rodríguez, J.M., Mateos, C., Zunino, A.: A two-phase energy-aware scheduling approach for cpu-intensive jobs in mobile grids. Journal of Grid Computing 15(1), 55–80 (2017)

    Article  Google Scholar 

  21. Islam, S., Keung, J., Lee, K., Liu, A.: Empirical prediction models for adaptive resource provisioning in the cloud. Futur. Gener. Comput. Syst. 28(1), 155–162 (2012)

    Article  Google Scholar 

  22. Jamshidi, P., Ahmad, A., Pahl, C.: Autonomic Resource Provisioning for Cloud-Based Software Proceedings of the 9Th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2014, pp 95–104. ACM, New York, NY, USA (2014)

    Google Scholar 

  23. Jiang, J., Lin, Y., Xie, G., Fu, L., Yang, J.: Time and energy optimization algorithms for the static scheduling of multiple workflows in heterogeneous computing system. Journal of Grid Computing 1–22 (2017)

  24. Jin, H., Wang, X., Wu, S., Di, S., Shi, X.: Towards optimized fine-grained pricing of iaas cloud platform. IEEE Transactions on cloud Computing (2014)

  25. Kejariwal, A.: Techniques for optimizing cloud footprint 2013 IEEE International Conference on Cloud Engineering (IC2E), pp 258–268 (2013)

    Chapter  Google Scholar 

  26. Lee, Y., Avizienis, R., Bishara, A., Xia, R., Lockhart, D., Batten, C., Asanovic, K.: Exploring the tradeoffs between programmability and efficiency in data-parallel accelerators 2011 38th Annual International Symposium on Computer Architecture (ISCA), pp 129–140 (2011)

    Google Scholar 

  27. Leitner, P., Inzinger, C., Hummer, W., Satzger, B., Dustdar, S.: Application-level performance monitoring of cloud services based on the complex event processing paradigm 2012 5th IEEE International Conference on Service-Oriented Computing and Applications (SOCA), pp 1–8 (2012)

    Google Scholar 

  28. Lim, H.C., Babu, S., Chase, J.S., Parekh, S.S.: Automated Control in Cloud Computing: Challenges and Opportunities Proceedings of the 1St Workshop on Automated Control for Datacenters and Clouds, ACDC ’09, pp 13–18. ACM, New York, NY, USA (2009)

    Chapter  Google Scholar 

  29. Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. Journal of Grid Computing 12(4), 559–592 (2014)

    Article  Google Scholar 

  30. Lu, L., Shi, X., Jin, H., Wang, Q., Yuan, D., Wu, S.: Morpho: A decoupled mapreduce framework for elastic cloud computing. Futur. Gener. Comput. Syst. 36, 80–90 (2014)

    Article  Google Scholar 

  31. Luo, L., Wu, W., Tsai, W.T., Di, D., Zhang, F.: Simulation of power consumption of cloud data centers. Simul. Model. Pract. Theory 39(0), 152–171 (2013). S.I.Energy efficiency in grids and clouds

    Article  Google Scholar 

  32. Miettinen, P., Vreeken, J.: Mdl4bmf: Minimum description length for boolean matrix factorization. ACM Trans. Knowl. Discov. Data 8(4), 18:1–18:31 (2014)

    Article  Google Scholar 

  33. Moore, L.R., Bean, K., Ellahi, T.: Transforming Reactive Auto-Scaling into Proactive Auto-Scaling Proceedings of the 3Rd International Workshop on Cloud Data and Platforms, CloudDP ’13, pp 7–12. ACM, New York, NY, USA (2013)

    Chapter  Google Scholar 

  34. Netto, M.A.S., Cardonha, C., Cunha, R.L.F., Assuncao, M.D.: Evaluating auto-scaling strategies for cloud computing environments IEEE 22nd International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems, MASCOTS 2014, Paris, France, September 9-11, 2014, p 2014. IEEE

  35. Nikolov, V., Kächele, S., Hauck, F.J., Rautenbach, D.: Cloudfarm: An Elastic Cloud Platform with Flexible and Adaptive Resource Management Proceedings of the 2014 IEEE/ACM 7Th International Conference on Utility and Cloud Computing, UCC ’14, pp 547–553. IEEE Computer Society, Washington, DC USA (2014)

    Chapter  Google Scholar 

  36. Nikravesh, A.Y., Ajila, S.A., Lung, C.-H.: Towards an Autonomic Auto-Scaling Prediction System for Cloud Resource Provisioning Proceedings of the 10Th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS ’15, pp 35–45. IEEE Press, Piscataway, NJ, USA (2015)

    Google Scholar 

  37. Niu, S., Zhai, J., Ma, X., Tang, X., Chen, W.: Cost-effective Cloud Hpc Resource Provisioning by Building Semi-Elastic Virtual Clusters Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC ’13, pp 56:1–56:12. ACM, New York, NY, USA (2013)

    Google Scholar 

  38. Orgerie, A.-C., De Assuncao, M.D., Lefevre, L.: A survey on techniques for improving the energy efficiency of large-scale distributed systems. ACM Comput. Surv. 46(4), 1–31 (2014)

    Article  Google Scholar 

  39. Padoin, E. L., de Oliveira, D. A. G., Velho, P., Navaux, P. O. A.: Time-to-solution and energy-to-solution: a comparison between arm and xeon 2012 Third Workshop on Applications for Multi-Core Architectures (WAMCA), pp 48–53 (2012)

  40. Rajan, D., Canino, A., Izaguirre, J.A., Thain, D.: Converting a High Performance Application to an Elastic Cloud Application Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science, CLOUDCOM ’11, pp 383–390. IEEE Computer Society, Washington, DC, USA (2011)

    Chapter  Google Scholar 

  41. Raveendran, A., Bicer, T., Agrawal, G.: A framework for elastic execution of existing mpi programs 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum (IPDPSW), pp 940–947 (2011)

    Chapter  Google Scholar 

  42. Righi, R., Rodrigues, V., Andre daCosta, C., Galante, G., Bona, L., Ferreto, T.: Autoelastic: Automatic resource elasticity for high performance applications in the cloud. IEEE Transactions on Cloud Computing, PP(99):1–1 (2015)

  43. Roy, N., Dubey, A., Gokhale, A.: Efficient autoscaling in the cloud using predictive models for workload forecasting Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing, CLOUD ’11, pp 500–507. IEEE Computer Society, Washington, DC, USA (2011)

    Chapter  Google Scholar 

  44. Sladescu, M., Fekete, A.: Event Aware Elasticity Control for Cloud Applications. Technical Report The University of Sydney, Sydney, Australia (2012)

    Google Scholar 

  45. Li, T., Kothapalli, S., Chen, L., Hussaini, O., Bissiri, R., Chen, Z.: A survey of power and energy efficient techniques for high performance numerical linear algebra operations. Parallel Comput. 40(10), 559–573 (2014)

    Article  MathSciNet  Google Scholar 

  46. Tian, Y., Lin, C., Li, K.: Managing performance and power consumption tradeoff for multiple heterogeneous servers in cloud computing. Clust. Comput. 17(3), 943–955 (2014)

    Article  Google Scholar 

  47. Tighe, M., Bauer, M.: Topology and application aware dynamic vm management in the cloud. Journal of Grid Computing 1–22 (2017)

  48. Tripodi, E., Musolino, A., Rizzo, R., Raugi, M.: Numerical integration of coupled equations for high-speed electromechanical devices. IEEE Trans. Magn. 51(3), 1–4 (2015)

    Article  Google Scholar 

  49. Wen, X., Gu, G., Li, Q., Gao, Y., Zhang, X.: Comparison of open-source cloud management platforms: Openstack and opennebula 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), pp 2457–2461 (2012)

    Chapter  Google Scholar 

  50. Yazdanov, L., Fetzer, C.: Vertical Scaling for Prioritized Vms Provisioning Proceedings of the 2012 Second International Conference on Cloud and Green Computing, CGC ’12, pp 118–125. IEEE Computer Society, Washington, DC, USA (2012)

    Chapter  Google Scholar 

Download references

Acknowledgements

The authors would like to thank to the following Brazilian Agencies: CNPq, CAPES and FAPERGS.

Author information

Authors and Affiliations

  1. Applied Computing Graduate Program, UNISINOS, São Leopoldo, Brazil

    Vinicius Facco Rodrigues, Rodrigo da Rosa Righi, Gustavo Rostirolla, Jorge Luis Victória Barbosa & Cristiano André da Costa

  2. INATEL, Santa Rita do Sapucaí, Brazil

    Antônio Marcos Alberti

  3. Xi’an Jiaotong Liverpool University, Suzhou, China

    Victor Chang

Authors
  1. Vinicius Facco Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodrigo da Rosa Righi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gustavo Rostirolla
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jorge Luis Victória Barbosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cristiano André da Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Antônio Marcos Alberti
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Victor Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rodrigo da Rosa Righi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodrigues, V.F., da Rosa Righi, R., Rostirolla, G. et al. Towards Enabling Live Thresholding as Utility to Manage Elastic Master-Slave Applications in the Cloud. J Grid Computing 15, 535–556 (2017). https://doi.org/10.1007/s10723-017-9405-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-017-9405-3

Keywords

Search

Navigation