Skip to main content
SpringerLink
Log in

An approach for scaling cloud resource management

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Given its current development trajectory, the complexity of cloud computing ecosystems are evolving to where traditional resource management strategies will struggle to remain fit for purpose. These strategies have to cope with ever-increasing numbers of heterogeneous resources, a proliferation of new services, and a growing user-base with diverse and specialized requirements. This growth not only significantly increases the number of parameters needed to make good decisions, it increases the time needed to take these decisions. Consequently, traditional resource management systems are increasingly prone to poor decisions making. Devolving resources management decisions to the local environment of that resource can dramatically increase the speed of decisions making; moreover, the cost of gathering global information can thus be eliminated; saving communication costs. Experimental data, provided in this paper, illustrate that extant cloud deployments can be used as effective vehicles for devolved decision making. This finding strengthens the case for the proposed paradigm shift, since it does not require a change to the architecture of existing cloud systems. This shift would result in systems in which resources decide for themselves how best they can be used. This paper takes this idea to its logical conclusion and proposes a system for supporting self-managing resources in cloud environments. It introduces the concept of coalitions, consisting of collaborating resources, formed for the purpose of service delivery. It suggests the utility of restricting the interactions between the end-user and the cloud service provider to a well-defined services interface. It shows how clouds can be considered functionally, as engines for delivering an appropriate set of resources in response to service requests. And finally, since modern applications are increasingly constructed from sophisticated workflows of complex components, it shows how combinatorial auctions can be used to effectively deliver packages of resources to support those workflows.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Compute-optimized instance with 32 vCPU and 60 GiB memory.

  2. For \(N=5\) and \(N=6\) the stirling numbers of the second kind are respectively 1, 15, 25, 10, 1 and 1, 31, 90, 65, 15, 1.

  3. \(\mathcal {S}(40,14)=3.5859872255621803491428554E+34\)

References

  1. Barossso, L.A., Clidaras, J., Hözle, U.: The Datacenter as a Computer; an Introduction to the Design of Warehouse-Scale Machines. Morgan & Claypool, San Rafael (2013)

    Google Scholar 

  2. Chang, V., Wills, G., De Roure, D.: A review of cloud business models and sustainability. In: Proceedings of the IEEE 3rd International Conference on Cloud Computing, pp. 43–50. (2010)

  3. Paya, A., Marinescu, D.C.: Energy-aware load balancing and application scaling for the cloud ecosystem. In: IEEE Transaction on Cloud Computing. (2015)

  4. Blackburn, M., Hawkins, A.: Unused server survey results analysis. www.thegreengrid.org/media/WhitePapers/Unused%20Server%20Study_WP_101910_v1.ashx?lang=en. Accessed 6 Dec 2013

  5. Marinescu, D.C., Paya, A., Morrison, J.P., Healy, P.: Distributed hierarchical control versus an economic model for cloud resource management. arXiv:1503.01061 (2015)

  6. Marinescu, D.C., Paya, A., Morrison, J.P.: A cloud reservation system for big data applications. In: IEEE Transaction on Parallel and Distributed Computing. (2016)

  7. Marinescu, D.C.: Complex Systems and Clouds: A Self-Organization and Self-Management Perspective. Morgan Kaufmann, Burlington (2016)

    Google Scholar 

  8. Müller, I., Kowalczyk, R., Braun, P.: Towards agent-based coalition formation for service composition. In: Proceedings IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 73–80. (2006)

  9. Niyato, D., Vasilakos, A., Kun, Z.: Resource and revenue sharing with coalition formation of cloud providers: game theoretic approach. In: Proceedings IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 215–224. (2011)

  10. Chaisiri, S., Lee, B., Niyato, D.: Optimization of resource provisioning cost in cloud computing. IEEE Trans. Serv. Comput. 5(2), 164–177 (2012)

    Article  Google Scholar 

  11. Li, C., Sycara, K.: Algorithm for combinatorial coalition formation and payoff division in an electronic marketplace. In: Proceedings AAMAS02—First Joint International Conference on Autonomous Agents and Multiagent Systems, pp. 120–127. (2002)

  12. Mashayekhy, L., Nejad, M.M., Grosu, D.: Cloud federations in the sky: formation game and mechanisms. IEEE Trans. Cloud Comput. 3(1), 14–27 (2014)

    Article  Google Scholar 

  13. Sandholm, T.W., Larson, K.S., Andersson, M., Shehory, O., Tohm, F.: Coalition structure generation with worst case guarantees. Artif. Intell. 111(1–2), 209–238 (1999)

  14. Rahwan, T., Ramchurn, S.D., Jennings, N.R., Giovannucci, A.: An anytime algorithm for optimal coalition structure generation. J. Artif. Intell. Res. 34, 521–567 (2009)

    MathSciNet  MATH  Google Scholar 

  15. Greco, G., Malizia, E., Palopoli, L., Scarello, F.: On the complexity of the core over coalition structures. In: Proceedings of the 22 International Joint Conference on Artificial Intelligence, pp. 216–221. (2011)

  16. Shehory, O., Kraus, S.: Methods for task allocation via agent coalition formation. Artif. Intell. 101(1–2), 165–200 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  17. Marinescu, D.C., Paya, A., Morrison, J.P.: Coalition formation and combinatorial auctions; applications to self-organization and self-management in utility computing. arXiv:1406.7487 (2015)

  18. Ausubel, L., Cramton, P., Milgrom, P.: The clock-proxy auction: a practical combinatorial auction design. In: Cramton, P., Shoham, Y., Steinberg, R. (eds.) Combinatorial Auctions. MIT Press, Cambridge (2006)

    Google Scholar 

  19. Bradic, I.: Towards self-manageable cloud services. In: Proceedings of the 33 International Conference on Computer Software and Applications, pp. 128–133. (2009)

  20. Marinescu, D.C.: Cloud Computing. Theory and Practice. Morgan Kaufmann, New York (2013)

    Google Scholar 

  21. Paton, N., de Arago, M.A.T., Lee, K., Fernandes, A.A.A., Sakellariou, R.R.: Optimizing utility in cloud computing through autonomic workload execution. Bull. Tech. Comm. Data Eng. 32(1), 51–58 (2009)

    Google Scholar 

  22. Sommerville, I., Cliff, D., Calinescu, R., Keen, J., Kelly, T., Kwiatowska, M., McDermid, J., Paige, R.: Large-scale IT complex systems. Commun. ACM 55(7), 71–77 (2012)

    Article  Google Scholar 

  23. Van, H.N., Tran, F.D., Menaud, J.M.: Autonomic virtual resource management for service hosting platforms. In: Software Engineering Challenges of Cloud Computing, ICSE Workshop at CLOUD09, pp. 1–8. (2009)

  24. Minsky, M.: Computation: Finite and Infinite Machines. Prentice Hall, New York (1967)

    MATH  Google Scholar 

  25. Gell-Mann, M.: Simplicity and complexity in the description of nature. Eng. Sci. I(3), 3–9 (1988)

    Google Scholar 

  26. Mayer, M.W.: Architecting principles for system of systems. Syst. Eng. 1(4), 267–274 (1998)

    Article  Google Scholar 

  27. Marinescu, D.C., Siegel, H.J., Morrison, J.P.: Options and commodity markets for computing resources. In: Buyya, R., Bubendorf, K. (eds.) Market Oriented Grid and Utility Computing, pp. 89–120. Wiley, New York (2009)

    Chapter  Google Scholar 

  28. Marinescu, D.C., Paya, A., Morrison, J.P., Healy, P.: An auction-driven, self-organizing cloud delivery model. http://arxiv.org/pdf/1312.2998v1.pdf. (2013)

Download references

Acknowledgements

The work reported in this paper was partially supported by NSF CCR Grant 1525943 “Is the Simulation of Quantum Many-Body Systems Feasible on the Cloud?” to Dan C. Marinescu and collaborators and by a Grant from the EU H2020 program to J. P. Morrison for the CloudLightning consortium.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Central Florida, Orlando, FL, USA

    Dan C. Marinescu & Ashkan Paya

  2. Computer Science Department, University College Cork, Cork, Ireland

    John P. Morrison

  3. Computer Science Department, Old Dominion University, Norfolk, VA, USA

    Stephen Olariu

Authors
  1. Dan C. Marinescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ashkan Paya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John P. Morrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephen Olariu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashkan Paya.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marinescu, D.C., Paya, A., Morrison, J.P. et al. An approach for scaling cloud resource management. Cluster Comput 20, 909–924 (2017). https://doi.org/10.1007/s10586-016-0700-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-016-0700-8

Keywords

Search

Navigation