Skip to main content
SpringerLink
Log in
Book cover

Research Advances in Cloud Computing pp 237–261Cite as

Recent Developments in Resource Management in Cloud Computing and Large Computing Clusters

  • Chapter
  • First Online:

  • 2603 Accesses

Abstract

Cloud computing and large computing clusters consist of a large number of computing resources of different types ranging from storage, CPU, memory, I/O to network bandwidth. Cloud computing exposes resources as a single access point to end users through the use of virtualization technologies. A major issue in cloud computing is how to properly allocate cloud resources to different users or frameworks accessing the cloud. There are a lot of complex, diverse, and heterogeneous workloads that need to coexist in the cloud and large-scale compute clusters, thus the need for finding efficient means of assigning resources to the different users or workloads. Millions of jobs need to be scheduled in a small amount of time, so there is a need for a resource management and scheduling mechanism that can minimize latency and maximize efficiency. Cloud resource management involves allocating computing, processing, storage, and networking resources to cloud users, in such a way that their demands and performance objectives are met. Cloud providers need to ensure efficient and effective resource provisioning while being constrained by Service Level Agreements (SLAs). This chapter gives the differences and similarities between resource management in cloud computing and cluster computing, and provide detailed information about different types of scheduling approaches and open research issues.

This is a preview of subscription content, log in via an institution.

References

  1. Amatriain, X., & Griffiths, D. (2004). Free software in education is it a viable alternative? In Proceedings of 7th IMAC Conference on Localization and Globalization in Technology Design Use and Transfer as a Subject of Engineering (Vol. 7(1)).

    Google Scholar 

  2. Awad, A., El-Hefnawy, N., & Abdel_kader, H. (2015). Enhanced particle swarm optimization for task scheduling in cloud computing environments. Procedia Computer Science, 65, 920–929.

    Google Scholar 

  3. Batat, A., & Feitelson, D. G. (2000). Gang scheduling with memory considerations. In Proceedings 14th International Parallel and Distributed Processing Symposium, IPDPS (pp. 109–114).

    Google Scholar 

  4. Boutin, E., Ekanayake, J., Lin, W., Shi, B., Zhou, J., Qian, Z., Wu, M., & Zhou, L. (2014). Apollo: Scalable and coordinated scheduling for cloud-scale computing. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), Broomfield, CO (pp. 285–300). USENIX Association.

    Google Scholar 

  5. Chang, G., & Su, H. (2003). Fifo scheduling time sharing. US Patent App. 10/163,047.

    Google Scholar 

  6. Chaskar, H. M., & Madhow, U. (2003). Fair scheduling with tunable latency: A round-robin approach. IEEE/ACM Transactions on Networking, 11(4), 592–601.

    Article  Google Scholar 

  7. Chen, H., & Guo, W. (2015). Real-Time Task Scheduling Algorithm for Cloud Computing Based on Particle Swarm Optimization (pp. 141–152). Cham: Springer International Publishing.

    Google Scholar 

  8. Chen, H., Wang, F., Helian, N., & Akanmu, G. (2013). User-priority guided min-min scheduling algorithm for load balancing in cloud computing. In 2013 National Conference on Parallel Computing Technologies (PARCOMPTECH) (pp. 1–8).

    Google Scholar 

  9. Convolbo, M. W., & Chou, J. (2016). Cost-aware dag scheduling algorithms for minimizing execution cost on cloud resources. The Journal of Supercomputing, 72(3), 985–1012.

    Article  Google Scholar 

  10. Danna, E., Hassidim, A., Kaplan, H., Kumar, A., Mansour, Y., Raz, D., et al. (2012). Upward max min fairness. In INFOCOM, 2012 Proceedings IEEE (pp. 837–845).

    Google Scholar 

  11. Dasgupta, K., Mandal, B., Dutta, P., Mandal, J. K., & Dam, S. (2013). A genetic algorithm (ga) based load balancing strategy for cloud computing. Procedia Technology, 10(Complete), 340–347.

    Google Scholar 

  12. Delgado, P., Dinu, F., Kermarrec, A.-M., & Zwaenepoel, W. (2015). Hawk: Hybrid datacenter scheduling. In Proceedings of the 2015 USENIX Annual Technical Conference (pp. 499–510).

    Google Scholar 

  13. Delimitrou, C., & Kozyrakis, C. (2013). Paragon: Qos-aware scheduling for heterogeneous datacenters. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’13, New York, NY, USA (pp. 77–88). ACM.

    Google Scholar 

  14. Delimitrou, C., & Kozyrakis, C. (2014). Quasar: Resource-efficient and qos-aware cluster management. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’14, New York, NY, USA (pp. 127–144). ACM.

    Google Scholar 

  15. Delimitrou, C., Sanchez, D., & Kozyrakis, C. (2015). Tarcil: Reconciling scheduling speed and quality in large shared clusters. In Proceedings of the Sixth ACM Symposium on Cloud Computing (pp. 97–110).

    Google Scholar 

  16. Dong, Z., Liu, N., & Rojas-Cessa, R. (2015). Greedy scheduling of tasks with time constraints for energy-efficient cloud-computing data centers. Journal of Cloud Computing: Advances, Systems and Applications, 4(1), 5.

    Article  Google Scholar 

  17. Frachtenberg, E., Petrini, F., Coll, S., Feng, W., Modeling, C., & Group, I. Gang scheduling with lightweight user-level communication.

    Google Scholar 

  18. Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S., & Stoica, I. (2011). Dominant resource fairness: Fair allocation of multiple resource types. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, NSDI’11, Berkeley, CA, USA (pp. 323–336). USENIX Association.

    Google Scholar 

  19. Grandl, R., Ananthanarayanan, G., Kandula, S., Rao, S., & Akella, A. (2014). Multi-resource packing for cluster schedulers. ACM SIGCOMM Computer Communication Review, 44(4), 455–466.

    Article  Google Scholar 

  20. Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A. D., Katz, R., et al. (2011). Mesos: A platform for fine-grained resource sharing in the data center. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, NSDI’11, Berkeley, CA, USA (pp. 295–308). USENIX Association.

    Google Scholar 

  21. Hunt, P., Konar, M., Junqueira, F. P., & Reed, B. (2010). Zookeeper: Wait-free coordination for internet-scale systems. In Proceedings of the 2010 USENIX Conference on USENIX Annual Technical Conference, USENIXATC’10, Berkeley, CA, USA (pp. 11–11). USENIX Association.

    Google Scholar 

  22. Kapgate, D. (2014). Improved round robin algorithm for data center selection in cloud computing. International Journal of Engineering Sciences and Research Technology, 3(2), 686–691.

    Google Scholar 

  23. Karanasos, K., Rao, S., Curino, C., Douglas, C., Chaliparambil, K., Fumarola, G. M., et al. (2015). Mercury: Hybrid centralized and distributed scheduling in large shared clusters. In Usenix-Atc (pp. 485–497).

    Google Scholar 

  24. Karatza, H. D. (2006). Scheduling gangs in a distributed system. International Journal of Simulation: Systems, Science and Technology, 7(1), 15–22.

    Google Scholar 

  25. Kargahi, M., & Movaghar, A. (2006). A method for performance analysis of earliest-deadline-first scheduling policy. The Journal of Supercomputing, 37(2), 197–222.

    Article  Google Scholar 

  26. Kaur, R., Kaur, G., & Scholar, R. (2016). A review on efficient hybrid framework for scheduling in cloud computing. International Journal of Engineering Science and Computing, 6(7), 8698–8700.

    Google Scholar 

  27. Kaur, R., & Kinger, S. (2014). Article: Enhanced genetic algorithm based task scheduling in cloud computing. International Journal of Computer Applications, 101(14), 1–6.

    Article  Google Scholar 

  28. Lamport, L., et al. (2001). Paxos made simple. ACM Sigact News, 32(4), 18–25.

    Google Scholar 

  29. Leith, D. J., Cao, Q., & Subramanian, V. G. (2012). Max-min fairness in 802.11 mesh networks. IEEE/ACM Transactions on Networking, 20(3), 756–769.

    Article  Google Scholar 

  30. Li, K., Xu, G., Zhao, G., Dong, Y., & Wang, D. (2011). Cloud task scheduling based on load balancing ant colony optimization. In 2011 Sixth Annual ChinaGrid Conference (pp. 3–9). IEEE.

    Google Scholar 

  31. Liu, C. Y., Zou, C. M., & Wu, P. (2014). A task scheduling algorithm based on genetic algorithm and ant colony optimization in cloud computing. In 2014 13th International Symposium on Distributed Computing and Applications to Business, Engineering and Science (DCABES) (pp. 68–72).

    Google Scholar 

  32. Liu, G., Li, J., & Xu, J. (2013). An improved min-min algorithm in cloud computing (pp. 47–52). Berlin: Springer.

    Google Scholar 

  33. Liu, K., Jin, H., Chen, J., Liu, X., Yuan, D., & Yang, Y. (2010). A compromised-time-cost scheduling algorithm in swindew-c for instance-intensive cost-constrained workflows on a cloud computing platform. International Journal of High Performance Computer Application, 24(4), 445–456.

    Article  Google Scholar 

  34. Lupetti, S., & Zagorodnov, D. (2006). Data popularity and shortest-job-first scheduling of network transfers. In International Conference on Digital Telecommunications (ICDT’06) (pp. 26–26).

    Google Scholar 

  35. Mitzenmacher, M. (2001). The power of two choices in randomized load balancing. IEEE Transactions on Parallel and Distributed Systems, 12(10), 1094–1104.

    Article  Google Scholar 

  36. Mohammadi, F., Jamali, S., & Bekravi, M. (2014). Survey on job scheduling algorithms in cloud computing. International Journal of Emerging Trends & Technology in Computer Science (IJETTCS), 3(2), 151–154.

    Google Scholar 

  37. Ousterhout, K., Wendell, P., Zaharia, M., & Stoica, I. (2013). Sparrow: Distributed, low latency scheduling. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, SOSP ’13, New York, NY, USA (69–84). ACM.

    Google Scholar 

  38. Pandey, S., Wu, L., Guru, S. M., & Buyya, R. (2010). A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In 2010 24th IEEE International Conference on Advanced Information Networking and Applications (pp. 400–407).

    Google Scholar 

  39. Petrini, F., & Feng, W.-C. (2000). Improved resource utilization with buffered coscheduling. Journal of Parallel Algorithms and Applications (Special Issue), 16(2–3),

    Google Scholar 

  40. Potts, C. N., & Kovalyov, M. Y. (2000). Scheduling with batching: A review. European Journal of Operational Research, 120(2), 228–249.

    Google Scholar 

  41. Ruchita, P., & Moni, C. (2016). Analysis of various task scheduling algorithms in cloud computing. International Research Journal of Engineering and Technology (IRJET), 3(3), 493–496.

    Google Scholar 

  42. Saifullah, A., Ferry, D., Li, J., Agrawal, K., Lu, C., & Gill, C. D. (2014). Parallel real-time scheduling of dags. IEEE Transactions on Parallel and Distributed Systems, 25(12), 3242–3252.

    Article  Google Scholar 

  43. Sakellariou, R., & Zhao, H. (2004). A hybrid heuristic for DAG scheduling on heterogeneous systems. In Parallel and Distributed Processing Symposium, 2004. Proceedings. 18th International (pp. 111–123).

    Google Scholar 

  44. Schwarzkopf, M., & Konwinski, A. (2013). Omega: Flexible, scalable schedulers for large compute clusters. In EuroSys ’13 Proceedings of the 8th ACM European Conference on Computer Systems (pp. 351–364).

    Google Scholar 

  45. Selvarani, S., & Sadhasivam, G. S. (2010). Improved cost-based algorithm for task scheduling in cloud computing. In 2010 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC) (pp. 1–5).

    Google Scholar 

  46. Shimpy, E., & Sidhu, J. (2014). Different scheduling algorithms in different cloud environment. International Journal of Advanced Research in Computer and Communication Engineering, 3(9), 2278–1021.

    Google Scholar 

  47. Singh, S., & Chana, I. (2016). A survey on resource scheduling in cloud computing: Issues and challenges. Journal of Grid Computing (pp. 1–48).

    Google Scholar 

  48. Sobalvarro, P., Pakin, S., Weihl, W. E., & Chien, A. A. (1998). Dynamic coscheduling on workstation clusters. In Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, IPPS/SPDP ’98, London, UK (pp. 231–256). Springer.

    Google Scholar 

  49. Tang, J. M., Luo, L., Wei, K. M., Guo, X., & Ji, X. Y. (2015). A heuristic resource scheduling algorithm of cloud computing based on polygons correlation calculation. In Proceedings—12th IEEE International Conference on E-Business Engineering, ICEBE 2015 (pp. 365–370).

    Google Scholar 

  50. Thomas, A., Krishnalal, G., & Jagathy Raj, V. P. (2015). Credit based scheduling algorithm in cloud computing environment. Procedia Computer Science, 46(Icict 2014), 913–920.

    Google Scholar 

  51. Tilak, S., & Patil, P. D. (2012). A survey of various scheduling algorithms in cloud environment. International Journal of Engineering Inventions, 1(2), 36–39.

    Google Scholar 

  52. Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015). Large-scale cluster management at Google with Borg. In Proceedings of the Tenth European Conference on Computer Systems—EuroSys ’15 (pp. 1–17).

    Google Scholar 

  53. Vignesh, V., Kumarand, S., & Jaisankar, N. (2013). Resource management and scheduling in cloud environment. International Journal of Scientific and Research Publications, 3(6), 1–6.

    Google Scholar 

  54. Wang, W., Li, B., & Liang, B. (2013). Dominant resource fairness in cloud computing systems with heterogeneous servers. CoRR. arXiv:abs/1308.0083.

  55. White, T. (2012). Hadoop: The definitive guide. O’Reilly Media, Inc.

    Google Scholar 

  56. Wiseman, Y., & Feitelson, D. G. (2003). Paired gang scheduling. IEEE Transactions on Parallel and Distributed Systems, 14(6), 581–592.

    Article  Google Scholar 

  57. Wu, X., Deng, M., Zhang, R., Zeng, B., & Zhou, S. (2013). A task scheduling algorithm based on qos-driven in cloud computing (Vol. 17, pp. 1162–1169).

    Google Scholar 

  58. Xu, M., Cui, L., Wang, H., & Bi, Y. (2009). A multiple qos constrained scheduling strategy of multiple workflows for cloud computing. In 2009 IEEE International Symposium on Parallel and Distributed Processing with Applications (pp. 629–634). IEEE.

    Google Scholar 

  59. Yang, X., & Vaidya, N. H. (2002). Priority scheduling in wireless ad hoc networks. In Proceedings of the 3rd ACM International Symposium on Mobile Ad Hoc Networking & Amp; Computing, MobiHoc ’02, New York, NY, USA (pp. 71–79). ACM.

    Google Scholar 

  60. Zhan, Z.-H., Liu, X.-F., Gong, Y.-J., Zhang, J., Chung, H. S.-H., & Li, Y. (2015a). Cloud computing resource scheduling and a survey of its evolutionary approaches. ACM Computing Survey, 47(4), 63:1–63:33.

    Google Scholar 

  61. Zhan, Z.-H., Liu, X.-F., Gong, Y.-J., Zhang, J., Chung, H. S.-H., & Li, Y. (2015b). Cloud computing resource scheduling and a survey of its evolutionary approaches. ACM Computing Surveys, 47(4), 1–33.

    Article  Google Scholar 

  62. Zhang, Z., Li, C., Tao, Y., Yang, R., Tang, H., & Xu, J. (2014). Fuxi: A fault-tolerant resource management and job scheduling system at internet scale. Proceedings of VLDB Endowment, 7(13), 1393–1404.

    Article  Google Scholar 

  63. Zuo, L., Shu, L., Dong, S., Zhu, C., & Hara, T. (2015). A multi-objective optimization scheduling method based on the ant colony algorithm in cloud computing. IEEE Access, 3, 2687–2699.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. McGill University, Montreal, QC, Canada

    Richard Olaniyan & Muthucumaru Maheswaran

Authors
  1. Richard Olaniyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muthucumaru Maheswaran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard Olaniyan .

Editor information

Editors and Affiliations

  1. School of Engineering and Applied Science, Ahmedabad University, Ahmedabad, Gujarat, India

    Sanjay Chaudhary

  2. Department of Computer Science and Engineering, Central University of Rajasthan, Ajmer, Rajasthan, India

    Gaurav Somani

  3. School of Computing and Information Systems, The University of Melbourne, Melbourne, Victoria, Australia

    Rajkumar Buyya

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Olaniyan, R., Maheswaran, M. (2017). Recent Developments in Resource Management in Cloud Computing and Large Computing Clusters. In: Chaudhary, S., Somani, G., Buyya, R. (eds) Research Advances in Cloud Computing. Springer, Singapore. https://doi.org/10.1007/978-981-10-5026-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-5026-8_10

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-5025-1

  • Online ISBN: 978-981-10-5026-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation