Abstract
Cloud computing can enable the unraveling of new scientific breakthroughs. We will eventually arrive to compute overwhelmingly large sizes of information, larger than we ever thought about it. Better scheduling algorithms are the key to process Big Data. This paper presents a load balancing scheduling algorithm for Many Task Computing using the computational resources from Cloud, in order to process a huge number of tasks with a finite number of resources. As such, the algorithm can be also used for Big Data, because it scales easily for big applications if we put a load balancing algorithm on top of virtual machines. We impose an upper bound of one for the maximum nodes that can carry an arbitrary job without executing it and we show that this statement holds by simulating the algorithm in MTS2 (Many Task Scheduling Simulator). We also show that the algorithm’s overlay performs even better when there are multiple nodes and we discuss about choosing the best local scheduling policy for the working nodes.
The research presented in this paper is supported by the following projects: StorEdge (GNaC 2018 ARUT - AU11-18-07), ROBIN (PN-III-P1-1.2-PCCDI-2017-0734), NETIO ForestMon/Tel-MONAER (53/05.09.2016, SMIS2014+ 105976) and SPERO (PN-III-P2-2.1-SOL-2016-03-0046, 3Sol/2017). We would like to thank the reviewers for their time and expertise, constructive comments and valuable insight.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bessis, N., Sotiriadis, S., Cristea, V., Pop, F.: Modelling requirements for enabling meta-scheduling in inter-clouds and inter-enterprises. In: 2011 Third International Conference on Intelligent Networking and Collaborative Systems, pp. 149–156. IEEE (2011)
Bonneau, J.: Algorithms for Big Data, 30 June 2014
Calheiros, R.N., Ranjan, R., Beloglazov, A., Rose, C.A.F.D., Buyya, R.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw.: Pract. Exp. 41, 23–50 (2010)
Chang, E., Roberts, R.: An improved algorithm for decentralized extrema-finding in circular configurations of processes. Commun. ACM 22(5), 281–283 (1979)
Gulati, A., Shanmuganathan, G., Ahmad, I., Holler, A.: Cloud scale resource management: challenges and techniques. In: HotCloud 2011 Proceedings of the 3rd USENIX Conference on Hot Topics in Cloud Computing, Berkeley, CA, USA, p. 3. USENIX Association (2011)
Iordache, G.V., Boboila, M.S., Pop, F., Stratan, C., Cristea, V.: A decentralized strategy for genetic scheduling in heterogeneous environments. Multiagent Grid Syst. 3(4), 355–367 (2007)
Krieder, S.J., et al.: Design and evaluation of the GeMTC framework for GPU-enabled many-task computing. In: HPDC 2014. ACM (2014)
Lua, E.K., Crowcroft, J., Pias, M., Sharma, R., Lim, S.: A survey and comparison of peer-to-peer overlay network schemes. Commun. Surv. Tutor. 7(2), 72–93 (2005)
Moise, D., Moise, E., Pop, F., Cristea, V.: Resource coallocation for scheduling tasks with dependencies, in grid. arXiv preprint arXiv:1106.5309 (2011)
Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. SIGCOMM Comput. Commun. Rev. 31(4), 149–160 (2001)
Wang, K., Brandstatter, K., Raicu, I.: SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascale. In: HPC 2013. ACM (2013)
Wang, K., Brandstatter, K., Raicu, I.: SimMatrix: simulator for many-task computing execution fabric at exascale. In: Proceedings of the High Performance Computing Symposium, HPC 2013, San Diego, CA, USA, pp. 9:1–9:9. Society for Computer Simulation International (2013)
Xu, Y., Suarez, A., Zhao, M.: IBIS : Interposed big-data I/O scheduler. In: Proceedings of the 22nd International Symposium on High-performance Parallel and Distributed Computing, pp. 109–110. ACM, New York, June 2013
Zhang, Q., Cheng, L., Boutaba, R.: Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1(1), 7–18 (2010)
Zhao, Y., Raicu, I., Foster, I., Hategan, M., Nefedova, V., Wilde, M.: Realizing fast, scalable and reliable scientific computations in grid environments. In: Grid Computing Research Progress. Nova Publisher (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Oncioiu, AR., Pop, F., Esposito, C. (2019). Asymptotic Load Balancing Algorithm for Many Task Scheduling. In: Palattella, M., Scanzio, S., Coleri Ergen, S. (eds) Ad-Hoc, Mobile, and Wireless Networks. ADHOC-NOW 2019. Lecture Notes in Computer Science(), vol 11803. Springer, Cham. https://doi.org/10.1007/978-3-030-31831-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-31831-4_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31830-7
Online ISBN: 978-3-030-31831-4
eBook Packages: Computer ScienceComputer Science (R0)