Skip to main content
Springer Nature Link
Log in

An experimental study of scheduling algorithms for many-task applications

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The paper studies the performance of algorithms for scheduling of many-task applications in distributed computing systems. Two important classes of such applications are considered: bags-of-tasks and workflows. The comparison of algorithms is performed on the basis of discrete-event simulation for various application cases and system configurations. The developed simulation framework based on SimGrid toolkit provides the necessary tools for implementation of scheduling algorithms, generation of synthetic systems and applications, execution of simulation experiments and analysis of results. This allowed to perform a large number of experiments in a reasonable amount of time and to ensure reproducible results. The conducted experiments demonstrate the dependence of the performance of studied algorithms on various application and system characteristics. While confirming the performance advantage of advanced static algorithms, the presented results reveal some interesting insights. In particular, the accuracy of the used network model helped to demonstrate the limitations of simple analytical models for scheduling of data-intensive parallel applications with static algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. https://github.com/alexmnazarenko/pysimgrid.

  2. https://github.com/frs69wq/daggen.

References

  1. Arabnejad H, Barbosa JG (2014) List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans Parallel Distrib Syst 25(3):682–694

    Article  Google Scholar 

  2. Arabnejad H, Barbosa JG, Prodan R (2016) Low-time complexity budget-deadline constrained workflow scheduling on heterogeneous resources. Future Gener Comput Syst 55:29–40

    Article  Google Scholar 

  3. Armstrong R, Hensgen D, Kidd T (1998) The relative performance of various mapping algorithms is independent of sizable variances in run-time predictions. In: 1998 Seventh Heterogeneous Computing Workshop. (HCW 98) Proceedings. IEEE, pp 79–87

  4. Bharathi S, Chervenak A, Deelman E, Mehta G, Su MH, Vahi K (2008) Characterization of scientific workflows. In: 2008 Third Workshop on Workflows in Support of Large-Scale Science, pp 1–10

  5. Bittencourt LF, Sakellariou R, Madeira ERM (2010) Dag scheduling using a lookahead variant of the heterogeneous earliest finish time algorithm. In: 2010 18th Euromicro Conference on Parallel, Distributed and Network-Based Processing, pp 27–34. https://doi.org/10.1109/PDP.2010.56

  6. Casanova H, Giersch A, Legrand A, Quinson M, Suter F (2014) Versatile, scalable, and accurate simulation of distributed applications and platforms. J Parallel Distrib Comput 74(10):2899–2917

    Article  Google Scholar 

  7. Chen W, Deelman E (2012) Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-science (e-science). IEEE, pp 1–8

  8. Freund RF, Gherrity M, Ambrosius S, Campbell M, Halderman M, Hensgen D, Keith E, Kidd T, Kussow M, Lima JD et al (1998) Scheduling resources in multi-user, heterogeneous, computing environments with smartnet. In: 1998 Seventh Heterogeneous Computing Workshop. (HCW 98) Proceedings. IEEE, pp 184–199

  9. Graham RL, Lawler EL, Lenstra JK, Kan AR (1979) Optimization and approximation in deterministic sequencing and scheduling: a survey. Ann Discrete Math 5:287–326

    Article  MathSciNet  Google Scholar 

  10. Hunold S, Rauber T, Suter F (2008) Scheduling dynamic workflows onto clusters of clusters using postponing. In: 8th IEEE International Symposium on Cluster Computing and the Grid. CCGRID’08. IEEE, pp 669–674

  11. Maheswaran M, Ali S, Siegal HJ, Hensgen D, Freund RF (1999) Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems. In: Eighth Heterogeneous Computing Workshop. (HCW’99) Proceedings. IEEE, pp 30–44

  12. Nazarenko A, Sukhoroslov O (2017) An experimental study of workflow scheduling algorithms for heterogeneous systems. In: Malyshkin V (ed) Parallel computing technologies. Springer, Cham, pp 327–341

    Chapter  Google Scholar 

  13. Raicu I, Foster IT, Zhao Y (2008) Many-task computing for grids and supercomputers. In: Workshop on Many-Task Computing on Grids and Supercomputers. MTAGS 2008. IEEE, pp 1–11

  14. Sih GC, Lee EA (1993) A compile-time scheduling heuristic for interconnection-constrained heterogeneous processor architectures. IEEE Trans Parallel Distrib Syst 4(2):175–187

    Article  Google Scholar 

  15. Taylor IJ, Deelman E, Gannon DB, Shields M (2014) Workflows for e-Science: scientific workflows for grids. Springer, Incorporated

  16. Tobita T, Kasahara H (2002) A standard task graph set for fair evaluation of multiprocessor scheduling algorithms. J Sched 5(5):379–394

    Article  MathSciNet  Google Scholar 

  17. Topcuoglu H, Hariri S, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274. https://doi.org/10.1109/71.993206

    Article  Google Scholar 

  18. Velho P, Legrand A (2009) Accuracy study and improvement of network simulation in the simgrid framework. In: Proceedings of the 2nd International Conference on Simulation Tools and Techniques, p 13. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)

  19. Velho P, Schnorr LM, Casanova H, Legrand A (2013) On the validity of flow-level tcp network models for grid and cloud simulations. ACM Trans Model Comput Simul (TOMACS) 23(4):23

    Article  MathSciNet  Google Scholar 

  20. Yu J, Buyya R, Ramamohanarao K (2008) Workflow scheduling algorithms for grid computing. In: Xhafa F, Abraham A (eds) Metaheuristics for scheduling in distributed computing environments. Studies in computational intelligence, vol 146. Springer, Berlin, Heidelberg, pp 173–214

Download references

Author information

Authors and Affiliations

  1. Institute for Information Transmission Problems of the Russian Academy of Sciences, Moscow, Russia

    Oleg Sukhoroslov & Alexey Nazarenko

  2. National Research University Higher School of Economics, Moscow, Russia

    Oleg Sukhoroslov & Roman Aleksandrov

Authors
  1. Oleg Sukhoroslov
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Alexey Nazarenko
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Roman Aleksandrov
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Oleg Sukhoroslov.

Additional information

This work is supported by the Russian Science Foundation (Project No. 16-11-10352).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sukhoroslov, O., Nazarenko, A. & Aleksandrov, R. An experimental study of scheduling algorithms for many-task applications. J Supercomput 75, 7857–7871 (2019). https://doi.org/10.1007/s11227-018-2553-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2553-9

Keywords