Skip to main content
SpringerLink
Log in

Dynamic load balancing with adaptive factoring methods in scientific applications

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

To improve the performance of scientific applications with parallel loops, dynamic loop scheduling methods have been proposed. Such methods address performance degradations due to load imbalance caused by predictable phenomena like nonuniform data distribution or algorithmic variance, and unpredictable phenomena such as data access latency or operating system interference. In particular, methods such as factoring, weighted factoring, adaptive weighted factoring, and adaptive factoring have been developed based on a probabilistic analysis of parallel loop iterates with variable running times. These methods have been successfully implemented in a number of applications such as: N-Body and Monte Carlo simulations, computational fluid dynamics, and radar signal processing.

The focus of this paper is on adaptive weighted factoring (AWF), a method that was designed for scheduling parallel loops in time-stepping scientific applications. The main contribution of the paper is to relax the time-stepping requirement, a modification that allows the AWF to be used in any application with a parallel loop. The modification further allows the AWF to adapt to load imbalance that may occur during loop execution. Results of experiments to compare the performance of the modified AWF with the performance of the other loop scheduling methods in the context of three nontrivial applications reveal that the performance of the modified method is comparable to, and in some cases, superior to the performance of the most recently introduced adaptive factoring method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Balasubramaniam M, Barker K, Banicescu I, Chrisochoides N, Pabico JP, Cariño RL (2004) A novel dynamic load balancing library for cluster computing. In: Proceedings of the 3rd international symposium on parallel and distributed computing, in association with the international workshop on algorithms, models and tools for parallel computing on heterogeneous networks (ISPDC/HeteroPar’04). IEEE Computer Society Press, Los Alamitos, pp 346–352

    Chapter  Google Scholar 

  2. Banicescu I, Cariño RL (2005) Addressing the stochastic nature of scientific computations via dynamic loop scheduling. Electron Trans Numer Anal 21:66–80

    Google Scholar 

  3. Banicescu I, Cariño RL, Harvill JL, Lestrade JP (2005) Simulation of vector nonlinear time series on clusters. In: Proceedings of the 19th international parallel and distributed processing symposium—the 6th international workshop on parallel and distributed scientific and engineering computing (IPDPS-PDSEC 2005). IEEE Computer Society Press, Los Alamitos (on CD-ROM)

    Google Scholar 

  4. Banicescu I, Cariño RL, Harvill JL, Lestrade JP (2005) Computational challenges in vector functional coefficient autoregressive models. In: Proceedings of the international conference on computational science 2005 (ICCS 2005), part I. Springer, Berlin, pp 237–244

    Google Scholar 

  5. Banicescu I, Cariño RL, Harvill JL, Lestrade JP (2005) Vector nonlinear time-series analysis of gamma-ray burst datasets on heterogeneous clusters. Sci Program 13(2):415–422

    Google Scholar 

  6. Banicescu I, Flynn-Hummel S (1995) Balancing processor loads and exploiting data locality in n-body simulations. In: Proceedings of the 1995 ACM/IEEE conference on supercomputing. ACM, New York (on CD-ROM)

    Google Scholar 

  7. Banicescu I, Liu Z (2000) Adaptive factoring: dynamic scheduling method tuned to the rate of weight changes. In Proceedings of the high performance computing symposium (HPC 2000), pp 122–129

  8. Banicescu I, Velusamy V (2001) Performance of scheduling scientific applications with adaptive weighted factoring. In: Proceedings of the 15th IEEE international parallel and distributed processing symposium—10th heterogeneous computing workshop (IPDPS-HCW 2001). IEEE Computer Society Press, Los Alamitos (on CD-ROM)

    Google Scholar 

  9. Banicescu I, Velusamy V (2002) Load balancing highly irregular computations with the adaptive factoring. In: Proceedings of the 16th IEEE international parallel and distributed processing symposium—11th heterogeneous computing workshop (IPDPS-HCW 2002). IEEE Computer Society Press, Los Alamitos

    Google Scholar 

  10. Banicescu I, Velusamy V, Devaprasad J (2003) On the scalability of dynamic scheduling scientific applications with adaptive weighted factoring. Clust Comput: J Netw Softw Tools Appl 6(3):215–226

    Google Scholar 

  11. BATSE. (2005) Burst and transient source experiment BATSE home page. http://www.batse.msfc.nasa.gov/batse

  12. Bohm D (1952) A suggested interpretation of the quantum theory in terms of “hidden” variables. Phys Rev 85:166–193

    Article  MathSciNet  Google Scholar 

  13. Brook RG, Oppenheimer PE, Weatherford CA, Banicescu I, Zhu J (2001) Solving the hydrodynamic formulation of quantum mechanics: A parallel MLS method. Int J Quantum Chem 85(4–5):263–271

    Article  Google Scholar 

  14. Cariño RL, Banicescu I (2002) Load balancing parallel loops on message-passing systems. In: Akl SG, Gonzales T (eds) Proceedings of the 14th IASTED international conference on parallel and distributed computing and systems (PDCS 2004). Acta Press, Calgary, pp 362–367

    Google Scholar 

  15. Cariño RL, Banicescu I (2005) A framework for statistical analysis of datasets on heterogeneous clusters. In: Proceedings of the 2005 IEEE international conference on cluster computing. IEEE, New York (on CD-ROM)

    Google Scholar 

  16. Cariño RL, Banicescu I (2005) A load balancing tool for distributed parallel loops. Clust Comput 8(4):313–321

    Article  Google Scholar 

  17. Cariño RL, Banicescu I, Rauber T, Runger G (2004) Dynamic loop scheduling on processor groups. In Proceedings of the 17th international conference on parallel and distributed computing systems (PDCS 2004), pp 78–84. International Society for Computers and Their Applications

  18. Cariño RL (1992) Numerical integration over finite regions using extrapolation by nonlinear sequence transformations. PhD thesis, La Trobe University, Melbourne, Australia

  19. Cariño RL, Banicescu I (2006) A Dynamic load Balancing tool For one- and Two-dimensional parallel loops. In: Proceedings of the 5th international symposium on parallel and distributed computing. IEEE Computer Society, Los Alamitos, pp 107–114

    Chapter  Google Scholar 

  20. Cariño RL, Banicescu I, Lim H, Williams N, Kim S (2006) Simulation of a hybrid model for image denoising. In: Proceedings of the 20th international parallel and distributed processing symposium. IEEE Computer Society, Los Alamitos (on CD-ROM)

    Google Scholar 

  21. Cariño RL, Banicescu I, Vadapalli RK, Weatherford CA, Zhu J (2003) Parallel adaptive quantum trajectory Method for Wavepacket simulation. In: Proceedings of the 17th international parallel and distributed Processing Symposium—4th workshop on parallel and distributed scientific and engineering applications. IEEE Computer Society Press, Los Alamitos (on CD-ROM)

    Google Scholar 

  22. Cariño RL, Banicescu I, Vadapalli RK, Weatherford CA, Zhu J (2004) Message passing parallel adaptive quantum trajectory method. Kluwer Academic, Dordrecht, pp 127–139

    Google Scholar 

  23. Flynn-Hummel S, Schmidt J, Uma RN, Wein J (1996) Load-sharing in heterogeneous systems via weighted factoring. In: Proceedings of the 8th annual ACM symposium on parallel algorithms and architectures, pp 318–328

  24. Flynn-Hummel S, Schonberg E, Flynn LE (1992) Factoring: A method for scheduling parallel loops. Commun ACM 35(8):90–101

    Article  Google Scholar 

  25. Harvill JL, Ray BK (2005) A note on multi-step forecasting with functional coefficient autoregressive models. Int J Forecast 21(4):717–727

    Article  Google Scholar 

  26. Harvill JL, Ray BK (2006) Functional coefficient autoregressive models for vector time series. Comput Stat Data Anal 50(12):3547–3566

    Article  MathSciNet  MATH  Google Scholar 

  27. HETE. (2005) High energy transient explorer HETE home page. http://space.mit.edu/HETE

  28. Kruskal CP, Weiss A (1985) Allocating independent subtasks on parallel processors. IEEE Trans Softw Eng 11(10):1001–1016

    Article  Google Scholar 

  29. Lopreore CL, Wyatt RE (1999) Quantum wave packet dynamics with trajectories. Phys Rev Lett 82:5190–5193

    Article  Google Scholar 

  30. Luke E, Banicescu I, Li J (1998) The optimal effectiveness metric for parallel application analysis. Inf. Process. Lett., special issue on parallel models 66(5):223–229

    MATH  MathSciNet  Google Scholar 

  31. Lyness J, Kaganove J (1976) Comments on the nature of automatic quadrature routines. ACM Trans Math Softw 2:65–81

    Article  MATH  MathSciNet  Google Scholar 

  32. Polychronopoulos CD, Kuck DJ (1987) Guided self-scheduling: a practical scheduling scheme for parallel supercomputers. IEEE Trans Comput 36(12):1425–1439

    Article  Google Scholar 

  33. Cariño RL, Robinson I, de Doncker E (1994) Adaptive cubature over a collection of triangles using the d-transformation. J Comput Appl Math 50:171–183

    Article  MATH  MathSciNet  Google Scholar 

  34. Tzen TH, Ni LM (1993) Trapezoid self-scheduling: a practical scheduling scheme for parallel computers. IEEE Trans Parallel Distrib Syst 4(1):87–98

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Advanced Vehicular Systems—HPCC, Mississippi State University, Mississippi State, MS, 39762, USA

    Ricolindo L. Cariño

  2. Department of Computer Science and Engineering, and Center for Computational Sciences—HPCC, Mississippi State University, Mississippi State, MS, 39762, USA

    Ioana Banicescu

Authors
  1. Ricolindo L. Cariño
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioana Banicescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricolindo L. Cariño.

Additional information

This work was partially supported by the National Science Foundation Grants: #9984465, #0081303, #0313274, #0085969 and #0132618.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cariño, R.L., Banicescu, I. Dynamic load balancing with adaptive factoring methods in scientific applications. J Supercomput 44, 41–63 (2008). https://doi.org/10.1007/s11227-007-0148-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-007-0148-y

Keywords

Search

Navigation