Skip to main content
SpringerLink
Log in

A Load Balancing Tool for Distributed Parallel Loops

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Large scale applications typically contain parallel loops with many iterates. The iterates of a parallel loop may have variable execution times which translate into performance degradation of an application due to load imbalance. This paper describes a tool for load balancing parallel loops on distributed-memory systems. The tool assumes that the data for a parallel loop to be executed is already partitioned among the participating processors. The tool utilizes the MPI library for interprocessor coordination, and determines processor workloads by loop scheduling techniques. The tool was designed independent of any application; hence, it must be supplied with a routine that encapsulates the computations for a chunk of loop iterates, as well as the routines to transfer data and results between processors. Performance evaluation on a Linux cluster indicates that the tool reduces the cost of executing a simulated irregular loop without load balancing by up to 81%. The tool is useful for parallelizing sequential applications with parallel loops, or as an alternate load balancing routine for existing parallel applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. Biswas, S. Das, D. Harvey and L. Oliker, Portable parallel programming for dynamic load balancing of unstructured grid applications, in: Proceedings of 13th International Parallel Processing Symposium, San Juan, Puerto Rico (1999) pp. 338–342.

  2. I. Banicescu and S.F. Hummel, Balancing processor loads and exploiting data locality in n-Body Simulations, in: Proceedings of Supercomputing '95, San Diego CA, USA (1995) (On CDROM).

  3. I. Banicescu and V. Velusamy, Load balancing highly irregular computations with the adaptive factoring, in: Proceedings of the IEEE-International Parallel and Distributed Processing Symposium 2002—Heterogeneous Computing Workshop, Fort Lauderdale FL, USA (2002) (on CD-ROM).

  4. I. Banicescu, V. Velusamy and J. Devaprasad, On the scalability of dynamic scheduling scientific applications with adaptive weighted factoring, in: Cluster Computing, The Journal of Networks, Software Tools and Applications (2003) pp. 215–226.

  5. R.L. Carino and I. Banicescu, Load balancing parallel loops on message-passing systems, in: S.G. Akl and T. Gonzales (eds.), Proceedings of the 14th IASTED International Conference on Parallel and Distributed Computing and Systems, Cambridge MA, USA (2002) pp. 362–367.

  6. R.L. Carino, I. Banicescu, R.K. Vadapalli, C.A. Weatherford and J. Zhu, Message-passing parallel adaptive quantum trajectory method, in: L. T. Yang and Y. Pan (eds.), High Performance Scientific and Engineering Computing: Hardware/Software Support, Kluwer Academic Publishers (2004) pp. 127–139.

  7. J. Chen and V. Taylor, Mesh partitioning for distributed systems: exploring optimal number of partitions with local and remote communication, in: Proceedings of 9th SIAM Conference on Parallel Processing for Scientific Computing, San Antonio TX, USA (1999) (On CDROM).

  8. S. Flynn Hummel, J. Schmidt, R.N. Uma and J. Wein, Load-sharing in heterogeneous systems via weighted factoring, in: Proceedings of Symposium on Parallel Algorithms and Architectures, Padua, Italy (1996) pp. 318–328.

  9. S. Flynn Hummel, E. Schonberg and L.E. Flynn, Factoring: A method for scheduling parallel loops, Communications of the ACM 35(8) (1992) 90–101.

    Google Scholar 

  10. C. Kruskal and A. Weiss, Allocating independent subtasks on parallel processors, in: IEEE Trans. Software Eng SE-11(10) (1985) pp. 1001–1016.

    Google Scholar 

  11. E. Luke, I. Banicescu and J. Li, The optimal effectiveness metric for parallel application analysis, Information Processing Letters—Special Issue on Parallel Models, Elsevier, North Holland 66(5) (1998) 223-229.

    Google Scholar 

  12. B. Maerten, D. Roose, A. Basermann and J. Fingberg, DRAMA: A library for parallel dynamic load balancing of finite element applications, in: Proceedings of 9th SIAM Conference on Parallel Processing for Scientific Computing, San Antonio TX, USA (1999) (On CDROM).

  13. F.H. McMahon, The livermore fortran kernels: a computer test of the numerical performance range, lawrence livermore national laboratory, Livermore, California, UCRL-53745, 1986.

  14. C. Polychronopoulos and D. Kuck, Guided self-scheduling: A practical scheduling scheme for parallel supercomputers, IEEE Trans. on Computers C-36(12) (1987) 1425–1439.

    Google Scholar 

  15. K. Schloegel, G. Karypis and V. Kumar, Dynamic repartitioning of adaptively refined meshes, in: Proceedings of Supercomputing '98, Orlando FL, USA (1998) (On CDROM).

  16. J. Singh, Parallel Hierarchical N-body Methods and their Implications for Multiprocessors PhD thesis (Stanford University, 1993).

  17. A. Sohn and H. Simon, S-HARP: A scalable parallel dynamic partitioner for adaptive mesh-based computations, in: Proceedings of Supercomputing '98, Orlando FL, USA (1998) (On CDROM).

  18. M. Warren and J. Salmon, A parallel hashed oct tree n-body algorithm, in: Proceedings of Supercomputing '93, Portland OR, USA (1993) pp. 12–21.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, and Center for Computational Sciences ERC, Mississippi State University, Mississippi State, MS 39762

    Ricolindo L. Cariño & Ioana Banicescu

Authors
  1. Ricolindo L. Cariño
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioana Banicescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricolindo L. Cariño.

Additional information

Ricolindo L. Cariño finished his B.Sc. from the University of the Philippines Los Baños (UPLB) in 1983, and his Ph.D. from La Trobe University, Melbourne, Australia in 1993. He was an Associate Professor of Computer Science in UPLB, retiring early in 2002. Currently, he is a research faculty in the Center for Computational Sciences ERC, Mississippi State University. His interests include high performance computing for scientific applications.

Ioana Banicescu is associate professor in the Department of Computer Science at Mississippi State University (MSU), and chair of the Scientific Computing research focus group. She received the Dipl. in Engineering (Electronics and Telecommunications) from Polytechnic University, Bucharest - Romania, and the M.S. and the Ph.D. degrees in Computer Science from Polytechnic University. New York. She is the recipient the National Science Foundation (NSF) Early CAREER award in 2000, and three NSF Information Technology Awards. Ioana Banicescu's research interests include parallel algorithms, scientific computing, scheduling theory, load balancing algorithms, performance analysis, optimization, and prediction. She also holds a research faculty position in the Center for Computational Sciences at MSU, where she presently is involved in the development of dynamic scheduling algorithms for scalable parallelization and performance optimization of problems in computational sciences. She has given many invited talks at universities, government laboratories, and at various national and international forums in the United States and overseas. She has authored and co-authored more than 50 articles published in journals, books, and conference proceedings. Ioana Banicescu is a senior member of the IEEE Computer Society, ACM, SIAM and AMS, serves on two journal editorial boards, on organizing, steering and program committees of a number of international conferences, symposia and workshops, and on the Executive Board of the IEEE Technical Committee on Parallel Processing. She received the Hearing Professor of Engineering Award in 2001, the Outstanding Faculty Researcher Award of Bagley College of Engineering at MSU in 2004, and the University Faculty Research Award for College of Engineering in 2005.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cariño, R.L., Banicescu, I. A Load Balancing Tool for Distributed Parallel Loops. Cluster Comput 8, 313–321 (2005). https://doi.org/10.1007/s10586-005-4098-y

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-005-4098-y

Keywords

Search

Navigation