Definition
Load balancing in distributed memory systems is the process of redistributing work between hardware resources to improve performance, typically by moving work from overloaded resources to underloaded resources.
Discussion
In a parallel application, when work is distributed unevenly so that underloaded processors are forced to wait for overloaded processors, the application is suffering from load imbalance. Load imbalance is one of the key impediments in achieving high performance on large parallel machines, especially when solving highly dynamic and irregular problems. Load balancing is a technique that performs the task of distributing computation and communication load across the hardware resources of a parallel machine so that no single processor is overloaded. It can reduce processor idle time across the machine while also reducing communication costs by colocating related work on the same processor.
Balancing an application’s load involves making decisions about where...
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsBibliography
Devine K, Hendrickson B, Boman E, St. John M, Vaughan C (2000) Design of dynamic load-balancing tools for parallel applications. In: Proceedings of the international conference on supercomputing, Santa Fe, New Mexico, May 2000. ACM Press, New York
Kale LV, Zheng G (2009) Charm + + and AMPI: adaptive runtime strategies via migratable objects. In: Parashar M (ed) Advanced computational infrastructures for parallel and distributed applications. Wiley-Interscience, Hoboken, pp 265–282
Basermann A, Clinckemaillie J, Coupez T, Fingberg J, Digonnet H, Ducloux R, Gratien JM, Hartmann U, Lonsdale G, Maerten B, Roose D, Walshaw C (2000) Dynamic load balancing of finite element applications with the DRAMA library. Appl Math Model 25:83–98
Chombo Software Package for AMR Applications. (October 2010) http://seesar.lbl.gov/anag/chombo/. Accessed October 2010
Kumar V, Grama AY, Vempaty NR (1994) Scalable load balancing techniques for parallel computers. J Parallel Distrib Comput 22(1):60–79
Devine KD, Boman EG, Heaphy RT, Hendrickson BA, Teresco JD, Faik J, Flaherty JE, Gervasio LG (2005) New challenges in dynamic load balancing. Appl Numer Math 52(2–3): 133–152
Xu C, Lau FCM (1997) Load balancing in parallel computers theory and practice. Kluwer Academic Publishers, Boston
Zheng G (2005) Achieving high performance on extremely large parallel machines: performance prediction and load balancing
Zheng G, Meneses E, Bhatele A, Kale LV (2010) Hierarchical load balancing for Charm + + applications on large supercomputers. In: Proceedings of the third international workshop on parallel programming models and systems software for high-end computing (P2S2), San Diego
Chow YC, Kohler WH (1982) Models for dynamic load balancing in homogeneous multiple processor systems. IEEE Trans Comput c-36:667–679
Ni LM, Hwang K (1985) Optimal load balancing in a multiple processor system with many job classes. IEEE Trans Software Eng SE-11:491–496
Corradi A, Leonardi L, Zambonelli F (1999) Diffusive load balancing policies for dynamic applications. IEEE Concurrency 7(1):22–31
Willebeek-LeMair MC, Reeves AP (1993) Strategies for dynamic load balancing on highly parallel computers. IEEE Trans Parallel Distrib Syst 4(9):979–993
Sinha A, Kalé LV (1993) A load balancing strategy for prioritized execution of tasks. In: International parallel processing symposium, New Port Beach, CA, April 1993. IEEE Computer Society, Washington, DC, 230–237
Frigo M, Leiserson CE, Randall KH (1998) The implementation of the Cilk-5 multithreaded language. In: ACM SIGPLAN ’98 conference on programming language design and implementation (PLDI), Montreal, vol 33 of ACM SIGPLAN notices, ACM, New York, pp 212–223
Dinan J, Larkins DB, Sadayappan P, Krishnamoorthy S, Nieplocha J (2009) Scalable work stealing. In: SC ’09: proceedings of the conference on high performance computing networking, storage and analysis, Portland, OR. ACM, New York, pp 1–11
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this entry
Cite this entry
Becker, A., Zheng, G., Kalé, L.V. (2011). Load Balancing, Distributed Memory. In: Padua, D. (eds) Encyclopedia of Parallel Computing. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-09766-4_504
Download citation
DOI: https://doi.org/10.1007/978-0-387-09766-4_504
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-09765-7
Online ISBN: 978-0-387-09766-4
eBook Packages: Computer ScienceReference Module Computer Science and Engineering