Skip to main content
SpringerLink
Log in

On the Price of Heterogeneity in Parallel Systems

  • Published:
Theory of Computing Systems Aims and scope Submit manuscript

Abstract

Suppose we have a parallel or distributed system whose nodes have limited capacities, such as processing speed, bandwidth, memory, or disk space. How does the performance of the system depend on the amount of heterogeneity of its capacity distribution? We propose a general framework to quantify the worst-case effect of increasing heterogeneity in models of parallel systems. Given a cost function g(C,W) representing the system’s performance as a function of its nodes’ capacities C and workload W (such as the makespan of an optimum schedule of jobs W on machines C), we say that g has price of heterogeneity α when for any workload, cost cannot increase by more than a factor α if node capacities become arbitrarily more heterogeneous. The price of heterogeneity also upper bounds the “value of parallelism”: the maximum benefit obtained by increasing parallelism at the expense of decreasing processor speed. We give constant or logarithmic bounds on the price of heterogeneity of several well-known job scheduling and graph degree/diameter problems, indicating that in many cases, increasing heterogeneity can never be much of a disadvantage.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Planetlab: http://www.planet-lab.org/

  2. Almeida, V.A.F., Vasconcelos, I.M.M., Árabe, J.N.C., Menascé, D.A.: Using random task graphs to investigave the potential benefits of heterogeneity in parallel systems. In: Proc. ACM/IEEE Conference on Supercomputing (1992)

  3. Almeida, V., Menascé, D.: Cost-performance analysis of heterogeneity in supercomputer architectures. In: Proc. ACM/IEEE Conference on Supercomputing (1990)

  4. Chawathe, Y., Ratnasamy, S., Breslau, L., Lanham, N., Shenker, S.: Making Gnutella-like P2P systems scalable. In: Proceedings of ACM SIGCOMM (2003)

  5. Chudak, F.A., Shmoys, D.B.: Approximation algorithms for precedence-constrained scheduling problems on parallel machines that run at different speeds. In: Proc. 8th Ann. ACM-SIAM Symp. on Discrete Algorithms (SODA), pp. 581–590 (1997)

  6. Comellas, F., Delorme, C.: The (degree, diameter) problem for graphs: http://www-mat.upc.es/grup_de_grafs/grafs/taula_delta_d.html

  7. Foster, I., Iamnitchi, A.: On death, taxes, and the convergence of peer-to-peer and grid computing. In: Proc. IPTPS (2002)

  8. Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. Freeman, New York (1979)

    MATH  Google Scholar 

  9. Godfrey, B., Lakshminarayanan, K., Surana, S., Karp, R., Stoica, I.: Load balancing in dynamic structured P2P systems. In: Proc. IEEE INFOCOM, Hong Kong (2004)

  10. Brighten, P.G., Stoica, I.: Heterogeneity and load balance in distributed hash tables. In: Proc. IEEE INFOCOM (2005)

  11. Graham, R.L.: Bounds on multiprocessing timing anomalies. Bell Syst. Tech. J. 45, 1563–1581 (1966)

    Google Scholar 

  12. Koutsoupias, E.: Coordination mechanisms for congestion games. In: Sigact News, December 2004

  13. Ledlie, J., Seltzer, M.: Distributed, secure load balancing with skew, heterogeneity, and churn. In: Proc. INFOCOM (2005)

  14. Lowney, G.: Invited talk: Why intel is designing multi-core processors. In: Proc. SPAA (2006)

  15. Marshall, A.W., Olkin, I.: Inequalities: Theory of Majorization and its Applications. Academic Press, New York (1979)

    MATH  Google Scholar 

  16. Moreno, J.H.: Invited talk: Chip-level integration: The new frontier for microprocessor architecture. In: Proc. SPAA (2006)

  17. Park, K., Pai, V.: Comon: A monitoring infrastructure for PlanetLab. http://comon.cs.princeton.edu/

  18. Ratnasamy, S., Shenker, S., Stoica, I.: Routing algorithms for DHTs: Some open questions. In: Proc. IPTPS (2002)

  19. Saroiu, S., Krishna Gummadi, P., Gribble, S.D.: A measurement study of peer-to-peer file sharing systems. In: Proc. MMCN, San Jose, CA, USA, January 2002

  20. Stidham, S.: On the optimality of single-server queueing systems. Oper. Res. 18, 708–732 (1970)

    Article  MATH  MathSciNet  Google Scholar 

  21. Suri, S., Tóth, C.D., Zhou, Y.: Selfish load balancing and atomic congestion games. In: Proc. SPAA (2004)

  22. Wierman, A., Osogami, T., Harchol-Balter, M., Scheller-Wolf, A.: How many servers are best in a dual-priority FCFS system? Perform. Eval. 63(12), 1253–1272 (2006)

    Article  Google Scholar 

  23. Yang, X., de Veciana, G.: Service capacity of peer to peer networks. In: Proc. INFOCOM (2004)

Download references

Author information

Authors and Affiliations

  1. Computer Science Division, UC Berkeley, Berkeley, CA, 94720, USA

    P. Brighten Godfrey & Richard M. Karp

Authors
  1. P. Brighten Godfrey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard M. Karp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. Brighten Godfrey.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Godfrey, P.B., Karp, R.M. On the Price of Heterogeneity in Parallel Systems. Theory Comput Syst 45, 280–301 (2009). https://doi.org/10.1007/s00224-008-9102-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00224-008-9102-5

Keywords

Search

Navigation