Skip to main content
Springer Nature Link
Log in

Asynchronous Iterative Algorithms

  • Reference work entry
Encyclopedia of Parallel Computing

  • 635 Accesses

Synonyms

Asynchronous iterations; Asynchronous iterative computations

Definition

In iterative algorithms, a grid of data points are updated iteratively until some convergence criterion is met. The update of each data point depends on the latest updates of its neighboring points. Asynchronous iterative algorithms refer to a class of parallel iterative algorithms that are capable of relaxing strict data dependencies, hence not requiring the latest updates when they are not ready, while still ensuring convergence. Such relaxation may result in the use of inconsistent data which potentially may lead to an increased iteration count and hence increased computational operations. On the other hand, the time spent on waiting for the latest updates performed on remote processors may be reduced. Where waiting time dominates the computation, a parallel program based on an asynchronous algorithm may outperform its synchronous counterparts.

Discussion

Introduction

Scaling application performance to...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bibliography

  1. Bahi JM, Contassot-Vivier S, Couturier R, Vernier F (2005) A decentralized convergence detection algorithm for asynchronous parallel iterative algorithms. IEEE Trans Parallel Distrib Syst 16(1):4–13

    Article  Google Scholar 

  2. Bahi JM, Contassot-Vivier S, Couturier R (2002) Asynchronism for iterative algorithms in a global computing environment. In: 16th Annual International Symposium on high performance computing systems and applications (HPCS’02). IEEE, Moncton, Canada, pp 90–97

    Google Scholar 

  3. Bahi JM, Contassot-Vivier S, Couturier R (2003) Coupling dynamic load balancing with asynchronism in iterative algorithms on the computational Grid. In: 17th International Parallel and Distributed Processing Symposium (IPDPS’03), p 40. IEEE, Nice, France

    Google Scholar 

  4. Bahi JM, Contassot-Vivier S, Couturier R (2004) Performance comparison of parallel programming environments for implementing AIAC algorithms. In: 18th International Parallel and Distributed Processing Symposium (IPDPS’04). IEEE, Santa Fe, USA

    Google Scholar 

  5. Bahi JM, Contassot-Vivier S, Couturier R (2007) Parallel iterative algorithms: from sequential to Grid computing. Chapman & Hall/CRC, Boca Raton, FL

    Google Scholar 

  6. Bahi JM, Domas S, Mazouzi K (2004) Combination of Java and asynchronism for the Grid: a comparative study based on a parallel power method. In: 18th International Parallel and Distributed Processing Symposium (IPDPS ’04), pp 158a, 8. IEEE, Santa Fe, USA, April 2004

    Google Scholar 

  7. Bahi JM, Domas S, Mazouzi K (2004). Jace: a Java environment for distributed asynchronous iterative computations. In: 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing (EUROMICRO-PDP’04), pp 350–357. IEEE, Coruna, Spain

    Chapter  Google Scholar 

  8. Baudet GM (1978) Asynchronous iterative methods for multiprocessors. JACM 25(2):226–244

    Article  MATH  MathSciNet  Google Scholar 

  9. El Baz D (1996) A method of terminating asynchronous iterative algorithms on message passing systems. Parallel Algor Appl 9:153–158

    Article  MATH  Google Scholar 

  10. El Baz D (2007) Communication study and implementation analysis of parallel asynchronous iterative algorithms on message passing architectures. In: Parallel, distributed and network-based processing, 2007. PDP ’07. 15th EUROMICRO International Conference, pp 77–83. Weimar, Germany

    Chapter  Google Scholar 

  11. El Baz D, Gazen D, Jarraya M, Spiteri P, Miellou JC (1998) Flexible communication for parallel asynchronous methods with application to a nonlinear optimization problem. D’Hollander E, Joubert G et al (eds). In: Advances in Parallel Computing: Fundamentals, Application, and New Directions. North Holland, vol 12, pp 429–436

    Chapter  Google Scholar 

  12. El Baz D, Spiteri P, Miellou JC, Gazen D (1996) Asynchronous iterative algorithms with flexible communication for nonlinear network flow problems. J Parallel Distrib Comput 38(1):1–15

    Article  Google Scholar 

  13. Bertsekas DP (1983) Distributed asynchronous computation of fixed points. Math Program 27(1):107–120

    Article  MATH  MathSciNet  Google Scholar 

  14. Bertsekas DP, Tsitsiklis JN (1989) Parallel and distributed computation. Prentice-Hall, Englewood Cliffs, NJ

    MATH  Google Scholar 

  15. Blathras K, Szyld DB, Shi Y (1999) Timing models and local stopping criteria for asynchronous iterative algorithms. J Parallel Distrib Comput 58(3):446–465

    Article  Google Scholar 

  16. Blondel VD, Hendrickx JM, Olshevsky A, Tsitsiklis JN (2006) Convergence in multiagent coordination, consensus, and flocking. In: Decision and Control, 2005 and 2005 European Control Conference. CDC-ECC’05. 44th IEEE Conference on, pp 2996–3000

    Google Scholar 

  17. Chazan D, Miranker WL (1969) Chaotic relaxation. J Linear Algebra Appl 2:199–222

    Article  MATH  MathSciNet  Google Scholar 

  18. Couturier R, Domas S (2007) CRAC: a Grid environment to solve scientific applications with asynchronous iterative algorithms. In: Parallel and Distributed Processing Symposium, 2007. IPDPS 2007. IEEE International, p 1–8

    Google Scholar 

  19. Elsner L, Koltracht I, Neumann M (1990) On the convergence of asynchronous paracontractions with application to tomographic reconstruction from incomplete data. Linear Algebra Appl 130:65–82

    Article  MATH  MathSciNet  Google Scholar 

  20. Frommer A, Szyld DB (2000) On asynchronous iterations. J Comput Appl Math 123(1–2):201–216

    Article  MATH  MathSciNet  Google Scholar 

  21. Frommer A, Schwandt H, Szyld DB (1997) Asynchronous weighted additive schwarz methods. ETNA 5:48–61

    MATH  MathSciNet  Google Scholar 

  22. Frommer A, Szyld DB (1994) Asynchronous two-stage iterative methods. Numer Math 69(2):141–153

    Article  MATH  MathSciNet  Google Scholar 

  23. Liu L, Li Z (2010) Improving parallelism and locality with asynchronous algorithms. In: 15th ACM SIGPLAN Symposium on principles and practice of parallel programming (PPoPP), pp 213–222, Bangalore, India

    Chapter  Google Scholar 

  24. Lubachevsky B, Mitra D (1986) A chaotic, asynhronous algorithm for computing the fixed point of a nonnegative matrix of unit spectral radius. JACM 33(1):130–150

    Article  MATH  MathSciNet  Google Scholar 

  25. Miellou JC, El Baz D, Spiteri P (1998) A new class of asynchronous iterative algorithms with order intervals. Mathematics of Computation, 67(221):237–255

    Article  MATH  MathSciNet  Google Scholar 

  26. Moga AC, Dubois M (1996) Performance of asynchronous linear iterations with random delays. In: Proceedings of the 10th International Parallel Processing Symposium (IPPS ’96), pp 625–629

    Google Scholar 

  27. Song Y, Li Z (1999) New tiling techniques to improve cache temporal locality. ACM SIGPLAN Notices ACM SIGPLAN Conf Program Lang Design Implement 34(5):215–228

    Google Scholar 

  28. Spiteri P, Chau M (2002) Parallel asynchronous Richardson method for the solution of obstacle problem. In: Proceedings of 16th Annual International Symposium on High Performance Computing Systems and Applications, Moncton, Canada, pp 133–138

    Google Scholar 

  29. Strikwerda JC (2002) A probabilistic analysis of asynchronous iteration. Linear Algebra Appl 349(1–3):125–154

    Article  MATH  MathSciNet  Google Scholar 

  30. Su Y, Bhaya A, Kaszkurewicz E, Kozyakin VS (1998) Further results on convergence of asynchronous linear iterations. Linear Algebra Appl 281(1–3):11–24

    Article  MATH  MathSciNet  Google Scholar 

  31. Szyld DB (2001) Perspectives on asynchronous computations for fluid flow problems. First MIT Conference on Computational Fluid and Solid Mechanics, pp 977–980

    Chapter  Google Scholar 

  32. Szyld DB, Xu JJ (2000) Convergence of some asynchronous nonlinear multisplitting methods. Num Algor 25(1–4):347–361

    Article  MATH  MathSciNet  Google Scholar 

  33. Uresin A, Dubois M (1996) Effects of asynchronism on the convergence rate of iterative algorithms. J Parallel Distrib Comput 34(1):66–81

    Article  Google Scholar 

  34. Wolfe M (1989) More iteration space tiling. In: Proceedings of the 1989 ACM/IEEE conference on Supercomputing, p 664. ACM, Reno, NV

    Google Scholar 

  35. Kollias G, Gallopoulos E, Szyld DB (2006) Asynchronous iterative computations with Web information retrieval structures: The PageRank case. In: Joubert GR, Nagel WE, et al (Eds). Parallel Computing: Current and Future issues of High-End computing, NIC Series. John von Neumann-Institut für Computing, Jülich, Germany, vol 33, pp 309–316

    Google Scholar 

  36. Kollias G, Gallopoulos E (2007) Asynchronous Computation of PageRank computation in an interactive multithreading environment. In: Frommer A, Mahoney MW, Szyld DB (eds) Web Information Retrieval and Linear Algebra Algorithms, Dagstuhl Seminar Proceedings. IBFI, Schloss Dagstuhl, Germany, ISSN: 1862–4405

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Purdue University, West Lafayette, IN, USA

    Giorgos Kollias, Ananth Y. Grama & Zhiyuan Li (Dr.)

Authors
  1. Giorgos Kollias
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ananth Y. Grama
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiyuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Illinois at Urbana-Champaign, Urbana, IL, USA

    David Padua

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this entry

Cite this entry

Kollias, G., Grama, A.Y., Li, Z. (2011). Asynchronous Iterative Algorithms. In: Padua, D. (eds) Encyclopedia of Parallel Computing. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-09766-4_231

Download citation

Publish with us

Policies and ethics

Search

Navigation