Skip to main content
SpringerLink
Log in

Enabling scalable parallel implementations of structured adaptive mesh refinement applications

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Parallel implementations of dynamic structured adaptive mesh refinement (SAMR) methods lead to significant runtime management challenges that can limit their scalability on large systems. This paper presents a runtime engine that addresses the scalability of SAMR applications with localized refinements and high SAMR efficiencies on large numbers of processors (upto 1024 processors). The SAMR runtime engine augments hierarchical partitioning with bin-packing based load-balancing to manage the space-time heterogeneity of the SAMR grid hierarchy, and includes a communication substrate that optimizes the use of MPI non-blocking communication primitives. An experimental evaluation on the IBM SP2 supercomputer using the 3-D Richtmyer-Meshkov compressible turbulence kernel demonstrates the effectiveness of the runtime engine in improving SAMR scalability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. ASCI Alliance, http://www.llnl.gov/asci-alliances/asci-chicago.html, University of Chicago

  2. ASCI/ASAP Center, http://www.cacr.caltech.edu/ASAP, California Institute of Technology

  3. Bell J, Berger M, Saltzman J, Welcome M (1994) Three-dimensional adaptive mesh refinement for hyperbolic conservation laws. SIAM J Sci Comput 15(1):127–138

    Article  MATH  MathSciNet  Google Scholar 

  4. Berger M, Hedstrom G, Oliger J, Rodrigue G (1983) Adaptive mesh refinement for 1-dimensional gas dynamics. In: IMACS Trans Sci Comput. IMACS/North Holland, pp 43–47

  5. Berger M, Oliger J (1984) Adaptive mesh refinement for hyperbolic partial differential equations. J Comput Phys 53(March):484–512

    Article  MATH  MathSciNet  Google Scholar 

  6. Bryan G (1999) Fluids in the universe: adaptive mesh refinement in cosmology. Comput Sci Eng (March-April):46–53

  7. Chandra S, Sinha S, Parashar M, Zhang Y, Yang J, Hariri S (2002) Adaptive runtime management of SAMR applications. In: Sahni S, Prasanna V, Shukla U (eds), Proceedings of 9th international conference on high performance computing (HiPC’02), Lecture notes in computer science, Springer, Bangalore, India, vol 2552, December 2002, pp 564–574

  8. CHOMBO, http://seesar.lbl.gov/anag/chombo/, NERSC, ANAG of Lawrence Berkeley National Lab, CA, USA, 2003

  9. Choptuik M (1989) Experiences with an adaptive mesh refinement algorithm in numerical relativity. In: Evans C, Finn L, Hobill D (eds), Frontiers in numerical relativity. Cambridge University Press, London, pp 206–221

    Google Scholar 

  10. Hawley S, Choptuik M (2000) Boson stars driven to the brink of black hole formation. Phys Rev D 62:104024

    Article  Google Scholar 

  11. Hilbert D (1891) Uber die stetige Abbildung einer Linie auf Flachenstuck. Math Ann 38:459–460

    Article  MathSciNet  Google Scholar 

  12. Kohn S, SAMRAI Homepage, Structured adaptive mesh refinement applications infrastructure. http://www.llnl.gov/CASC/SAMRAI/, 1999

  13. Lan Z, Taylor V, Bryan G (2001) Dynamic load balancing for structured adaptive mesh refinement applications. In: Proceedings of international conference on parallel processing, Valencia, Spain, 2001, pp 571–579

  14. Lan Z, Taylor V, Bryan G (2001) Dynamic load balancing of SAMR applications on distributed systems. In: Proceedings of supercomputing conference (SC’01), Denver, CO, 2001

  15. Li X, Parashar M (2003) Dynamic load partitioning strategies for managing data of space and time heterogeneity in parallel SAMR applications. In: Kosch H, Boszormenyi L, Hellwagner H (eds), Proceedings of 9th international Euro-par conference (Euro-Par’03), Lecture Notes in Computer Science, Springer, Klagenfurt, Austria, vol 2790, August 2003, pp 181–188

  16. MacNeice P, Olson K, Mobarry C, de Fainchtein R, Packer C (2000) PARAMESH: a parallel adaptive mesh refinement community toolkit. Comput Phys Commun 126:330–354

    Article  MATH  Google Scholar 

  17. Moon B, Jagadish H, Faloutsos C, Saltz J (2001) Analysis of the clustering properties of the Hilbert space-filling curve. IEEE Trans Knowl Data Eng 13(1):124–141

    Article  Google Scholar 

  18. Norman M, Bryan G (1998) Cosmological adaptive mesh refinement. In: Miyama S, Tomisaka K (eds), Numerical astrophysics. Kluwer, Tokyo

  19. Parallel Environment (PE) for AIX V3R2.0, MPI programming guide, 2nd edn, December 2001

  20. Parashar M, Browne J (1996) On partitioning dynamic adaptive grid hierarchies. In: Proceedings of 29th annual Hawaii international conference on system sciences, January 1996, pp 604–613

  21. Parashar M, Browne J (1995) Distributed dynamic data structures for parallel adaptive mesh refinement. In: Proceedings of international conference for high performance computing, December 1995, pp 22–27

  22. Parashar M, Wheeler J, Pope G, Wang K, Wang P (1997) A new generation EOS compositional reservoir simulator: Part II—framework and multiprocessing. In: Society of petroleum engineering reservoir simulation symposium, Dallas, TX, June 1997

  23. Peano G (1890) Sur une courbe qui remplit toute une aire plane. Math Ann 36:157–160

    Article  MathSciNet  Google Scholar 

  24. Pember R, Bell J, Colella P, Crutchfield W, Welcome M (1993) Adaptive Cartesian grid methods for representing geometry in inviscid compressible flow. In: 11th AIAA computational fluid dynamics conference, Orlando, FL, July 1993

  25. Ray J, Najm H, Milne R, Devine K, Kempka S (2000) Triple flame structure and dynamics at the stabilization point of an unsteady lifted jet diffusion flame, Proc of Combust Inst 28(1):219–226

    Article  Google Scholar 

  26. Sagan H (1994) Space-filling curves. Springer

  27. Saif T (2004) Architecture specific communication optimizations for structured adaptive mesh refinement applications, M.S. thesis, Graduate School, Rutgers University

  28. Steensland J, Chandra S, Parashar M (2002) An application-centric characterization of domain-based SFC partitioners for parallel SAMR. IEEE Trans Parallel Distribut Syst 13(12):1275–1289

    Article  Google Scholar 

  29. Steinthorsson E, Modiano D (1995) Advanced methodology for simulation of complex flows using structured grid systems. In: Surface modeling, grid generation, and related issues in CFD solutions, NASA Conference Publication 3291, May 1995

  30. Wang P, Yotov I, Arbogast T, Dawson C, Parashar M, Sepehrnoori K (1997) A new generation EOS compositional reservoir simulator: Part I—formulation and discretization. In: Society of petroleum engineering reservoir simulation symposium, Dallas, TX, June 1997

  31. Wissink A, Hornung R, Kohn S, Smith S, Elliott N (2001) Large scale parallel structured AMR calculations using the SAMRAI framework. In: Proceedings of supercomputing conference (SC’01), Denver, CO, 2001

Download references

Author information

Authors and Affiliations

  1. The Applied Software Systems Laboratory, Department of Electrical and Computer Engineering, Organization to Rutgers, The State University of New Jersey, 94 Brett Road, Piscataway, NJ, 08854, USA

    Sumir Chandra, Xiaolin Li, Taher Saif & Manish Parashar

  2. Scalable Software Systems Laboratory, Department of Computer Science, Oklahoma State University, 219 MSCS, Stillwater, OK, 74078, USA

    Xiaolin Li

Authors
  1. Sumir Chandra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaolin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Taher Saif
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manish Parashar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sumir Chandra.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chandra, S., Li, X., Saif, T. et al. Enabling scalable parallel implementations of structured adaptive mesh refinement applications. J Supercomput 39, 177–203 (2007). https://doi.org/10.1007/s11227-007-0110-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-007-0110-z

Keywords

Search

Navigation